Commit Graph

263 Commits

Author SHA1 Message Date
Maksym Pavlenko
7efda48c53 devmapper: more precise way of checking if device is activated
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:26:46 -08:00
Maksym Pavlenko
37cdedc61c devmapper: add linux tags, fix build
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:26:46 -08:00
Maksym Pavlenko
0c6d194cce devmapper: add README and minor fixes
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
2218275ec9 devmapper: register plugin
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
cec72efc2a devmapper: add snapshotter
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
3a75882520 devmapper: add pool device manager
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
6e0ae68e17 devmapper: add snapshotter config
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
fcd9dc2749 devmapper: add pool metadata
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
809e5fd3b8 devmapper: add dmsetup
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
fe05e4d1a9 devmapper: add losetup
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Oliver Stenbom
7f5d7ff6b8 Update snapshots docs with garbage collector label
Signed-off-by: Oliver Stenbom <oliver@stenbom.eu>
2019-01-29 15:37:56 +00:00
Ace-Tang
7faaa64cf9 fix: miss remove temp file in createSnapshot
```
func foo() error {

defer func() {
    if err != nil {
	...
    }
}()
...
}
```
use defer func to do something when err not nil, if foo() not use
named error, `err != nil` can not catch all errors, since when err
re-defined in if condition, it is a new variable.

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-12-24 23:01:56 +08:00
zhangyue
7b1e6f323a fix: use func args instead of build new one
Signed-off-by: zhangyue <zy675793960@yeah.net>
2018-12-16 15:07:12 +08:00
Michael Crosby
16aaf6c065 Change unsupported snapshot warnings to INFO
Since there is no real action the user can do, these can safely be
informative that the underlying filesystem does not support a snapshot
plugin at boot.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-10-26 12:44:14 -04:00
Justin Terry (VM)
3f1d9b2c4f Revendor github.com/Microsoft/hcsshim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-15 13:38:24 -07:00
Justin Terry (VM)
00242e1668 Dismount sandbox VHD on snapshot remove
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-10 13:35:06 -07:00
Akihiro Suda
5349fa31df remove pkg/testutil/loopback_linux.go and use continuity/testutil/loopback
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-02 13:12:25 +09:00
Justin Terry (VM)
e373126bfb Fix race in lcow snapshot scratch.vhdx creation
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-25 09:35:46 -07:00
Justin Terry (VM)
7768ab1b5e Update runhcs-shim to use go-bindings
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-20 10:40:33 -07:00
John Howard
2586f3fbb9 boltdb/bolt --> go.etcd.io/bbolt
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-12 15:23:57 -07:00
Xiaodong Zhang
e6d787172c Fix some typo in runtime and snapshots
Signed-off-by: Xiaodong Zhang <a4012017@sina.com>
2018-09-08 08:31:42 +08:00
Michael Wan
92243ff72a bugfix: updatedAt timestamp file may be empty
Signed-off-by: Michael Wan <zirenwan@gmail.com>
2018-08-29 21:49:40 -04:00
Justin Terry (VM)
0110b3c0bc Introduce the Windows lcow diff/snaphotter
Implements the Windows lcow differ/snapshotter responsible for managing
the creation and lifetime of lcow containers on Windows.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-28 13:51:04 -07:00
Wei Fu
67b54c6670 Support >= 128 layers in overlayfs snapshots
Auto-detect longest common dir in lowerdir option and compact it if the
option size is hitting one page size. If does, Use chdir + CLONE to do
mount thing to avoid hitting one page argument buffer in linux kernel
mount.

Signed-off-by: Wei Fu <fhfuwei@163.com>
2018-08-07 10:59:36 +08:00
Akihiro Suda
7461739b58 native: set '/' permission to 0755
Unlike other snapshotters, the permission was previously set to 0700
unexpectedly.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-23 16:14:55 +09:00
Michael Crosby
e17969caad Merge pull request #2436 from thaJeztah/fix_gofmt
Workaround for gofmt change in Go 1.11
2018-07-17 14:52:29 -04:00
Rolf Neugebauer
651920405e test: Increase btrfs loopback device size to 128MB
With btrfs-prog 4.17 as shipped with Alpine 3.8 the btrfs tests
fail with:

  ERROR: '/dev/loop1' is too small to make a usable filesystem
  ERROR: minimum size for each btrfs device is 114294784

btrfs-prog commit 997f9977c243 ("btrfs-progs: mkfs: Prevent
temporary system chunk to use space in reserved 1M range")
changed the code to compute the minimum size for a btrfs size
and is the likely reason for this error.

Increase the size of the loop back device to 128MB for
the btrfs test.

With this fix, the containerd tests pass on Alpine 3.8,
tested in LinuxKit with kernel 4.14.53.

fixes #2447

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-07-08 10:53:03 +01:00
Sebastiaan van Stijn
383d750d4f Workaround for gofmt change in Go 1.11
Go 1.11 uses a different formatting for maps, and now
aligns values; running `gofmt -s -w` on this file resulted
in this diff;

```patch
 content/testsuite/testsuite.go | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/content/testsuite/testsuite.go b/content/testsuite/testsuite.go
index 974c7cb8ed..d9ae9dc160 100644
--- a/content/testsuite/testsuite.go
+++ b/content/testsuite/testsuite.go
@@ -365,8 +365,8 @@ func checkLabels(ctx context.Context, t *testing.T, cs content.Store) {

 	rootTime := time.Now().UTC().Format(time.RFC3339)
 	labels := map[string]string{
-		"k1": "v1",
-		"k2": "v2",
+		"k1":                    "v1",
+		"k2":                    "v2",
 		"containerd.io/gc.root": rootTime,
 	}

@@ -402,7 +402,7 @@ func checkLabels(ctx context.Context, t *testing.T, cs content.Store) {
 	}

 	info.Labels = map[string]string{
-		"k1": "v1",
+		"k1":                    "v1",
 		"containerd.io/gc.root": rootTime,
 	}
 	preUpdate = time.Now()
```

Adding a whitespace before the long key to make it format the same
on both Go 1.11 and older versions of Go.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-05 11:05:30 +02:00
Vincent Demeester
832b05ae67 Update tests to use gotest.tools angel
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-08 21:02:01 +02:00
Phil Estes
e9434a10bc Merge pull request #2341 from dmcgowan/move-client-content-snapshot
Move client implementations for content store and snapshotter
2018-05-25 13:47:39 -04:00
Michael Crosby
009ba4d797 Move testutils to pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:08:38 -04:00
Derek McGowan
48b0a022ef Rename remote snapshotter to proxy
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-08 14:45:57 -07:00
Derek McGowan
0d1807a43d Move client snapshotter to separate package
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-08 14:45:27 -07:00
Michael Wan
7fd6d5e2be fix words misspell
Signed-off-by: Michael Wan <zirenwan@gmail.com>
2018-04-23 00:09:42 -04:00
Christopher Jones
f1639bca4b testing: cleanup loopback devices on test skip
Fixes a case where loopback devices wouldn't get cleaned up
when a test was being skipped

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2018-04-18 12:39:21 -04:00
Akihiro Suda
83e35b3d3a enable native (formerly naive) snapshotter by default
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-02 13:29:19 +09:00
Derek McGowan
94cf25f7db overlay: fix cleanup directory deletion race
Fixes a bug where a writable transaction may create or make changes to
a directory while the cleanup is running, leading to removal of a
directory which will be referenced.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-03-14 13:36:49 -07:00
Stephen Day
220a479292 Merge pull request #1833 from dmcgowan/snapshot-gc-3rd-phase
snapshots/overlay: add overlay cleanup
2018-03-12 17:49:56 -07:00
Akihiro Suda
5cc915c26c overlay: add Supported() checker
This function is not called during plugin initialization (#2140),
but should be useful for downstream projects that uses overlayfs
snapshotter as a Go library.

Benchmark result on Ubuntu 17.10, GCE n1-standard-4:

BenchmarkOverlaySupportedOnExt4-4                    100          20490598 ns/op
BenchmarkOverlayUnsupportedOnFType0XFS-4           30000             39316 ns/op
BenchmarkOverlaySupportedOnFType1XFS-4               100          19287083 ns/op
BenchmarkOverlayUnsupportedOnFAT-4                   100          14217772 ns/op

i.e. the overhead is typically about 20 msec on this machine.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-06 16:53:21 +09:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Christopher Jones
6e3701141b [testing] use smaller filesize on 4KB pagesize systems
Use a smaller filesize on systems that have a default 4KB pagesize.

This is because mkfs.btrfs uses the default system pagesize as blocksize
when creating a device, so if the pagesize is larger, then the file needs
to be larger as well. This larger filesize is needed specifically for
systems where 64KB is default, such as ppc64le.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2018-02-15 15:33:46 -06:00
Daniel Nephin
3279acca82 Check timestamps in snapshot storage test suite
also use t.Helper()
convert assertions to canonical

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Daniel Nephin
37aa41b164 Cleanup after migration
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Daniel Nephin
ef48a0268e Migrate to gotestyourself/assert
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Brian Goff
c776b6d8d9 Use continuity fs package
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-07 12:40:52 -05:00
Derek McGowan
1fd2b5783a Add overlay options for making cleanup asynchronous
Allow configuring the overlay snapshotter to synchronously
or asynchronously do cleanup. When the driver is integrated
into a garbage collection system, the asynchronous cleanup
can reduce the time of removal and allow the longer disk
cleanup to be handled without locking the snapshotter.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-02-01 14:56:03 -08:00
Derek McGowan
14d402e289 Add overlay cleanup
Updates overlay remove to simply remove the reference, adds
a cleanup method for discarding the directory.
Updates snapshot create to setup the directory structure while
in the transaction, to prevent cleanup from removing directories
which are part of a create.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-01-30 16:17:41 -08:00
Christopher Jones
a8c5ff57e5 Cleanup loop devices after test failure
Cleans up loop devices if part of the test or mount process fails.

Also increases btrfs default file size to 650MB to accommodate
minimum btrfs size on ppc64le and s390x

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2018-01-30 14:44:36 -05:00
Christopher Jones
051ac5dd63 running tests in a container
This provides a dockerfile for building a container to run the containerd tests

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2018-01-30 14:44:30 -05:00
Darren Stahl
e981cc6289 Do not create new layer for View
Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 14:40:24 -08:00