Commit Graph

80 Commits

Author SHA1 Message Date
Eric Ren
6f463d3505 test/snapshots: umount before committing snapshot
For block device like devicemapper, umount before committing
active snapshot can sync data onto disk. Otherewise:

1. deactivating a block device in use will fail or IO error
when forced;
2. new snapshot on it will not catch the data not laid on disk yet.

Signed-off-by: Eric Ren <renzhen.rz@alibaba-linux.com>
2019-05-09 12:27:05 +08:00
renzhen.rz
3887053177 snapshots/devmapper: deactivate thin device after committed
1. reason to deactivate committed snapshot

The thin device will not be used for IO after committed,
and further thin snapshotting is OK using an inactive thin
device as origin. The benefits to deactivate are:
 - device is not unneccesary visible avoiding any unexpected IO;
 - save useless kernel data structs for maintaining active dm.

 Quote from kernel doc (Documentation/device-mapper/provisioning.txt):

"
  ii) Using an internal snapshot.

  Once created, the user doesn't have to worry about any connection
  between the origin and the snapshot.  Indeed the snapshot is no
  different from any other thinly-provisioned device and can be
  snapshotted itself via the same method.  It's perfectly legal to
  have only one of them active, and there's no ordering requirement on
  activating or removing them both.  (This differs from conventional
  device-mapper snapshots.)
"

2. an thinpool metadata bug is naturally removed

An problem happens when failed to suspend/resume origin thin device
when creating snapshot:

"failed to create snapshot device from parent vg0-mythinpool-snap-3"
error="failed to save initial metadata for snapshot "vg0-mythinpool-snap-19":
object already exists"

This issue occurs because when failed to create snapshot, the
snapshotter.store can be rollbacked, but the thin pool metadata
boltdb failed to rollback in PoolDevice.CreateSnapshotDevice(),
therefore metadata becomes inconsistent: the snapshotID is not
taken in snapshotter.store, but saved in pool metadata boltdb.

The cause is, in PoolDevice.CreateSnapshotDevice(), the defer calls
are invoked on "first-in-last-out" order. When the error happens
on the "resume device" defer call, the metadata is saved and
snapshot is created, which has no chance to be rollbacked.

Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
2019-05-09 10:58:21 +08:00
Derek McGowan
c10eb8d122
Merge pull request #3256 from dvrkps/redundant
Remove redundant error checks
2019-05-03 16:23:40 -07:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Davor Kapsa
eded188f4f Fix misspells
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 20:38:24 +02:00
Maksym Pavlenko
90085a7ac5 Add empty file to prevent build from failing
Go build fails on directory with only test files
https://github.com/golang/go/issues/8279

Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-04-02 17:35:19 -07:00
Maksym Pavlenko
8784eb6308 Move snapshotters benchmark to a separate package
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-04-02 14:42:21 -07:00
Maksym Pavlenko
d9526f5c4f Move CloudFormation template to contrib
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-04-01 13:34:48 -07:00
Maksym Pavlenko
515be73c82 Add snapshotters benchmark
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-04-01 11:13:27 -07:00
Michael Crosby
e7b6fea572
Merge pull request #3022 from mxpv/devmapper
Integrate device mapper snapshotter
2019-03-29 00:06:11 -04:00
Maksym Pavlenko
87289a0c62 devmapper: implement Usage
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2019-03-27 14:50:12 -07:00
Maksym Pavlenko
010b4da36f devmapper: implement dmsetup status
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2019-03-27 14:26:07 -07:00
Eric Hotinger
e13a4b67e8 Update the snapshotter docs to refer to the snapshots package for
WithLabels.

Signed-off-by: Eric Hotinger <ehotinger@gmail.com>
2019-02-28 15:03:37 -08:00
Maksym Pavlenko
208957ba3c
devmapper: proper cleanup in pool device test
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-22 12:51:27 -08:00
Maksym Pavlenko
734989c2a0
Update README
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-22 11:10:51 -08:00
Maksym Pavlenko
95f0a4903c
devmapper: rollback thin devices on error
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 17:40:10 -08:00
Maksym Pavlenko
adf5c640f4
devmapper: don't create or reload thin-pool from snapshotter
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:26:46 -08:00
Maksym Pavlenko
7efda48c53
devmapper: more precise way of checking if device is activated
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:26:46 -08:00
Maksym Pavlenko
37cdedc61c
devmapper: add linux tags, fix build
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:26:46 -08:00
Maksym Pavlenko
0c6d194cce
devmapper: add README and minor fixes
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
2218275ec9
devmapper: register plugin
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
cec72efc2a
devmapper: add snapshotter
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
3a75882520
devmapper: add pool device manager
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
6e0ae68e17
devmapper: add snapshotter config
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
fcd9dc2749
devmapper: add pool metadata
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
809e5fd3b8
devmapper: add dmsetup
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Maksym Pavlenko
fe05e4d1a9
devmapper: add losetup
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
Oliver Stenbom
7f5d7ff6b8 Update snapshots docs with garbage collector label
Signed-off-by: Oliver Stenbom <oliver@stenbom.eu>
2019-01-29 15:37:56 +00:00
Ace-Tang
7faaa64cf9 fix: miss remove temp file in createSnapshot
```
func foo() error {

defer func() {
    if err != nil {
	...
    }
}()
...
}
```
use defer func to do something when err not nil, if foo() not use
named error, `err != nil` can not catch all errors, since when err
re-defined in if condition, it is a new variable.

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-12-24 23:01:56 +08:00
zhangyue
7b1e6f323a fix: use func args instead of build new one
Signed-off-by: zhangyue <zy675793960@yeah.net>
2018-12-16 15:07:12 +08:00
Michael Crosby
16aaf6c065 Change unsupported snapshot warnings to INFO
Since there is no real action the user can do, these can safely be
informative that the underlying filesystem does not support a snapshot
plugin at boot.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-10-26 12:44:14 -04:00
Justin Terry (VM)
3f1d9b2c4f Revendor github.com/Microsoft/hcsshim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-15 13:38:24 -07:00
Justin Terry (VM)
00242e1668 Dismount sandbox VHD on snapshot remove
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-10 13:35:06 -07:00
Akihiro Suda
5349fa31df remove pkg/testutil/loopback_linux.go and use continuity/testutil/loopback
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-02 13:12:25 +09:00
Justin Terry (VM)
e373126bfb Fix race in lcow snapshot scratch.vhdx creation
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-25 09:35:46 -07:00
Justin Terry (VM)
7768ab1b5e Update runhcs-shim to use go-bindings
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-20 10:40:33 -07:00
John Howard
2586f3fbb9 boltdb/bolt --> go.etcd.io/bbolt
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-12 15:23:57 -07:00
Xiaodong Zhang
e6d787172c Fix some typo in runtime and snapshots
Signed-off-by: Xiaodong Zhang <a4012017@sina.com>
2018-09-08 08:31:42 +08:00
Michael Wan
92243ff72a bugfix: updatedAt timestamp file may be empty
Signed-off-by: Michael Wan <zirenwan@gmail.com>
2018-08-29 21:49:40 -04:00
Justin Terry (VM)
0110b3c0bc Introduce the Windows lcow diff/snaphotter
Implements the Windows lcow differ/snapshotter responsible for managing
the creation and lifetime of lcow containers on Windows.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-28 13:51:04 -07:00
Wei Fu
67b54c6670 Support >= 128 layers in overlayfs snapshots
Auto-detect longest common dir in lowerdir option and compact it if the
option size is hitting one page size. If does, Use chdir + CLONE to do
mount thing to avoid hitting one page argument buffer in linux kernel
mount.

Signed-off-by: Wei Fu <fhfuwei@163.com>
2018-08-07 10:59:36 +08:00
Akihiro Suda
7461739b58 native: set '/' permission to 0755
Unlike other snapshotters, the permission was previously set to 0700
unexpectedly.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-23 16:14:55 +09:00
Michael Crosby
e17969caad
Merge pull request #2436 from thaJeztah/fix_gofmt
Workaround for gofmt change in Go 1.11
2018-07-17 14:52:29 -04:00
Rolf Neugebauer
651920405e test: Increase btrfs loopback device size to 128MB
With btrfs-prog 4.17 as shipped with Alpine 3.8 the btrfs tests
fail with:

  ERROR: '/dev/loop1' is too small to make a usable filesystem
  ERROR: minimum size for each btrfs device is 114294784

btrfs-prog commit 997f9977c243 ("btrfs-progs: mkfs: Prevent
temporary system chunk to use space in reserved 1M range")
changed the code to compute the minimum size for a btrfs size
and is the likely reason for this error.

Increase the size of the loop back device to 128MB for
the btrfs test.

With this fix, the containerd tests pass on Alpine 3.8,
tested in LinuxKit with kernel 4.14.53.

fixes #2447

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-07-08 10:53:03 +01:00
Sebastiaan van Stijn
383d750d4f
Workaround for gofmt change in Go 1.11
Go 1.11 uses a different formatting for maps, and now
aligns values; running `gofmt -s -w` on this file resulted
in this diff;

```patch
 content/testsuite/testsuite.go | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/content/testsuite/testsuite.go b/content/testsuite/testsuite.go
index 974c7cb8ed..d9ae9dc160 100644
--- a/content/testsuite/testsuite.go
+++ b/content/testsuite/testsuite.go
@@ -365,8 +365,8 @@ func checkLabels(ctx context.Context, t *testing.T, cs content.Store) {

 	rootTime := time.Now().UTC().Format(time.RFC3339)
 	labels := map[string]string{
-		"k1": "v1",
-		"k2": "v2",
+		"k1":                    "v1",
+		"k2":                    "v2",
 		"containerd.io/gc.root": rootTime,
 	}

@@ -402,7 +402,7 @@ func checkLabels(ctx context.Context, t *testing.T, cs content.Store) {
 	}

 	info.Labels = map[string]string{
-		"k1": "v1",
+		"k1":                    "v1",
 		"containerd.io/gc.root": rootTime,
 	}
 	preUpdate = time.Now()
```

Adding a whitespace before the long key to make it format the same
on both Go 1.11 and older versions of Go.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-05 11:05:30 +02:00
Vincent Demeester
832b05ae67
Update tests to use gotest.tools angel
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-08 21:02:01 +02:00
Phil Estes
e9434a10bc
Merge pull request #2341 from dmcgowan/move-client-content-snapshot
Move client implementations for content store and snapshotter
2018-05-25 13:47:39 -04:00
Michael Crosby
009ba4d797 Move testutils to pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:08:38 -04:00
Derek McGowan
48b0a022ef
Rename remote snapshotter to proxy
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-08 14:45:57 -07:00
Derek McGowan
0d1807a43d
Move client snapshotter to separate package
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-08 14:45:27 -07:00