- reproducer
1. stop a container;
2. reboot, or dmsetup remove its corresponding dm device;
3. start the container, it will fail like:
"""
Error: failed to start containers: {"message":"failed to create container(4f33d2760760c41518a84821153ccdf7f80980b797b783cdd75178fc6ca0bf4b) on containerd: failed to create task for container(4f33d2760760c41518a84821153ccdf7f80980b797b783cdd75178fc6ca0bf4b): failed to mount rootfs component &{ext4 /dev/mapper/vg0-mythinpool-snap-2 []}: no such file or directory: unknown"}
"""
- how the fix works
activate the dm device if necessary, and give a warn msg:
"""
time="2019-08-21T22:44:08.422695797+08:00" level=warning msg="devmapper device \"vg0-mythinpool-snap-2\" marked as \"Activated\" but not active, activating it"
"""
Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
Enables showing debug logs in testing output.
For integration tests the client log output will show
in addition to daemon output, with timestamps for better
correlation.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This also refactors the lcow and windows
snapshotters to use go-winio's utility functions for checking the
filesystem type.
Signed-off-by: Eric Hotinger <ehotinger@gmail.com>
For block device like devicemapper, umount before committing
active snapshot can sync data onto disk. Otherewise:
1. deactivating a block device in use will fail or IO error
when forced;
2. new snapshot on it will not catch the data not laid on disk yet.
Signed-off-by: Eric Ren <renzhen.rz@alibaba-linux.com>
1. reason to deactivate committed snapshot
The thin device will not be used for IO after committed,
and further thin snapshotting is OK using an inactive thin
device as origin. The benefits to deactivate are:
- device is not unneccesary visible avoiding any unexpected IO;
- save useless kernel data structs for maintaining active dm.
Quote from kernel doc (Documentation/device-mapper/provisioning.txt):
"
ii) Using an internal snapshot.
Once created, the user doesn't have to worry about any connection
between the origin and the snapshot. Indeed the snapshot is no
different from any other thinly-provisioned device and can be
snapshotted itself via the same method. It's perfectly legal to
have only one of them active, and there's no ordering requirement on
activating or removing them both. (This differs from conventional
device-mapper snapshots.)
"
2. an thinpool metadata bug is naturally removed
An problem happens when failed to suspend/resume origin thin device
when creating snapshot:
"failed to create snapshot device from parent vg0-mythinpool-snap-3"
error="failed to save initial metadata for snapshot "vg0-mythinpool-snap-19":
object already exists"
This issue occurs because when failed to create snapshot, the
snapshotter.store can be rollbacked, but the thin pool metadata
boltdb failed to rollback in PoolDevice.CreateSnapshotDevice(),
therefore metadata becomes inconsistent: the snapshotID is not
taken in snapshotter.store, but saved in pool metadata boltdb.
The cause is, in PoolDevice.CreateSnapshotDevice(), the defer calls
are invoked on "first-in-last-out" order. When the error happens
on the "resume device" defer call, the metadata is saved and
snapshot is created, which has no chance to be rollbacked.
Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
```
func foo() error {
defer func() {
if err != nil {
...
}
}()
...
}
```
use defer func to do something when err not nil, if foo() not use
named error, `err != nil` can not catch all errors, since when err
re-defined in if condition, it is a new variable.
Signed-off-by: Ace-Tang <aceapril@126.com>
Since there is no real action the user can do, these can safely be
informative that the underlying filesystem does not support a snapshot
plugin at boot.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Implements the Windows lcow differ/snapshotter responsible for managing
the creation and lifetime of lcow containers on Windows.
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Auto-detect longest common dir in lowerdir option and compact it if the
option size is hitting one page size. If does, Use chdir + CLONE to do
mount thing to avoid hitting one page argument buffer in linux kernel
mount.
Signed-off-by: Wei Fu <fhfuwei@163.com>
With btrfs-prog 4.17 as shipped with Alpine 3.8 the btrfs tests
fail with:
ERROR: '/dev/loop1' is too small to make a usable filesystem
ERROR: minimum size for each btrfs device is 114294784
btrfs-prog commit 997f9977c243 ("btrfs-progs: mkfs: Prevent
temporary system chunk to use space in reserved 1M range")
changed the code to compute the minimum size for a btrfs size
and is the likely reason for this error.
Increase the size of the loop back device to 128MB for
the btrfs test.
With this fix, the containerd tests pass on Alpine 3.8,
tested in LinuxKit with kernel 4.14.53.
fixes#2447
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
Go 1.11 uses a different formatting for maps, and now
aligns values; running `gofmt -s -w` on this file resulted
in this diff;
```patch
content/testsuite/testsuite.go | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/content/testsuite/testsuite.go b/content/testsuite/testsuite.go
index 974c7cb8ed..d9ae9dc160 100644
--- a/content/testsuite/testsuite.go
+++ b/content/testsuite/testsuite.go
@@ -365,8 +365,8 @@ func checkLabels(ctx context.Context, t *testing.T, cs content.Store) {
rootTime := time.Now().UTC().Format(time.RFC3339)
labels := map[string]string{
- "k1": "v1",
- "k2": "v2",
+ "k1": "v1",
+ "k2": "v2",
"containerd.io/gc.root": rootTime,
}
@@ -402,7 +402,7 @@ func checkLabels(ctx context.Context, t *testing.T, cs content.Store) {
}
info.Labels = map[string]string{
- "k1": "v1",
+ "k1": "v1",
"containerd.io/gc.root": rootTime,
}
preUpdate = time.Now()
```
Adding a whitespace before the long key to make it format the same
on both Go 1.11 and older versions of Go.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Fixes a bug where a writable transaction may create or make changes to
a directory while the cleanup is running, leading to removal of a
directory which will be referenced.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This function is not called during plugin initialization (#2140),
but should be useful for downstream projects that uses overlayfs
snapshotter as a Go library.
Benchmark result on Ubuntu 17.10, GCE n1-standard-4:
BenchmarkOverlaySupportedOnExt4-4 100 20490598 ns/op
BenchmarkOverlayUnsupportedOnFType0XFS-4 30000 39316 ns/op
BenchmarkOverlaySupportedOnFType1XFS-4 100 19287083 ns/op
BenchmarkOverlayUnsupportedOnFAT-4 100 14217772 ns/op
i.e. the overhead is typically about 20 msec on this machine.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Use a smaller filesize on systems that have a default 4KB pagesize.
This is because mkfs.btrfs uses the default system pagesize as blocksize
when creating a device, so if the pagesize is larger, then the file needs
to be larger as well. This larger filesize is needed specifically for
systems where 64KB is default, such as ppc64le.
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
Allow configuring the overlay snapshotter to synchronously
or asynchronously do cleanup. When the driver is integrated
into a garbage collection system, the asynchronous cleanup
can reduce the time of removal and allow the longer disk
cleanup to be handled without locking the snapshotter.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Updates overlay remove to simply remove the reference, adds
a cleanup method for discarding the directory.
Updates snapshot create to setup the directory structure while
in the transaction, to prevent cleanup from removing directories
which are part of a create.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Cleans up loop devices if part of the test or mount process fails.
Also increases btrfs default file size to 650MB to accommodate
minimum btrfs size on ppc64le and s390x
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
This changes the Windows runtime to use the snapshotter and differ
created layers, and updates the ctr commands to use the snapshotter and differ.
Signed-off-by: Darren Stahl <darst@microsoft.com>
This implements the Windows snapshotter and diff Apply function.
This allows for Windows layers to be created, and layers to be pulled
from the hub.
Signed-off-by: Darren Stahl <darst@microsoft.com>
Implements btrfs usage using a double walking diff and
counting the result. Walking gives the most accurate
count and includes inode usage.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
