Commit Graph

36 Commits

Author SHA1 Message Date
ningmingxiao
6ff1a59345
fix when kernel version < 4.13rc1 by using index=off cause overlay test error
Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
2021-12-01 21:48:52 +08:00
Eng Zer Jun
50da673592
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-09-21 09:50:38 +08:00
Sebastiaan van Stijn
2ac9968401
replace uses of os/exec with golang.org/x/sys/execabs
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows. This issue also
affects Unix users who have “.” listed explicitly in their PATH and are running
“go get” outside of a module or with module mode disabled.

This issue is not limited to the go command itself, and can also affect binaries
that use `os.Command`, `os.LookPath`, etc.

From the related blogpost (ttps://blog.golang.org/path-security):

> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing

This patch replaces all uses of `os/exec` with `golang.org/x/sys/execabs`. While
some uses of `os/exec` should not be problematic (e.g. part of tests), it is
probably good to be consistent, in case code gets moved around.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-25 18:11:09 +02:00
Akihiro Suda
d3aa7ee9f0
Run go fmt with Go 1.17
The new `go fmt` adds `//go:build` lines (https://golang.org/doc/go1.17#tools).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-08-22 09:31:50 +09:00
ktock
67406b3732 overlay: add an optional label of upperdir location of each snapshot
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-07-28 09:52:51 +09:00
Sebastiaan van Stijn
708299ca40
Move RunningInUserNS() to its own package
This allows using the utility without bringing whole of "sys" with it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-23 11:29:53 +01:00
Sebastiaan van Stijn
ba8f9845ec
move overlay-checks to an overlayutils package
This allows using the utilities without importing the whole
snapshotter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-15 19:18:50 +01:00
Akihiro Suda
9ade247b38
overlay: support "userxattr" option (kernel 5.11)
The "userxattr" option is needed for mounting overlayfs inside a user namespace with kernel >= 5.11.

The "userxattr" option is NOT needed for the initial user namespace (aka "the host").

Also, Ubuntu (since circa 2015) and Debian (since 10) with kernel < 5.11 can mount the overlayfs in a user namespace without the "userxattr" option.

The corresponding kernel commit: 2d2f2d7322ff43e0fe92bf8cccdc0b09449bf2e1
> ovl: user xattr
>
> Optionally allow using "user.overlay." namespace instead of "trusted.overlay."
> ...
> Disable redirect_dir and metacopy options, because these would allow privilege escalation through direct manipulation of the
> "user.overlay.redirect" or "user.overlay.metacopy" xattrs.

Fix issue 5060

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-01 13:54:51 +09:00
Phil Estes
85d9fe3e8c
Adjust overlay tests to expect "index=off"
When running tests on any modern distro, this assumption will work. If
we need to make it work with kernels where we don't append this option
it will require some more involved changes.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2020-11-19 10:59:40 -05:00
Derek McGowan
70ffb12c1b
Separate overlay implementation from plugin
Put the overlay plugin in a separate package to allow the overlay package to be
used without needing to import and initialize the plugin.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2020-08-26 18:50:51 -07:00
Ashray Jain
5ed177a2da Add configurable overlayfs path
This allows configuring the location of the overlayfs snapshotter by
adding the following in config.toml
```
[plugins]
  [plugins.overlayfs]
    root_path = "/custom_location"
```

This is useful to isolate disk i/o for overlayfs from the rest of
containerd and prevent containers saturating disk i/o from negatively
affecting containerd operations and cause timeouts.

Signed-off-by: Ashray Jain <ashrayj@palantir.com>
2020-08-26 16:08:10 +01:00
Rudy Zhang
d36810d66d overlay: use index=off to fix EBUSY on mount
kernel version > 4.13rc1 support index=on feature, it will be failed
with EBUSY when trying to mount.

Related: https://github.com/moby/moby/pull/37993

Signed-off-by: Rudy Zhang <rudyflyzhang@gmail.com>
2020-06-08 15:51:15 +08:00
Akihiro Suda
a4b423b19b overlay: test actual Opts
AsynchronousRemove opt was untested while it is specified by default in the plugin init.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-07 13:52:13 +09:00
Lantao Liu
8f9eff4fec Support disk usage in windows snapshot.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-11-01 00:12:10 -07:00
Derek McGowan
66aa1d3ef6
Add snapshot walk implementations
Temporarily remove zfs and aufs until interface update

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-10-24 11:11:22 -07:00
bpopovschi
e8c14c07c6
Added filters to snapshots API
Signed-off-by: bpopovschi <zyqsempai@mail.ru>
2019-10-24 11:11:22 -07:00
Derek McGowan
5a0ff41c81
Update continuity vendor
Pulls in copy and fstest changes

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-08-16 15:49:47 -07:00
Ace-Tang
7faaa64cf9 fix: miss remove temp file in createSnapshot
```
func foo() error {

defer func() {
    if err != nil {
	...
    }
}()
...
}
```
use defer func to do something when err not nil, if foo() not use
named error, `err != nil` can not catch all errors, since when err
re-defined in if condition, it is a new variable.

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-12-24 23:01:56 +08:00
zhangyue
7b1e6f323a fix: use func args instead of build new one
Signed-off-by: zhangyue <zy675793960@yeah.net>
2018-12-16 15:07:12 +08:00
Akihiro Suda
5349fa31df remove pkg/testutil/loopback_linux.go and use continuity/testutil/loopback
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-02 13:12:25 +09:00
Xiaodong Zhang
e6d787172c Fix some typo in runtime and snapshots
Signed-off-by: Xiaodong Zhang <a4012017@sina.com>
2018-09-08 08:31:42 +08:00
Michael Wan
92243ff72a bugfix: updatedAt timestamp file may be empty
Signed-off-by: Michael Wan <zirenwan@gmail.com>
2018-08-29 21:49:40 -04:00
Michael Crosby
009ba4d797 Move testutils to pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:08:38 -04:00
Christopher Jones
f1639bca4b
testing: cleanup loopback devices on test skip
Fixes a case where loopback devices wouldn't get cleaned up
when a test was being skipped

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2018-04-18 12:39:21 -04:00
Derek McGowan
94cf25f7db
overlay: fix cleanup directory deletion race
Fixes a bug where a writable transaction may create or make changes to
a directory while the cleanup is running, leading to removal of a
directory which will be referenced.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-03-14 13:36:49 -07:00
Stephen Day
220a479292
Merge pull request #1833 from dmcgowan/snapshot-gc-3rd-phase
snapshots/overlay: add overlay cleanup
2018-03-12 17:49:56 -07:00
Akihiro Suda
5cc915c26c overlay: add Supported() checker
This function is not called during plugin initialization (#2140),
but should be useful for downstream projects that uses overlayfs
snapshotter as a Go library.

Benchmark result on Ubuntu 17.10, GCE n1-standard-4:

BenchmarkOverlaySupportedOnExt4-4                    100          20490598 ns/op
BenchmarkOverlayUnsupportedOnFType0XFS-4           30000             39316 ns/op
BenchmarkOverlaySupportedOnFType1XFS-4               100          19287083 ns/op
BenchmarkOverlayUnsupportedOnFAT-4                   100          14217772 ns/op

i.e. the overhead is typically about 20 msec on this machine.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-06 16:53:21 +09:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Brian Goff
c776b6d8d9 Use continuity fs package
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-07 12:40:52 -05:00
Derek McGowan
1fd2b5783a
Add overlay options for making cleanup asynchronous
Allow configuring the overlay snapshotter to synchronously
or asynchronously do cleanup. When the driver is integrated
into a garbage collection system, the asynchronous cleanup
can reduce the time of removal and allow the longer disk
cleanup to be handled without locking the snapshotter.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-02-01 14:56:03 -08:00
Derek McGowan
14d402e289
Add overlay cleanup
Updates overlay remove to simply remove the reference, adds
a cleanup method for discarding the directory.
Updates snapshot create to setup the directory structure while
in the transaction, to prevent cleanup from removing directories
which are part of a create.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-01-30 16:17:41 -08:00
Daniel Nephin
1cd32fa68d Cleanup extra returns in tests
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-01-23 14:09:22 -05:00
yason
db742c90c0 get wrong parents bucket in withSnapshotBucket
improve error message for CommitActive
Signed-off-by: yason <yan.xuean@zte.com.cn>
2017-12-02 23:01:29 +08:00
Derek McGowan
b28d7cdf1b
Update snapshot and content related log messages
Make the tense and casing consistent.
Add useful log messages in image service.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-12-01 14:57:34 -08:00
yason
bb02302c9c improve rollback for overlay.prepare
improve err message
Signed-off-by: yason <yan.xuean@zte.com.cn>
2017-12-02 00:09:44 +08:00
Jess Valarezo
9885edfc44 rename snapshot->snapshots pkg
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-29 14:55:02 -08:00