Commit Graph

304 Commits

Author SHA1 Message Date
Derek McGowan
7b2a918213
Generalize the plugin package
Remove containerd specific parts of the plugin package to prepare its
move out of the main repository. Separate the plugin registration
singleton into a separate package.

Separating out the plugin package and registration makes it easier to
implement external plugins without creating a dependency loop.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 21:22:32 -07:00
Derek McGowan
a80606bc2d
Move plugin type definitions to containerd plugins package
The plugins packages defines the plugins used by containerd.
Move all the types and properties to this package.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 20:52:56 -07:00
Akihiro Suda
9ca6fd9e6e
Merge pull request #9117 from kinvolk/rata/userns-chown-opt-in
Require opt-in for rootfs chown when idmap mounts is not supported
2023-09-28 02:34:41 +09:00
Derek McGowan
b5615caf11
Update go-toml to v2
Updates host file parsing to use new v2 method rather than the removed
toml.Tree.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-22 15:35:12 -07:00
Derek McGowan
508aa3a1ef
Move to use github.com/containerd/log
Add github.com/containerd/log to go.mod

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-22 07:53:23 -07:00
Rodrigo Campos
ec9e0dca91 overlay: Require opt-in if idmap mounts are not supported.
If we don't use idmap mounts, doing a chown per pod is very expensive:
it implies duplicating the container storage for the image for every pod
and the latency to start a new pod is affected too.

Let's make sure users are aware of this, by having them opt-in, for
snapshotters that we have a better solution (like overlayfs, that has
support for idmap mounts).

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-09-19 14:42:51 +02:00
Akihiro Suda
0ee2433c94
Merge pull request #5890 from artqzn/idmapped_mounts
RFC: Initial support of idmapped mount points
2023-09-05 20:41:05 +09:00
Akihiro Suda
e30a40eb65
Merge pull request #9016 from djdongjin/remove-most-logrus
Remove most logrus import
2023-09-05 16:09:12 +09:00
Ilya Hanov
295bcec074 snapshotter: implement unit tests for idmapped mounts for overlay
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
e49e6d6fd7 snapshotter: implement slow path for idmapped mounts check for overlay
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
723c88ce30 snapshotter: add "--remap-labels" support to overlayfs
Previously the only fuse-overlayfs supports "--remap-labels" option.
Since idmapped mounts were landed to Linux kernel v5.12 it becomes
possible to use it with overlayfs via mount_setattr() system call.

The changes are based on experimental patchset published by
Mauricio Vásquez #4734.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Artem Kuzin <artem.kuzin@huawei.com>
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
e8ddf669f5 snapshotter: support "remap-ids" capability for overlayfs
Previously remapping of a snapshotter has been done using
recursive chown.

Commit
31a6449734 added a support
for "remap-ids" capability which allows snapshotter internals do
remappings in case of idmapped mounts support to avoid recursive
chown and creating a new remapped snapshot.

Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Danny Canter
13ff185ba0 Blockfile: Enlighten blockfile copy on Darwin
The Go stdlib does not seem to have an efficient os.File.ReadFrom
routine for other platforms like it does on Linux with
copy_file_range. For Darwin at least we can use clonefile
in its place, otherwise if we have a sparse file we'd have
a fun surprise with the io.Copy approach..

We should see if there's other platforms that we can enhance here.
I've forgotten what's the right route on Windows.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-08-31 20:18:48 -07:00
Jin Dong
fc45365fa1 Remove most logrus
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-26 14:31:53 -04:00
Danny Canter
34f32043bc Blockfile: Expose RecreateScratch option
This was the only option not configurable from the toml for the plugin.
This is useful if you want to restart containerd and try a different
blockfile/size for the snapshotter.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-08-24 13:28:22 -07:00
Sebastiaan van Stijn
11a5dd8265
snapshots: replace some fmt.Sprintfs with strconv
Teeny-tiny optimizations:

    BenchmarkSprintf-10       37735996    32.31  ns/op  0 B/op  0 allocs/op
    BenchmarkItoa-10         591945836     2.031 ns/op  0 B/op  0 allocs/op
    BenchmarkFormatUint-10   593701444     2.014 ns/op  0 B/op  0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 10:10:59 +02:00
Fu Wei
738c153573
Merge pull request #8992 from djdongjin/remove-hashicorp-multierror
Remove hashicorp/go-multierror dependency
2023-08-23 13:13:51 +08:00
Jin Dong
6d8329d7f5 Remove unnecessary joinError unwrap
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-21 19:03:41 -07:00
Jin Dong
cd8c8ae4bc Remove hashicorp/go-multierror
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-20 17:59:45 -07:00
Robbie Buxton
23c95359aa Add WithMetaStore to overlay snapshotter to allow bringing your own
Signed-off-by: Robbie Buxton <138501839+rbpdt@users.noreply.github.com>
2023-08-17 19:28:44 +01:00
Ben Foster
f3daf32c73 Fix ro mount option being passed
"ro" was not parsed out of the string, so it was passed as part of data
to mount().
This would lead to mount() returning an invalid argument code.
Separate out the "ro" option, much like "userxattr", which will allow
the MS-RDONLY mountflag to get set.

Signed-off-by: Ben Foster <bpfoster@gmail.com>
2023-07-20 17:08:01 -04:00
James Jenkins
5c480d9c88 Modify loopback size
Modify the loopback size in the blockfile snapshotter test setup.
Set the loopback size to 16MB when the page size is greater than 4096.

Signed-off-by: James Jenkins <James.Jenkins@ibm.com>
2023-07-10 11:06:24 -04:00
Danny Canter
b3ab1f26c4 Snapshots: Add From/ToProto helpers for types
Helpers to convert from snapshot types to their protobuf structures and
vice-versa appear three times. It seems sane to just expose this facility
in the snapshots pkg. From/ToKind weren't used anywhere but doesn't hurt to
round out the types by exposing them.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-06-28 12:17:52 -07:00
Danny Canter
55a8102ec1 mount: Add From/ToProto helpers
Helpers to convert from containerd's [Mount] to its protobuf structure for
[Mount] and vice-versa appear three times. It seems sane to just expose
this facility in /mount.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-06-28 04:03:18 -07:00
Wei Fu
a207b430e3 snapshots/blockfile: fix lint issue
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-06-17 08:42:02 +08:00
Wei Fu
7de95cbc4c snapshots/blockfile: deflaky the testsuite
* Use direct-io mode to reduce IO.

* Add testViewHook helper to recovery the backing file since the ext4
  might need writable permission to handle recovery. If the backing file
  needs recovery and it's for View snapshot, the readonly mount will
  cause error.

* Use 8 MiB as capacity to reduce the IO.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-06-15 23:54:06 +08:00
Wei Fu
6dfb16f99a snapshots|pkg: umount without DETACH and nosync after umount
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-06-15 23:53:47 +08:00
Derek McGowan
d115129d1e
Add configurable mount options to overlay
Allows default mount options to be provided through configuration.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-06-12 09:17:07 -07:00
Derek McGowan
f49789fd54
Merge pull request #8594 from fuweid/fix-blockfile-mount
snapshots/blockfile: use loop if options is empty
2023-05-30 08:33:09 -07:00
Cardy.Tang
9702d4aeec snapshots/blockfile: use passed in dst in scratchGenerator
Signed-off-by: Cardy.Tang <zuniorone@gmail.com>
2023-05-30 11:42:19 +08:00
Wei Fu
bb82f3683e snapshots/blockfile: use loop if options is empty
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-05-29 23:26:39 +08:00
Derek McGowan
7beaa5e859
Add mount options to blockfile snapshotter
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-23 10:56:20 -07:00
Cardy.Tang
b9f2e48e37 feat: make overlay sync removal configurable
Signed-off-by: Cardy.Tang <zuniorone@gmail.com>
2023-05-19 16:32:10 +08:00
Akihiro Suda
878132923d
Merge pull request #8511 from dmcgowan/blockfile-snapshotter
Add blockfile snapshotter
2023-05-18 11:52:37 +09:00
Derek McGowan
1be571b14f
Add sync before unmount on snapshotter layer test
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-17 14:25:27 -07:00
Derek McGowan
0c386e2ebd
Add blockfile snapshotter
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-17 14:25:27 -07:00
Akihiro Suda
5dbae38191
snapshots/testsuite: Rename: fix fuse-overlayfs incompatibility
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-05-11 16:59:53 +09:00
Derek McGowan
14d22efccc
Merge pull request #8254 from gabriel-samfira/fix-access-denied-on-vhdx-mount
Fix access denied on mounted vhdx root
2023-05-10 08:39:36 -07:00
Maksym Pavlenko
6f34da5f80 Cleanup logrus imports
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-05-05 11:54:14 -07:00
Gabriel Adrian Samfira
6f0714efcb Use RunWithPrivileges
RunWithPrivileges() will enable privileges will lock a thread, change
privileges, and run the function passed in, within that thread. This
allows us to limit the scope in which we enable privileges and avoids
accidentally enabling privileges in threads that should never have them.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-06 17:06:50 -07:00
Gabriel Adrian Samfira
b41ca11598 Fix access denied on mounted vhdx root
It seems that in certain situations, like having the containerd root
and state on a file system hosted on a mounted VHDX, we need
SeSecurityPrivilege when opening a file with winio.ACCESS_SYSTEM_SECURITY.
This happens in the base layer writer in hcsshim when adding a new file.

Enabling SeSecurityPrivilege allows the containerd root to be hosted on
a vhdx.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-06 17:06:50 -07:00
Gabriel Adrian Samfira
8538e7a2ac Improve error messages and remove check
* Improve error messages
  * remove a check for the existance of unmount target. We probably
    should not mask that the target was missing.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-04 12:07:34 -07:00
Gabriel Adrian Samfira
6a5b4c9c24 Remove "bind" code path from diff
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-03 08:11:35 -07:00
Gabriel Adrian Samfira
d373ebc4de Properly mount base layers
As opposed to a writable layer derived from a base layer, the volume
path of a base layer, once activated and prepared will not be a WCIFS
volume, but the actual path on disk to the snapshot. We cannot directly
mount this folder, as that would mean a client may gain access and
potentially damage important metadata files that would render the layer
unusabble.

For base layers we need to mount the Files folder which must exist in
any valid base windows-layer.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-02 08:35:34 -07:00
Gabriel Adrian Samfira
ca5605b4a2 Skip parent layer options on bind mounts
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-01 08:49:24 -07:00
Gabriel Adrian Samfira
4012c1b853 Remove escalated privileges
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:17:35 -07:00
Gabriel Adrian Samfira
95687a9324 Fix go.mod, simplify boolean logic, add logging
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:16:56 -07:00
Gabriel Adrian Samfira
feb637f92d Fix layer comparison and enable read-only checks
fstest.CheckDirectoryEqual checks if any files in the diff matches a
list of known metadataFiles. This only happens if we specify the initial
layer as the first parameter and the mutated layer as the second.

This also enables the read-only view checks, as the bind filter allows
us to mount a layer as ro.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:18 -07:00
Gabriel Adrian Samfira
36dc2782c4 Use bind filer for mounts
The bind filter supports bind-like mounts and volume mounts. It also
allows us to have read-only mounts.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
7b36becd2d Run Windows snapshotter through the test suite
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00