Commit Graph

299 Commits

Author SHA1 Message Date
Derek McGowan
508aa3a1ef Move to use github.com/containerd/log
Add github.com/containerd/log to go.mod

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-22 07:53:23 -07:00
Akihiro Suda
0ee2433c94 Merge pull request #5890 from artqzn/idmapped_mounts
RFC: Initial support of idmapped mount points
2023-09-05 20:41:05 +09:00
Akihiro Suda
e30a40eb65 Merge pull request #9016 from djdongjin/remove-most-logrus
Remove most logrus import
2023-09-05 16:09:12 +09:00
Ilya Hanov
295bcec074 snapshotter: implement unit tests for idmapped mounts for overlay
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
e49e6d6fd7 snapshotter: implement slow path for idmapped mounts check for overlay
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
723c88ce30 snapshotter: add "--remap-labels" support to overlayfs
Previously the only fuse-overlayfs supports "--remap-labels" option.
Since idmapped mounts were landed to Linux kernel v5.12 it becomes
possible to use it with overlayfs via mount_setattr() system call.

The changes are based on experimental patchset published by
Mauricio Vásquez #4734.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Artem Kuzin <artem.kuzin@huawei.com>
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
e8ddf669f5 snapshotter: support "remap-ids" capability for overlayfs
Previously remapping of a snapshotter has been done using
recursive chown.

Commit
31a6449734 added a support
for "remap-ids" capability which allows snapshotter internals do
remappings in case of idmapped mounts support to avoid recursive
chown and creating a new remapped snapshot.

Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Danny Canter
13ff185ba0 Blockfile: Enlighten blockfile copy on Darwin
The Go stdlib does not seem to have an efficient os.File.ReadFrom
routine for other platforms like it does on Linux with
copy_file_range. For Darwin at least we can use clonefile
in its place, otherwise if we have a sparse file we'd have
a fun surprise with the io.Copy approach..

We should see if there's other platforms that we can enhance here.
I've forgotten what's the right route on Windows.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-08-31 20:18:48 -07:00
Jin Dong
fc45365fa1 Remove most logrus
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-26 14:31:53 -04:00
Danny Canter
34f32043bc Blockfile: Expose RecreateScratch option
This was the only option not configurable from the toml for the plugin.
This is useful if you want to restart containerd and try a different
blockfile/size for the snapshotter.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-08-24 13:28:22 -07:00
Sebastiaan van Stijn
11a5dd8265 snapshots: replace some fmt.Sprintfs with strconv
Teeny-tiny optimizations:

    BenchmarkSprintf-10       37735996    32.31  ns/op  0 B/op  0 allocs/op
    BenchmarkItoa-10         591945836     2.031 ns/op  0 B/op  0 allocs/op
    BenchmarkFormatUint-10   593701444     2.014 ns/op  0 B/op  0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 10:10:59 +02:00
Fu Wei
738c153573 Merge pull request #8992 from djdongjin/remove-hashicorp-multierror
Remove hashicorp/go-multierror dependency
2023-08-23 13:13:51 +08:00
Jin Dong
6d8329d7f5 Remove unnecessary joinError unwrap
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-21 19:03:41 -07:00
Jin Dong
cd8c8ae4bc Remove hashicorp/go-multierror
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-20 17:59:45 -07:00
Robbie Buxton
23c95359aa Add WithMetaStore to overlay snapshotter to allow bringing your own
Signed-off-by: Robbie Buxton <138501839+rbpdt@users.noreply.github.com>
2023-08-17 19:28:44 +01:00
Ben Foster
f3daf32c73 Fix ro mount option being passed
"ro" was not parsed out of the string, so it was passed as part of data
to mount().
This would lead to mount() returning an invalid argument code.
Separate out the "ro" option, much like "userxattr", which will allow
the MS-RDONLY mountflag to get set.

Signed-off-by: Ben Foster <bpfoster@gmail.com>
2023-07-20 17:08:01 -04:00
James Jenkins
5c480d9c88 Modify loopback size
Modify the loopback size in the blockfile snapshotter test setup.
Set the loopback size to 16MB when the page size is greater than 4096.

Signed-off-by: James Jenkins <James.Jenkins@ibm.com>
2023-07-10 11:06:24 -04:00
Danny Canter
b3ab1f26c4 Snapshots: Add From/ToProto helpers for types
Helpers to convert from snapshot types to their protobuf structures and
vice-versa appear three times. It seems sane to just expose this facility
in the snapshots pkg. From/ToKind weren't used anywhere but doesn't hurt to
round out the types by exposing them.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-06-28 12:17:52 -07:00
Danny Canter
55a8102ec1 mount: Add From/ToProto helpers
Helpers to convert from containerd's [Mount] to its protobuf structure for
[Mount] and vice-versa appear three times. It seems sane to just expose
this facility in /mount.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-06-28 04:03:18 -07:00
Wei Fu
a207b430e3 snapshots/blockfile: fix lint issue
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-06-17 08:42:02 +08:00
Wei Fu
7de95cbc4c snapshots/blockfile: deflaky the testsuite
* Use direct-io mode to reduce IO.

* Add testViewHook helper to recovery the backing file since the ext4
  might need writable permission to handle recovery. If the backing file
  needs recovery and it's for View snapshot, the readonly mount will
  cause error.

* Use 8 MiB as capacity to reduce the IO.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-06-15 23:54:06 +08:00
Wei Fu
6dfb16f99a snapshots|pkg: umount without DETACH and nosync after umount
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-06-15 23:53:47 +08:00
Derek McGowan
d115129d1e Add configurable mount options to overlay
Allows default mount options to be provided through configuration.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-06-12 09:17:07 -07:00
Derek McGowan
f49789fd54 Merge pull request #8594 from fuweid/fix-blockfile-mount
snapshots/blockfile: use loop if options is empty
2023-05-30 08:33:09 -07:00
Cardy.Tang
9702d4aeec snapshots/blockfile: use passed in dst in scratchGenerator
Signed-off-by: Cardy.Tang <zuniorone@gmail.com>
2023-05-30 11:42:19 +08:00
Wei Fu
bb82f3683e snapshots/blockfile: use loop if options is empty
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-05-29 23:26:39 +08:00
Derek McGowan
7beaa5e859 Add mount options to blockfile snapshotter
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-23 10:56:20 -07:00
Cardy.Tang
b9f2e48e37 feat: make overlay sync removal configurable
Signed-off-by: Cardy.Tang <zuniorone@gmail.com>
2023-05-19 16:32:10 +08:00
Akihiro Suda
878132923d Merge pull request #8511 from dmcgowan/blockfile-snapshotter
Add blockfile snapshotter
2023-05-18 11:52:37 +09:00
Derek McGowan
1be571b14f Add sync before unmount on snapshotter layer test
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-17 14:25:27 -07:00
Derek McGowan
0c386e2ebd Add blockfile snapshotter
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-17 14:25:27 -07:00
Akihiro Suda
5dbae38191 snapshots/testsuite: Rename: fix fuse-overlayfs incompatibility
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-05-11 16:59:53 +09:00
Derek McGowan
14d22efccc Merge pull request #8254 from gabriel-samfira/fix-access-denied-on-vhdx-mount
Fix access denied on mounted vhdx root
2023-05-10 08:39:36 -07:00
Maksym Pavlenko
6f34da5f80 Cleanup logrus imports
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-05-05 11:54:14 -07:00
Gabriel Adrian Samfira
6f0714efcb Use RunWithPrivileges
RunWithPrivileges() will enable privileges will lock a thread, change
privileges, and run the function passed in, within that thread. This
allows us to limit the scope in which we enable privileges and avoids
accidentally enabling privileges in threads that should never have them.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-06 17:06:50 -07:00
Gabriel Adrian Samfira
b41ca11598 Fix access denied on mounted vhdx root
It seems that in certain situations, like having the containerd root
and state on a file system hosted on a mounted VHDX, we need
SeSecurityPrivilege when opening a file with winio.ACCESS_SYSTEM_SECURITY.
This happens in the base layer writer in hcsshim when adding a new file.

Enabling SeSecurityPrivilege allows the containerd root to be hosted on
a vhdx.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-06 17:06:50 -07:00
Gabriel Adrian Samfira
8538e7a2ac Improve error messages and remove check
* Improve error messages
  * remove a check for the existance of unmount target. We probably
    should not mask that the target was missing.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-04 12:07:34 -07:00
Gabriel Adrian Samfira
6a5b4c9c24 Remove "bind" code path from diff
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-03 08:11:35 -07:00
Gabriel Adrian Samfira
d373ebc4de Properly mount base layers
As opposed to a writable layer derived from a base layer, the volume
path of a base layer, once activated and prepared will not be a WCIFS
volume, but the actual path on disk to the snapshot. We cannot directly
mount this folder, as that would mean a client may gain access and
potentially damage important metadata files that would render the layer
unusabble.

For base layers we need to mount the Files folder which must exist in
any valid base windows-layer.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-02 08:35:34 -07:00
Gabriel Adrian Samfira
ca5605b4a2 Skip parent layer options on bind mounts
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-01 08:49:24 -07:00
Gabriel Adrian Samfira
4012c1b853 Remove escalated privileges
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:17:35 -07:00
Gabriel Adrian Samfira
95687a9324 Fix go.mod, simplify boolean logic, add logging
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:16:56 -07:00
Gabriel Adrian Samfira
feb637f92d Fix layer comparison and enable read-only checks
fstest.CheckDirectoryEqual checks if any files in the diff matches a
list of known metadataFiles. This only happens if we specify the initial
layer as the first parameter and the mutated layer as the second.

This also enables the read-only view checks, as the bind filter allows
us to mount a layer as ro.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:18 -07:00
Gabriel Adrian Samfira
36dc2782c4 Use bind filer for mounts
The bind filter supports bind-like mounts and volume mounts. It also
allows us to have read-only mounts.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
7b36becd2d Run Windows snapshotter through the test suite
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
ac30eabbdc Fix misspelling of 'Native' as 'Naive'
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
639c5799a2 Add paired 'mount' log for 'unmount'
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
8395e3a89a Don't use all-upper-case filenames in snapshot tests
NTFS, when presented with an all-caps filename, assumes you are just
being loud for no reason, and instead stores an all-lower-case filename.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
909730decb Skip tests that do not apply to WCOW on Windows
Filesystem permissions and ownership are not modifiable via an image
mount.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
469c13997a Ensure mounts are unmounted before leaving the test
This is necessary on Windows, as it's not possible to delete a snapshot
while it is still mounted, even if the mount-point has been deleted.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:17 -07:00