Commit Graph

14496 Commits

Author SHA1 Message Date
Derek McGowan
74e205f1e7
Merge pull request #8792 from zhuangqh/fix/writer-deadlock
content: reduce the contention between ref lock and boltdb lock
2023-09-06 09:58:07 -07:00
Akihiro Suda
e44d9e1c4f
Merge pull request #9061 from AkihiroSuda/crun-1.8.7
CI: bump up crun to 1.8.7
2023-09-07 01:32:48 +09:00
Phil Estes
827a960303
Merge pull request #9050 from vernou/fix-windows-installation-script
fix(docs): fix on the windows installation script
2023-09-06 12:07:00 -04:00
Akihiro Suda
0041996ca0
CI: bump up crun to 1.8.7
https://github.com/containers/crun/compare/1.8.3...1.8.7

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-09-06 23:45:56 +09:00
VERNOU Cédric
4bc0b26602
Revert the removal of the parameter Container
Signed-off-by: VERNOU Cédric <1659796+vernou@users.noreply.github.com>
2023-09-06 14:56:21 +02:00
Akhil Mohan
6ec0d4a3ad
prevent ctr from creating tags with forbidden characters
check if the target tag that is to be created using ctr image tag
is valid and does not contain any forbidden characters.

Signed-off-by: Akhil Mohan <makhil@vmware.com>
2023-09-06 15:47:52 +05:30
jerryzhuang
a4bdbf7844 content: reduce the contention between ref lock and boltdb lock
tryLock only once to reduce the amount of time the lock is held

Signed-off-by: jerryzhuang <zhuangqhc@gmail.com>
2023-09-05 23:34:13 +08:00
Fu Wei
5c37d3827b
Merge pull request #9055 from fuweid/fix-fuzz-issue
contrib/fuzz: fix redeclared issue
2023-09-05 23:03:57 +08:00
Derek McGowan
b11439fc4b
Merge pull request #9034 from thaJeztah/replace_reference
replace reference/docker for github.com/distribution/reference v0.5.0
2023-09-05 06:52:29 -07:00
Akihiro Suda
0ee2433c94
Merge pull request #5890 from artqzn/idmapped_mounts
RFC: Initial support of idmapped mount points
2023-09-05 20:41:05 +09:00
Wei Fu
abfabb6d82 contrib/fuzz: fix redeclared issue
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-05 19:04:19 +08:00
Akihiro Suda
e30a40eb65
Merge pull request #9016 from djdongjin/remove-most-logrus
Remove most logrus import
2023-09-05 16:09:12 +09:00
Ilya Hanov
295bcec074 snapshotter: implement unit tests for idmapped mounts for overlay
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
9d01ed1c32 integration: add test for idmapped mounts
Signed-off-by: Alexey Perevalov <alexey.perevalov@huawei.com>
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
e49e6d6fd7 snapshotter: implement slow path for idmapped mounts check for overlay
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
1555a31bf6 mount: support idmapped mount points
This patch introduces idmapped mounts support for
container rootfs.

The idmapped mounts support was merged in Linux kernel 5.12
torvalds/linux@7d6beb7.
This functionality allows to address chown overhead for containers that
use user namespace.

The changes are based on experimental patchset published by
Mauricio Vásquez #4734.
Current version reiplements support of idmapped mounts using Golang.

Performance measurement results:
Image           idmapped mount  recursive chown
BusyBox         00.135          04.964
Ubuntu          00.171          15.713
Fedora          00.143          38.799

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Artem Kuzin <artem.kuzin@huawei.com>
Signed-off-by: Alexey Perevalov <alexey.perevalov@huawei.com>
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
723c88ce30 snapshotter: add "--remap-labels" support to overlayfs
Previously the only fuse-overlayfs supports "--remap-labels" option.
Since idmapped mounts were landed to Linux kernel v5.12 it becomes
possible to use it with overlayfs via mount_setattr() system call.

The changes are based on experimental patchset published by
Mauricio Vásquez #4734.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Artem Kuzin <artem.kuzin@huawei.com>
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Ilya Hanov
e8ddf669f5 snapshotter: support "remap-ids" capability for overlayfs
Previously remapping of a snapshotter has been done using
recursive chown.

Commit
31a6449734 added a support
for "remap-ids" capability which allows snapshotter internals do
remappings in case of idmapped mounts support to avoid recursive
chown and creating a new remapped snapshot.

Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
2023-09-05 01:23:30 +03:00
Derek McGowan
817391989f
Merge pull request #9044 from thaJeztah/update_uuid
vendor: github.com/google/uuid v1.3.1
2023-09-04 14:30:46 -07:00
VERNOU Cédric
ef6b91947e
fix(docs): fix on the windows installation script
Modify the parameter `-Path` to reference a folder, so `Copy-Item` create the destination folder.
Remove "-Container:$false" that flatten the hierarchy folder.

Signed-off-by: VERNOU Cédric <1659796+vernou@users.noreply.github.com>
2023-09-02 13:19:50 +02:00
Fu Wei
e2bf34feaf
Merge pull request #9033 from dcantah/sberror-include-id
CRI: Include sandbox ID in failed to recover error
2023-09-02 10:48:34 +08:00
Akihiro Suda
74705ae4f4
Merge pull request #9045 from thaJeztah/less_libcontainer
remove uses of github.com/runc/libcontainer/cgroups
2023-09-02 07:56:41 +09:00
Akihiro Suda
5a6c32da38
Merge pull request #9041 from fuweid/enable-fuzz-build
.github: enable fuzz build
2023-09-02 07:47:32 +09:00
Phil Estes
6f2855ef0f
Merge pull request #9036 from kiashok/updateRunhcsVersion 2023-09-01 14:00:28 -04:00
Derek McGowan
dc8b0d80b9
Merge pull request #9042 from dcantah/darwin-blockfile-copyfile
Blockfile: Enlighten blockfile copy on Darwin
2023-09-01 07:18:44 -07:00
Sebastiaan van Stijn
5d31e93787
pkg/systemd: use sync.Once for systemd detection
This brings over the enhancement from a506630e57.

We don't expect the systemd state to change while containerd is running,
so we can use a `sync.Once` for this, to prevent stat'ing each time.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-01 12:14:56 +02:00
Sebastiaan van Stijn
7d0ab4fc2c
remove uses of github.com/runc/libcontainer/cgroups
runc considers libcontainer to be "unstable" (not for external use),
so we try not to use it. Commit ed47d6ba76
brought back the dependency on other parts of libcontainer, but looks to
be only depending on a single utility, which in itself was borrowed from
github.com/coreos/go-systemd to not introduce CGO code in the same package.

This patch copies the version from github.com/coreos/go-systemd (adding
proper attribution, although the function is pretty trivial).

runc is in process of moving the libcontainer/user package to an external
module, which means we can remove the dependency on libcontainer entirely
in the near future. There is one more use of `libcontainer` in our vendor
tree; it looks like CDI is depending on one utility (devices.DeviceFromPath);
a943033a8b/vendor/github.com/container-orchestrated-devices/container-device-interface/pkg/cdi/container-edits_unix.go (L38)

We should remove the dependency on that utility, and add a CI check to
prevent bringing it back.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-01 12:10:55 +02:00
Sebastiaan van Stijn
9bc6441c21
vendor: github.com/google/uuid v1.3.1
Contains some performance improvements:

full diff: https://github.com/google/uuid/compare/v1.3.0...v1.3.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-01 11:49:50 +02:00
Danny Canter
13ff185ba0 Blockfile: Enlighten blockfile copy on Darwin
The Go stdlib does not seem to have an efficient os.File.ReadFrom
routine for other platforms like it does on Linux with
copy_file_range. For Darwin at least we can use clonefile
in its place, otherwise if we have a sparse file we'd have
a fun surprise with the io.Copy approach..

We should see if there's other platforms that we can enhance here.
I've forgotten what's the right route on Windows.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-08-31 20:18:48 -07:00
Wei Fu
a3e8503ed2 .github: enable fuzz build
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-01 10:36:54 +08:00
Kazuyoshi Kato
a943033a8b
Merge pull request #9013 from fish98/main
fuzzing: fix ossfuzz building error
2023-08-31 16:10:43 -07:00
Kazuyoshi Kato
a617005750
Merge pull request #9028 from hinshun/fix/import-unpack-wait
Add missing unpacker.Wait for image import
2023-08-31 16:09:42 -07:00
Derek McGowan
cd705af89b
metadata: add image delete events during garbage collection
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 15:10:13 -07:00
Derek McGowan
3f9756c184
gc: add support for image expiration
Update the garbage collector to support image expiration along with
support for image leasing. This allows making images collectible during
garbage collection and using a lease to prevent removal of an image.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 15:10:13 -07:00
Derek McGowan
b32638e21b
ctr: pull or fetch image metadata by default
The metadata is small and useful for viewing all platforms
for an image and enabling push back to the same registry.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 11:07:52 -07:00
Derek McGowan
20f79300cf
push: inherit distribution sources from parent
When a blob does not exist locally, rather than erroring on info
lookup, inherit the parent distribution sources. Push is able
to succeed even if the blob does not exist locally when a cross
repository mount is done. This is a common operation pushing a
multi-platform image to the same registry but different namespace.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 11:07:51 -07:00
Derek McGowan
24aca53fa0
Update use of content.Infoprovider
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 11:04:33 -07:00
Derek McGowan
f8c789f319
content: add InfoProvider interface
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 11:04:32 -07:00
Kirtana Ashok
02d519ad64 Update hcsshim tag in runhcs-version script
Signed-off-by: Kirtana Ashok <kiashok@microsoft.com>
2023-08-31 10:07:14 -07:00
Danny Canter
a2817ca16d CRI: Include sandbox ID in failed to load error
The failed to recover state message didn't include the ID making this
not as useful as it could be..

This additionally moves some of the other logs to include the id for
the sandbox/container as a field instead of part of a format string.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-08-31 10:07:07 -07:00
Derek McGowan
b2f46463b5
Merge pull request #9031 from thaJeztah/revert_log_var
Revert "log: define G() as a function instead of a variable"
2023-08-31 06:55:41 -07:00
Sebastiaan van Stijn
4923470902
replace reference/docker for github.com/distribution/reference v0.5.0
The reference/docker package was a fork of github.com/distribution/distribution,
which could not easily be used as a direct dependency, as it brought many other
dependencies with it.

The "reference' package has now moved to a separate repository, which means
we can replace the local fork, and use the upstream implementation again.

The new module was extracted from the distribution repository at commit:
b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 15:54:50 +02:00
TTFISH
26dcae2f6b fix ossfuzz building error
Signed-off-by: TTFISH <jiongchiyu@gmail.com>
2023-08-31 20:07:05 +08:00
Derek McGowan
f9608c0027
Merge pull request #9030 from kiashok/updateShimTag-main
Update hcsshim tag to v0.12.0-rc.0
2023-08-30 16:13:15 -07:00
Sebastiaan van Stijn
19d6c37a3f
Revert "log: define G() as a function instead of a variable"
This reverts commit 778ac302b2.

(slightly modified, due to changes that were merged after that).

The reverted commit had two elements;

- Make `G` an actual function to improve the documentation
- Prevent `G` from being overwritten externally

From the commit that's reverted:

> The `G` variable is exported, and not expected to be overwritten
> externally. Defining it as a function also documents it as a function
> on https://pkg.go.dev, instead of a variable; https://pkg.go.dev/github.com/containerd/containerd@v1.6.22/log#pkg-variables

While it's unclear if the ability to replace the implementation was
_intentional_, it's this part that some external consumers were (ab)using.

We should look into that part in a follow-up, and design for this, for
example by providing a utility to replace the logger, and properly document
that.

In the meantime, let's revert the change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-30 21:46:21 +02:00
Kirtana Ashok
fd5d92a7f5 Update hcsshim tag versioning to v0.12.0-rc.0
hcsshim tags v0.10.* is deprecated, so using the new
v0.12.0-rc.* versioning for hcsshim tags on containerd/main

Signed-off-by: Kirtana Ashok <kiashok@microsoft.com>
2023-08-29 17:41:20 -07:00
Edgar Lee
779875a057 Add missing unpacker.Wait for image import
- For remote snapshotters, the unpack phase serves as an important step for
  preparing the remote snapshot. With the missing unpacker.Wait, the
  snapshotter `Prepare` context is always canceled.
- This patch allows remote snapshotter based archives to be imported via
  the transfer service or `ctr image import`

Signed-off-by: Edgar Lee <edgarhinshunlee@gmail.com>
2023-08-29 15:34:20 -07:00
Samuel Karp
b0a7f08aa7
Merge pull request #9021 from akhilerm/make-configurable-dependencies
make repositories of install dependencies configurable
2023-08-28 23:55:48 -07:00
Samuel Karp
cb532a87ef
Merge pull request #9019 from dcantah/hcsshim-version
Windows: Supply windows shim version via file
2023-08-28 23:48:32 -07:00
Derek McGowan
4f5db2bc03
metadata: format unit test output
Make it easier to see list differences in gc node test failure output.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-28 16:52:25 -07:00