Commit Graph

1609 Commits

Author SHA1 Message Date
BinBin He
79f781d009 Refactor: Removing inherently flaky and unused SourceDateEpochOrNow function.
Signed-off-by: BinBin He <BinSquare@users.noreply.github.com>
2023-09-17 08:34:26 -07:00
Derek McGowan
31b6cdfd10 Merge pull request #8493 from DataDog/image-verifier-bindir-plugin
Add image verifier transfer service plugin system based on a binary directory
2023-09-14 06:37:17 -07:00
Phil Estes
3f315fcabf Merge pull request #9095 from thaJeztah/isolate_platform 2023-09-14 08:31:50 -04:00
Rodrigo Campos
2e13d39546 pkg/process: Only use idmap mounts if runc supports it
runc, as mandated by the runtime-spec, ignores unknown fields in the
config.json. This is unfortunate for cases where we _must_ enable that
feature or fail.

For example, if we want to start a container with user namespaces and
volumes, using the uidMappings/gidMappings field is needed so the
UID/GIDs in the volume don't end up with garbage. However, if we don't
fail when runc will ignore these fields (because they are unknown to
runc), we will just start a container without using the mappings and the
UID/GIDs the container will persist to volumes the hostUID/GID, that can
change if the container is re-scheduled by Kubernetes.

This will end up in volumes having "garbage" and unmapped UIDs that the
container can no longer change. So, let's avoid this entirely by just
checking that runc supports idmap mounts if the container we are about
to create needs them.

Please note that the "runc features" subcommand is only run when we are
using idmap mounts. If idmap mounts are not used, the subcommand is not
run and therefore this should not affect containers that don't use idmap
mounts in any way.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-09-13 16:44:54 +02:00
Rodrigo Campos
a81f80884b Revert "cri: Throw an error if idmap mounts is requested"
This reverts commit 7e6ab84884.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-09-13 16:44:54 +02:00
Rodrigo Campos
ab5b43fe80 cri/sbserver: Pass down UID/GID mappings to OCI runtime
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-09-13 16:44:54 +02:00
Sebastiaan van Stijn
e916d77c81 platforms: move ToProto, FromProto to api/types
These utilities resulted in the platforms package to have the containerd
API as dependency. As this package is used in many parts of the code, as
well as external consumers, we should try to keep it light on dependencies,
with the potential to make it a standalone module.

These utilities were added in f3b7436b61,
which has not yet been included in a release, so skipping deprecation
and aliases for these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-13 16:44:52 +02:00
Rodrigo Campos
e0b2b17de3 cri/server: Add tests for the linux-specific parts of VolumeMounts()
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-09-13 16:42:31 +02:00
Rodrigo Campos
10cb112e4a cri/server: Add tests for ContainerMounts()
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-09-13 16:42:31 +02:00
Rodrigo Campos
97dfa7f556 cri/server: Pass down uidMappings to OCI runtime
When the kubelet sends the uid/gid mappings for a mount, just pass them
down to the OCI runtime.

OCI runtimes support this since runc 1.2 and crun 1.8.1.

And whenever we add mounts (container mounts or image spec volumes) and
userns are requested by the kubelet, we use those mappings in the mounts
so the mounts are idmapped correctly. If no userns is used, we don't
send any mappings which just keeps the current behavior.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-09-13 16:42:31 +02:00
Ethan Lowman
ac1d556b92 Add image verifier transfer service plugin system based on a binary directory
Signed-off-by: Ethan Lowman <ethan.lowman@datadoghq.com>
2023-09-07 18:45:02 -04:00
Maksym Pavlenko
c13f47a3ae Merge pull request #9029 from dmcgowan/push-inherit-distribution-sources
push: inherit distribution sources from parent
2023-09-07 12:46:18 -07:00
Derek McGowan
b11439fc4b Merge pull request #9034 from thaJeztah/replace_reference
replace reference/docker for github.com/distribution/reference v0.5.0
2023-09-05 06:52:29 -07:00
Akihiro Suda
e30a40eb65 Merge pull request #9016 from djdongjin/remove-most-logrus
Remove most logrus import
2023-09-05 16:09:12 +09:00
Fu Wei
e2bf34feaf Merge pull request #9033 from dcantah/sberror-include-id
CRI: Include sandbox ID in failed to recover error
2023-09-02 10:48:34 +08:00
Sebastiaan van Stijn
5d31e93787 pkg/systemd: use sync.Once for systemd detection
This brings over the enhancement from a506630e57.

We don't expect the systemd state to change while containerd is running,
so we can use a `sync.Once` for this, to prevent stat'ing each time.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-01 12:14:56 +02:00
Sebastiaan van Stijn
7d0ab4fc2c remove uses of github.com/runc/libcontainer/cgroups
runc considers libcontainer to be "unstable" (not for external use),
so we try not to use it. Commit ed47d6ba76
brought back the dependency on other parts of libcontainer, but looks to
be only depending on a single utility, which in itself was borrowed from
github.com/coreos/go-systemd to not introduce CGO code in the same package.

This patch copies the version from github.com/coreos/go-systemd (adding
proper attribution, although the function is pretty trivial).

runc is in process of moving the libcontainer/user package to an external
module, which means we can remove the dependency on libcontainer entirely
in the near future. There is one more use of `libcontainer` in our vendor
tree; it looks like CDI is depending on one utility (devices.DeviceFromPath);
a943033a8b/vendor/github.com/container-orchestrated-devices/container-device-interface/pkg/cdi/container-edits_unix.go (L38)

We should remove the dependency on that utility, and add a CI check to
prevent bringing it back.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-01 12:10:55 +02:00
Derek McGowan
24aca53fa0 Update use of content.Infoprovider
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 11:04:33 -07:00
Danny Canter
a2817ca16d CRI: Include sandbox ID in failed to load error
The failed to recover state message didn't include the ID making this
not as useful as it could be..

This additionally moves some of the other logs to include the id for
the sandbox/container as a field instead of part of a format string.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-08-31 10:07:07 -07:00
Sebastiaan van Stijn
4923470902 replace reference/docker for github.com/distribution/reference v0.5.0
The reference/docker package was a fork of github.com/distribution/distribution,
which could not easily be used as a direct dependency, as it brought many other
dependencies with it.

The "reference' package has now moved to a separate repository, which means
we can replace the local fork, and use the upstream implementation again.

The new module was extracted from the distribution repository at commit:
b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 15:54:50 +02:00
Edgar Lee
779875a057 Add missing unpacker.Wait for image import
- For remote snapshotters, the unpack phase serves as an important step for
  preparing the remote snapshot. With the missing unpacker.Wait, the
  snapshotter `Prepare` context is always canceled.
- This patch allows remote snapshotter based archives to be imported via
  the transfer service or `ctr image import`

Signed-off-by: Edgar Lee <edgarhinshunlee@gmail.com>
2023-08-29 15:34:20 -07:00
Jin Dong
fc45365fa1 Remove most logrus
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-26 14:31:53 -04:00
Akihiro Suda
f48bbef193 Merge pull request #8994 from mxpv/cri
Use sandboxed CRI by default
2023-08-24 13:42:58 +09:00
Phil Estes
8e7a25856b Merge pull request #8998 from dmcgowan/image-inspect
ctr: images inspect
2023-08-23 14:12:56 -04:00
Maksym Pavlenko
c3f3cad287 Use sandboxed CRI by default
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-08-23 08:50:40 -07:00
Sebastiaan van Stijn
b76cd4d9fd replace some fmt.Sprintfs with strconv
Teeny-tiny optimizations:

    BenchmarkSprintf-10       37735996    32.31  ns/op  0 B/op  0 allocs/op
    BenchmarkItoa-10         591945836     2.031 ns/op  0 B/op  0 allocs/op
    BenchmarkFormatUint-10   593701444     2.014 ns/op  0 B/op  0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 16:43:02 +02:00
Derek McGowan
78308b4a44 Add manifest printer library
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-23 06:52:35 -07:00
Sebastiaan van Stijn
d7bc8694be pkg/cri: replace some fmt.Sprintfs with strconv
Teeny-tiny optimizations:

    BenchmarkSprintf-10       37735996    32.31  ns/op  0 B/op  0 allocs/op
    BenchmarkItoa-10         591945836     2.031 ns/op  0 B/op  0 allocs/op
    BenchmarkFormatUint-10   593701444     2.014 ns/op  0 B/op  0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 10:10:56 +02:00
Fu Wei
738c153573 Merge pull request #8992 from djdongjin/remove-hashicorp-multierror
Remove hashicorp/go-multierror dependency
2023-08-23 13:13:51 +08:00
Derek McGowan
2bac6ffb79 Merge pull request #8663 from helen-frank/feature/MergeSortedStringSlices
MergeStringSlices use sets
2023-08-22 16:31:28 -07:00
Phil Estes
de066a37dc Merge pull request #8935 from lengrongfu/feat/add-metrics-for-dropped-events
add metrics for discarding events
2023-08-22 09:09:31 -04:00
Fu Wei
3ffde050a4 Merge pull request #8988 from kinvolk/rata/userns-fix-platform
cri: Fix sandbox_mode "shim"
2023-08-22 16:40:34 +08:00
Derek McGowan
b8f32e863c Merge pull request #8951 from kiashok/exposeCommitMemWindows
Populate commit memory for windows memory usage stats
2023-08-21 15:42:34 -07:00
Jin Dong
cd8c8ae4bc Remove hashicorp/go-multierror
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-20 17:59:45 -07:00
Rodrigo Campos
d09f7cbe00 cri: Fix sandbox_mode "shim"
This is a partial revert of "cri/sbserver: Use platform instead of GOOS
for userns detection".

While what that commit did is 100% the right thing to do, when the
sandbox_mode is "shim" all controller.XXX() calls are RPCs and the
controller.Create() call initializes the controller. Therefore, things
like "getSandboxController()" don't work in the case of "shim"
sandbox_mode until after the controller.Create().

Due to this asymmetry and the lack of tests for shim mode, we didn't
catch it before.

This patch just reverts that commit so that the Create() and
getSandboxController() calls remain where they were, and just relies on
the config Linux section as a hack to detect if the pod sandbox will use
user namespaces or not.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-08-18 15:13:10 +02:00
Enrico Weigelt, metux IT consult
0c1ad52eac cri: spec_linux: drop unused retvals
cgroupv1HasHugetlb() and cgroupv2HasHugetlb() may return errors, but nobody
(there's just one call site anyways) ever cares. So drop the unnecessary code.

Signed-off-by: Enrico Weigelt, metux IT consult <info@metux.net>
2023-08-17 18:52:37 +02:00
Fu Wei
ba852faf41 Merge pull request #8954 from fuweid/fix-shim-leak 2023-08-17 08:16:20 +08:00
Kirtana Ashok
e2ce4f58f6 Populate commit memory for windows memory usage stats
Signed-off-by: Kirtana Ashok <kiashok@microsoft.com>
2023-08-15 16:48:22 -07:00
Kirtana Ashok
823e0420eb Fix transfer service dependencies:
- Fill OSVersion field of ocispec.Platform for windows OS in
transfer service plugin init()
- Do not return error from transfer service ReceiveStream if
stream.Recv() returned context.Canceled error

Signed-off-by: Kirtana Ashok <kiashok@microsoft.com>
2023-08-15 15:32:51 -07:00
Wei Fu
8dcb2a6e6d pkg/cri/sbserver: fix leaked shim issue for podsandbox mode
Fixes: #7496 #8931

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-08-11 17:43:51 +08:00
Wei Fu
72bc63d83d pkg/cri/server: fix leaked shim issue
Fixes: #7496 #8931

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-08-11 17:43:51 +08:00
Kirtana Ashok
a645ff2e68 Update dependencies after protobuf update in hcsshim
Signed-off-by: Kirtana Ashok <kiashok@microsoft.com>
(cherry picked from commit d129b6f890bceb56b050bbb23ad330bb5699f78c)
Signed-off-by: Kirtana Ashok <kiashok@microsoft.com>
2023-08-09 11:56:45 -07:00
rongfu.leng
54baf766e5 add metrics for discarding events
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
2023-08-09 09:56:26 +08:00
Fu Wei
2b2195c36b Merge pull request #8722 from marquiz/devel/cgroup-driver-autoconfig
cri: implement RuntimeConfig rpc
2023-08-04 16:09:34 +08:00
Rodrigo Campos
c80a3ecafd cri/sbserver: Use platform instead of GOOS for userns detection
In the sbserver we should not use the GOOS, as windows hosts can run
linux containers. On the sbserver we should use the platform param.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-08-02 12:32:05 +02:00
Rodrigo Campos
2d64ab8d79 cri: Don't use rel path for image volumes
Runc 1.1 throws a warning when using rel destination paths, and runc 1.2
is planning to thow an error (i.e. won't start the container).

Let's just make this an abs path in the only place it might not be: the
mounts created due to `VOLUME` directives in the Dockerfile.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-07-31 12:33:54 +02:00
Markus Lehtonen
ed47d6ba76 cri: implement RuntimeConfig rpc
The rpc only reports one field, i.e. the cgroup driver, to kubelet.
Containerd determines the effective cgroup driver by looking at all
runtime handlers, starting from the default runtime handler (the rest in
alphabetical order), and returning the cgroup driver setting of the
first runtime handler that supports one. If no runtime handler supports
cgroup driver (i.e. has a config option for it) containerd falls back to
auto-detection, returning systemd if systemd is running and cgroupfs
otherwise.

This patch implements the CRI server side of Kubernetes KEP-4033:
https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4033-group-driver-detection-over-cri

Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
2023-07-28 13:50:43 +03:00
Phil Estes
81895d22c9 Merge pull request #8867 from Iceber/pinned_image_label
cri: fix using the labels to pin image
2023-07-27 09:51:23 -04:00
Iceber Gu
7f7ba31b64 cri: fix using the pinned label to pin image
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2023-07-26 12:26:00 +08:00
Akihiro Suda
4807571352 pkg/epoch: fix Y2038 on 32-bit hosts
`strconv.Itoa(int(tm.Unix()))` rounds the time to 32-bit int on 32-bit hosts

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-07-26 13:17:39 +09:00