Commit Graph

146 Commits

Author SHA1 Message Date
Abel Feng
15887d7efc sandbox: add update api for controller
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-06-14 02:31:51 +00:00
Derek McGowan
5d6a94a54c
Merge pull request #10243 from dmcgowan/update-shim-manager-config
Cleanup shim manager configuration
2024-06-03 17:22:36 +00:00
Maksym Pavlenko
6383a1ce76
Merge pull request #10258 from bzsuni/bz/update/pause
update pause image to 3.10
2024-05-28 18:01:23 +00:00
Sebastiaan van Stijn
446e63579c
remove uses of platforms.Platform alias
Commit 3c8469a782 removed uses of the api
types.Platform type from public interfaces, instead using the type from
the OCI image spec.

For convenience, it also introduced an alias in the platforms package.
While this alias allows packages that already import containerd's
platforms package (now a separate module), it may also cause confusion
(it's not clear that it's an alias for the OCI type), and for packages
that do not depend on containerd's platforms package / module may now
be resulting in an extra dependency.

Let's remove the use of this alias, and instead use the OCI type directly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-28 14:56:30 +02:00
bzsuni
22f2af40c0 update pause image to 3.10
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
2024-05-25 08:17:46 +08:00
Derek McGowan
ca59fb0b41
Cleanup shim manager configuration
Keep platforms configuration on task manager and add environment config
for shims.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-22 11:06:53 -07:00
Fu Wei
857dc6f89e
Merge pull request #10162 from dmcgowan/cleanup-local-transfer
Cleanup local transfer interface
2024-05-03 10:40:46 +00:00
Derek McGowan
fe01cad201
Cleanup local transfer interface
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-02 15:45:49 -07:00
Derek McGowan
2ac2b9c909
Make api a Go sub-module
Allow the api to stay at the same v1 go package name and keep using a
1.x version number. This indicates the API is still at 1.x and allows
sharing proto types with containerd 1.6 and 1.7 releases.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-02 11:03:00 -07:00
Derek McGowan
e1b94c0e7d
Move protobuf package under pkg
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-02 10:52:03 -07:00
Derek McGowan
4a45507772
Move runc options to api directory
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-05-02 10:52:00 -07:00
Fu Wei
a91b05d99c
Merge pull request #9736 from abel-von/sandbox-task-0201
sandbox: Store bootstrap parameters in sandbox metadata and shim get them from sandbox metadata rather than other shim's bootstrap.json file.
2024-05-02 15:35:29 +00:00
Maksym Pavlenko
2ec82c4703
Merge pull request #10128 from xinyangge-db/lockless_sync
Perform file sync outside of lock on Commit
2024-05-01 21:46:43 +00:00
Abel Feng
de38490ed6 sandbox: merge address and protocol to one url
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-04-30 15:28:00 +08:00
Abel Feng
c3b306240e add task api endpoint in task create options
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-04-30 15:22:44 +08:00
Abel Feng
b1fefccc78 sandbox: store endpoint in cri sandboxStore
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-04-30 15:20:03 +08:00
Maksym Pavlenko
7feb1f327d
Merge pull request #9853 from abel-von/make-shim-independent
sandbox: make an independent shim plugin
2024-04-29 21:07:21 +00:00
Sebastiaan van Stijn
8a8c3e2215
pkg/cri/server/base: log CRI config as embedded JSON
Use the JSON-encoded representation of the config used, which allows
users to reconstruct a (valid) config file from the logs, which may be
more useful for debugging purposes than the internal (Go) representation.

Before this:

    INFO[2023-12-07T15:33:39.914626385Z] starting cri plugin                           config="{PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}"

After this:

    INFO[2023-12-07T15:27:15.862946138Z] starting cri plugin                           config="{\"containerd\":{\"snapshotter\":\"overlayfs\",\"defaultRuntimeName\":\"runc\",\"runtimes\":{\"runc\":{\"runtimeType\":\"io.containerd.runc.v2\",\"runtimePath\":\"\",\"PodAnnotations\":null,\"ContainerAnnotations\":null,\"options\":{\"BinaryName\":\"\",\"CriuImagePath\":\"\",\"CriuWorkPath\":\"\",\"IoGid\":0,\"IoUid\":0,\"NoNewKeyring\":false,\"Root\":\"\",\"ShimCgroup\":\"\"},\"privileged_without_host_devices\":false,\"privileged_without_host_devices_all_devices_allowed\":false,\"baseRuntimeSpec\":\"\",\"cniConfDir\":\"\",\"cniMaxConfNum\":0,\"snapshotter\":\"\",\"sandboxer\":\"podsandbox\"}},\"disableSnapshotAnnotations\":true,\"discardUnpackedLayers\":false,\"ignoreBlockIONotEnabledErrors\":false,\"ignoreRdtNotEnabledErrors\":false},\"cni\":{\"binDir\":\"/opt/cni/bin\",\"confDir\":\"/etc/cni/net.d\",\"maxConfNum\":1,\"setupSerially\":false,\"confTemplate\":\"\",\"ipPref\":\"\"},\"registry\":{\"configPath\":\"\",\"mirrors\":null,\"configs\":null,\"auths\":null,\"headers\":null},\"imageDecryption\":{\"keyModel\":\"node\"},\"disableTCPService\":true,\"streamServerAddress\":\"127.0.0.1\",\"streamServerPort\":\"0\",\"streamIdleTimeout\":\"4h0m0s\",\"enableSelinux\":false,\"selinuxCategoryRange\":1024,\"sandboxImage\":\"registry.k8s.io/pause:3.9\",\"statsCollectPeriod\":10,\"enableTLSStreaming\":false,\"x509KeyPairStreaming\":{\"tlsCertFile\":\"\",\"tlsKeyFile\":\"\"},\"maxContainerLogSize\":16384,\"disableCgroup\":false,\"disableApparmor\":false,\"restrictOOMScoreAdj\":false,\"maxConcurrentDownloads\":3,\"disableProcMount\":false,\"unsetSeccompProfile\":\"\",\"tolerateMissingHugetlbController\":true,\"disableHugetlbController\":true,\"device_ownership_from_security_context\":false,\"ignoreImageDefinedVolumes\":false,\"netnsMountsUnderStateDir\":false,\"enableUnprivilegedPorts\":true,\"enableUnprivilegedICMP\":true,\"enableCDI\":false,\"cdiSpecDirs\":[\"/etc/cdi\",\"/var/run/cdi\"],\"imagePullProgressTimeout\":\"5m0s\",\"drainExecSyncIOTimeout\":\"0s\",\"containerdRootDir\":\"/var/lib/docker/containerd/daemon\",\"containerdEndpoint\":\"/var/run/docker/containerd/containerd.sock\",\"rootDir\":\"/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri\",\"stateDir\":\"/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri\"}"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-29 13:10:54 +02:00
Sebastiaan van Stijn
f62edda5a2
pkg/cri/server/base: use structured log for CRI plugin startup
Log the config as a field instead of as part of the log message.

Before this:

    INFO[2023-12-07T14:58:43.515360429Z] loading plugin                                id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1
    INFO[2023-12-07T14:58:43.515787512Z] loading plugin                                id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1
    INFO[2023-12-07T14:58:43.515974429Z] loading plugin                                id=io.containerd.internal.v1.cri type=io.containerd.internal.v1
    INFO[2023-12-07T14:58:43.516037887Z] Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}

After this:

    INFO[2023-12-07T15:33:39.914112719Z] loading plugin                                id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1
    INFO[2023-12-07T15:33:39.914526135Z] loading plugin                                id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1
    INFO[2023-12-07T15:33:39.914580427Z] loading plugin                                id=io.containerd.internal.v1.cri type=io.containerd.internal.v1
    INFO[2023-12-07T15:33:39.914626385Z] starting cri plugin                           config="{PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-29 13:10:51 +02:00
Xinyang Ge
4167416754 Perform file sync outside of lock on Commit
Signed-off-by: Xinyang Ge <xinyang.ge@databricks.com>
2024-04-26 05:42:01 -07:00
Abel Feng
a12acedfad sandbox: make a independent shim plugin
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-04-24 14:27:20 +08:00
Shuaiyi Zhang
e461a59ae6 fix migrateConfig for io.containerd.cri.v1.images
Signed-off-by: Shuaiyi Zhang <zhang_syi@qq.com>
2024-04-23 12:59:50 +00:00
Danny Canter
32caaee484 Snapshotters: Export the root path
Some of the snapshotters that allow you to change their root location
were already doing this, this just makes all of them follow the same
pattern.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2024-04-14 06:24:33 -07:00
Brian Goff
e41e9e11b5 transfer: Platform matcher should match multiple platforms
This allows arm64 to pull armhf images.
Before this change the transfer service would reject pulls for armhf on
an arm64 machine, or indeed any such platform variant mismatches.

I would argue that its a bit weird for the transfer service to reject a
pull at all since there are legitamate reasons to want to pull images
for other architectures, however that's a more philosophical change.

In the case where I ran into this, I have an arm64 machine running
an armhf containerd in an armhf container (for running some basic sanity
checks during packaging).
Tests started failing once `ctr` was moved to use the transfer service
by default.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-03-21 14:59:17 +00:00
Akihiro Suda
5a23e8878c
Merge pull request #9917 from AkihiroSuda/mv-testutil
mv internal/testutil pkg/testutil
2024-03-12 12:01:16 +00:00
zhanluxianshen
47d13767f4
Clean typos in plugins.
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
2024-03-08 07:33:20 +00:00
Amit Barve
994fdd74e5 Don't create new scratch VHD per image for CimFS
CimFS layers don't need to create a new scratch VHD per image. The scratch VHDs used with CimFS are empty so
we can just create one base VHD and one differencing VHD and copy it for every scratch snapshot.
(Note that UVM VHDs are still unique per image because the VHD information is embedded in the UVM BCD during
import)

Signed-off-by: Amit Barve <ambarve@microsoft.com>
2024-03-06 04:18:17 -08:00
Maksym Pavlenko
7d2bc0620b
Merge pull request #9911 from dmcgowan/introspection-split
Cleanup introspection interface
2024-03-05 03:20:45 +00:00
Akihiro Suda
d9b9160ae1
mv internal/testutil pkg/testutil
The package is consumed by several snapshotter plugins

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-03-04 17:00:39 +09:00
Derek McGowan
1bf781d8eb
Cleanup introspection interface
Split service proxy from service plugin.
Make introspection service easier for clients to use.
Update service proxy to support grpc and ttrpc.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-03-01 23:07:42 -08:00
Derek McGowan
9128ee0a91
Move nri packages to plugin and internal
NRI is still newer and mostly used by CRI plugin. Keep the package in
internal to allow for interfaces as the project matures.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-29 21:37:36 -08:00
Fu Wei
6333db7701
Merge pull request #9891 from dmcgowan/move-events-plugin
Move events to plugins and core
2024-02-29 12:45:25 +00:00
Fu Wei
2cdf012387
Merge pull request #9617 from abel-von/sandbox-plugin-0109
sandbox: use sandboxService in CRI plugin instead of calling controller API directly
2024-02-28 15:49:12 +00:00
Derek McGowan
72f21833b1
Move events to plugins and core
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-27 22:09:20 -08:00
Abel Feng
0f1d27412f sandbox: add methods to sandboxService
so that we cri service don't have to get sandbox controller everytime it
needs to call sandbox controller api.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-02-26 10:10:11 +08:00
Wei Fu
a2768f19d9 plugins/sandbox: move local plugin into services
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-02-24 20:45:17 +08:00
Maksym Pavlenko
33e544e94a
Merge pull request #9800 from austinvazquez/update-golangci-lint
Update golangci-lint version to v1.56.1 and resolve warnings
2024-02-22 21:22:56 +00:00
Maksym Pavlenko
67ff3dbc8d
Merge pull request #9795 from catap/prevent-zero-timer
Prevent GC from schedule itself with 0 period.
2024-02-21 21:15:00 +00:00
Austin Vazquez
f44d90be4a
Rename variables and snapshotter option to resolve lint warnings
Signed-off-by: Austin Vazquez <macedonv@amazon.com>
2024-02-19 15:14:00 +00:00
Derek McGowan
f8f1e5f7a6
Add container monitor plugin type for restart
Adds a plugin type for container monitor.
Rename the task monitor type to avoid confusion.
Add config migration for new plugin types to pass existing migration
tests.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-15 21:54:40 -08:00
Kirill A. Korinsky
c8766123d9
Prevent GC from schedule itself with 0 period.
On startup `gcTimeSum` might work fast and return `0`, so on this case
the algorithm turns in infinity loop which simple consume CPU on timer
which fires without any interval.

Use `5ms` as fallback to have interval `245ms` for that case.

Closes: https://github.com/containerd/containerd/issues/5089

Signed-off-by: Kirill A. Korinsky <kirill@korins.ky>
2024-02-15 11:32:53 +01:00
Derek McGowan
a086125ae3
Move config version to version package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-14 21:09:44 -08:00
Maksym Pavlenko
128f249f0e
Merge pull request #9790 from dmcgowan/mv-transfer-packages
Move transfer and unpack packages to core
2024-02-10 16:49:55 +00:00
Akihiro Suda
b466b7ef85
Merge pull request #9747 from AkihiroSuda/revert-9713
Revert "cri: make read-only mounts recursively read-only"
2024-02-08 10:29:03 +00:00
Derek McGowan
f46aea6187
Move transfer and unpack packages
Packages related to transfer and unpacking provide core interfaces which
use other core interfaces and part of common functionality.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-07 22:40:15 -08:00
Derek McGowan
86530c0afb
Move image event publishing to metadata store
The metadata store is in the best place to handle events directly after
the database has been updated. This prevents every user of the image
store interface from having to know whether or not they are responsible
for publishing events and avoid double events if the grpc local service
is used.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-06 21:24:55 -08:00
Samuel Karp
0125a42fb5
Merge pull request #9729 from mxpv/duration
Remove duplicated TOML duration parsers
2024-02-05 07:43:51 +00:00
Akihiro Suda
6670695836
Revert "cri: make read-only mounts recursively read-only"
Revert PR 9713, as it appeared to break the compatibility too much
https://github.com/kubernetes/enhancements/pull/3858#issuecomment-1925441072

This reverts commit b2f254fff0.

> Conflicts:
>	internal/cri/opts/spec_linux_opts.go

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-04 01:17:14 +09:00
Samuel Karp
96bf529cbf
Merge pull request #9742 from mxpv/envelope
Move Message proto to types
2024-02-03 06:32:01 +00:00
Derek McGowan
a896610da1
Merge pull request #9718 from jsturtevant/transfer-service-windows
Add a default differ for Windows that matches the snapshotter when using transfer service
2024-02-02 20:38:26 +00:00