Commit Graph

21 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
8a8c3e2215
pkg/cri/server/base: log CRI config as embedded JSON
Use the JSON-encoded representation of the config used, which allows
users to reconstruct a (valid) config file from the logs, which may be
more useful for debugging purposes than the internal (Go) representation.

Before this:

    INFO[2023-12-07T15:33:39.914626385Z] starting cri plugin                           config="{PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}"

After this:

    INFO[2023-12-07T15:27:15.862946138Z] starting cri plugin                           config="{\"containerd\":{\"snapshotter\":\"overlayfs\",\"defaultRuntimeName\":\"runc\",\"runtimes\":{\"runc\":{\"runtimeType\":\"io.containerd.runc.v2\",\"runtimePath\":\"\",\"PodAnnotations\":null,\"ContainerAnnotations\":null,\"options\":{\"BinaryName\":\"\",\"CriuImagePath\":\"\",\"CriuWorkPath\":\"\",\"IoGid\":0,\"IoUid\":0,\"NoNewKeyring\":false,\"Root\":\"\",\"ShimCgroup\":\"\"},\"privileged_without_host_devices\":false,\"privileged_without_host_devices_all_devices_allowed\":false,\"baseRuntimeSpec\":\"\",\"cniConfDir\":\"\",\"cniMaxConfNum\":0,\"snapshotter\":\"\",\"sandboxer\":\"podsandbox\"}},\"disableSnapshotAnnotations\":true,\"discardUnpackedLayers\":false,\"ignoreBlockIONotEnabledErrors\":false,\"ignoreRdtNotEnabledErrors\":false},\"cni\":{\"binDir\":\"/opt/cni/bin\",\"confDir\":\"/etc/cni/net.d\",\"maxConfNum\":1,\"setupSerially\":false,\"confTemplate\":\"\",\"ipPref\":\"\"},\"registry\":{\"configPath\":\"\",\"mirrors\":null,\"configs\":null,\"auths\":null,\"headers\":null},\"imageDecryption\":{\"keyModel\":\"node\"},\"disableTCPService\":true,\"streamServerAddress\":\"127.0.0.1\",\"streamServerPort\":\"0\",\"streamIdleTimeout\":\"4h0m0s\",\"enableSelinux\":false,\"selinuxCategoryRange\":1024,\"sandboxImage\":\"registry.k8s.io/pause:3.9\",\"statsCollectPeriod\":10,\"enableTLSStreaming\":false,\"x509KeyPairStreaming\":{\"tlsCertFile\":\"\",\"tlsKeyFile\":\"\"},\"maxContainerLogSize\":16384,\"disableCgroup\":false,\"disableApparmor\":false,\"restrictOOMScoreAdj\":false,\"maxConcurrentDownloads\":3,\"disableProcMount\":false,\"unsetSeccompProfile\":\"\",\"tolerateMissingHugetlbController\":true,\"disableHugetlbController\":true,\"device_ownership_from_security_context\":false,\"ignoreImageDefinedVolumes\":false,\"netnsMountsUnderStateDir\":false,\"enableUnprivilegedPorts\":true,\"enableUnprivilegedICMP\":true,\"enableCDI\":false,\"cdiSpecDirs\":[\"/etc/cdi\",\"/var/run/cdi\"],\"imagePullProgressTimeout\":\"5m0s\",\"drainExecSyncIOTimeout\":\"0s\",\"containerdRootDir\":\"/var/lib/docker/containerd/daemon\",\"containerdEndpoint\":\"/var/run/docker/containerd/containerd.sock\",\"rootDir\":\"/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri\",\"stateDir\":\"/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri\"}"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-29 13:10:54 +02:00
Sebastiaan van Stijn
f62edda5a2
pkg/cri/server/base: use structured log for CRI plugin startup
Log the config as a field instead of as part of the log message.

Before this:

    INFO[2023-12-07T14:58:43.515360429Z] loading plugin                                id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1
    INFO[2023-12-07T14:58:43.515787512Z] loading plugin                                id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1
    INFO[2023-12-07T14:58:43.515974429Z] loading plugin                                id=io.containerd.internal.v1.cri type=io.containerd.internal.v1
    INFO[2023-12-07T14:58:43.516037887Z] Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}

After this:

    INFO[2023-12-07T15:33:39.914112719Z] loading plugin                                id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1
    INFO[2023-12-07T15:33:39.914526135Z] loading plugin                                id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1
    INFO[2023-12-07T15:33:39.914580427Z] loading plugin                                id=io.containerd.internal.v1.cri type=io.containerd.internal.v1
    INFO[2023-12-07T15:33:39.914626385Z] starting cri plugin                           config="{PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: PodAnnotations:[] ContainerAnnotations:[] Options:map[BinaryName: CriuImagePath: CriuWorkPath: IoGid:0 IoUid:0 NoNewKeyring:false Root: ShimCgroup:] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: Sandboxer:podsandbox}] DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.9 StatsCollectPeriod:10 EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:true EnableUnprivilegedICMP:true EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-04-29 13:10:51 +02:00
Shuaiyi Zhang
e461a59ae6 fix migrateConfig for io.containerd.cri.v1.images
Signed-off-by: Shuaiyi Zhang <zhang_syi@qq.com>
2024-04-23 12:59:50 +00:00
Derek McGowan
9128ee0a91
Move nri packages to plugin and internal
NRI is still newer and mostly used by CRI plugin. Keep the package in
internal to allow for interfaces as the project matures.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-29 21:37:36 -08:00
Abel Feng
0f1d27412f sandbox: add methods to sandboxService
so that we cri service don't have to get sandbox controller everytime it
needs to call sandbox controller api.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2024-02-26 10:10:11 +08:00
Derek McGowan
a086125ae3
Move config version to version package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-14 21:09:44 -08:00
Akihiro Suda
b466b7ef85
Merge pull request #9747 from AkihiroSuda/revert-9713
Revert "cri: make read-only mounts recursively read-only"
2024-02-08 10:29:03 +00:00
Derek McGowan
86530c0afb
Move image event publishing to metadata store
The metadata store is in the best place to handle events directly after
the database has been updated. This prevents every user of the image
store interface from having to know whether or not they are responsible
for publishing events and avoid double events if the grpc local service
is used.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-02-06 21:24:55 -08:00
Akihiro Suda
6670695836
Revert "cri: make read-only mounts recursively read-only"
Revert PR 9713, as it appeared to break the compatibility too much
https://github.com/kubernetes/enhancements/pull/3858#issuecomment-1925441072

This reverts commit b2f254fff0.

> Conflicts:
>	internal/cri/opts/spec_linux_opts.go

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-04 01:17:14 +09:00
Maksym Pavlenko
bbac058cf3 Move CRI from pkg/ to internal/
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2024-02-02 10:12:08 -08:00
Akihiro Suda
b2f254fff0
cri: make read-only mounts recursively read-only
Prior to this commit, `readOnly` volumes were not recursively read-only and
could result in compromise of data;
e.g., even if `/mnt` was mounted as read-only, its submounts such as
`/mnt/usbstorage` were not read-only.

This commit utilizes runc's "rro" bind mount option to make read-only bind
mounts literally read-only. The "rro" bind mount options is implemented by
calling `mount_setattr(2)` with `MOUNT_ATTR_RDONLY` and `AT_RECURSIVE`.

The "rro" bind mount options requires kernel >= 5.12, with runc >= 1.1 or
a compatible runtime such as crun >= 1.4.

When the "rro" bind mount options is not available, containerd falls back
to the legacy non-recursive read-only mounts by default.

The behavior is configurable via `/etc/containerd/config.toml`:
```toml
version = 2
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  # treat_ro_mounts_as_rro ("Enabled"|"IfPossible"|"Disabled")
  # treats read-only mounts as recursive read-only mounts.
  # An empty string means "IfPossible".
  # "Enabled" requires Linux kernel v5.12 or later.
  # This configuration does not apply to non-volume mounts such as "/sys/fs/cgroup".
  treat_ro_mounts_as_rro = ""
```

Replaces:
- kubernetes/enhancements issue 3857
- kubernetes/enhancements PR 3858

Note: this change does not affect non-CRI clients such as ctr, nerdctl, and Docker/Moby.
RRO mounts have been supported since nerdctl v0.14 (containerd/nerdctl PR 511)
and Docker v25 (moby/moby PR 45278).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-01 09:39:36 +09:00
Derek McGowan
64b4778fc2
Add deprecation warnings to CRI image server configuration
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-28 23:14:59 -08:00
Derek McGowan
65b3922df7
Split streaming config from runtime config
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-28 23:14:59 -08:00
Derek McGowan
58ff9d368d
Move cri plugin to plugins subpackage
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-28 20:57:19 -08:00
Derek McGowan
9795677fe9
Move cri base plugin to CRI runtime service
Create new plugin type for CRI runtime and image services.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-28 20:57:18 -08:00
Derek McGowan
e79ec7a095
Remove deprecated platforms package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-23 09:14:03 -08:00
Derek McGowan
70ed2696fa
Move events to pkg/events
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:54:36 -08:00
Derek McGowan
fcd39ccc53
Move snapshots to core/snapshots
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:54:09 -08:00
Derek McGowan
ce41d1c90a
Move services/server to cmd/containerd/server
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:52:48 -08:00
Derek McGowan
1a1e0e8c81
Move metadata to core/metadata
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:52:03 -08:00
Derek McGowan
3baf5edb8b
Separate the CRI image config from the main plugin config
This change simplifies the CRI plugin dependencies by not requiring the
CRI image plugin to depend on any other CRI components. Since other CRI
plugins depend on the image plugin, this allows prevents a dependency
cycle for CRI configurations on a base plugin.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-11 09:55:09 -08:00