A nil CRIImplementation field can cause a nil pointer dereference and
panic during startup recovery.
Prior to this change, the nri.API struct would have a nil cri
(CRIImplementation) field after nri.NewAPI until nri.Register was
called. Register is called mid-way through initialization of the CRI
plugin, but recovery for containers occurs prior to that. Container
recovery includes establishing new exit monitors for existing containers
that were discovered. When a container exits, NRI plugins are given the
opportunity to be notified about the lifecycle event, and this is done
by accessing that CRIImplementation field inside the nri.API. If a
container exits prior to nri.Register being called, access to the
CRIImplementation field can cause a panic.
Here's the call-path:
* The CRI plugin starts running
[here](ae71819c4f/pkg/cri/server/service.go (L222))
* It then [calls into](ae71819c4f/pkg/cri/server/service.go (L227))
`recover()` to recover state from previous runs of containerd
* `recover()` then attempts to recover all containers through
[`loadContainer()`](ae7d74b9e2/internal/cri/server/restart.go (L175))
* When `loadContainer()` finds a container that is still running, it waits
for the task (internal containerd object) to exit and sets up
[exit monitoring](ae7d74b9e2/internal/cri/server/restart.go (L391))
* Any exit that then happens must be
[handled](ae7d74b9e2/internal/cri/server/events.go (L145))
* Handling an exit includes
[deleting the Task](ae7d74b9e2/internal/cri/server/events.go (L188))
and specifying [`nri.WithContainerExit`](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L348))
to [notify](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L356))
any subscribed NRI plugins
* NRI plugins need to know information about the pod (not just the sandbox),
so before a plugin is notified the NRI API package
[queries the Sandbox Store](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L232))
through the CRI implementation
* The `cri` implementation member field in the `nri.API` struct is set as part of the
[`Register()`](ae7d74b9e2/internal/cri/nri/nri_api_linux.go (L66)) method
* The `nri.Register()` method is only called
[much further down in the CRI `Run()` method](ae71819c4f/pkg/cri/server/service.go (L279))
Signed-off-by: Samuel Karp <samuelkarp@google.com>
Unfortunately, this is a rather large diff, but perhaps worth a one-time
"rip off the bandaid" for v2. This patch removes the use of "gocontext"
as alias for stdLib's "context", and uses "cliContext" for uses of
cli.context.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Debian has started building packages with usernamespaces
to disable network access and similar isolation features. The
containerd package executes a unit test that fails in that
scenario, see https://bugs.debian.org/1070411
The code contains a conditional on whether it is running in
usernamepsace. This commit expands the unit test to cover
this behavior; it was previously untested.
The easiest way to reproduce this issue is to prefix the test
invocaiton with 'unshare -nr go test [...]'
Signed-off-by: Reinhard Tartler <siretart@gmail.com>
Commit 3c8469a782 removed uses of the api
types.Platform type from public interfaces, instead using the type from
the OCI image spec.
For convenience, it also introduced an alias in the platforms package.
While this alias allows packages that already import containerd's
platforms package (now a separate module), it may also cause confusion
(it's not clear that it's an alias for the OCI type), and for packages
that do not depend on containerd's platforms package / module may now
be resulting in an extra dependency.
Let's remove the use of this alias, and instead use the OCI type directly.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
sandbox address should be in the form of
<ttrpc|grpc>+<unix|vsock|hvsock>://<uds-path|vsock-cid:vsock-port|uds-path:hvsock-port>
for example: ttrpc+hvsock:///run/test.hvsock:1024
or: grpc+vsock://1111111:1024
and the Stdin/Stdout/Stderr will add a `streaming_id` as a parameter of the url
result form is:
<ttrpc|grpc>+<unix|vsock|hvsock>://<uds-path|vsock-cid:vsock-port|uds-path:hvsock-port>?streaming_id=<stream-id>
for example ttrpc+hvsock:///run/test.hvsock:1024?streaming_id=111111
or grpc+vsock://1111111:1024?streaming_id=222222
Signed-off-by: Abel Feng <fshb1988@gmail.com>
remote sandbox controller may restart, the Wait call should be retried
if it is an grpc disconnetion error.
Signed-off-by: Abel Feng <fshb1988@gmail.com>
Allow the api to stay at the same v1 go package name and keep using a
1.x version number. This indicates the API is still at 1.x and allows
sharing proto types with containerd 1.6 and 1.7 releases.
Signed-off-by: Derek McGowan <derek@mcg.dev>
This is a non-functional change, that fixes the following typos:
* Snashotter -> Snapshotter
* expectSnapshotter -> expectedSnapshotter
* expectErr -> expectedErr
* exiting-runtime -> existing-runtime
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This includes migrating from cdi.GetRegistry() to cdi.Configure() and
using top-level cdi Refresh and InjectDevices functions as applicable.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
We are currently in the process of developing a feature to facilitate guest image pulling
on confidential-containers, and we would be grateful for containerd's support in this endeavor.
It would greatly assist our efforts if containerd could provide the pause image name and
add it into the annotations.
Fixes: #9418
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Fixes#10013. It seems we can end up in a spot where the sandbox store still
has a listing for a pod, whereas containerds underlying store has removed it.
It might be better to shield the caller (k8s) from these transient errors.
Signed-off-by: Danny Canter <danny@dcantah.dev>
This pacakge is only used internally in the cri package, which is an internal
packages, so we can make the utility internal as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This pacakge is only used internally in the cri package, which is an internal
packages, so we can make the utility internal as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For the first version of containerd's "Forensic Container Checkpointing"
support the error message if the CRIU binary is not found was
deliberately wrong to not break Kubernetes e2e_node tests.
Now that the e2e_node tests have been adapted, containerd can return the
correct error message.
Signed-off-by: Adrian Reber <areber@redhat.com>
This connects the new CRI ContainerCheckpoint RPC to the existing
internal checkpoint functions. With this commit it is possible
to checkpoint a container in Kubernetes using the Forensic Container
Checkpointing KEP (#2008):
# curl X POST "https://localhost:10250/checkpoint/namespace/podId/container"
Which will result in containerd creating a checkpoint in the location
specified by Kubernetes (usually /var/lib/kubelet/checkpoints).
This is a Linux only feature because CRIU only exists on Linux.
Rewritten with the help of Phil Estes.
Signed-off-by: Phil Estes <estesp@gmail.com>
Signed-off-by: Adrian Reber <areber@redhat.com>
Split service proxy from service plugin.
Make introspection service easier for clients to use.
Update service proxy to support grpc and ttrpc.
Signed-off-by: Derek McGowan <derek@mcg.dev>
NRI is still newer and mostly used by CRI plugin. Keep the package in
internal to allow for interfaces as the project matures.
Signed-off-by: Derek McGowan <derek@mcg.dev>
so that we cri service don't have to get sandbox controller everytime it
needs to call sandbox controller api.
Signed-off-by: Abel Feng <fshb1988@gmail.com>
Since kubernetes 1.30, the kubelet will query the runtime handlers
features and only start pods with userns if the runtime handler used for
that pod supports it.
Let's expose the user namespace support to the kubelet.
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
As `setupSandboxFiles` was done in sandbox controller, it is difficult
here to know if the sandbox controller has done and where the host path
in. Make sure the host path exists before adding them to linux container
mounts, otherwise, the container would generate some unnecessary mounts.
Signed-off-by: Zhang Tianyang <burning9699@gmail.com>
See kubernetes/enhancements issue 3857 (PR 3858).
Replaces PR 9713 `cri: make read-only mounts recursively read-only`
Unlike PR 9713, this PR does not automatically upgrade RO mounts to RRO.
Test depends on:
- kubernetes-sigs/cri-tools PR 1344
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Propagate the deprecation list to CRI runtime conditions.
The propagated conditions are visible via `crictl info`,
but not visible via `kubectl get nodes -o yaml` yet, although
the CRI API says "These conditions will be exposed to users to help
them understand the status of the system".
https://github.com/kubernetes/cri-api/blob/v0.29.1/pkg/apis/runtime/v1/api.proto#L1505-L1509
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
We added support for userns but we weren't showing it in the
podSandboxStatus.
Let's just show the whole nsOpts, so we don't forget in the future
either if something else inside there changes.
Please note that this will expose the content of nsOpts.TargetId that we
weren't exposing before. But that seemed like a bug to me.
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
Maybe this is better?
The metadata store is in the best place to handle events directly after
the database has been updated. This prevents every user of the image
store interface from having to know whether or not they are responsible
for publishing events and avoid double events if the grpc local service
is used.
Signed-off-by: Derek McGowan <derek@mcg.dev>
