Commit Graph

13315 Commits

Author SHA1 Message Date
Samuel Karp
423c7ad4fe
Merge pull request #9211 from UiPath/use-loop-configure 2023-10-16 23:40:58 -07:00
Derek McGowan
e4639ad18b
Add exports to proxy plugin config
Allows external plugins to define exports.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-16 21:25:57 -07:00
Kohei Tokunaga
3986f80c35
go.mod: bump up github.com/opencontainers/image-spec to v1.1.0-rc5
Release note: https://github.com/opencontainers/image-spec/releases/tag/v1.1.0-rc5

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-17 10:19:55 +09:00
Derek McGowan
aef2ebc76a
Merge pull request #9250 from thaJeztah/bump_x_net
vendor: golang.org/x/net v0.17.0
2023-10-16 15:42:53 -07:00
Kazuyoshi Kato
14c50204e1
Merge pull request #9247 from thaJeztah/bump_grpc
vendor: google.golang.org/grpc v1.57.1
2023-10-16 14:48:01 -07:00
Sebastiaan van Stijn
f7c9e99422
vendor: golang.org/x/net v0.17.0
full diff: https://github.com/golang/text/compare/v0.13.0...v0.17.0

This fixes the same CVE as go1.21.3 and go1.20.10;

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 21:50:24 +02:00
Sebastiaan van Stijn
c3652540c7
vendor: golang.org/x/text v0.13.0
full diff: https://github.com/golang/text/compare/v0.11.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 21:48:18 +02:00
Sebastiaan van Stijn
ff602c2133
vendor: golang.org/x/sys v0.13.0
full diff: https://github.com/golang/sys/compare/v0.10.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 21:47:15 +02:00
Sebastiaan van Stijn
39b168cdb7
vendor: google.golang.org/grpc v1.57.1
server: prohibit more than MaxConcurrentStreams handlers from running at once
(CVE-2023-44487).

In addition to this change, applications should ensure they do not leave running
tasks behind related to the RPC before returning from method handlers, or should
enforce appropriate limits on any such work.

- https://github.com/grpc/grpc-go/compare/v1.57.0...v1.57.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-16 20:50:18 +02:00
Maksym Pavlenko
eb9ce4fd64
Merge pull request #9246 from GoodDaisy/main
Fix typos
2023-10-16 09:04:54 -07:00
Daisy Rong
930ee552e0 Fix typos
Signed-off-by: Daisy Rong <zrong0405@gmail.com>
2023-10-16 22:14:09 +08:00
Abel Feng
3ef300ca75 sandbox: remove global variable of podsandbox controller
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 21:17:50 +08:00
Abel Feng
7bca70c0c3 sandbox: do not call Connect when loadShim
The ShimManager.Start() will call loadShim() to get the existing shim if SandboxID
is specified for a container, but shimTask.PID() is called in loadShim,
which will call Connect() of Task API with the ID of a task that is not
created yet(containerd is getting the shim and Task API address to call
Create, so the task is not created yet).
In this commit we change the logic of loadShim() to get the shim without calling
Connect() of the not created container ID.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 21:17:50 +08:00
Abel Feng
d2d434b7d6 sandbox: add all sandbox information to Create method
When call sandbox controller to create sandbox, we change the param from
sandbox id to total sandbox object to git all information to controller,
so that sandbox controller do not rely on the sandbox store anymore,
this is more decouple for the sandbox controller plugin inside
containerd, and it is neccesary for remote sandbox controller plugins as
it is not able to get sandbox from the sandbox store anymore.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 21:17:50 +08:00
Abel Feng
0707f68690 sandbox: ignore not found error when remove sandbox
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 21:17:50 +08:00
Abel Feng
2951fb6dc6 sandbox: support more sandbox controllers
make containerd extensible to support more sandbox controllers
registered into containerd by config.
we change the default sandbox controller plugin's name from "local" to "shim".
to make sure we can get the controller by the plugin name it registered into
containerd.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 21:17:44 +08:00
Abel Feng
8b35976850 sandbox: add Sandboxer field to sandbox metadata
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 20:49:36 +08:00
Abel Feng
69e501e7cd sandbox: change SandboxMode to Sandboxer
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 20:49:36 +08:00
Abel Feng
f372b3501b sandbox: add sandboxer field of sandbox requests
As we are going to support more kinds of sandboxers, we have to tell
containerd which sandboxer used to manipulate a specific sandbox.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 20:49:36 +08:00
Alexandru Matei
a782fd6da2 Use LOOP_CONFIGURE when creating loop devices
LOOP_CONFIGURE is a new ioctl that is a lot faster than
the LOOP_SET_FD+LOOP_SET_STATUS64 calls

Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
2023-10-16 13:02:12 +03:00
Samuel Karp
de92d012bc
Merge pull request #9242 from ktock/compress-1.17.1 2023-10-15 22:42:51 -07:00
Samuel Karp
dbe82b7f6e
Merge pull request #9238 from fuweid/deprecated-go-plugin 2023-10-15 22:15:56 -07:00
Samuel Karp
cc9389a3fe
Merge pull request #9215 from qiutongs/docker-schema1-label 2023-10-15 22:07:51 -07:00
Qiutong Song
7712375630 Add a new image label if it is docker schema 1
Signed-off-by: Qiutong Song <songqt01@gmail.com>
2023-10-16 04:11:33 +00:00
Jiang Liu
5ad6f34329 CRI: use (snapshotter_id, snapshot_key) to uniquely identify snapshots
Before snapshotter per runtime, CRI only supports a global snapshotter.
So a snapshot can be uniquely identified by `snapshot_key`. With snapshotter
per runtime enabled, there may be multiple snapshotters used by CRI. So only
(snapshotter_id, snapshot_key) can uniquely identify a snapshot.
Also extends CRI/store/snapshot/Store to support multiple snapshotters.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
2023-10-16 10:21:10 +08:00
Kohei Tokunaga
41b2b2a7b8
go.mod: bump up github.com/klauspost/compress from v1.17.0 to v1.17.1
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-16 10:38:00 +09:00
Wei Fu
4febb08528 deprecated: go-plugin library as runtime plugin
We, containerd, suggest user to use binary plugins or RPC-based plugins.
Since go plugin has too many restrictions, I'm not sure that how many users
use the go plugin to extend the core function in the production.

Based on the fact that we put a lot of effort to make external plugins
better, suggest to deprecate go-plugin type plugin in v2.0 and remove it
in v2.1

REF: https://github.com/containerd/containerd/pull/556

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-16 08:55:07 +08:00
Mike Brown
23573965ff
Merge pull request #9234 from dmcgowan/fix-basic-auth-credential-error
remotes: add handling for missing basic auth credentials
2023-10-13 16:41:22 -05:00
Derek McGowan
51c9ffe468
remotes: add handling for missing basic auth credentials
When a credential handler is provided but no basic auth credentials
are provided, handle the error specifically rather than treating
the credentials as not implemented. This allows a clearer error to
be provided to users rather than a confusing not implemented error
or generic unauthorized error.

Add unit tests for the basic auth case.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-13 12:42:19 -07:00
Maksym Pavlenko
c49e6a7a1b
Merge pull request #9200 from kzys/igzip
Use Intel ISA-L's igzip if available
2023-10-13 11:13:01 -07:00
Derek McGowan
15bf23df09
Merge pull request #9231 from fuweid/update-release.md
RELEASES.md: mark legacy CRI as removed status
2023-10-13 05:59:53 -07:00
Samuel Karp
8ed0255f1b
Merge pull request #9205 from lengrongfu/feat/add-cpu-burst 2023-10-12 23:55:00 -07:00
Wei Fu
4f339b5b58 RELEASES.md: mark legacy CRI as removed status
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-13 14:21:24 +08:00
Akihiro Suda
9fcc3beb59
Merge pull request #9230 from mxpv/exit
Exit shim when shutdown manager is done
2023-10-13 14:51:20 +09:00
Akihiro Suda
0cd7bdd4a5
Merge pull request #9214 from dmcgowan/generalize-plugin-library
Generalize plugin library
2023-10-13 14:50:38 +09:00
Derek McGowan
0a4f792117
Remove use of v1 runtime plugin type for Unix
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 21:22:33 -07:00
Derek McGowan
71f8b4357e
Move dynamic plugins to a subpackage
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 21:22:32 -07:00
Derek McGowan
7b2a918213
Generalize the plugin package
Remove containerd specific parts of the plugin package to prepare its
move out of the main repository. Separate the plugin registration
singleton into a separate package.

Separating out the plugin package and registration makes it easier to
implement external plugins without creating a dependency loop.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 21:22:32 -07:00
Derek McGowan
a80606bc2d
Move plugin type definitions to containerd plugins package
The plugins packages defines the plugins used by containerd.
Move all the types and properties to this package.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 20:52:56 -07:00
Derek McGowan
cb969085f5
Temporarily remove zfs as built in plugin
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 20:39:22 -07:00
Maksym Pavlenko
2486c12987 Exit shim when shutdown manager is done
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-10-12 19:53:05 -07:00
Akihiro Suda
a7333657af
Merge pull request #9228 from mxpv/cri
🪦 RIP legacy CRI
2023-10-13 08:43:41 +09:00
Akihiro Suda
9c367d09f1
Merge pull request #9223 from dmcgowan/config-migration-v1
Introduce top level config migration
2023-10-13 08:43:13 +09:00
Maksym Pavlenko
1b31993240 Rename sbserver to server
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-10-12 15:46:57 -07:00
Derek McGowan
4edc6c2d99
Add note about configuration versions in releases
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 14:52:44 -07:00
Maksym Pavlenko
c5c94dc86c Fix CRI tests panic on CI
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-10-12 14:35:27 -07:00
Maksym Pavlenko
a3c2c1e4da Clean shell scripts
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-10-12 10:30:46 -07:00
Maksym Pavlenko
fa1d3a9ccb Fix dependencies after remove
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-10-12 10:30:46 -07:00
Wei Fu
3742f7f0db idmapped: use pidfd to avoid pid reuse issue
It's followup for #5890.

The containerd-shim process depends on the mount package to init rootfs
for container. For the container enable user namespace, the mount
package needs to fork child process to get the brand-new user namespace.
However, there are two reapers in one process (described by the
following list) and there are race-condition cases.

1. mount package
2. sys.Reaper as global one which watch all the SIGCHLD.

=== [kill(2)][kill] the wrong process ===

Currently, we use pipe to ensure that child process is alive. However,
the pide file descriptor can be hold by other process, which the child
process cannot exit by self. We should use [kill(2)][kill] to ensure the
child process. But we might kill the wrong process if the child process
might be reaped by containerd-shim and the PID might be reused by other
process.

=== [waitid(2)][waitid] on the wrong child process ===

```
containerd-shim process:

Goroutine 1(GetUsernsFD):                   Goroutine 2(Reaper)

1. Ready to wait for child process X

                                            2. Received SIGCHLD from X

                                            3. Reaped the zombie child process X

                                            (X has been reused by other child process)

4. Wait on process X

The goroutine 1 will be stuck until the process X has been terminated.
```

=== open `/proc/X/ns/user` on the wrong child process ===

There is also pid-reused risk between opening `/proc/$pid/ns/user` and
writing `/proc/$pid/u[g]id_map`.

```
containerd-shim process:

Goroutine 1(GetUsernsFD):                   Goroutine 2(Reaper)

1. Fork child process X

2. Write /proc/X/uid_map,gid_map

                                            3. Received SIGCHLD from X

                                            4. Reaped the zombie child process X

                                            (X has been reused by other process)

5. Open /proc/X/ns/user file as usernsFD

The usernsFD links to the wrong X!!!
```

In order to fix the race-condition, we should use [CLONE_PIDFD][clone2] (Since
Linux v5.2).

When we fork child process `X`, the kernel will return a process file
descriptor `X_PIDFD` referencing to child process `X`. With the pidfd, we can
use [pidfd_send_signal(2)][pidfd_send_signal] (Since Linux v5.1)
to send signal(0) to ensure the child process `X` is alive. If the `X` has
terminated and its PID has been recycled for another process. The
pidfd_send_signal fails with the error ESRCH.

Therefore, we can open `/proc/X/{ns/user,uid_map,gid_map}` file
descriptors as first and then use pidfd_send_signal to check the process
is still alive. If so, we can ensure the file descriptors are valid and
reference to the child process `X`. Even if the `X` PID has been reused
after pidfd_send_signal call, the file descriptors are still valid.

```code
X, pidfd = clone2(CLONE_PIDFD)

usernsFD = open /proc/X/ns/user
uidmapFD = open /proc/X/uid_map
gidmapFD = open /proc/X/gid_map

pidfd_send_signal pidfd, signal(0)
  return err if no such process

== When we arrive here, we can ensure usernsFD/uidmapFD/gidmapFD are correct
== even if X has been reused after pidfd_send_signal call.

update uid/gid mapping by uidmapFD/gidmapFD
return usernsFD
```

And the [waitid(2)][waitid] also supports pidfd type (Since Linux 5.4).
We can use pidfd type waitid to ensure we are waiting for the correct
process. All the PID related race-condition issues can be resolved by
pidfd.

```bash
➜  mount git:(followup-idmapped) pwd
/home/fuwei/go/src/github.com/containerd/containerd/mount

➜  mount git:(followup-idmapped) sudo go test -test.root -run TestGetUsernsFD -count=1000 -failfast -p 100  ./...
PASS
ok      github.com/containerd/containerd/mount  3.446s
```

[kill]: <https://man7.org/linux/man-pages/man2/kill.2.html>
[clone2]: <https://man7.org/linux/man-pages/man2/clone.2.html>
[pidfd_send_signal]: <https://man7.org/linux/man-pages/man2/pidfd_send_signal.2.html>
[waitid]: <https://man7.org/linux/man-pages/man2/waitid.2.html>

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-13 00:56:55 +08:00
Maksym Pavlenko
2332b9ba10 Remove legacy CRI from CI
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-10-12 09:18:55 -07:00