Commit Graph

14496 Commits

Author SHA1 Message Date
Sebastiaan van Stijn
734219a21a
update to go1.21.5, go1.20.12
go1.21.5 (released 2023-12-05) includes security fixes to the go command,
and the net/http and path/filepath packages, as well as bug fixes to the
compiler, the go command, the runtime, and the crypto/rand, net, os, and
syscall packages. See the Go 1.21.5 milestone on our issue tracker for
details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.5+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.4...go1.21.5

from the security mailing:

[security] Go 1.21.5 and Go 1.20.12 are released

Hello gophers,

We have just released Go versions 1.21.5 and 1.20.12, minor point releases.

These minor releases include 3 security fixes following the security policy:

- net/http: limit chunked data overhead

  A malicious HTTP sender can use chunk extensions to cause a receiver
  reading from a request or response body to read many more bytes from
  the network than are in the body.

  A malicious HTTP client can further exploit this to cause a server to
  automatically read a large amount of data (up to about 1GiB) when a
  handler fails to read the entire body of a request.

  Chunk extensions are a little-used HTTP feature which permit including
  additional metadata in a request or response body sent using the chunked
  encoding. The net/http chunked encoding reader discards this metadata.
  A sender can exploit this by inserting a large metadata segment with
  each byte transferred. The chunk reader now produces an error if the
  ratio of real body to encoded bytes grows too small.

  Thanks to Bartek Nowotarski for reporting this issue.

  This is CVE-2023-39326 and Go issue https://go.dev/issue/64433.

- cmd/go: go get may unexpectedly fallback to insecure git

  Using go get to fetch a module with the ".git" suffix may unexpectedly
  fallback to the insecure "git://" protocol if the module is unavailable
  via the secure "https://" and "git+ssh://" protocols, even if GOINSECURE
  is not set for said module. This only affects users who are not using
  the module proxy and are fetching modules directly (i.e. GOPROXY=off).

  Thanks to David Leadbeater for reporting this issue.

  This is CVE-2023-45285 and Go issue https://go.dev/issue/63845.

- path/filepath: retain trailing \ when cleaning paths like \\?\c:\

  Go 1.20.11 and Go 1.21.4 inadvertently changed the definition of the
  volume name in Windows paths starting with \\?\, resulting in
  filepath.Clean(\\?\c:\) returning \\?\c: rather than \\?\c:\ (among
  other effects). The previous behavior has been restored.

  This is an update to CVE-2023-45283 and Go issue https://go.dev/issue/64028.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:34:44 +01:00
Sebastiaan van Stijn
2122004525
update to go1.21.4, go1.20.11
go1.21.4 (released 2023-11-07) includes security fixes to the path/filepath
package, as well as bug fixes to the linker, the runtime, the compiler, and
the go/types, net/http, and runtime/cgo packages. See the Go 1.21.4 milestone
on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.21.4+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.21.3...go1.21.4

from the security mailing:

[security] Go 1.21.4 and Go 1.20.11 are released

Hello gophers,

We have just released Go versions 1.21.4 and 1.20.11, minor point releases.

These minor releases include 2 security fixes following the security policy:

- path/filepath: recognize `\??\` as a Root Local Device path prefix.

  On Windows, a path beginning with `\??\` is a Root Local Device path equivalent
  to a path beginning with `\\?\`. Paths with a `\??\` prefix may be used to
  access arbitrary locations on the system. For example, the path `\??\c:\x`
  is equivalent to the more common path c:\x.

  The filepath package did not recognize paths with a `\??\` prefix as special.

  Clean could convert a rooted path such as `\a\..\??\b` into
  the root local device path `\??\b`. It will now convert this
  path into `.\??\b`.

  `IsAbs` did not report paths beginning with `\??\` as absolute.
  It now does so.

  VolumeName now reports the `\??\` prefix as a volume name.

  `Join(`\`, `??`, `b`)` could convert a seemingly innocent
  sequence of path elements into the root local device path
  `\??\b`. It will now convert this to `\.\??\b`.

  This is CVE-2023-45283 and https://go.dev/issue/63713.

- path/filepath: recognize device names with trailing spaces and superscripts

  The `IsLocal` function did not correctly detect reserved names in some cases:

  - reserved names followed by spaces, such as "COM1 ".
  - "COM" or "LPT" followed by a superscript 1, 2, or 3.

  `IsLocal` now correctly reports these names as non-local.

  This is CVE-2023-45284 and https://go.dev/issue/63713.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-12-05 23:31:28 +01:00
Samuel Karp
d55bfab176
Merge pull request #9465 from ambarve/sn_cleanup
Don't block snapshot garbage collection on Remove failures
2023-12-05 21:33:43 +00:00
Fu Wei
9a7130d132
Merge pull request #9461 from kinvolk/rata/idmap
pkg/process: Check using idmap mount options too
2023-12-05 15:43:14 +00:00
Amit Barve
ad96fded4c Don't block snapshot garbage collection on Remove failures
If a snapshot removal fails (during garbage collection), the entire garbage collection operation is
cancelled. This is problematic because once cleanup of any snapshot fails no other snapshots will be cleaned
and the disk usage will just keep increasing.
Solution is to return snapshot removal errors wrapped as "ErrFailedPrecondition" errors. The garbage
collectors continues cleanup if the error is of this type.

Signed-off-by: Amit Barve <ambarve@microsoft.com>
2023-12-04 15:00:00 -08:00
Fu Wei
87bf39a7f5
Merge pull request #9445 from dmcgowan/update-contributing-root-packages
Add core and internal root packages to contributing
2023-12-04 18:22:14 +00:00
Rodrigo Campos
8bbce8d37f pkg/process: Check using idmap mount options too
The runtime-spec just merged this PR:
	https://github.com/opencontainers/runtime-spec/pull/1224

This means that it is now possible to request idmap mounts by specifying
"idmap" or "ridmap" in the mount options, without any mappings.

Let's add a check to see if they are requested in that way too.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
2023-12-04 15:17:59 +01:00
Maksym Pavlenko
47163c3c00
Merge pull request #9391 from abel-von/sandbox-plugin-1117
sandbox: Move CRI Image Service and CRI Base seperate plugins
2023-12-01 22:42:22 +00:00
Maksym Pavlenko
40ec4b9bce
Merge pull request #9410 from ktock/continuity-0.4.3
go.mod: github.com/containerd/continuity v0.4.3
2023-12-01 22:41:30 +00:00
Maksym Pavlenko
63609d33ca
Merge pull request #9434 from abel-von/add-integration-test
sandbox: add cri integration test case for upgrade
2023-12-01 22:40:35 +00:00
Maksym Pavlenko
e2303c267e
Merge pull request #9414 from ZhangShuaiyi/fix/tomlext_MarshalText
tomlext.Duration add MarshalText method
2023-12-01 22:36:53 +00:00
Maksym Pavlenko
a1a8a666fd
Merge pull request #9430 from ktock/fsnotify-1.7.0
Bump up github.com/fsnotify/fsnotify to v1.7.0
2023-12-01 22:35:01 +00:00
Maksym Pavlenko
6f405e89f8
Merge pull request #9447 from dcantah/dial-grpc-shim-socket
runtime/v2: net.Dial gRPC shim sockets before trying grpc
2023-12-01 22:32:41 +00:00
Derek McGowan
96aaf5a3b2
Add core and internal root packages
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-12-01 11:26:25 -08:00
Abel Feng
c0363754fb sandbox: get runtime info from sandbox or container
For backward compatibility, we should get runtimeInfo from sandbox in
db, or get it from the sandbox container in db.
Note that this is a temporary solution and we will remove the Container field in
Sandbox in cri cache, and replace it with a SandboxInsantance of type
containerd.Sandbox interface.

Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-12-01 15:58:27 +08:00
Abel Feng
95d2a3b7c0 upgrade: add cri upgrade test case
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-12-01 15:58:16 +08:00
Danny Canter
0bc9633414 runtime/v2: net.Dial gRPC shim sockets before trying grpc
This is mostly to workaround an issue with gRPC based shims after containerd
restart. If a shim dies while containerd is also down/restarting, on reboot
grpc.DialContext with our current set of DialOptions will make us wait for
100 seconds per shim even if the socket no longer exists or has no listener.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2023-11-30 19:37:43 -08:00
Kazuyoshi Kato
8459273f80
Merge pull request #9444 from AkihiroSuda/move-pkg-process
mv pkg/process cmd/containerd-shim-runc-v2/process
2023-12-01 00:08:34 +00:00
Derek McGowan
b8e32595ba
Merge pull request #8166 from laurazard/gzip-gstd-compression
Support gzip and zstd HTTP transport compression to fetch remote resources
2023-11-30 18:05:55 +00:00
Abel Feng
e1b4958663 sandbox: keep rootDir and stateDir compatible
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-11-30 23:06:53 +08:00
Abel Feng
b0fef6738f sandbox: migrate sandbox_mode to sandboxer
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-11-30 23:06:53 +08:00
Abel Feng
c8012b6d74 sandbox: make a clear dependency of cri plugins
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-11-30 23:06:41 +08:00
Laura Brehm
2ead895094
Support gzip and zstd HTTP transport compression to fetch remote resources
Co-authored-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-11-30 13:51:33 +00:00
Akihiro Suda
8e567aa581
mv pkg/process cmd/containerd-shim-runc-v2/process
The package is quite specific to runc and only imported by
containerd-shim-runc-v2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-11-30 21:50:04 +09:00
Derek McGowan
58af8cda54
Merge pull request #9422 from fuweid/upgrade-exec
integration: add new binary upgrade case
2023-11-29 22:37:51 +00:00
Derek McGowan
066ab793f2
Merge pull request #9428 from yibozhuang/containerd-stress-cri-endpoint
containerd-stress: use config address for CRI test
2023-11-28 22:01:55 +00:00
Kohei Tokunaga
a813097dcc
Bump up github.com/fsnotify/fsnotify to v1.7.0
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-11-28 17:07:23 +09:00
Yibo Zhuang
c384e37177 containerd-stress: use config address for CRI test
This change removes the hard-coded containerd endpoint
for CRI test and use the address in the config which would
honor the CLI flag.

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2023-11-27 17:51:02 -08:00
Wei Fu
d4c828c2ca integration: add new binary upgrade case
Add Exec-to-Existing-Running-Container case.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-11-26 22:38:02 +08:00
Samuel Karp
787943dc10
Merge pull request #9412 from profnandaa/fix/dialer_function_windows
pkg/dialer: minor fix on dialer function for windows
2023-11-23 00:38:51 +00:00
Anthony Nandaa
8e91edb71e fix(pkg/dialer): minor fix on dialer function for windows
This commit fixes the dialer function to make sure that
"npipe://" prefix is trimmed, just like the way it is done
in the Unix counterpart, `./dialer_unix.go:50`

This will also unblock some downstream work going on in
buildkit; setting up integration tests to run on Windows.

Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
2023-11-22 04:25:11 -08:00
Shuaiyi Zhang
ad3f8c563b tomlext.Duration add MarshalText method
Signed-off-by: Shuaiyi Zhang <zhang_syi@qq.com>
2023-11-22 19:28:46 +08:00
Samuel Karp
6ed8f01aa1
Merge pull request #9384 from rokkiter/fix-ticker-leak
fix ticker leak
2023-11-22 05:41:57 +00:00
Derek McGowan
d682b16f9c
Merge pull request #9280 from vvoland/push-inherit-distribution-source
push: always inherit distribution sources from parent
2023-11-22 05:31:35 +00:00
Akihiro Suda
ab63218858
Merge pull request #9407 from ktock/golangci-lint-1.55.2
Bump up golangci-lint to v1.55.2 and tiny fix on .golangci.yml
2023-11-22 04:12:03 +00:00
Kohei Tokunaga
6da015284c
go.mod: github.com/containerd/continuity v0.4.3
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-11-22 12:26:35 +09:00
Kohei Tokunaga
cd348e6ef9
.golangci.yml: remove directories that don't exist
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-11-22 00:14:35 +09:00
Kohei Tokunaga
4b556a6e01
Bump up golangci-lint to v1.55.2
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-11-22 00:13:57 +09:00
Samuel Karp
f12e84b5c5
Merge pull request #9369 from fuweid/fix-pull-progress
fix: ImagePull should close http connection if there is no available data to read.
2023-11-20 19:02:21 +00:00
Paweł Gronowski
c51d4f9b69
push: always inherit distribution sources from parent
Propagate parent distribution source labels to each of its children even
if they're not missing. This allows to cross-repo mount blobs when the
child content has different distribution source label from its
parent manifest/index. This could happen when different parts of image
were fetched from different sources.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-11-20 15:06:16 +01:00
Maksym Pavlenko
e15c246550 Move CRI image service into a separate plugin
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-11-20 09:41:27 +08:00
Wei Fu
80dd779deb remotes/docker: close connection if no more data
Close connection if no more data. It's to fix false alert filed by image
pull progress.

```
dst = OpenWriter (--> Content Store)

src = Fetch
        Open (--> Registry)
        Mark it as active request

Copy(dst, src) (--> Keep updating total received bytes)

   ^
   |  (Active Request > 0, but total received bytes won't be updated)
   v

defer src.Close()
content.Commit(dst)
```

Before migrating to transfer service, CRI plugin doesn't limit global
concurrent downloads for ImagePulls. Each ImagePull requests have 3 concurrent
goroutines to download blob and 1 goroutine to unpack blob. Like ext4
filesystem [1][1], the fsync from content.Commit may sync unrelated dirty pages
into disk. The host is running under IO pressure, and then the content.Commit
will take long time and block other goroutines. If httpreadseeker
doesn't close the connection after io.EOF, this connection will be
considered as active. The pull progress reporter reports there is no
bytes transfered and cancels the ImagePull.

The original 1-minute timeout[2][2] is from kubelet settting. Since CRI-plugin
can't limit the total concurrent downloads, this patch is to update 1-minute
to 5-minutes to prevent from unexpected cancel.

[1]: https://lwn.net/Articles/842385/
[2]: https://github.com/kubernetes/kubernetes/blob/release-1.23/pkg/kubelet/config/flags.go#L45-L48

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-11-18 10:23:05 +08:00
Wei Fu
7f410ae05a integration: reproduce #9347
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-11-18 10:23:05 +08:00
Wei Fu
2e9686c054 fix: deflake TestCRIImagePullTimeout/HoldingContentOpenWriter
The new active request is filed and there is no bytes read yet when the
progress reporter just wakes up. If the timeout / 2 is less than the
minPullProgressReportInternal, it's easy to file false alert.

We should remove the minPullProgressReportInternal limit.

Fixes: #8024

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-11-18 10:23:05 +08:00
Samuel Karp
349c8d12c8
Merge pull request #9393 from AkihiroSuda/ci-fedora39
CI: update Fedora to 39
2023-11-17 21:24:47 +00:00
Akihiro Suda
4aa932337c
CI: update Fedora to 39
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-11-18 05:28:09 +09:00
Derek McGowan
01f3937248
Merge pull request #9392 from fuweid/fix-ci
fix: podsandbox depends on Lease plugin
2023-11-17 18:00:01 +00:00
Wei Fu
f6c4de6b53 fix: podsandbox depends on Lease plugin
introduced by 09723a6175

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-11-17 23:29:04 +08:00
rokkiter
1b65fe2646 fix ticker leak
Signed-off-by: rokkiter <yongen.pan@daocloud.io>
2023-11-17 10:21:57 +08:00
Phil Estes
7ea66e3af7
Merge pull request #9376 from austinvazquez/update-getting-started-for-v2
Update documentation for containerd v2.0 packages
2023-11-16 23:46:46 +00:00