Commit Graph

1860 Commits

Author SHA1 Message Date
Craig Ingram
07a0b5419c (cherry picked from commit de1341c201ffb0effebbf51d00376181968c8779) 2025-03-11 14:52:44 +00:00
Lei Liu
0ce93e16a9 prevent oom watcher depend on shim pkg.
Signed-off-by: Lei Liu <liulei.pt@bytedance.com>
2025-02-26 01:46:25 +00:00
Derek McGowan
f58939c33d
Remove deprecated WithCDIDevices in oci spec opts
This function has been moved to prevent an unintended dependency on CDI.

Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit bdc847f1eb535a6728b6db3f2619d2a5ed0edbb9)
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-17 08:58:34 -08:00
Derek McGowan
3d53430fe1
Move CDI device spec out of the OCI package
The CDI device injection spec opt was mistakenly added to the OCI
package which brought in an unintended dependency on CDI and its
transitive dependencies.

Signed-off-by: Derek McGowan <derek@mcg.dev>
(cherry picked from commit e20f7f4a2425c005d85855abfd4556d7b4ccbf87)
Signed-off-by: Derek McGowan <derek@mcg.dev>
2025-01-14 14:16:18 -08:00
Derek McGowan
73f57acb0d Update differ to handle zstd media types
The differ should be able to generate zstd compressed layers when
provided with the zstd media type.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-11-27 16:35:58 +00:00
Akihiro Suda
588b7a100a
testutil: avoid conflict with continuity/testutil
```
=== FAIL: core/metadata  (0.00s)
/tmp/go-build142702247/b1061/metadata.test flag redefined: test.root
panic: /tmp/go-build142702247/b1061/metadata.test flag redefined: test.root

goroutine 1 [running]:
flag.(*FlagSet).Var(0xc0000d6000, {0x132a040, 0x190f3f3}, {0x11b4291, 0x9}, {0x11c7690, 0x1e})
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:1028 +0x54d
flag.BoolVar(...)
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:755
github.com/containerd/containerd/v2/pkg/testutil.init.0()
	/home/runner/work/containerd/containerd/pkg/testutil/helpers.go:36 +0x6b
FAIL	github.com/containerd/containerd/v2/core/metadata	0.028s
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-10-28 11:58:28 +09:00
Sebastiaan van Stijn
fddeb6f3cb
pkg/protobuf: fix typo in godoc
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-26 14:18:53 +02:00
Mike Baynton
347423a114 Request 'allow' setgroups when spawning new userns
Signed-off-by: Mike Baynton <mike@mbaynton.com>
2024-10-17 15:37:36 -05:00
Brian Goff
b85909cd4c shim: Move pprof server to plugin
Makes the pprof server a plugin and also gates by the `shim_tracing`
build tag (like otel is).
With this change, `net/http` is no longer a dependency in the shim.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-03 00:12:50 +00:00
Brian Goff
b2681dfbdb shim: Move ttrpc interceptors to plugins
This makes it so we don't need to import otelttrpc unless the shim is
compiled with the `shim_tracing` build tag.
This way otel is no longer compiled into the binary at all unless
`shim_tracing` is set.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-10-03 00:12:47 +00:00
Brian Goff
17d4a1357e Propagate trace contexts to shims
This adds trace context propagation over the grpc/ttrpc calls to a shim.

It also adds the otlp plugin to the runc shim so that it will send
traces to the configured tracer (which is inherited from containerd's
config).
It doesn't look like this is adding any real overhead to the runc shim's
memory usage, however it does add 2MB to the binary size.
As such this is gated by a build tag `shim_tracing`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-09-30 21:44:16 +00:00
Akihiro Suda
8c64a2f6a1
Merge pull request #10607 from fuweid/pin-userns
internal/cri: simplify netns setup with pinned userns
2024-09-19 01:05:41 +00:00
Wei Fu
fd3f3d5a13 pkg/sys: add GetUsernsForNamespace interface
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-09-11 07:21:43 +08:00
Wei Fu
490e45a08a pkg/sys: Add UnshareAfterEnterUserns function
It allows to disassociate parts of its execution context within a user
namespace.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-09-11 07:21:39 +08:00
Samuel Karp
5f37a2c205
fifosync: cross-process synchronization
Signed-off-by: Samuel Karp <samuelkarp@google.com>
2024-09-05 12:48:28 -07:00
Akihiro Suda
cae19b14f3
Merge pull request #10658 from darwin-containers/reorganize-mount-unmount
Reorganize mount/unmount code so it is easier to add Darwin-specific implementation
2024-09-03 01:51:24 +00:00
Akihiro Suda
26d6fd0c3f
Merge pull request #10611 from fuweid/getridof-hall-of-shame
core/mount: use ptrace instead of go:linkname
2024-08-30 17:07:33 +00:00
Marat Radchenko
bfc1465a2c Reorganize mount/unmount code so it is easier to add Darwin-specific implementation
After these changes, in order to add Darwin bind-mount implementation, one only needs:
* Adjust HasBindMounts definition in mount.go
* Provide implementation in mount_darwin.go

There was no consensus on adding dependency on bindfs, that seems to be the only working solution for bind-mounts on Darwin as of today, in https://github.com/containerd/containerd/pull/8789, that's why the actual implementation is not added in current PR.

As a bonus, Linux FUSE-related code was moved to a separate file and possibly could be reused on FreeBSD, though this needs testing.

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
2024-08-30 15:25:06 +03:00
Jin Dong
d83184ced1 avoid repeated calls in Tricolor gc
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-08-26 23:35:24 -07:00
Jin Dong
6b97a08eee add benchmark
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2024-08-26 23:35:04 -07:00
Wei Fu
3cd8f9734d core/mount: use ptrace instead of go:linkname
The Go runtime has started to [lock down future uses of linkname][1] since
go1.23. In the go source code, containerd project has been marked in the
comment, [hall of shame][2]. Well, the go:linkname is used to fork no-op
subprocess efficiently. However, since that comment, I would like to use
ptrace and remove go:linkname in the whole repository.

With go1.22 `go:linkname`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 2440            533320 ns/op            1145 B/op         43 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 342           3661616 ns/op           11562 B/op        421 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  2.983s
```

With go1.22 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1785            739557 ns/op            3948 B/op         68 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 328           4024300 ns/op           39601 B/op        671 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.104s
```

With go1.23 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1815            723252 ns/op            4220 B/op         69 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 319           3957157 ns/op           42351 B/op        682 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.051s
```

Diff:

The `ptrace` is slower than `go:linkname` mode. However, it's accepctable.

```
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
                                    │ go122-golinkname │             go122-ptrace              │             go123-ptrace              │
                                    │      sec/op      │    sec/op     vs base                 │    sec/op     vs base                 │
BatchRunGetUsernsFD_Concurrent1-16        533.3µ ± ∞ ¹   739.6µ ± ∞ ¹        ~ (p=1.000 n=1) ²   723.3µ ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16       3.662m ± ∞ ¹   4.024m ± ∞ ¹        ~ (p=1.000 n=1) ²   3.957m ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                   1.397m         1.725m        +23.45%                   1.692m        +21.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │              go122-ptrace               │              go123-ptrace               │
                                    │       B/op       │     B/op       vs base                  │     B/op       vs base                  │
BatchRunGetUsernsFD_Concurrent1-16       1.118Ki ± ∞ ¹   3.855Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   4.121Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16      11.29Ki ± ∞ ¹   38.67Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   41.36Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
geomean                                  3.553Ki         12.21Ki        +243.65%                   13.06Ki        +267.43%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │             go122-ptrace             │             go123-ptrace             │
                                    │    allocs/op     │  allocs/op   vs base                 │  allocs/op   vs base                 │
BatchRunGetUsernsFD_Concurrent1-16         43.00 ± ∞ ¹   68.00 ± ∞ ¹        ~ (p=1.000 n=1) ²   69.00 ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16        421.0 ± ∞ ¹   671.0 ± ∞ ¹        ~ (p=1.000 n=1) ²   682.0 ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                    134.5         213.6        +58.76%                   216.9        +61.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
```

[1]: <https://github.com/golang/go/issues/67401>
[2]: <https://github.com/golang/go/blob/release-branch.go1.23/src/runtime/proc.go#L4820>

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-26 21:19:50 +08:00
Akhil Mohan
ebc47359ea
use format string when using printf like commands
As per https://github.com/golang/go/issues/60529, printf like commands with
non-constant format strings and no args give an error in govet

Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
2024-08-14 17:04:53 +05:30
Maksym Pavlenko
0b02e0c225
Merge pull request #7616 from swagatbora90/trace-cri-runtime
Add tracing spans to CRI runtime service apis
2024-08-09 18:24:47 +00:00
Sebastiaan van Stijn
9776047243
migrate to github.com/moby/sys/userns
Commit 8437c567d8 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-08-08 12:48:54 +02:00
Swagat Bora
83c26962c2 Remove extra span created in the instrumented service layer
Signed-off-by: Swagat Bora <sbora@amazon.com>
2024-08-05 02:34:27 +00:00
Swagat Bora
c0cdcb34f1 Add spans to CRI runtime service and related client methods
This adds otel spans to CRI service mainly targeting mutating apis which includes:
* Sandbox apis - RunPodSandbox, StopPodSandbox, RemovePodSandbox
* Container apis - CreateContainer, StartContainer, StopContainer, RemoveContainer
* Attach, Exec and Exec Sync
* Containerd client methods: container.go, client.go, process.go and task.go

Signed-off-by: Swagat Bora <sbora@amazon.com>
2024-08-05 02:34:20 +00:00
Kazuyoshi Kato
2ddd3db952
Merge pull request #9858 from w13915984028/fixdebug
Add file name to device type check failure message
2024-07-26 21:20:38 +00:00
Akihiro Suda
7a1f6b2891
Merge pull request #10509 from samuelkarp/pr-9966-follow-up
deprecation: update warnings for CRI config fields
2024-07-26 17:40:59 +00:00
Sebastiaan van Stijn
eddd90d7c1
remove deprecated pkg/userns
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-26 09:48:40 +02:00
Sebastiaan van Stijn
8437c567d8
pkg/userns: deprecate and migrate to github.com/moby/sys/user/userns
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].

This patch deprecates the containerd fork of that package, and adds it as
an alias for the moby/sys/user/userns package.

[1]: 3778ae603c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-26 09:47:50 +02:00
Samuel Karp
f9146c0516
deprecation: update warnings for CRI config fields
Some CRI config properties had removal postponed until v2.1 in
https://github.com/containerd/containerd/pull/9966. Update the
associated deprecation warnings to match the new removal version.

Signed-off-by: Samuel Karp <samuelkarp@google.com>
2024-07-25 14:25:39 -07:00
Danny Canter
b41bb6df73 Avoid potential reallocs by pre-sizing some slices
There's a couple spots where we know exactly how large
the destination buffer should be, so pre-size these to
avoid any reallocs to a higher capacity.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2024-07-19 13:05:49 -07:00
Colin O'Dell
c8254dfae9
Include filename in error
Signed-off-by: Colin O'Dell <colinodell@gmail.com>
2024-07-16 12:14:36 -04:00
Samuel Karp
0262714edb
Merge pull request #10412 from TinaMor/tinamor/dev
[Windows] Set stderr to empty string when using terminal on Windows
2024-07-15 16:29:08 +00:00
Akhil Mohan
300fd770a0
use typeurl funcs for marshalling anypb.Any
Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
2024-07-10 22:26:27 +05:30
TinaMor
eaa1afe634 Set stderr to empty string when using terminal on Windows.
Windows HCSShim requires that stderr is an empty string when using terminal.
	Reference: 200feabd85/cmd/containerd-shim-runhcs-v1/service_internal.go (L127)

Signed-off-by: Christine Murimi <mor.tina@outlook.com>
2024-07-08 09:21:20 +03:00
Derek McGowan
23170e20df
Move fsverity package to internal
This functionality is not directly related to containerd and could move
to external package at some point.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-07-01 09:21:24 -07:00
Akihiro Suda
7103e381f1
Merge pull request #10007 from Jenkins-J/fsverity-content-verification
Fsverity content verification
2024-06-29 06:32:17 +00:00
Sebastiaan van Stijn
a723c0c6ea
pkg/reference: remove deprecated SplitObject
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-28 19:06:30 +02:00
James Jenkins
ef98c71985 Implement fsverity functionality
Implement calls to the fsverity kernel module, allowing containerd to
enable fsverity on blob data in the content store. This causes fsverity
to veirfy the integrity of blob data when the blob is read.

Signed-off-by: James Jenkins <James.Jenkins@ibm.com>
2024-06-28 09:50:44 -04:00
Sebastiaan van Stijn
fec33aa735
pkg/reference: deprecate SplitObject
The behavior of this function is quite counter-intuitive, as it preserves
the delimiter in the result, and its use for external consumers would be
very limited.

Spec.Digest no longer uses this function, and it appears that BuildKit is
currently the only (publicly visible) external consumer of it.

This patch deprecates the function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 22:11:42 +02:00
Sebastiaan van Stijn
a5fce38f31
pkg/reference: Spec.Digest(): inline SplitObject code
Inline the relevant code from SplitObject, as we're only interested
in the digest portion.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 12:05:23 +02:00
Sebastiaan van Stijn
42145950bb
pkg/reference: SplitObject: add proper GoDoc
The behavior of this function is quite counter-intuitive, as it preserves
the delimiter in the result. This function should probably have been an
internal function, as its use for external consumers would be very limited,
but let's at least document the (surprising) behavior for those that are
considering to use it.

It appears that BuildKit is currently the only (publicly visible) external
consumer of this function; I am planning to inline its functionality in
Spec.Digest() and to deprecate this function so that it can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 12:05:23 +02:00
Sebastiaan van Stijn
74a6156ac2
pkg/reference: SplitObject: zero allocations
Before / After:

    BenchmarkSplitObject-10        2785656    428.1 ns/op     416 B/op    13 allocs/op
    BenchmarkSplitObjectNew-10    13510520     88.2 ns/op       0 B/op     0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 12:05:10 +02:00
Sebastiaan van Stijn
799bca97f2
pkg/reference: Spec.String(): use string-concatenation instead of sprintf
These were straight concatenations of strings; reduce some allocations by
removing fmt.Sprintf for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-27 12:04:22 +02:00
Henry Wang
243b803a19 Add pprof to runc-shim
Signed-off-by: Henry Wang <henwang@amazon.com>
2024-06-20 23:12:31 +00:00
Sebastiaan van Stijn
75fd7a5a7f
pkg/shim: remove logrus imports
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-17 12:35:36 +02:00
Sebastiaan van Stijn
587ee80f61
pkg/tracing: LogrusHook.Fire: micro-optimisation
Check span.IsRecording first, as it's a more lightweight check than
span.SpanContext().IsValid()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-14 12:12:22 +02:00
Sebastiaan van Stijn
ccf7938126
pkg/tracing: remove direct use of github.com/sirupsen/logrus
While the hook is intended to be used with logrus, we don't need to have
the direct import; use the aliases provided by the containerd/log module
instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-14 12:08:51 +02:00
Sebastiaan van Stijn
4203e2de8d
pkg/tracing/plugin: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-14 11:57:27 +02:00