github/containerd
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity
14,554 Commits 3 Branches 4 Tags
a2d9d4fd556970c39d1fe80d94a77a1aa025c032
Commit Graph

1856 Commits

Author SHA1 Message Date
Derek McGowan
73f57acb0d Update differ to handle zstd media types 
The differ should be able to generate zstd compressed layers when
provided with the zstd media type.

Signed-off-by: Derek McGowan <derek@mcg.dev>
 2024-11-27 16:35:58 +00:00
Akihiro Suda
588b7a100a testutil: avoid conflict with continuity/testutil 
```
=== FAIL: core/metadata  (0.00s)
/tmp/go-build142702247/b1061/metadata.test flag redefined: test.root
panic: /tmp/go-build142702247/b1061/metadata.test flag redefined: test.root

goroutine 1 [running]:
flag.(*FlagSet).Var(0xc0000d6000, {0x132a040, 0x190f3f3}, {0x11b4291, 0x9}, {0x11c7690, 0x1e})
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:1028 +0x54d
flag.BoolVar(...)
	/opt/hostedtoolcache/go/1.23.2/x64/src/flag/flag.go:755
github.com/containerd/containerd/v2/pkg/testutil.init.0()
	/home/runner/work/containerd/containerd/pkg/testutil/helpers.go:36 +0x6b
FAIL	github.com/containerd/containerd/v2/core/metadata	0.028s
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
 2024-10-28 11:58:28 +09:00
Sebastiaan van Stijn
fddeb6f3cb pkg/protobuf: fix typo in godoc 
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-10-26 14:18:53 +02:00
Mike Baynton
347423a114 Request 'allow' setgroups when spawning new userns 
Signed-off-by: Mike Baynton <mike@mbaynton.com>
 2024-10-17 15:37:36 -05:00
Brian Goff
b85909cd4c shim: Move pprof server to plugin 
Makes the pprof server a plugin and also gates by the `shim_tracing`
build tag (like otel is).
With this change, `net/http` is no longer a dependency in the shim.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
 2024-10-03 00:12:50 +00:00
Brian Goff
b2681dfbdb shim: Move ttrpc interceptors to plugins 
This makes it so we don't need to import otelttrpc unless the shim is
compiled with the `shim_tracing` build tag.
This way otel is no longer compiled into the binary at all unless
`shim_tracing` is set.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
 2024-10-03 00:12:47 +00:00
Brian Goff
17d4a1357e Propagate trace contexts to shims 
This adds trace context propagation over the grpc/ttrpc calls to a shim.

It also adds the otlp plugin to the runc shim so that it will send
traces to the configured tracer (which is inherited from containerd's
config).
It doesn't look like this is adding any real overhead to the runc shim's
memory usage, however it does add 2MB to the binary size.
As such this is gated by a build tag `shim_tracing`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
 2024-09-30 21:44:16 +00:00
Akihiro Suda
8c64a2f6a1 Merge pull request #10607 from fuweid/pin-userns 
internal/cri: simplify netns setup with pinned userns
 2024-09-19 01:05:41 +00:00
Wei Fu
fd3f3d5a13 pkg/sys: add GetUsernsForNamespace interface 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2024-09-11 07:21:43 +08:00
Wei Fu
490e45a08a pkg/sys: Add UnshareAfterEnterUserns function 
It allows to disassociate parts of its execution context within a user
namespace.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2024-09-11 07:21:39 +08:00
Samuel Karp
5f37a2c205 fifosync: cross-process synchronization 
Signed-off-by: Samuel Karp <samuelkarp@google.com>
 2024-09-05 12:48:28 -07:00
Akihiro Suda
cae19b14f3 Merge pull request #10658 from darwin-containers/reorganize-mount-unmount 
Reorganize mount/unmount code so it is easier to add Darwin-specific implementation
 2024-09-03 01:51:24 +00:00
Akihiro Suda
26d6fd0c3f Merge pull request #10611 from fuweid/getridof-hall-of-shame 
core/mount: use ptrace instead of go:linkname
 2024-08-30 17:07:33 +00:00
Marat Radchenko
bfc1465a2c Reorganize mount/unmount code so it is easier to add Darwin-specific implementation 
After these changes, in order to add Darwin bind-mount implementation, one only needs:
* Adjust HasBindMounts definition in mount.go
* Provide implementation in mount_darwin.go

There was no consensus on adding dependency on bindfs, that seems to be the only working solution for bind-mounts on Darwin as of today, in https://github.com/containerd/containerd/pull/8789, that's why the actual implementation is not added in current PR.

As a bonus, Linux FUSE-related code was moved to a separate file and possibly could be reused on FreeBSD, though this needs testing.

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
 2024-08-30 15:25:06 +03:00
Jin Dong
d83184ced1 avoid repeated calls in Tricolor gc 
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
 2024-08-26 23:35:24 -07:00
Jin Dong
6b97a08eee add benchmark 
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
 2024-08-26 23:35:04 -07:00
Wei Fu
3cd8f9734d core/mount: use ptrace instead of go:linkname 
The Go runtime has started to [lock down future uses of linkname][1] since
go1.23. In the go source code, containerd project has been marked in the
comment, [hall of shame][2]. Well, the go:linkname is used to fork no-op
subprocess efficiently. However, since that comment, I would like to use
ptrace and remove go:linkname in the whole repository.

With go1.22 `go:linkname`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 2440            533320 ns/op            1145 B/op         43 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 342           3661616 ns/op           11562 B/op        421 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  2.983s
```

With go1.22 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1785            739557 ns/op            3948 B/op         68 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 328           4024300 ns/op           39601 B/op        671 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.104s
```

With go1.23 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1815            723252 ns/op            4220 B/op         69 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 319           3957157 ns/op           42351 B/op        682 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.051s
```

Diff:

The `ptrace` is slower than `go:linkname` mode. However, it's accepctable.

```
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
                                    │ go122-golinkname │             go122-ptrace              │             go123-ptrace              │
                                    │      sec/op      │    sec/op     vs base                 │    sec/op     vs base                 │
BatchRunGetUsernsFD_Concurrent1-16        533.3µ ± ∞ ¹   739.6µ ± ∞ ¹        ~ (p=1.000 n=1) ²   723.3µ ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16       3.662m ± ∞ ¹   4.024m ± ∞ ¹        ~ (p=1.000 n=1) ²   3.957m ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                   1.397m         1.725m        +23.45%                   1.692m        +21.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │              go122-ptrace               │              go123-ptrace               │
                                    │       B/op       │     B/op       vs base                  │     B/op       vs base                  │
BatchRunGetUsernsFD_Concurrent1-16       1.118Ki ± ∞ ¹   3.855Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   4.121Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16      11.29Ki ± ∞ ¹   38.67Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   41.36Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
geomean                                  3.553Ki         12.21Ki        +243.65%                   13.06Ki        +267.43%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │             go122-ptrace             │             go123-ptrace             │
                                    │    allocs/op     │  allocs/op   vs base                 │  allocs/op   vs base                 │
BatchRunGetUsernsFD_Concurrent1-16         43.00 ± ∞ ¹   68.00 ± ∞ ¹        ~ (p=1.000 n=1) ²   69.00 ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16        421.0 ± ∞ ¹   671.0 ± ∞ ¹        ~ (p=1.000 n=1) ²   682.0 ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                    134.5         213.6        +58.76%                   216.9        +61.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
```

[1]: <https://github.com/golang/go/issues/67401>
[2]: <https://github.com/golang/go/blob/release-branch.go1.23/src/runtime/proc.go#L4820>

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2024-08-26 21:19:50 +08:00
Akhil Mohan
ebc47359ea use format string when using printf like commands 
As per https://github.com/golang/go/issues/60529, printf like commands with
non-constant format strings and no args give an error in govet

Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
 2024-08-14 17:04:53 +05:30
Maksym Pavlenko
0b02e0c225 Merge pull request #7616 from swagatbora90/trace-cri-runtime 
Add tracing spans to CRI runtime service apis
 2024-08-09 18:24:47 +00:00
Sebastiaan van Stijn
9776047243 migrate to github.com/moby/sys/userns 
Commit 8437c567d8 migrated the use of the
userns package to the github.com/moby/sys/user module.

After further discussion with maintainers, it was decided to move the
userns package to a separate module, as it has no direct relation with
"user" operations (other than having "user" in its name).

This patch migrates our code to use the new module.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-08-08 12:48:54 +02:00
Swagat Bora
83c26962c2 Remove extra span created in the instrumented service layer 
Signed-off-by: Swagat Bora <sbora@amazon.com>
 2024-08-05 02:34:27 +00:00
Swagat Bora
c0cdcb34f1 Add spans to CRI runtime service and related client methods 
This adds otel spans to CRI service mainly targeting mutating apis which includes:
* Sandbox apis - RunPodSandbox, StopPodSandbox, RemovePodSandbox
* Container apis - CreateContainer, StartContainer, StopContainer, RemoveContainer
* Attach, Exec and Exec Sync
* Containerd client methods: container.go, client.go, process.go and task.go

Signed-off-by: Swagat Bora <sbora@amazon.com>
 2024-08-05 02:34:20 +00:00
Kazuyoshi Kato
2ddd3db952 Merge pull request #9858 from w13915984028/fixdebug 
Add file name to device type check failure message
 2024-07-26 21:20:38 +00:00
Akihiro Suda
7a1f6b2891 Merge pull request #10509 from samuelkarp/pr-9966-follow-up 
deprecation: update warnings for CRI config fields
 2024-07-26 17:40:59 +00:00
Sebastiaan van Stijn
eddd90d7c1 remove deprecated pkg/userns 
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-07-26 09:48:40 +02:00
Sebastiaan van Stijn
8437c567d8 pkg/userns: deprecate and migrate to github.com/moby/sys/user/userns 
The userns package in libcontainer was integrated into the moby/sys/user
module at commit [3778ae603c706494fd1e2c2faf83b406e38d687d][1].

This patch deprecates the containerd fork of that package, and adds it as
an alias for the moby/sys/user/userns package.

[1]: 3778ae603c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-07-26 09:47:50 +02:00
Samuel Karp
f9146c0516 deprecation: update warnings for CRI config fields 
Some CRI config properties had removal postponed until v2.1 in
https://github.com/containerd/containerd/pull/9966. Update the
associated deprecation warnings to match the new removal version.

Signed-off-by: Samuel Karp <samuelkarp@google.com>
 2024-07-25 14:25:39 -07:00
Danny Canter
b41bb6df73 Avoid potential reallocs by pre-sizing some slices 
There's a couple spots where we know exactly how large
the destination buffer should be, so pre-size these to
avoid any reallocs to a higher capacity.

Signed-off-by: Danny Canter <danny@dcantah.dev>
 2024-07-19 13:05:49 -07:00
Colin O'Dell
c8254dfae9 Include filename in error 
Signed-off-by: Colin O'Dell <colinodell@gmail.com>
 2024-07-16 12:14:36 -04:00
Samuel Karp
0262714edb Merge pull request #10412 from TinaMor/tinamor/dev 
[Windows] Set stderr to empty string when using terminal on Windows
 2024-07-15 16:29:08 +00:00
Akhil Mohan
300fd770a0 use typeurl funcs for marshalling anypb.Any 
Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
 2024-07-10 22:26:27 +05:30
TinaMor
eaa1afe634 Set stderr to empty string when using terminal on Windows. 
Windows HCSShim requires that stderr is an empty string when using terminal.
	Reference: 200feabd85/cmd/containerd-shim-runhcs-v1/service_internal.go (L127)

Signed-off-by: Christine Murimi <mor.tina@outlook.com>
 2024-07-08 09:21:20 +03:00
Derek McGowan
23170e20df Move fsverity package to internal 
This functionality is not directly related to containerd and could move
to external package at some point.

Signed-off-by: Derek McGowan <derek@mcg.dev>
 2024-07-01 09:21:24 -07:00
Akihiro Suda
7103e381f1 Merge pull request #10007 from Jenkins-J/fsverity-content-verification 
Fsverity content verification
 2024-06-29 06:32:17 +00:00
Sebastiaan van Stijn
a723c0c6ea pkg/reference: remove deprecated SplitObject 
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-28 19:06:30 +02:00
James Jenkins
ef98c71985 Implement fsverity functionality 
Implement calls to the fsverity kernel module, allowing containerd to
enable fsverity on blob data in the content store. This causes fsverity
to veirfy the integrity of blob data when the blob is read.

Signed-off-by: James Jenkins <James.Jenkins@ibm.com>
 2024-06-28 09:50:44 -04:00
Sebastiaan van Stijn
fec33aa735 pkg/reference: deprecate SplitObject 
The behavior of this function is quite counter-intuitive, as it preserves
the delimiter in the result, and its use for external consumers would be
very limited.

Spec.Digest no longer uses this function, and it appears that BuildKit is
currently the only (publicly visible) external consumer of it.

This patch deprecates the function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-27 22:11:42 +02:00
Sebastiaan van Stijn
a5fce38f31 pkg/reference: Spec.Digest(): inline SplitObject code 
Inline the relevant code from SplitObject, as we're only interested
in the digest portion.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-27 12:05:23 +02:00
Sebastiaan van Stijn
42145950bb pkg/reference: SplitObject: add proper GoDoc 
The behavior of this function is quite counter-intuitive, as it preserves
the delimiter in the result. This function should probably have been an
internal function, as its use for external consumers would be very limited,
but let's at least document the (surprising) behavior for those that are
considering to use it.

It appears that BuildKit is currently the only (publicly visible) external
consumer of this function; I am planning to inline its functionality in
Spec.Digest() and to deprecate this function so that it can be removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-27 12:05:23 +02:00
Sebastiaan van Stijn
74a6156ac2 pkg/reference: SplitObject: zero allocations 
Before / After:

    BenchmarkSplitObject-10        2785656    428.1 ns/op     416 B/op    13 allocs/op
    BenchmarkSplitObjectNew-10    13510520     88.2 ns/op       0 B/op     0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-27 12:05:10 +02:00
Sebastiaan van Stijn
799bca97f2 pkg/reference: Spec.String(): use string-concatenation instead of sprintf 
These were straight concatenations of strings; reduce some allocations by
removing fmt.Sprintf for this.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-27 12:04:22 +02:00
Henry Wang
243b803a19 Add pprof to runc-shim 
Signed-off-by: Henry Wang <henwang@amazon.com>
 2024-06-20 23:12:31 +00:00
Sebastiaan van Stijn
75fd7a5a7f pkg/shim: remove logrus imports 
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-17 12:35:36 +02:00
Sebastiaan van Stijn
587ee80f61 pkg/tracing: LogrusHook.Fire: micro-optimisation 
Check span.IsRecording first, as it's a more lightweight check than
span.SpanContext().IsValid()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-14 12:12:22 +02:00
Sebastiaan van Stijn
ccf7938126 pkg/tracing: remove direct use of github.com/sirupsen/logrus 
While the hook is intended to be used with logrus, we don't need to have
the direct import; use the aliases provided by the containerd/log module
instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-14 12:08:51 +02:00
Sebastiaan van Stijn
4203e2de8d pkg/tracing/plugin: rename var that collided with import 
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-14 11:57:27 +02:00
Sebastiaan van Stijn
e2e09b384a pkg/tracing: rename func that shadowed builtin, rm makeSpanName 
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2024-06-14 11:56:06 +02:00
Akihiro Suda
86b8a88241 Remove pkg/seed 
Since Go 1.20, math/rand does not need explicit seeding:
https://go.dev/doc/go1.20#minor_library_changes

Go <= 1.19 is no longer supported due to EOL.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
 2024-06-13 08:50:28 +09:00
Derek McGowan
ca59fb0b41 Cleanup shim manager configuration 
Keep platforms configuration on task manager and add environment config
for shims.

Signed-off-by: Derek McGowan <derek@mcg.dev>
 2024-05-22 11:06:53 -07:00
Henry Wang
d9dc2811ae fix: delete sockets on shim exit 
Signed-off-by: Henry Wang <henwang@amazon.com>
 2024-05-08 21:53:39 +00:00