Commit Graph

422 Commits

Author SHA1 Message Date
Kazuyoshi Kato
80b825ca2c Remove gogoproto.stdtime
This commit removes gogoproto.stdtime, since it is not supported by
Google's official toolchain
(see https://github.com/containerd/containerd/issues/6564).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-19 13:39:30 +00:00
Derek McGowan
be60973a30 Merge pull request #6703 from mxpv/s
Sandbox API
2022-04-18 20:55:06 -07:00
Derek McGowan
fe8da6dcaf Move lease manager plugin to separate package
Create lease plugin type to separate lease manager from services plugin.
This allows other service plugins to depend on the lease manager.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-04-15 11:08:47 -07:00
Fu Wei
e5f4c29a2b Merge pull request #6792 from Junnplus/typo 2022-04-11 11:08:39 +08:00
Ye Sijun
6d69aca561 make consistent for checkpoint path
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2022-04-09 17:28:17 +08:00
Maksym Pavlenko
85a49e4ee7 [sandbox] Cleanup interfaces
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:34:50 -07:00
Maksym Pavlenko
de49745723 [sandbox] Fix CI
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:34:48 -07:00
Maksym Pavlenko
b7a36950f6 [Sandbox] Add Wait and PID
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:48 -07:00
Maksym Pavlenko
0d165e6544 Restore sandboxes on daemon restart
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:48 -07:00
Maksym Pavlenko
35195737e7 [sandbox] Register shim plugin after #6301
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:48 -07:00
Maksym Pavlenko
982de8a5d5 Launch sandboxed containers from task service
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
00f7a6bf2b [sandbox] Address PR review comments
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
6343fe3ea2 [sandbox] Implement sandbox controller
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
df234edeb4 [sandbox] Add controller service
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
a433beb79f [sandbox] Update vendor
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
0e77f758a4 [sandbox] Implement store service
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Derek McGowan
ffddd4446c Merge pull request #6761 from kzys/bbolt-freelist
Disable writing freelist to make the file robust against data corruptions
2022-04-05 21:36:49 -07:00
Maksym Pavlenko
871b6b6a9f Use testify
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-01 18:17:58 -07:00
Kazuyoshi Kato
6da3183105 Disable writing freelist to make the file robust against data corruptions
A bbolt database has a freelist to track all pages that are available
for allocation. However writing the list takes some time and reading
the list sometimes panics.

This commit sets NoFreelistSync true to skipping the freelist entirely,
following what etcd does.

https://github.com/etcd-io/etcd/blob/v3.5.2/server/mvcc/backend/config_linux.go#L31

Fixes #4838.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-01 21:06:42 +00:00
Phil Estes
3633cae64b Merge pull request #6706 from kzys/typeurl-upgrade
Use typeurl.Any instead of github.com/gogo/protobuf/types.Any
2022-03-25 10:38:46 -04:00
Kazuyoshi Kato
96b16b447d Use typeurl.Any instead of github.com/gogo/protobuf/types.Any
This commit upgrades github.com/containerd/typeurl to use typeurl.Any.
The interface hides gogo/protobuf/types.Any from containerd's Go client.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-03-24 20:50:07 +00:00
Sebastiaan van Stijn
c091d48cb9 Use cgroups.AddProc() for cgroups v1
All occurrences only passed a PID, so we can use this utility to make
the code more symmetrical with their cgroups v2 counterparts.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-23 15:25:09 +01:00
Kazuyoshi Kato
067611fdea Remove enumvalue_customname, goproto_enum_prefix and enum_customname
This commit removes gogoproto.enumvalue_customname,
gogoproto.goproto_enum_prefix and gogoproto.enum_customname.

All of them make proto-generated Go code more idiomatic, but we already
don't use these enums in our external-surfacing types and they are anyway
not supported by Google's official toolchain (see #6564).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-03-21 19:48:16 +00:00
Kazuyoshi Kato
3eeeb9429a Remove gogoproto.customtype
gogoproto.customtype is used to have go-digest.Digest instead of string.
While it is convinient, protoc-gen-go doesn't support the extension
and that blocks #6564.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-03-18 23:14:44 +00:00
Fu Wei
79d7df70d6 Merge pull request #6681 from Juneezee/test/t.TempDir 2022-03-16 14:54:16 +08:00
Oleg Atamanenko
fdb746442b document log level and format
Signed-off-by: Oleg Atamanenko <oleg.atamanenko@gmail.com>
2022-03-15 21:00:58 -07:00
Eng Zer Jun
18ec2761c0 test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 14:03:50 +08:00
Henry Wang
b8bf504e94 Enable gosec linter for golangci-lint
`gosec` linter is able to identify issues described in #6584

e.g.

$ git revert 54e95e6b88
[gosec dfc8ca1ec] Revert "fix Implicit memory aliasing in for loop"
 2 files changed, 2 deletions(-)

$ make check
+ proto-fmt
+ check
GOGC=75 golangci-lint run
containerstore.go:192:54: G601: Implicit memory aliasing in for loop. (gosec)
		containers = append(containers, containerFromProto(&container))
		                                                   ^
image_store.go:132:42: G601: Implicit memory aliasing in for loop. (gosec)
		images = append(images, imageFromProto(&image))
		                                       ^
make: *** [check] Error 1

I also disabled following two settings which prevent the linter to show a complete list of issues.

* max-issues-per-linter (default 50)
* max-same-issues (default 3)

Furthermore enabling gosec revealed many other issues. For now I blacklisted the ones except G601.

Will create separate tasks to address them one by one moving next.

Signed-off-by: Henry Wang <henwang@amazon.com>
2022-03-14 22:50:54 +00:00
Derek McGowan
a1ba38056e Update caching logic to avoid map access
Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-01-11 22:11:15 -08:00
Iceber Gu
ce3871966e services/introspection: support to show introspection grpc service
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2022-01-11 22:11:15 -08:00
haoyun
bbe46b8c43 feat: replace github.com/pkg/errors to errors
Signed-off-by: haoyun <yun.hao@daocloud.io>
Co-authored-by: zounengren <zouyee1989@gmail.com>
2022-01-07 10:27:03 +08:00
Markus Lehtonen
f4a191917b cri: annotations for controlling RDT class
Use goresctrl for parsing container and pod annotations related to RDT.

In practice, from the users' point of view, this patchs adds support for
a container annotation and two separate pod annotations for controlling
the RDT class of containers.

Container annotation can be used by a CRI client:
  "io.kubernetes.cri.rdt-class"

Pod annotations for specifying the RDT class in the K8s pod spec level:
  "rdt.resources.beta.kubernetes.io/pod"
  (pod-wide default for all containers within)

  "rdt.resources.beta.kubernetes.io/container.<container_name>"
  (container-specific overrides)

Annotations are intended as an intermediate step before the CRI API
supports RDT.

Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
2022-01-04 09:27:54 +02:00
Markus Lehtonen
eae14688c6 tasks: add Linux rdt support
Add support for configuring the Linux resctrl pseudo-filesystem with
goresctrl library. The functionality is integrated in the
"io.containerd.service.v1.tasks-service" plugin.

Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
2022-01-04 09:27:54 +02:00
Kazuyoshi Kato
2ee3ce510c Use insecure.NewCredentials instead of grpc.WithInsecure
grpc.WithInsecure is being deprecated.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-16 23:10:06 +00:00
Phil Estes
330961c2d5 Merge pull request #6358 from jonyhy96/feat-error
refactor: functions for error log and error return
2021-12-14 10:16:54 -05:00
Phil Estes
6e9e759553 Merge pull request #6225 from jonyhy96/feat-blot-open-timeout
feat: add timeout for bolt open
2021-12-13 11:22:28 -05:00
haoyun
dd26d3d092 feat: support custom timeout for blot open
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 17:02:37 +08:00
haoyun
c0d07094be feat: Errorf usage
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 14:31:53 +08:00
Maksym Pavlenko
6bccd67e84 Revert shim plugin migration
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-12-02 10:35:15 -08:00
Maksym Pavlenko
af93ea7f1e Merge pull request #6241 from jterry75/main
Add support for TMP override on toml
2021-11-18 16:43:51 -08:00
Justin Terry
63895de455 Add support for TMP override on toml
When running containerd as a service it may be hard to
override the TMP location of the process. This is especially
true on Windows when running containerd in SCM. This change
allows you to set the 'temp' location in the config.toml when
the service starts up that overrides its TEMP/TMP/TMPDIR usage.

This is helpful on Linux as well but it primarily solves the
performance issue on Windows when running containerd across
volumes. IE: If you configure your data/root paths on a volume
other than the SystemDrive the snapshotter does a temporary unpack
on the SystemDrive and then has to copy contents of that data
to the snapshot folder on the destination volume. By alinging the
tmp with the destination it is a simple move operation instead of
a copy operation.

Signed-off-by: Justin Terry <jlterry@amazon.com>
2021-11-16 10:43:48 -08:00
Maksym Pavlenko
6870f3b1b8 Support custom runtime path when launching tasks
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-11-09 13:31:46 -08:00
Maksym Pavlenko
a3d298193c Fix backward compatibility with old task shims
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-11-01 07:37:01 -07:00
Maksym Pavlenko
33786ee4d2 Add plugin dependency between shim and shim services
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-11-01 07:37:00 -07:00
Maksym Pavlenko
7c4ead285d Add task manager
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-11-01 07:36:58 -07:00
Maksym Pavlenko
2d5d3541e6 Rename task manager to shim manager
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-11-01 07:36:34 -07:00
Phil Estes
0800b45dd2 Merge pull request #6145 from tonistiigi/content-close-fix
content: close stream after commit request
2021-10-28 10:02:33 -04:00
Tonis Tiigi
010a9e2bca content: close stream after commit request
This allows both client and server to determine when
stream is completed. Previously this was done by
closesend call from the client side but that is async
so client can not assume that the server side also
already thinks that stream is complete.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-21 19:24:28 -07:00
Brian Goff
130a9c7ddb Ensure namespace is proxied to grpc/ttrpc plugins
Before this change we only ever had the grpc incoming medata set so when
we make a request to a shim or a grpc plugin the namespace is not sent
over the RPC.

I need this for github.com/cpuguy83/systemdshim, and I am sure there are
other use-cases where this would be needed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-10-16 00:09:50 +00:00
Derek McGowan
18d483b236 Update cgroups to v1.0.2
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-10-08 17:48:33 -07:00