Commit Graph

246 Commits

Author SHA1 Message Date
Derek McGowan
9db21401c4 Switch to github.com/containerd/plugin
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-11-01 23:01:42 -07:00
Derek McGowan
5fdf55e493 Update go module to github.com/containerd/containerd/v2
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-29 20:52:21 -07:00
Wei Fu
337cc21719 pkg/cri: should ignore no sandbox bucket
The sandbox might be recovered from v1.x release. It doesn't have
metadata bucket. We should ignore the not-found error.

How to reproduce the issue:

```bash
➜  containerd git:(main) sudo ctr version
Client:
  Version:  1.6.22
  Revision: 8165feabfdfe38c65b599c4993d227328c231fca
  Go version: go1.19.11

Server:
  Version:  1.6.22
  Revision: 8165feabfdfe38c65b599c4993d227328c231fca
  UUID: be4216aa-8a2e-4305-9186-efeacd2d9a17

➜  containerd git:(main) cat /tmp/pod.json
{
    "metadata": {
        "name": "nginx-sandbox",
        "namespace": "default",
        "attempt": 1,
        "uid": "hdishd83djaidwnduwk28bcsb"
    },
    "log_directory": "/tmp",
    "linux": {
    }
}

➜  containerd git:(main) sudo crictl runp /tmp/pod.json
616ea1cc657c57e80abf74e707a8177878ac2ec1ab7c346b4adb7bc0fadf986e
➜  containerd git:(main) sudo crictl pods
POD ID              CREATED             STATE               NAME                NAMESPACE           ATTEMPT             RUNTIME
616ea1cc657c5       9 seconds ago       Ready               nginx-sandbox       default             1                   (default)

➜  containerd git:(main) make BUILDTAGS=no_btrfs
➜  containerd git:(main) sudo PREFIX=/usr make install
+ install bin/ctr bin/containerd bin/containerd-stress bin/containerd-shim-runc-v2

➜  containerd git:(main) sudo systemctl restart containerd
➜  containerd git:(main) sudo ctr version
Client:
  Version:  v1.7.0-943-g980767551
  Revision: 9807675518
  Go version: go1.20.10

Server:
  Version:  v1.7.0-943-g980767551
  Revision: 9807675518
  UUID: be4216aa-8a2e-4305-9186-efeacd2d9a17

➜  containerd git:(main) sudo crictl stopp  616ea1cc657c5
Stopped sandbox 616ea1cc657c5

➜  containerd git:(main) sudo crictl rmp 616ea1cc657c5
E1019 14:03:37.885162 2052643 remote_runtime.go:295] "RemovePodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to remove sandbox metadata from store: failed to delete sandbox \"616ea1cc657c57e80abf74e707a8177878ac2ec1ab7c346b4adb7bc0fadf986e\": bucket not found" podSandboxID="616ea1cc657c5"
removing the pod sandbox "616ea1cc657c5": rpc error: code = Unknown desc = failed to remove sandbox metadata from store: failed to delete sandbox "616ea1cc657c57e80abf74e707a8177878ac2ec1ab7c346b4adb7bc0fadf986e": bucket not found
```

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-20 15:20:18 +08:00
Abel Feng
8b35976850 sandbox: add Sandboxer field to sandbox metadata
Signed-off-by: Abel Feng <fshb1988@gmail.com>
2023-10-16 20:49:36 +08:00
Derek McGowan
7b2a918213 Generalize the plugin package
Remove containerd specific parts of the plugin package to prepare its
move out of the main repository. Separate the plugin registration
singleton into a separate package.

Separating out the plugin package and registration makes it easier to
implement external plugins without creating a dependency loop.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 21:22:32 -07:00
Derek McGowan
a80606bc2d Move plugin type definitions to containerd plugins package
The plugins packages defines the plugins used by containerd.
Move all the types and properties to this package.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 20:52:56 -07:00
Akihiro Suda
66aab638da Merge pull request #8989 from dmcgowan/add-image-delete-target
Add image delete target
2023-10-05 10:53:57 -07:00
Kohei Tokunaga
6169433b6b Fix linter issues for golangci-lint 1.54.2
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-03 21:56:47 +09:00
Derek McGowan
508aa3a1ef Move to use github.com/containerd/log
Add github.com/containerd/log to go.mod

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-22 07:53:23 -07:00
Derek McGowan
2ce971d890 Add delete target to image remove
Adds atomicity to image delete when deleting from a list.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-19 17:23:33 -07:00
Derek McGowan
cd705af89b metadata: add image delete events during garbage collection
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 15:10:13 -07:00
Derek McGowan
3f9756c184 gc: add support for image expiration
Update the garbage collector to support image expiration along with
support for image leasing. This allows making images collectible during
garbage collection and using a lease to prevent removal of an image.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 15:10:13 -07:00
Derek McGowan
4f5db2bc03 metadata: format unit test output
Make it easier to see list differences in gc node test failure output.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-28 16:52:25 -07:00
Fu Wei
e350c8ca94 Merge pull request #8711 from djdongjin/sb-metadata
Minor changes to sandbox metadata
2023-06-25 15:16:43 +08:00
Alex Couture-Beil
f3df7f7391 log when a lease expires
This makes it easier to debug GC-related issues, where a reference is
unexpectedly released.

Signed-off-by: Alex Couture-Beil <alex@mofo.ca>
2023-06-21 16:00:02 -07:00
Jin Dong
a01118bbed Minor changes to sandbox metadata
1. Reduce one bucket.Bucket call.
2. Use update helper instead of db.Update as in other functions.

Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2023-06-20 20:41:17 +00:00
Jin Dong
1fb835f042 Remove unnecessary label bucket loop
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2023-06-17 19:26:29 +00:00
Sebastiaan van Stijn
4bb709c018 avoid "any" as variable name
Avoid shadowing / confusion with Go's "any" built-in type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-10 13:49:06 +02:00
Derek McGowan
2a60fe5a60 Remove events from init context
Events from the init context have been replaced by the events plugin

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-31 09:35:03 -07:00
Fu Wei
773874caea Merge pull request #8547 from zhaojizhuang/bolt
[sandbox] Optimize metadata Schema notes to correctly match metadata structures
2023-05-23 15:32:47 +08:00
zhaojizhuang
82deabf9db Optimize metadata Schema notes to correctly match metadata structures
1. Add sandboxes object
2. Optimize indentation of parent and child objects

Signed-off-by: zhaojizhuang <571130360@qq.com>
2023-05-22 16:18:58 +08:00
mstmdev
cdaa4025e9 Fix some typos
Signed-off-by: Pan Yibo <mstmdev@gmail.com>
2023-05-16 10:12:50 +08:00
Akihiro Suda
ff4acdc42e metadata: add comments about Image.CreatedAt
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-03-08 03:23:01 +09:00
Akihiro Suda
4ced1fa69e Merge pull request #8188 from dmcgowan/fix-streaming-gc-deadlock
Fix streaming manager deadlock on collection
2023-03-02 10:25:05 +09:00
Derek McGowan
5c6e9f83d4 Fix streaming manager deadlock on collection
Ensure that lock is released and stream is closed.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-03-01 09:40:59 -08:00
Henry Wang
cf46d3c6fc Treat sandboxes as root gc resources and scan referenced objects
Signed-off-by: Henry Wang <henwang@amazon.com>
2023-02-27 19:28:28 +00:00
Akihiro Suda
b61988670c go.mod: github.com/containerd/typeurl/v2 v2.1.0
Changes: https://github.com/containerd/typeurl/compare/7f6e6d160d67...v2.1.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-02-11 23:39:52 +09:00
Henry Wang
b9bd10c14e use local variable for rt when iterating collectors
Signed-off-by: Henry Wang <henwang@amazon.com>
2023-01-13 21:50:04 +00:00
Fu Wei
5fc727224e Merge pull request #7861 from dmcgowan/cleanup-context
Add cleanup package for context management during cleanup
2023-01-05 13:18:31 +08:00
Derek McGowan
b550526ccd Use cleanup.Background instead of context.Background for cleanup
Use the cleanup context to re-use values from the original context

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-01-04 11:22:24 -08:00
Maksym Pavlenko
06bfcd658c Enable dupword linter
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-01-03 12:47:16 -08:00
Derek McGowan
2c573de6d3 Move snapshot event publishing into metadata store
Removes the snapshot event publishing from the snapshot service.

Adds an option to metadata db to add a publisher. Adds event
publishing to prepare, commit, and remove snapshot operations.
Adds remove snapshot event to garbage collection.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-12-19 17:05:28 -08:00
Akihiro Suda
75b09ac4a7 images: support specifying SourceDateEpoch via ctx
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-12-12 09:02:35 +09:00
Derek McGowan
dcf5687cab Add streaming service
Adds a service capable of streaming Any objects bi-directionally.
This can be used by services to send data, received data, or to
initiate requests from server to client.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-11-30 12:55:56 -08:00
Sebastiaan van Stijn
eaedadbed0 replace strings.Split(N) for strings.Cut() or alternatives
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:

```go
func BenchmarkSplit(b *testing.B) {
        b.ReportAllocs()
        data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
        for i := 0; i < b.N; i++ {
                for _, s := range data {
                        _ = strings.SplitN(s, "=", 2)[0]
                }
        }
}

func BenchmarkCut(b *testing.B) {
        b.ReportAllocs()
        data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
        for i := 0; i < b.N; i++ {
                for _, s := range data {
                        _, _, _ = strings.Cut(s, "=")
                }
        }
}
```

    BenchmarkSplit
    BenchmarkSplit-10            8244206               128.0 ns/op           128 B/op          4 allocs/op
    BenchmarkCut
    BenchmarkCut-10             54411998                21.80 ns/op            0 B/op          0 allocs/op

While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints; for cases where an specific number of items is expected, I used `strings.SplitN()`
with a suitable limit. This prevents (theoretical) unlimited splits.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-07 10:02:25 +01:00
Kazuyoshi Kato
407703f092 Make checkContainerTimestamps less strict on Windows
This assertion is flaky on Windows.
Because of Go, Windows' time.Now resolution is lower than Linux.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-08-31 17:37:57 +00:00
Maksym Pavlenko
ca3b9b50fe Run gofmt 1.19
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-08-04 18:18:33 -07:00
Maksym Pavlenko
e47c433d57 Add sandbox store helpers
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-07-28 14:17:39 -07:00
Maksym Pavlenko
d97b754a5b Cleanup metadata tests
This commit replaces func returns with t.Cleanup,
which makes API and tests slightly easier to maintain.

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-06-27 14:42:22 -07:00
Derek McGowan
99e210c50a Move metadata plugin registration to seperate package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-06-22 17:38:41 -07:00
Derek McGowan
c4e29027d4 Merge pull request #6937 from mythi/sandbox-errors
sandbox: replace github.com/pkg/errors with native errors
2022-05-26 10:44:15 -07:00
Iceber Gu
8d95f2b599 fix comments on metadata schema
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2022-05-19 22:52:51 +08:00
Mikko Ylinen
523d069a25 sandbox: replace github.com/pkg/errors with native errors
PR #6366 implemented a tree-wide change to replace github.com/pkg/errors
to errors. The new sandbox API PR #6703 had few errors.Wrap*() leftovers
and pulled github.com/pkg/errors back. This commit replaces those
leftovers by following the pattern in #6366.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2022-05-12 17:09:45 +03:00
Kazuyoshi Kato
2bfc2a587b Merge pull request #6804 from dmcgowan/metadata-collectible-resources
Add collectible resources to metadata gc
2022-05-02 11:24:39 -07:00
Taeho Nam
b5370b0406 Fix comment for metadata/db.go
Fix comment

Signed-off-by: Taeho Nam <thn7440@gmail.com>
2022-04-28 13:32:45 +09:00
Kazuyoshi Kato
7a4f81d8ba Fix tests
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-22 15:41:05 +00:00
Derek McGowan
3b82f9e33c metadata: use resource max and end on registration
Ensure the registered resource type does not conflict
with existing resource types or over the max.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-04-19 14:25:31 -07:00
Kazuyoshi Kato
88c0c7201e Consolidate gogo/protobuf dependencies under our own protobuf package
This would make gogo/protobuf migration easier.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-19 15:53:36 +00:00
Derek McGowan
8367f69fb5 Add collectible resources to metadata gc
Adds a registration function to metadata which allows plugins to
register resources to be garbage collected. These resources allow
defining resources types which are ephemeral and stored outside the
metadata plugin without extending it. The garbage collection of these
resources will not fail the metadata gc process if their removal fails.
These resources may be referenced by existing metadata store resources
but may not be used to reference metadata store resources for the purpose
of preventing garbage collection.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-04-12 18:59:18 -07:00
Maksym Pavlenko
d0b32c0539 [sandbox] Migrate from gogo to Any
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:34:50 -07:00