Commit Graph

242 Commits

Author SHA1 Message Date
Derek McGowan
7b2a918213
Generalize the plugin package
Remove containerd specific parts of the plugin package to prepare its
move out of the main repository. Separate the plugin registration
singleton into a separate package.

Separating out the plugin package and registration makes it easier to
implement external plugins without creating a dependency loop.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 21:22:32 -07:00
Derek McGowan
a80606bc2d
Move plugin type definitions to containerd plugins package
The plugins packages defines the plugins used by containerd.
Move all the types and properties to this package.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-12 20:52:56 -07:00
Akihiro Suda
66aab638da
Merge pull request #8989 from dmcgowan/add-image-delete-target
Add image delete target
2023-10-05 10:53:57 -07:00
Kohei Tokunaga
6169433b6b
Fix linter issues for golangci-lint 1.54.2
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-03 21:56:47 +09:00
Derek McGowan
508aa3a1ef
Move to use github.com/containerd/log
Add github.com/containerd/log to go.mod

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-22 07:53:23 -07:00
Derek McGowan
2ce971d890
Add delete target to image remove
Adds atomicity to image delete when deleting from a list.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-19 17:23:33 -07:00
Derek McGowan
cd705af89b
metadata: add image delete events during garbage collection
Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 15:10:13 -07:00
Derek McGowan
3f9756c184
gc: add support for image expiration
Update the garbage collector to support image expiration along with
support for image leasing. This allows making images collectible during
garbage collection and using a lease to prevent removal of an image.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-31 15:10:13 -07:00
Derek McGowan
4f5db2bc03
metadata: format unit test output
Make it easier to see list differences in gc node test failure output.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-08-28 16:52:25 -07:00
Fu Wei
e350c8ca94
Merge pull request #8711 from djdongjin/sb-metadata
Minor changes to sandbox metadata
2023-06-25 15:16:43 +08:00
Alex Couture-Beil
f3df7f7391
log when a lease expires
This makes it easier to debug GC-related issues, where a reference is
unexpectedly released.

Signed-off-by: Alex Couture-Beil <alex@mofo.ca>
2023-06-21 16:00:02 -07:00
Jin Dong
a01118bbed Minor changes to sandbox metadata
1. Reduce one bucket.Bucket call.
2. Use update helper instead of db.Update as in other functions.

Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2023-06-20 20:41:17 +00:00
Jin Dong
1fb835f042 Remove unnecessary label bucket loop
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2023-06-17 19:26:29 +00:00
Sebastiaan van Stijn
4bb709c018
avoid "any" as variable name
Avoid shadowing / confusion with Go's "any" built-in type.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-10 13:49:06 +02:00
Derek McGowan
2a60fe5a60
Remove events from init context
Events from the init context have been replaced by the events plugin

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-05-31 09:35:03 -07:00
Fu Wei
773874caea
Merge pull request #8547 from zhaojizhuang/bolt
[sandbox] Optimize metadata Schema notes to correctly match metadata structures
2023-05-23 15:32:47 +08:00
zhaojizhuang
82deabf9db Optimize metadata Schema notes to correctly match metadata structures
1. Add sandboxes object
2. Optimize indentation of parent and child objects

Signed-off-by: zhaojizhuang <571130360@qq.com>
2023-05-22 16:18:58 +08:00
mstmdev
cdaa4025e9 Fix some typos
Signed-off-by: Pan Yibo <mstmdev@gmail.com>
2023-05-16 10:12:50 +08:00
Akihiro Suda
ff4acdc42e
metadata: add comments about Image.CreatedAt
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-03-08 03:23:01 +09:00
Akihiro Suda
4ced1fa69e
Merge pull request #8188 from dmcgowan/fix-streaming-gc-deadlock
Fix streaming manager deadlock on collection
2023-03-02 10:25:05 +09:00
Derek McGowan
5c6e9f83d4
Fix streaming manager deadlock on collection
Ensure that lock is released and stream is closed.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-03-01 09:40:59 -08:00
Henry Wang
cf46d3c6fc Treat sandboxes as root gc resources and scan referenced objects
Signed-off-by: Henry Wang <henwang@amazon.com>
2023-02-27 19:28:28 +00:00
Akihiro Suda
b61988670c
go.mod: github.com/containerd/typeurl/v2 v2.1.0
Changes: https://github.com/containerd/typeurl/compare/7f6e6d160d67...v2.1.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-02-11 23:39:52 +09:00
Henry Wang
b9bd10c14e use local variable for rt when iterating collectors
Signed-off-by: Henry Wang <henwang@amazon.com>
2023-01-13 21:50:04 +00:00
Fu Wei
5fc727224e
Merge pull request #7861 from dmcgowan/cleanup-context
Add cleanup package for context management during cleanup
2023-01-05 13:18:31 +08:00
Derek McGowan
b550526ccd
Use cleanup.Background instead of context.Background for cleanup
Use the cleanup context to re-use values from the original context

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-01-04 11:22:24 -08:00
Maksym Pavlenko
06bfcd658c Enable dupword linter
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-01-03 12:47:16 -08:00
Derek McGowan
2c573de6d3
Move snapshot event publishing into metadata store
Removes the snapshot event publishing from the snapshot service.

Adds an option to metadata db to add a publisher. Adds event
publishing to prepare, commit, and remove snapshot operations.
Adds remove snapshot event to garbage collection.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-12-19 17:05:28 -08:00
Akihiro Suda
75b09ac4a7
images: support specifying SourceDateEpoch via ctx
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-12-12 09:02:35 +09:00
Derek McGowan
dcf5687cab
Add streaming service
Adds a service capable of streaming Any objects bi-directionally.
This can be used by services to send data, received data, or to
initiate requests from server to client.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-11-30 12:55:56 -08:00
Sebastiaan van Stijn
eaedadbed0
replace strings.Split(N) for strings.Cut() or alternatives
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:

```go
func BenchmarkSplit(b *testing.B) {
        b.ReportAllocs()
        data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
        for i := 0; i < b.N; i++ {
                for _, s := range data {
                        _ = strings.SplitN(s, "=", 2)[0]
                }
        }
}

func BenchmarkCut(b *testing.B) {
        b.ReportAllocs()
        data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
        for i := 0; i < b.N; i++ {
                for _, s := range data {
                        _, _, _ = strings.Cut(s, "=")
                }
        }
}
```

    BenchmarkSplit
    BenchmarkSplit-10            8244206               128.0 ns/op           128 B/op          4 allocs/op
    BenchmarkCut
    BenchmarkCut-10             54411998                21.80 ns/op            0 B/op          0 allocs/op

While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints; for cases where an specific number of items is expected, I used `strings.SplitN()`
with a suitable limit. This prevents (theoretical) unlimited splits.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-07 10:02:25 +01:00
Kazuyoshi Kato
407703f092 Make checkContainerTimestamps less strict on Windows
This assertion is flaky on Windows.
Because of Go, Windows' time.Now resolution is lower than Linux.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-08-31 17:37:57 +00:00
Maksym Pavlenko
ca3b9b50fe Run gofmt 1.19
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-08-04 18:18:33 -07:00
Maksym Pavlenko
e47c433d57 Add sandbox store helpers
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-07-28 14:17:39 -07:00
Maksym Pavlenko
d97b754a5b Cleanup metadata tests
This commit replaces func returns with t.Cleanup,
which makes API and tests slightly easier to maintain.

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-06-27 14:42:22 -07:00
Derek McGowan
99e210c50a
Move metadata plugin registration to seperate package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-06-22 17:38:41 -07:00
Derek McGowan
c4e29027d4
Merge pull request #6937 from mythi/sandbox-errors
sandbox: replace github.com/pkg/errors with native errors
2022-05-26 10:44:15 -07:00
Iceber Gu
8d95f2b599 fix comments on metadata schema
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2022-05-19 22:52:51 +08:00
Mikko Ylinen
523d069a25 sandbox: replace github.com/pkg/errors with native errors
PR #6366 implemented a tree-wide change to replace github.com/pkg/errors
to errors. The new sandbox API PR #6703 had few errors.Wrap*() leftovers
and pulled github.com/pkg/errors back. This commit replaces those
leftovers by following the pattern in #6366.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2022-05-12 17:09:45 +03:00
Kazuyoshi Kato
2bfc2a587b
Merge pull request #6804 from dmcgowan/metadata-collectible-resources
Add collectible resources to metadata gc
2022-05-02 11:24:39 -07:00
Taeho Nam
b5370b0406
Fix comment for metadata/db.go
Fix comment

Signed-off-by: Taeho Nam <thn7440@gmail.com>
2022-04-28 13:32:45 +09:00
Kazuyoshi Kato
7a4f81d8ba Fix tests
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-22 15:41:05 +00:00
Derek McGowan
3b82f9e33c
metadata: use resource max and end on registration
Ensure the registered resource type does not conflict
with existing resource types or over the max.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-04-19 14:25:31 -07:00
Kazuyoshi Kato
88c0c7201e Consolidate gogo/protobuf dependencies under our own protobuf package
This would make gogo/protobuf migration easier.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-19 15:53:36 +00:00
Derek McGowan
8367f69fb5
Add collectible resources to metadata gc
Adds a registration function to metadata which allows plugins to
register resources to be garbage collected. These resources allow
defining resources types which are ephemeral and stored outside the
metadata plugin without extending it. The garbage collection of these
resources will not fail the metadata gc process if their removal fails.
These resources may be referenced by existing metadata store resources
but may not be used to reference metadata store resources for the purpose
of preventing garbage collection.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-04-12 18:59:18 -07:00
Maksym Pavlenko
d0b32c0539 [sandbox] Migrate from gogo to Any
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:34:50 -07:00
Maksym Pavlenko
17a2aaded3 [sandbox] Add ctr support
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:48 -07:00
Maksym Pavlenko
00f7a6bf2b [sandbox] Address PR review comments
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
d7ece87243 [sandbox] Save sandbox ID to container's store
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
cab7d5b3d2 [sandbox] Implement metadata store
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00