Removes the snapshot event publishing from the snapshot service.
Adds an option to metadata db to add a publisher. Adds event
publishing to prepare, commit, and remove snapshot operations.
Adds remove snapshot event to garbage collection.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Adds a service capable of streaming Any objects bi-directionally.
This can be used by services to send data, received data, or to
initiate requests from server to client.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:
```go
func BenchmarkSplit(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_ = strings.SplitN(s, "=", 2)[0]
}
}
}
func BenchmarkCut(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_, _, _ = strings.Cut(s, "=")
}
}
}
```
BenchmarkSplit
BenchmarkSplit-10 8244206 128.0 ns/op 128 B/op 4 allocs/op
BenchmarkCut
BenchmarkCut-10 54411998 21.80 ns/op 0 B/op 0 allocs/op
While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints; for cases where an specific number of items is expected, I used `strings.SplitN()`
with a suitable limit. This prevents (theoretical) unlimited splits.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adds a registration function to metadata which allows plugins to
register resources to be garbage collected. These resources allow
defining resources types which are ephemeral and stored outside the
metadata plugin without extending it. The garbage collection of these
resources will not fail the metadata gc process if their removal fails.
These resources may be referenced by existing metadata store resources
but may not be used to reference metadata store resources for the purpose
of preventing garbage collection.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Provide a flag which configures a lease to only hold
reference to its given references and ignore label references
during garbage collection rooted from the lease.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Currently the objects which can retain content from labels
are limited. This limitation has required clients to work
around this and and in some cases add outside reference
counting (e.g. buildkit keeping content for snapshots).
Updated the logic to treat content and snapshot labels equally
and simplified the code in the process.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The noop-loop does nothing.
The containerd doesn't have any snapshotter buckets at the beginning.
If user uses specific dir as rootfs to create container, and sets
`snapshotter` key to the container, like `overlayfs` by mistake,
the gc scheduler will try to scan the snapshotter and panic.
In order to avoid this case, remove the noop-loop here.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Allow content ingests to be cleaned up during gc.
Use a default expiration on content ingests or make
use of the lease expiration when provided.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Allow setting an expiration label to have the garbage
collector remove an item after the specified time.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Allows linking to multiple snapshots within the same snapshotter.
Adds support for using slash to separate content as well for consistency.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Marks and sweeps unreferenced objects.
Add snapshot cleanup to metadata.
Add content garbage collection
Add dirty flags for snapshotters and content store which
are set on deletion and used during the next garbage collection.
Cleanup content store backend when content metadata is removed.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
