Add three methods for lease service so that the client can use it to
manage the resource by lease, not just gc.root label. With the following
methods, it is easy for client to maintain their own cache system.
```
- AddResource(context.Context, Lease, Resource) error
- RemoveResource(context.Context, Lease, Resource) error
- ListResources(context.Context, Lease) ([]Resource, error)
```
And the resource is to be
```golang
type Resource {
ID string
Type string
}
```
For the snapshots, the Type field will be formatted by
snapshots/%{type}, like snapshots/overlayfs.
fix: #3295
Signed-off-by: Wei Fu <fuweid89@gmail.com>
The namespaced writer Commit method must always have a
namespace in the context as checked by the removeIngestLease
function, resulting in a panic when not provided.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Extend the adaptImage function with a case for handling the annotations
so they can be used in the filter adaptors for fieldpaths.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Currently the objects which can retain content from labels
are limited. This limitation has required clients to work
around this and and in some cases add outside reference
counting (e.g. buildkit keeping content for snapshots).
Updated the logic to treat content and snapshot labels equally
and simplified the code in the process.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Adds comment showing the structure of the buckets
as stored in boltdb. Includes all bucket names, key names
and descriptions of the stored values.
Moves the metadata comment to package level to be exposed
in the godoc overview.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The noop-loop does nothing.
The containerd doesn't have any snapshotter buckets at the beginning.
If user uses specific dir as rootfs to create container, and sets
`snapshotter` key to the container, like `overlayfs` by mistake,
the gc scheduler will try to scan the snapshotter and panic.
In order to avoid this case, remove the noop-loop here.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.
close: #2945
Signed-off-by: Wei Fu <fuweid89@gmail.com>
This changeset modifies the metadata store to allow one to set a
"content sharing policy" that defines how blobs are shared between
namespaces in the content store.
The default mode "shared" will make blobs available in all namespaces
once it is pulled into any namespace. The blob will be pulled into
the namespace if a writer is opened with the "Expected" digest that
is already present in the backend.
The alternative mode, "isolated" requires that clients prove they have
access to the content by providing all of the content to the ingest
before the blob is added to the namespace.
Both modes share backing data, while "shared" will reduce total
bandwidth across namespaces, at the cost of allowing access to any
blob just by knowing its digest.
Note: Most functional codes and changelog of this commit originate from
Stephen J Day <stephen.day@docker.com>, see
40455aade8Fixes#1713Fixes#2865
Signed-off-by: Eric Lin <linxiulei@gmail.com>
In many cases code is calling errors.Wrapf with an arbitrary string
instead of a format string. This causes confusing errors when the
wrapped error message contains '%' characters.
This change replaces such calls with calls to errors.Wrap.
Signed-off-by: John Starks <jostarks@microsoft.com>
The local store could end up in a state where the writer is
closed but the reference is locked after a commit on an
existing object.
Cleans up Commit logic to always close the writer even after
an error occurs, guaranteeing the reference is unlocked after commit.
Adds a test to the content test suite to verify this behavior.
Updates the content store interface definitions to clarify the behavior.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Allow content ingests to be cleaned up during gc.
Use a default expiration on content ingests or make
use of the lease expiration when provided.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Allow setting an expiration label to have the garbage
collector remove an item after the specified time.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.
No OCI descriptor field is written to the store.
No change on gRPC API.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Allows linking to multiple snapshots within the same snapshotter.
Adds support for using slash to separate content as well for consistency.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Allows the client to choose the context to finish the lease.
This allows the client to switch contexts when the main context
used to the create the lease may have been cancelled.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Update content ingests to use content from another namespace.
Ingests must be committed to make content available and the
client will see the sharing as an ingest which has already
been fully written to, but not completed.
Updated the database version to change the ingest record in
the database from a link key to an object with a link and
expected value. This expected value is used to indicate that
the content already exists and an underlying writer may
not yet exist.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Let the test runners choose the namespaces and
wrap the contexts. This allows the test suite to create
multiple contexts without worrying about namespacing
or leasing in the contexts.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Updates overlay remove to simply remove the reference, adds
a cleanup method for discarding the directory.
Updates snapshot create to setup the directory structure while
in the transaction, to prevent cleanup from removing directories
which are part of a create.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
