Use single instance of content store instead of
creating new one for each collection. Using new
instance and wrapping causes failures.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Ensures that all callers and the garbage collector are using
the same lock instances to prevent cleanup of objects
during creation.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Marks and sweeps unreferenced objects.
Add snapshot cleanup to metadata.
Add content garbage collection
Add dirty flags for snapshotters and content store which
are set on deletion and used during the next garbage collection.
Cleanup content store backend when content metadata is removed.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Updates metadata plugin to require content and
snapshotter plugins be loaded and initializes with
those plugins, keeping the metadata database structure
static after initialization. Service plugins now only
require metadata plugin access snapshotter or content
stores through metadata, which was already required
behavior of the services.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Adds back links from parent to children in order to prevent
deletion of a referenced snapshot in a namespace.
Avoid removing snapshot during metadata delete to
prevent shared namespaces from being mistakenly deleted.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
To allow for updating extensions without collisions, we have moved to
using a map type that can be explicitly selected via the field path for
updates. This ensures that multiple parties can operate on their
extensions without stepping on each other's toes or incurring an
inordinate number of round trips.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This field allows a client to store specialized information in the
container metadata rather than having to store this itself and keep
the data in sync with containerd.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Content commit is updated to take in a context, allowing
content to be committed within the same context the writer
was in. This is useful when commit may be able to use more
context to complete the action rather than creating its own.
An example of this being useful is for the metadata implementation
of content, having a context allows tests to fully create
content in one database transaction by making use of the context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The testsuite alters global state by setting the umask, avoid
running the testsuite in parallel and move umask manipulation
to the test suite level to individual tests may run in parallel.
Added better error messaging and handling.
Removed reliance on testing object for handling cleanup failure.
When a cleanup error occurred, it would fail the test but the log
would get skipped. With this change the failure will show up for
the running test.
Update test unmounting to set the detach flag, avoiding races with
btrfs which may see the device as busy when attempting to unmount.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This fixes a few bugs in the container store related to reading and
writing fields. Specifically, on update, the full field set wasn't being
returned to the caller, making it appear that the store was corrupted.
We now return the correctly updated field and store the missing field
that was omitted in the original implementation. In course, we also have
defined the update semantics of each field, as well as whether or not
they are required.
The big addition here is really the container metadata testsuite. It
covers listing, filtering, creates, updates and deletes in a vareity of
scenarios.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This could render tasks for a container unresolvalbe. If there is a use
case for changing the runtime of a container, we should think it through
carefully.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Add commit options which allow for setting labels on commit.
Prevents potential race between garbage collector reading labels
after commit and labels getting set.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
After some analysis, it was found that Content.Reader was generally
redudant to an io.ReaderAt. This change removes `Content.Reader` in
favor of a `Content.ReaderAt`. In general, `ReaderAt` can perform better
over interfaces with indeterminant latency because it avoids remote
state for reads. Where a reader is required, a helper is provided to
convert it into an `io.SectionReader`.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Moves label and timestamp bolt functions to subpackage
for use outside the metadata package without importing
metadata package.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The snapshot test suite is designed to run against the snapshotter
interface, run the test suite for metadata and client implementations
of the snapshotter interface.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Because the lock on an ingest ref being held regardless of whether a
writer was in use, resuming an existing ingest proved impossible. We now
defer writer locking to the content store backend, where the lock will
be released automatically on closing the writer or on restarting
containerd.
There are still cases where a writer can be abandoned but not closed,
leaving an active ingest, but this is extremely rare and can be resolved
with a daemon restart.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Add content test suite with test for writer.
Update fs and metadata implementations to use test suite.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
After review, there are cases where having common requirements for
namespaces and identifiers creates contention between applications. One
example is that it is nice to have namespaces comply with domain name
requirement, but that does not allow underscores, which are required for
certain identifiers.
The namespaces validation has been reverted to be in line with RFC 1035.
Existing identifiers has been modified to allow simply alpha-numeric
identifiers, while limiting adjacent separators.
We may follow up tweaks for the identifier charset but this split should
remove the hard decisions.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Update list content command to support filters
Add label subcommand to content in dist tool to update labels
Add uncompressed label on unpack
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
