This moves both the Mount type and mountinfo into a single mount
package.
This also opens up the root of the repo to hold the containerd client
implementation.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Remove rootfs service in place of snapshot service. Adds
diff service for extracting and creating diffs. Diff
creation is not yet implemented. This service allows
pulling or creating images without needing root access to
mount. Additionally in the future this will allow containerd
to ensure extractions happen safely in a chroot if needed.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
No need to have a util package under a package... er, package.
We should move most of this functionality to continuity.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Rollback was not being called when the function was called with
a key which does not exist. This failure to call rollback kept
the database open and caused all new requests to block.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The split between provider and ingester was a long standing division
reflecting the client-side use cases. For the most part, we were
differentiating these for the algorithms that operate them, but it made
instantation and use of the types challenging. On the server-side, this
distinction is generally less important. This change unifies these types
and in the process we get a few benefits.
The first is that we now completely access the content store over GRPC.
This was the initial intent and we have now satisfied this goal
completely. There are a few issues around listing content and getting
status, but we resolve these with simple streaming and regexp filters.
More can probably be done to polish this but the result is clean.
Several other content-oriented methods were polished in the process of
unification. We have now properly seperated out the `Abort` method to
cancel ongoing or stalled ingest processes. We have also replaced the
`Active` method with a single status method.
The transition went extremely smoothly. Once the clients were updated to
use the new methods, every thing worked as expected on the first
compile.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This adds pause and unpause to containerd's execution service and the
same commands to the `ctr` client.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
To allow the querying of usage for snapshots, we define a new method on
the snapshotter to query the resources in use by a single snapshot.
Conversely, it can be said that if the snapshot was deleted, the
reported amount of usage would be recovered.
There are few problems with this model in the implementation of btrfs
that need to be worked out. In btrfs, it is hard to resolve the amount
of data usage with the use of quotas but these may report valuables that
are incompatible with the model.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
When starting up a snapshot driver on subsequent runs, the
mkdir call will return an exist error, this can be safely
ignored.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
We need to set +x on the overlay dirs or after dropping from root to a
non-root user an eperm will happen on exec or other file access
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Ensure undo failures are logged in a consistent way.
Remove created subvolume on commit failure on create.
Ensure that removed value get logged on delete failure.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Adds a zero byte separator on composite key.
Stores child key in back reference to allow following.
Add parent bucket to get/create bucket functions.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Two test cases added
- Prepare, Stat on new layer, should return Active layer.
- Prepare & Commit , Stat on new layer, should return Committed layer
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Allow creating actives without an upper directory for
capturing changes. Actives without the upper directory
will always be mounted read only. Read only actives
must have a parent.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
After reviewing this closer, we removed a few buffers in favor
`(*exec.Command).Output` and simplified the loop file creation method to
just use `(*os.File).Truncate`.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Updates the btrfs snapshotter to meet the interface and current tests.
Mostly, we merge the keyspace into a common index. Like with the overlay
driver, we will still need to do more verification work to ensure
idempotence of key collisions.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
With the change to the snapshotter interface, we've now updated the
overlay driver to follow the conventions of the current test suite. To
support key unification, an hashed index was added to active and
committed directories. We still need to do some testing around
collisions, but we'll leave that for a future PR.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
After receiving feedback on the `snapshot.Driver` interface, it was
found that the behavior of active and committed snapshots was confusing.
We attempt to clean this up by doing the following:
1. Define the concept of "active" and "committed" snapshots and their
lifecycle relationship. Active snapshots can be created from a parent.
Committed snapshots can only be created from active snapshots.
2. Only committed snapshots can be a parent.
3. Unify the keyspace of snapshots. For common operations, such as
removal and stat, we only have a single method that works for both
active and committed snapshots. For methods that take one or the other,
the restriction is called out. `Remove` and `Delete` are consolidated
for this purpose.
4. Define the `Info` data type to include name, parent, kind and
readonly state. This allows us to collect `Exists` and `Parent` into a
single method `Stat` and simplifies the `Walk` method, eliding `Active`.
5. The `Driver` has been renamed to `Snapshotter` due to the overuse of
the term `Driver`.
Effectively, we now have snapshots that are either active or committed.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
We now include btrfs in the snapshot driver test suite. This includes
the addition of parent links and name hashing into the btrfs driver.
We'll probably endup replacing this with a common metadata store, as
these relationships are generally identical between implementations.
A small bug was discovered in the delete implementation in the course
testing, so the btrfs package has been updated with a fix.
The overlay driver was modified accordingly with the btrfs driver to use
`Driver` as the exported type of each driver packge.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
We now define the `snapshot.Driver` interface based on earlier work.
Many details of the model are worked out, such as snapshot lifecycle and
parentage of commits against "Active" snapshots.
The impetus of this change is to provide a snapshot POC that does a
complete push/pull workflow. The beginnings of a test suite for snapshot
drivers is included that we can use to verify the assumptions of
drivers. The intent is to port the existing tests over to this test
suite and start scaling contributions and test to the snapshot driver
subsystem.
There are still some details that need to be worked out, such as listing
and metadata access. We can do this activity as we further integrate
with tooling.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Also remove the target from the Mount struct because it should not be
used at all. The target can be variable and set by a caller, not by the
snapshot drivers.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
