When client uses Pull action to pull image, it will limit the number of
manifest as one. But Unpack action will call Manifest to traverse all
the manifests including non-dowloaded one. If the platform has more than
one manifest, the Pull with unpack action will fail. And also, there is
no need to read non-best matched manifest. Therefore, the Manifest can
do the sort earlier.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Remove annotation prefix and add multiple index records
for manifests with multiple image names. This makes the
custom annotation more consistent with the OCI image
annotation. Additionally, ensure the OCI image annotation
always represents the tag (partial image name) as recommended
by the specification. The containerd image name annotation
will always contain the full image name.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Ensure the manifest which gets created using the Docker
compatibility code compresses the blob before creating
the manifest. This ensures consistency with manifests
used by Docker.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Using the cri utility caused other project to have
containerd/cri as a dependency, only for this utility.
The new `reference.ParseDockerRef` function does the
same (other than having a different name).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Avoid filtering manifests by platform when a non-index
manifest is provided to the Manifest function.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Adds a manifest filter for pulling which ensures only one
manifest from a manifest list is pulled even when multiple matches.
Removes unused filter platform list.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Adds a new platform interface for matching and comparing platforms.
This new interface allows both filtering and ordering of platforms
to support running multiple platform and choosing the best platform.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.
No OCI descriptor field is written to the store.
No change on gRPC API.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Fix issue where manifest content must always be fetched
even if it is already fully downloaded or shared locally.
Simplify children label setting and platform filtering.
Prevent getting a fetcher when content shared locally.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This linter checks for unnecessary type convertions.
Some convertions are whitelisted because their type is different
on 32bit platforms
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
- Use lease API (previoisly, GC was not supported)
- Refactored interfaces for ease of future Docker v1 importer support
For usage, please refer to `ctr images import --help`.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Synchronous image delete provides an option image delete to wait
until the next garbage collection deletes after an image is removed
before returning success to the caller.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The `Check` function returns information about an image's content components
over a content provider. From this information, one can tell which content is
required, present or missing to run an image.
The utility can be demonstrated with the `check` command:
```console
$ ctr images check
REF TYPE DIGEST STATUS SIZE
docker.io/library/alpine:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d incomplete (1/2) 1.5 KiB/1.9 MiB
docker.io/library/postgres:latest application/vnd.docker.distribution.manifest.v2+json sha256:2f8080b9910a8b4f38ff5a55a82e77cb43d88bdbb16d723c71d18493590832e9 complete (13/13) 99.3 MiB/99.3 MiB
docker.io/library/redis:alpine application/vnd.docker.distribution.manifest.v2+json sha256:e633cded055a94202e4ccccb8125b7f383cd6ee56527ab890db643383a2647dd incomplete (6/7) 8.1 MiB/10.0 MiB
docker.io/library/ubuntu:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:60f835698ea19e8d9d3a59e68fb96fb35bc43e745941cb2ea9eaf4ba3029ed8a unavailable (0/?) 0.0 B/?
docker.io/trollin/busybox:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:54a6424f7a2d5f4f27b3d69e5f9f2bc25fe9087f0449d3cb4215db349f77feae complete (2/2) 699.9 KiB/699.9 KiB
```
The above shows us that we have two incomplete images and one that is
unavailable. The incomplete images are those that we know the complete
size of all content but some are missing. "Unavailable" means that the
check could not get enough information about the image to get its full
size.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Add support for downloading layers with external URLs and
foreign/non-distributable mediatypes. This ensures that encountered
windows images are downloaded correctly. We still need to filter out the
extra windows resources when pulling linux, but this is a step towards
correctly supporting multi-platform images.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Fixes pulling of multi-arch images by limiting the expansion
of the index by filtering to the current default platform.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
After some analysis, it was found that Content.Reader was generally
redudant to an io.ReaderAt. This change removes `Content.Reader` in
favor of a `Content.ReaderAt`. In general, `ReaderAt` can perform better
over interfaces with indeterminant latency because it avoids remote
state for reads. Where a reader is required, a helper is provided to
convert it into an `io.SectionReader`.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The primary feature we get with this PR is support for filters and
labels on the image metadata store. In the process of doing this, the
conventions for the API have been converged between containers and
images, providing a model for other services.
With images, `Put` (renamed to `Update` briefly) has been split into a
`Create` and `Update`, allowing one to control the behavior around these
operations. `Update` now includes support for masking fields at the
datastore-level across both the containers and image service. Filters
are now just string values to interpreted directly within the data
store. This should allow for some interesting future use cases in which
the datastore might use the syntax for more efficient query paths.
The containers service has been updated to follow these conventions as
closely as possible.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This PR ensures that we can pull images with manifest lists, aka OCI
indexes. After this change, when pulling such an image, the resources
will all be available for creating the image.
Further support is required to do platform based selection for rootfs
creation, so such images may not yet be runnable. This is mostly useful
for checkpoint transfers, which use an OCI index for assembling the
component set.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The implementations for the storage of metadata have been merged into a
single metadata package where they can share storage primitives and
techniques. The is a requisite for the addition of namespaces, which
will require a coordinated layout for records to be organized by
namespace.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Working from feedback on the existing implementation, we have now
introduced a central metadata object to represent the lifecycle and pin
the resources required to implement what people today know as
containers. This includes the runtime specification and the root
filesystem snapshots. We also allow arbitrary labeling of the container.
Such provisions will bring the containerd definition of container closer
to what is expected by users.
The objects that encompass today's ContainerService, centered around the
runtime, will be known as tasks. These tasks take on the existing
lifecycle behavior of containerd's containers, which means that they are
deleted when they exit. Largely, there are no other changes except for
naming.
The `Container` object will operate purely as a metadata object. No
runtime state will be held on `Container`. It only informs the execution
service on what is required for creating tasks and the resources in use
by that container. The resources referenced by that container will be
deleted when the container is deleted, if not in use. In this sense,
users can create, list, label and delete containers in a similar way as
they do with docker today, without the complexity of runtime locks that
plagues current implementations.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Update go-runc to 49b2a02ec1ed3e4ae52d30b54a291b75
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add shim to restore creation
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Keep checkpoint path in service
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add C/R to non-shim build
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Checkpoint rw and image
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Pause container on bind checkpoints
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Return dump.log in error on checkpoint failure
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Pause container for checkpoint
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Update runc to 639454475cb9c8b861cc599f8bcd5c8c790ae402
For checkpoint into to work you need runc version
639454475cb9c8b861cc599f8bcd5c8c790ae402 + and criu 3.0 as this is what
I have been testing with.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Move restore behind create calls
This remove the restore RPCs in favor of providing the checkpoint
information to the `Create` calls of a container. If provided, the
container will be created/restored from the checkpoint instead of an
existing container.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Regen protos after rebase
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Server and Client images of the image store are now provided. We have
created an image metadata interface and converted the bolt functions to
implement that interface over an transaction. A remote client
implementation is provided that implements the same interface.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This adds very simple deletion of images by name. We still need to
consider the approach to handling image name, so this may change. For
the time being, it allows one to delete an image entry in the metadata
database.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Allow usage of the experimental docker resolver as a package. There are
very few changes to the consuming code, demonstrating the effectiveness
of the abstraction. This move will allow future contributions to a more
featured resolver implementation.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
As a demonstration of the power of the visitor implementation, we now
report the image size in the `dist images` command. This is the size of
the packed resources as would be pushed into a remote. A similar method
could be added to calculate the unpacked size.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
