For remote snapshotter cases it's quite often there is need to pass extra info
from client (for instance - registry URL to query remote layer from, credentials, etc).
This commit slightly extends WithPullSnapshotter to pass extra labels to a snapshotter.
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
When pull image with unpack option, the fetch action will defer blobs
download until unpack. If create image record in ImageService before
blobs download, the following requests to use image will fail because
there is still missing blobs download.
In order to fix concurrent issue, need to create image record after
blobs download.
Fix: #3937
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Though containerd gives ChainID to backend snapshotters during unpack for
searching snapshots to be skipped downloading the contents, ChainID isn't enough
for some snapshotters which require additional information of layers.
Some examples are remote snapshotters which is based on stargz filesystem
(requires image-related information to query the contents to docker registry)
and those which is based on CernVM-FS (requires manifest digest, etc. for
providing squashed rootfs).
This commit solves this issue by enabling a handler to inject additional
information of layers to snapshotters during unpack.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Moves the content fetching into the unpack process
and defers the download until the snapshot needs it
and is ready to apply. As soon as a layer is reached
which requires fetching, all remaining layers are
fetched.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
- Add `all-manifests` option to both `ctr content fetch` and `ctr
images pull`. By default it is false.
- This option ties to `AppendDistributionSourceLabel` in client.
Signed-off-by: Yu Yi <yiyu@google.com>
We can use cross repository push feature to reuse the existing blobs in
the same registry. Before make push fast, we know where the blob comes
from.
Use the `containerd.io/distribution.source. = [,]` as label format. For
example, the blob is downloaded by the docker.io/library/busybox:latest
and the label will be
containerd.io/distribution.source.docker.io = library/busybox
If the blob is shared by different repos in the same registry, the repo
name will be appended, like:
containerd.io/distribution.source.docker.io = library/busybox,x/y
NOTE:
1. no need to apply for legacy docker image schema1.
2. the concurrent fetch actions might miss some repo names in label, but
it is ok.
3. it is optional. no need to add label if the engine only uses images
not push.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Gives clients more control of the pull process, allowing
the client to operate on a descriptor after it has been
pulled. This could be useful for filtering output or
tracking children before they dispatched to. This can
also be used to call custom unpackers to have visibility
into a pulled config in parallel to the downloads.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
