Even though application/octet-stream issue has been fixed in docker,
there exists lots of images which contains the invalid mediatype.
In order to pull those images, containerd client side modifies the
manifest content before insert/update image reference.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
containerd should cache empty label for docker schema1 image.
if not, the original empty layer will be non-empty layer and the image
config will be changed too. in this case, the image ID will be changed.
check the blob empty label to avoid changing image ID when repull docker
schema1 image.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.
No OCI descriptor field is written to the store.
No change on gRPC API.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Updates blob writer helper to use new open and ensure
unavailable errors are always handled.
Removes duplication of unavailable handling code.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This fix adds support for image registries that expect authentication for POST /v2/token such as used by the GET. E.g., JFrog Artifactory y has been observed to respond with a 401 (Unauthorized) in that case. Adding 401 in addition to the current handling of 405 and 404 in the resolver solves the authentication problem. Finally, this enables image pulls also for Artifactory.
Signed-off-by: Ruediger Maass <ruediger.maass@de.ibm.com>
Schema1 manifests did not set a size in the digest for the blobs,
breaking the expectations of the update http seeking reader. Now
the http seeker has been updated to support unknown size as a
value of negative 1 and the schema1 puller sets the unknown size
accordingly.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Currently pushing a new tag to a manifest which already
exists in the registry skips the tag push because it
only checks that the manifest exists. This updates the
logic to instead check if the tag exists and is at the
same digest.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
To support resumable download, the fetcher for a remote must implement
`io.Seeker`. If implemented the `content.Copy` function will detect the
seeker and begin from where the download was terminated by a previous
attempt.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
To allow concurrent pull of images of the v1 persuasion, we need to
backoff when multiple pullers are trying to operate on the same
resource. The back off logic is ported to v1 pull to match the behavior
for other images.
A little randomness is also added to the backoff to prevent thundering
herd and to reduce expected recovery time.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Prevents a server from sending a large response causing containerd to
allocate too much RAM and potentially OOM.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Add support for downloading layers with external URLs and
foreign/non-distributable mediatypes. This ensures that encountered
windows images are downloaded correctly. We still need to filter out the
extra windows resources when pulling linux, but this is a step towards
correctly supporting multi-platform images.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Content commit is updated to take in a context, allowing
content to be committed within the same context the writer
was in. This is useful when commit may be able to use more
context to complete the action rather than creating its own.
An example of this being useful is for the metadata implementation
of content, having a context allows tests to fully create
content in one database transaction by making use of the context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Support registries returning 204 or 200 in place of 201/202.
Ensure body is closed when request is retried.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Add commit options which allow for setting labels on commit.
Prevents potential race between garbage collector reading labels
after commit and labels getting set.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
After some analysis, it was found that Content.Reader was generally
redudant to an io.ReaderAt. This change removes `Content.Reader` in
favor of a `Content.ReaderAt`. In general, `ReaderAt` can perform better
over interfaces with indeterminant latency because it avoids remote
state for reads. Where a reader is required, a helper is provided to
convert it into an `io.SectionReader`.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Now that we have most of the services required for use with containerd,
it was found that common patterns were used throughout services. By
defining a central `errdefs` package, we ensure that services will map
errors to and from grpc consistently and cleanly. One can decorate an
error with as much context as necessary, using `pkg/errors` and still
have the error mapped correctly via grpc.
We make a few sacrifices. At this point, the common errors we use across
the repository all map directly to grpc error codes. While this seems
positively crazy, it actually works out quite well. The error conditions
that were specific weren't super necessary and the ones that were
necessary now simply have better context information. We lose the
ability to add new codes, but this constraint may not be a bad thing.
Effectively, as long as one uses the errors defined in `errdefs`, the
error class will be mapped correctly across the grpc boundary and
everything will be good. If you don't use those definitions, the error
maps to "unknown" and the error message is preserved.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
These interfaces allow us to preserve both the checking of error "cause"
as well as messages returned from the gRPC API so that the client gets
full error reason instead of a default "metadata: not found" in the case
of a missing image.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
The size and throwaway fields in the history can bother be
omitted, making the emptiness of a layer ambiguous. In these
cases download and check whether the content is empty.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Split resolver to only return a name with separate methods
for getting a fetcher and pusher. Add implementation for
push.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Tests resolving, fetching, and using the various authentication
methods supported by the Docker registry protocol.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Allow usage of the experimental docker resolver as a package. There are
very few changes to the consuming code, demonstrating the effectiveness
of the abstraction. This move will allow future contributions to a more
featured resolver implementation.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
