In the refactor from 926b9c72f61b5be6bf8d952512f1d0932fbaf898, the error
handling was substantially reworked, and changed the types of errors
returned.
Notably, in the case of a network error, instead of propogating the
error through to return from pushWriter.Write (as previously), it would
be propagated through to pushWriter.Commit - however, this is too late,
since we've already closed the io.Pipe by the time we would have reached
this function. Therefore, we get the generic error message "io:
read/write on closed pipe" for *every network error*.
This patch corrects this behavior to ensure that the correct error
object is always returned as early as possible, by checking the error
result after writing and detecting a closed pipe.
Additionally, we do some additional hardening - specifically we prevent
falling through when resetting the content or detecting errors, and
update the tests to explicitly check for the ErrReset message.
Signed-off-by: Justin Chadwell <me@jedevc.com>
when a put request is retried due to the response from registry,
the body of the request should be seekable. A dynamic pipe is added
to the body so that the content of the body can be read again.
Currently a maximum of 5 resets are allowed, above which will fail the
request. A new error ErrReset is introduced which informs that a
reset has occured and request needs to be retried.
also added tests for Copy() and push() to test the new functionality
Signed-off-by: Akhil Mohan <makhil@vmware.com>
`dockerPusher` provides `pushWriter` which implements `content.Writer`.
However, even if `pushWriter` become abort status (i.e. `Close()` is called
before `Commit()`), `dockerPusher` doesn't recognise that status and treats that
writer as on-going.
This behaviour doesn't allow the client to retry an aborted push.
This commit fixes this issue.
This commit also adds an test to ensure that the issue is fixed.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Discovered this while using HTTP tracing via OpenTelemetry inside of
buildkitd, where the trace spans were not being reported for the
registry PUT http requests. The spans are only reported on the Close
for the Body, after adding these Close calls, the spans are reported as
expected.
Signed-off-by: coryb <cbennett@netflix.com>
The `(dockerPusher).Push` method uses a `StatusTracker` to check if an
upload already happened, before repeating the upload. However, there is
no provision for failure handling. If a PUT request returns an error,
the `StatusTracker` will still see the upload as if it happened
successfully. Add a status boolean so that only successful uploads
short-circuit `Push`.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
`docker.Authorizer` requires library clients to configure scope via context.
It is helpful for the clients to use the helper (currently private) functions
for generating scope string and to use that function with the combination of
other scope-related ones (e.g. `docker.WithScope`).
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
This accomplishes a few long-standing TODO items, but also helps users
in showing exact registry error messages
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
Dependencies may be switching to use the new `%w` formatting
option to wrap errors; switching to use `errors.Is()` makes
sure that we are still able to unwrap the error and detect the
underlying cause.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When pushing a manifest list, all manifests should be pushed by digest
and only the final manifest pushed by tag. The Pusher was preventing
this by mistakenly disallowing objects to contain a digest. When objects
have a digest, only push tags associated with that digest.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Go documentation says
`Use of GetBody still requires setting Body`.
This change ensures the body is always set in
addition to GetBody. This fixes a bug where
sometimes the body is nil.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Adds support for registry mirrors
Adds support for multiple pull endpoints
Adds capabilities to limit trust in public mirrors
Fixes user agent header missing
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
With distribution source label in content store, select the longest
common prefix components as condidate mount blob source and try to push
with mount blob.
Fix#2964
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Currently pushing a new tag to a manifest which already
exists in the registry skips the tag push because it
only checks that the manifest exists. This updates the
logic to instead check if the tag exists and is at the
same digest.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Content commit is updated to take in a context, allowing
content to be committed within the same context the writer
was in. This is useful when commit may be able to use more
context to complete the action rather than creating its own.
An example of this being useful is for the metadata implementation
of content, having a context allows tests to fully create
content in one database transaction by making use of the context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Support registries returning 204 or 200 in place of 201/202.
Ensure body is closed when request is retried.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Add commit options which allow for setting labels on commit.
Prevents potential race between garbage collector reading labels
after commit and labels getting set.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Now that we have most of the services required for use with containerd,
it was found that common patterns were used throughout services. By
defining a central `errdefs` package, we ensure that services will map
errors to and from grpc consistently and cleanly. One can decorate an
error with as much context as necessary, using `pkg/errors` and still
have the error mapped correctly via grpc.
We make a few sacrifices. At this point, the common errors we use across
the repository all map directly to grpc error codes. While this seems
positively crazy, it actually works out quite well. The error conditions
that were specific weren't super necessary and the ones that were
necessary now simply have better context information. We lose the
ability to add new codes, but this constraint may not be a bad thing.
Effectively, as long as one uses the errors defined in `errdefs`, the
error class will be mapped correctly across the grpc boundary and
everything will be good. If you don't use those definitions, the error
maps to "unknown" and the error message is preserved.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
These interfaces allow us to preserve both the checking of error "cause"
as well as messages returned from the gRPC API so that the client gets
full error reason instead of a default "metadata: not found" in the case
of a missing image.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Split resolver to only return a name with separate methods
for getting a fetcher and pusher. Add implementation for
push.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
