This makes it easy for share functionality across tools to prevent
pushing layers that are not supposed to be re-distributed.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
`dockerPusher` provides `pushWriter` which implements `content.Writer`.
However, even if `pushWriter` become abort status (i.e. `Close()` is called
before `Commit()`), `dockerPusher` doesn't recognise that status and treats that
writer as on-going.
This behaviour doesn't allow the client to retry an aborted push.
This commit fixes this issue.
This commit also adds an test to ensure that the issue is fixed.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Test are being added for GenerateTokenOptions to cover multiple scope cases and the error cases handled.
Signed-off-by: Jacob MacElroy <jacob@okteto.com>
Currently scopes added to token options are added with all scopes included in space delimited string. This changes it so that each scope is added to the string slice as a separate string. This seems to be the desire behavior based on the fact that a string slice is used and the usage of this function in github.com/moby/buildkit.
Signed-off-by: Jacob MacElroy <jacob@okteto.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Currently, containerd doesn't restart pull when it encounters unexpected EOF of
blob strem withtout error codes.
There are cases where this lead to pull failure.
This commit tries to fix this issue.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Remove build tags which are already implied by the name of the file.
Ensures build tags are used consistently
Signed-off-by: Derek McGowan <derek@mcg.dev>
Discovered this while using HTTP tracing via OpenTelemetry inside of
buildkitd, where the trace spans were not being reported for the
registry PUT http requests. The spans are only reported on the Close
for the Body, after adding these Close calls, the spans are reported as
expected.
Signed-off-by: coryb <cbennett@netflix.com>
Adds support for mirrors which are non-compliant with the
OCI distribution specification but have previously mirrored
content with a namespace prefix after the API root `/v2`.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Previously, containerd uses Go's default UA "Go-http-client/1.1" while authenticating with registry.
This commit changes it to the pattern like "containerd/v1.5.2" which is used for all other requests.
Signed-off-by: Ethan Chen <pov@mahou-shoujo.moe>
Allows redirects to be authorized if authorization is provided
for the redirected to host. The authorization will always go
to the redirect and never to the referrer.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The `(dockerPusher).Push` method uses a `StatusTracker` to check if an
upload already happened, before repeating the upload. However, there is
no provision for failure handling. If a PUT request returns an error,
the `StatusTracker` will still see the upload as if it happened
successfully. Add a status boolean so that only successful uploads
short-circuit `Push`.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
This allows a pusher to be used for more than one tag without creating a
new resolver/pusher. The current implementation checks the ref key
tracker status based on type and hash and will skip the push even if the
repository reference is unique.
Signed-off-by: Phil Estes <estesp@amazon.com>
- changed the `MatchLocalhost` function in remotes/docker/registry.go
- Make use of SplitHostPort to split host and port number
- Added additional tests for modified code in remotes/docker/registry_test.go
- Note: this does not handle mathcing of IP addresses in octal, decimal or hex format or a mix of these.
Fixes: #5129
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
The certs dir parsing was skipping over files instead of reading them,
as such the certs would never load.
It was also stating the file name rather than the full path for cert
pairs.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
`docker.Authorizer` requires library clients to configure scope via context.
It is helpful for the clients to use the helper (currently private) functions
for generating scope string and to use that function with the combination of
other scope-related ones (e.g. `docker.WithScope`).
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
This accomplishes a few long-standing TODO items, but also helps users
in showing exact registry error messages
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
Proxy registries are designed to serve content from upstreams.
However, the proxy hostname will usually not match the hostname
of the upstream, requiring the proxy to only use a single
upstream or use its own pattern matching to determine the upstream.
To solve this issue, the client will pass along the namespace which
is being used for the request, allowing mirrors to easily map
to multiple upstreams. This query parameter can safely be ignored
if multiple upstreams are not supported.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Authorizer interface can’t be really implemented because
scopes are passed in on a side channel via private value in context.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
When a server is specified at the top level, there is a bug
that prevents the keys from being checked properly.
When no server is provided, the server attempts to parse
with an empty host, leaving partial values and a defaulted
skip verify configuration.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The `DualStack` option was deprecated in Go 1.12, and is now enabled by default
(through commit github.com/golang/go@efc185029bf770894defe63cec2c72a4c84b2ee9).
> The Dialer.DualStack field is now meaningless and documented as deprecated.
>
> To disable fallback, set FallbackDelay to a negative value.
The default `FallbackDelay` is 300ms; to make this more explicit, this patch
sets `FallbackDelay` to the default value.
Note that Docker Hub currently does not support IPv6 (DNS for registry-1.docker.io
has no AAAA records, so we should not hit the 300ms delay).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Dependencies may be switching to use the new `%w` formatting
option to wrap errors; switching to use `errors.Is()` makes
sure that we are still able to unwrap the error and detect the
underlying cause.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If there is not specific host config, like ctr does, the resolver will
fail to get host path. And this patch is to add default host config if
needs.
And default config host config should have all caps for pull and push.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Moved registry host configuration to the config package
and allows support of loading configurations from a
directory when the hosts are being resolved.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Add configuration toml file format and configuration
function to configure registry hosts from a directory
based configuration. Compatible with Docker registry
certificate loading.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Add `remotes/certutil` functions for loading `ca.crt`, `client.cert`, and `client.key` into `tls.Config` from a directory like `/etc/docker/certs.d/<hostname>.
See https://docs.docker.com/engine/security/certificates/ .
Client applications including CRI plugin are expected to configure the resolver using these functions.
As an example, the `ctr` tool is extended to support `ctr images pull --certs-dir=/etc/docker/certs.d example.com/foo/bar:baz`.
Tested with Harbor 1.8.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This commit improves the fallback behaviour when resolving and
fetching images with multiple hosts. If an error is encountered
when resolving and fetching images, and more than one host is being
used, we will try the same operation on the next host. The error
from the first host is preserved so that if all hosts fail, we can
display the error from the first host.
fixes#3850
Signed-off-by: Alex Price <aprice@atlassian.com>
Registries may allow using token authorization without
explicitly setting the scope. This may cover use cases where
no scope is required for an endpoint or the registry is only
covering authentication using the token. This aligns with the
oauth2 spec which specifies the scope as optional.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Avoid directly handling media types with "+" attributes,
instead handling the base and passing through the full
media type to the appropriate stream processor or decompression.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Currently auth.docker.io uses a custom auth scope for (docker) plugins
`repository(plugin):<repo>:<perms>`.
This makes it impossible to use containerd distribution tooling to fetch
plugins without also supplying a totally custom authorizer.
This changes allows clients to set the correct scope on the context.
It's a little bit nasty but "works".
I'm also a bit suspect of some a couple of these unexported context
functrions. Before the primary one used `contextWithRepositoryScope`
overwrites any scope value and there is another one that appends the
scope value.
With this change they both append...
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Adds a method for setting a custom media type key prefix used by the
fetch handler.
This allows both overwriting a built-in prefix (for reasons?) as well as
supplying a custom media type.
I added this because I was getting an error on `FetchHandler` when
pulling docker plugin images which have their own media type.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
