Commit Graph

27 Commits

Author SHA1 Message Date
Fu Wei
0975ec0908
Merge pull request #10342 from dmcgowan/add-mutex-fallback-host
Adds a mutex to protect fallback host
2024-06-17 13:13:57 +00:00
Maksym Pavlenko
ab61734e3d
Merge pull request #10231 from jedevc/add-get-token-span
auth: add span to FetchToken helpers
2024-06-14 19:33:52 +00:00
Derek McGowan
38e2f00382
Adds a mutex to protect fallback host
Race detector complains about concurrent access such as with Dispatch on
push.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-06-14 10:43:39 -07:00
Derek McGowan
d23c4b8b53
Use unix and windows specific connection error checks
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-06-05 14:39:57 -07:00
Derek McGowan
02b6c6939f
Allow fallback across default ports
When no port is specified, allow falling back from 443 to 80 when
http is specified along with a TLS configuration.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-06-05 14:39:57 -07:00
Justin Chadwell
9831a62d72 auth: add span to FetchToken helpers
Before this, during a call to the docker resolver, we would generate
span wrappers for each HTTPRequest correctly, however, as the docker
resolver reaches out to the docker authorizer, it could create HTTP
requests (for fetching tokens) that would not be wrapped in any span.

This can result in rather confusing traces, e.g. something like:

	remotes.docker.resolver.HTTPRequest
		HTTP HEAD (fetch index, fails with 401)
	HTTP GET (fetch token)
	remotes.docker.resolver.HTTPRequest
		HTTP HEAD (fetch index)
	remotes.docker.resolver.HTTPRequest
		HTTP GET (fetch manifest)

By adding a span into the FetchToken, this trace becomes a little easier
to consume:

	remotes.docker.resolver.HTTPRequest
		HTTP HEAD (fetch index, fails with 401)
	remotes.docker.resolver.FetchToken
		HTTP GET (fetch token)
	remotes.docker.resolver.HTTPRequest
		HTTP HEAD (fetch index)
	remotes.docker.resolver.HTTPRequest
		HTTP GET (fetch manifest)

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-05-15 15:54:37 +01:00
zouyee
11d8beff80 optimize error logs by providing absolute file paths
Signed-off-by: zouyee <zouyee1989@gmail.com>
2024-04-30 09:08:01 +08:00
Derek McGowan
5e470e1cae
Update HTTPFallback to handle tls handshake timeout
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-04-22 18:53:27 -07:00
Kohei Tokunaga
63d5573a38
remote: Fix HTTPFallback fails when pushing manifest
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2024-04-03 00:28:31 +09:00
Derek McGowan
b8654e36f4
Merge pull request #8379 from jedevc/docker-pusher-concurrency
Fix various timing issues with docker pusher
2024-02-21 17:59:56 +00:00
Akihiro Suda
99721c27e1
Disable the support for Schema 1 images
Schema 1 (`application/vnd.docker.distribution.manifest.v1+prettyjws`) has been
officially deprecated since containerd v1.7 (PR 6884).

We have planned to remove the support for Schema 1 in containerd v2.0, but this
removal may still surprise some users.
So, in containerd v2.0 we will just disable it by default.

The support for Schema 1 can be still enabled by setting an environment variable
`CONTAINERD_ENABLE_DEPRECATED_PULL_SCHEMA_1_IMAGE=1`, however, this workaround
will be completely removed in containerd v2.1.

Schema 2 was introduced in Docker 1.10 (Feb 2016), so most users should
have been already using Schema 2 or OCI.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-02-15 11:11:35 +09:00
Justin Chadwell
a9152ebf89 copy: prevent potential deadlock if close before fully written
We also need an additional check to avoid setting both the error and
response which can create a race where they can arrive in the receiving
thread in either order.

If we hit an error, we don't need to send the response.

> There is a condition where the registry (unexpectedly, not to spec)
> returns 201 or 204 on the put before the body is fully written. I would
> expect that the http library would issue close and could fall into a
> deadlock here. We could just read respC and call setResponse. In that
> case ErrClosedPipe would get returned and Commit shouldn't be called
> anyway.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-03 10:20:11 +01:00
Justin Chadwell
b48e1141eb copy: setError should imply Close
If sending two messages from goroutine X:

	a <- 1
	b <- 2

And receiving them in goroutine Y:

	select {
	case <- a:
	case <- b:
	}

Either branch of the select can trigger first - so when we call
.setError and .Close next to each other, we don't know whether the done
channel will close first or the error channel will receive first - so
sometimes, we get an incorrect error message.

We resolve this by not sending both signals - instead, we can have
.setError *imply* .Close, by having the pushWriter call .Close on
itself, after receiving an error.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-03 10:20:11 +01:00
Justin Chadwell
651cfa2a2c pushWriter: refactor reset pipe logic into separate function
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-03 10:18:05 +01:00
Justin Chadwell
9d7641ff3e copy: improve error detection from closed pipes
If we get io.ErrClosedPipe in pushWriter.Write, there are three possible
scenarios:

- The request has failed, we need to attempt a reset, so we can expect a
  new pipe incoming on pipeC.
- The request has failed, we don't need to attempt a reset, so we can
  expect an incoming error on errC.
- Something else externally has called Close, so we can expect the done
  channel to be closed.

This patch ensures that we block for as long as possible (while still
handling each of the above cases, so we avoid hanging), to make sure
that we properly return an appropriate error message each time.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-03 10:18:05 +01:00
Justin Chadwell
91a50f70b7 copy: check if writer was closed before setting a pipe
If Close is called externally before a request is attempted, then we
will accidentally attempt to send to a closed channel, causing a panic.

To avoid this, we can check to see if Close has been called, using a
done channel. If this channel is ever done, we drop any incoming errors,
requests or pipes - we don't need them, since we're done.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-03 10:18:05 +01:00
Justin Chadwell
4660f63033 copy: remove wrapping io.NopCloser from push writer pipe
io.Pipe produces a PipeReader and a PipeWriter - a close on the write
side, causes an error on both the read and write sides, while a close on
the read side causes an error on only the read side. Previously, we
explicitly prohibited closing from the read side.

However, http.Request.Body requires that "calling Close should unblock a
Read waiting for input". Our reader will not do this - calling close
becomes a no-op. This can cause a deadlock because client.Do may never
terminate in some circumstances.

We need the Reader side to close its side of the pipe as well, which it
already does using the go standard library - otherwise, we can hang
forever, writing to a pipe that will never be closed.

Allowing the requester to close the body should be safe - we never reuse
the same reader between requests, as the result of body() will never be
reused by the guarantees of the standard library.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-02-03 10:18:05 +01:00
Phil Estes
f5f84a9c75
Merge pull request #8735 from iain-macdonald/iain-macdonald/issue-6377
remotes/docker/authorizer.go: refresh OAuth tokens when they expire
2024-01-29 16:18:56 +00:00
Derek McGowan
fb9b59a843
Switch to new errdefs package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-25 22:18:45 -08:00
Iain Macdonald
af6a90bf5c remotes/docker/authorizer.go: invalidate auth tokens when they expire.
Signed-off-by: Iain Macdonald <xiainx@gmail.com>
2024-01-23 09:58:57 -08:00
Akihiro Suda
e9f2bba1f2
remotes: FetchByDigest: propagate media type from config to desc
A media type string passed via `WithMediaType()` was not propagated
to a descriptor returned by `FetchByDigest()`.

Follow-up to PR 8744

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-18 23:50:21 +09:00
Derek McGowan
8f0eb26311
Move tracing to pkg/tracing
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:56:25 -08:00
Derek McGowan
fdb8a527c9
Move reference to pkg/reference
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:55:58 -08:00
Derek McGowan
b76236bb45
Move labels to pkg/labels
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:55:30 -08:00
Derek McGowan
44a836c9b5
Move errdefs to pkg/errdefs
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:54:45 -08:00
Derek McGowan
8e14c39e80
Move archive to pkg/archive
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:54:18 -08:00
Derek McGowan
0dabf6f154
Move remotes to core/remotes
Signed-off-by: Derek McGowan <derek@mcg.dev>
2024-01-17 09:52:21 -08:00