All components need to use a consistent `semconv` version or OTel
will emit errors about "cannot merge resource due to conflicting Schema URL".
Switch to the appropriate semconv version, which requires dropping
usage of `httpconv`. Instead, the upstream HTTP client hooks are
used directly. (The lower-level functions are no longer exported by
OTel.)
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Close connection if no more data. It's to fix false alert filed by image
pull progress.
```
dst = OpenWriter (--> Content Store)
src = Fetch
Open (--> Registry)
Mark it as active request
Copy(dst, src) (--> Keep updating total received bytes)
^
| (Active Request > 0, but total received bytes won't be updated)
v
defer src.Close()
content.Commit(dst)
```
Before migrating to transfer service, CRI plugin doesn't limit global
concurrent downloads for ImagePulls. Each ImagePull requests have 3 concurrent
goroutines to download blob and 1 goroutine to unpack blob. Like ext4
filesystem [1][1], the fsync from content.Commit may sync unrelated dirty pages
into disk. The host is running under IO pressure, and then the content.Commit
will take long time and block other goroutines. If httpreadseeker
doesn't close the connection after io.EOF, this connection will be
considered as active. The pull progress reporter reports there is no
bytes transfered and cancels the ImagePull.
The original 1-minute timeout[2][2] is from kubelet settting. Since CRI-plugin
can't limit the total concurrent downloads, this patch is to update 1-minute
to 5-minutes to prevent from unexpected cancel.
[1]: https://lwn.net/Articles/842385/
[2]: https://github.com/kubernetes/kubernetes/blob/release-1.23/pkg/kubelet/config/flags.go#L45-L48
Signed-off-by: Wei Fu <fuweid89@gmail.com>
When the HTTP fallback is used, the scheme changes from HTTPS to HTTP
which can cause a mismatch on redirect, causing the authorizer to get
stripped out. Since the redirect host must match the redirect host in
this case, credentials are only sent to the same origin host that
returned the redirect.
This fixes an issue for a push getting a 401 unauthorized on the PUT
request even though credentials are available.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The TLS fallback should only be used when the protocol is ambiguous due
to provided TLS configurations and defaulting to http. Do not add TLS
configurations when defaulting to http. When the port is 80 or will be
defaulted to 80, there is no protocol ambiguity and TLS fallback should
not be used.
Signed-off-by: Derek McGowan <derek@mcg.dev>
When a credential handler is provided but no basic auth credentials
are provided, handle the error specifically rather than treating
the credentials as not implemented. This allows a clearer error to
be provided to users rather than a confusing not implemented error
or generic unauthorized error.
Add unit tests for the basic auth case.
Signed-off-by: Derek McGowan <derek@mcg.dev>
`MountedFrom` was prefixed with the whole target repository instead of
just the registry hostname.
Also adjust the test cases to use the registry hostname.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When a endpoint is configured for http and has a tls configuration,
always try to the tls connection and fallback to http when the tls
connections fails from receiving an http response. This fixes an issue
with default localhost endpoints which get defaulted to http with
insecure tls also configured but are using tls.
Signed-off-by: Derek McGowan <derek@mcg.dev>
The reference/docker package was a fork of github.com/distribution/distribution,
which could not easily be used as a direct dependency, as it brought many other
dependencies with it.
The "reference' package has now moved to a separate repository, which means
we can replace the local fork, and use the upstream implementation again.
The new module was extracted from the distribution repository at commit:
b9b19409cf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
From the Go docs:
"For a nil slice, the number of iterations is 0." [1]
Both `info.RootFS` and `host.clientPairs` are slices. Therefore, an
additional nil check for before the loop is unnecessary.
[1]: https://go.dev/ref/spec#For_range
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Previously, we would return the first non-404 error from a host.
This is logical, however, it can result in confusing errors to the
user:
- e.g. we have an HTTP host, and an HTTPS host.
If the image does not exist, we return "http: server gave HTTP
response to HTTPS client". This is technically correct, however, the
user is easily confused - the most relevant error in this case is the
404 error.
- e.g. we have a broken HTTP host that returns 5XX errors, and a HTTP
host with authentication.
On the request for an image, we return the 5XX error directly.
However, we have a host later on which returned an authentication
error which is now hidden from the user.
Note: this *can* be resolved by changing the order of hosts passed in,
however this requires 1. knowing ahead of time which hosts are going to
return certain errors and 2. this is often not desirable, we'd prefer
to use HTTPS if it's available, and only then fallback to HTTP.
To resolve this, we assign each possible error during resolution a
"priority" that marks how far through the image resolution process a
host/path combo got. Then we return the error with the highest priority,
which is much more likely to be the most relevant error to the user.
The ranking of priority then is (from lowest to highest):
- Underlying transport errors (TLS, TCP, etc)
- 404 errors
- Other 4XX/5XX errors
- Manifest rejection (due to max size exceeded)
Signed-off-by: Justin Chadwell <me@jedevc.com>
Go deprecation comments must be formatted to have an empty comment line before
them. Fix the formatting to make sure linters and editors detect that these
are deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes it possible to check whether content didn't actually need to
be pushed to the remote registry and was cross-repo mounted or already
existed.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Add a public const for "containerd.io/distribution.source" in `labels`
package and replace hardcoded usages.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
In the refactor from 926b9c72f61b5be6bf8d952512f1d0932fbaf898, the error
handling was substantially reworked, and changed the types of errors
returned.
Notably, in the case of a network error, instead of propogating the
error through to return from pushWriter.Write (as previously), it would
be propagated through to pushWriter.Commit - however, this is too late,
since we've already closed the io.Pipe by the time we would have reached
this function. Therefore, we get the generic error message "io:
read/write on closed pipe" for *every network error*.
This patch corrects this behavior to ensure that the correct error
object is always returned as early as possible, by checking the error
result after writing and detecting a closed pipe.
Additionally, we do some additional hardening - specifically we prevent
falling through when resetting the content or detecting errors, and
update the tests to explicitly check for the ErrReset message.
Signed-off-by: Justin Chadwell <me@jedevc.com>
