Commit Graph

249 Commits

Author SHA1 Message Date
Derek McGowan
51c9ffe468
remotes: add handling for missing basic auth credentials
When a credential handler is provided but no basic auth credentials
are provided, handle the error specifically rather than treating
the credentials as not implemented. This allows a clearer error to
be provided to users rather than a confusing not implemented error
or generic unauthorized error.

Add unit tests for the basic auth case.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-13 12:42:19 -07:00
Derek McGowan
5b097a0817
Merge pull request #9187 from vvoland/remotes-docker-mounted-fix
remotes/docker: Fix MountedFrom prefixed with target repository
2023-10-03 09:33:35 -07:00
Phil Estes
399c9d8326
Merge pull request #8861 from jedevc/docker-resolver-relevant-error
docker: return most relevant error from docker resolution
2023-10-03 10:40:52 -04:00
Paweł Gronowski
859c2651e5
remotes/docker: Fix MountedFrom prefixed with target repository
`MountedFrom` was prefixed with the whole target repository instead of
just the registry hostname.

Also adjust the test cases to use the registry hostname.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-10-03 16:40:10 +02:00
Derek McGowan
e8770b1716
Merge pull request #9155 from neersighted/image_cleanup_2
Use mediatype helpers in more places
2023-10-03 07:11:12 -07:00
Derek McGowan
79772a0dd4
remotes: always try to establish tls connection when tls configured
When a endpoint is configured for http and has a tls configuration,
always try to the tls connection and fallback to http when the tls
connections fails from receiving an http response. This fixes an issue
with default localhost endpoints which get defaulted to http with
insecure tls also configured but are using tls.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-10-02 15:29:11 -07:00
Bjorn Neergaard
cdba61603c
remotes: use mediatype helpers
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-09-27 13:12:51 -06:00
Derek McGowan
b5615caf11
Update go-toml to v2
Updates host file parsing to use new v2 method rather than the removed
toml.Tree.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-22 15:35:12 -07:00
Derek McGowan
508aa3a1ef
Move to use github.com/containerd/log
Add github.com/containerd/log to go.mod

Signed-off-by: Derek McGowan <derek@mcg.dev>
2023-09-22 07:53:23 -07:00
Phil Estes
25198a0557
Merge pull request #8330 from vvoland/remotes-docker-status-exists-mounted
remotes/docker: Add Mounted and Exists push status
2023-09-13 13:14:39 -04:00
Derek McGowan
b11439fc4b
Merge pull request #9034 from thaJeztah/replace_reference
replace reference/docker for github.com/distribution/reference v0.5.0
2023-09-05 06:52:29 -07:00
Akihiro Suda
e30a40eb65
Merge pull request #9016 from djdongjin/remove-most-logrus
Remove most logrus import
2023-09-05 16:09:12 +09:00
Sebastiaan van Stijn
4923470902
replace reference/docker for github.com/distribution/reference v0.5.0
The reference/docker package was a fork of github.com/distribution/distribution,
which could not easily be used as a direct dependency, as it brought many other
dependencies with it.

The "reference' package has now moved to a separate repository, which means
we can replace the local fork, and use the upstream implementation again.

The new module was extracted from the distribution repository at commit:
b9b19409cf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-31 15:54:50 +02:00
Eng Zer Jun
34def8b33b
Remove redundant nil check
From the Go docs:

  "For a nil slice, the number of iterations is 0." [1]

Both `info.RootFS` and `host.clientPairs` are slices. Therefore, an
additional nil check for before the loop is unnecessary.

[1]: https://go.dev/ref/spec#For_range

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-08-27 17:56:27 +08:00
Jin Dong
fc45365fa1 Remove most logrus
Signed-off-by: Jin Dong <jin.dong@databricks.com>
2023-08-26 14:31:53 -04:00
Sebastiaan van Stijn
710d22366d
removes/docker: replace some fmt.Sprintfs with strconv
Teeny-tiny optimizations:

    BenchmarkSprintf-10       37735996    32.31  ns/op  0 B/op  0 allocs/op
    BenchmarkItoa-10         591945836     2.031 ns/op  0 B/op  0 allocs/op
    BenchmarkFormatUint-10   593701444     2.014 ns/op  0 B/op  0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-23 10:10:59 +02:00
Derek McGowan
1aa710d0fc
Merge pull request #8694 from yankay/fix-push-image-rate
Fix ctr push image can't show right rate
2023-08-22 16:39:35 -07:00
Sebastiaan van Stijn
e2ad5a985e
removes/docker: remove unnecessary conversion (unconvert)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-30 22:35:31 +02:00
Justin Chadwell
a1cdf60cb6 docker: add test to ensure not found error is returned by resolve
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-26 11:33:50 +01:00
Justin Chadwell
b4814a29d4 docker: return most relevant error from docker resolution
Previously, we would return the first non-404 error from a host.

This is logical, however, it can result in confusing errors to the
user:

- e.g. we have an HTTP host, and an HTTPS host.

  If the image does not exist, we return "http: server gave HTTP
  response to HTTPS client". This is technically correct, however, the
  user is easily confused - the most relevant error in this case is the
  404 error.

- e.g. we have a broken HTTP host that returns 5XX errors, and a HTTP
  host with authentication.

  On the request for an image, we return the 5XX error directly.
  However, we have a host later on which returned an authentication
  error which is now hidden from the user.

Note: this *can* be resolved by changing the order of hosts passed in,
however this requires 1. knowing ahead of time which hosts are going to
return certain errors and 2. this is often not desirable, we'd prefer
to use HTTPS if it's available, and only then fallback to HTTP.

To resolve this, we assign each possible error during resolution a
"priority" that marks how far through the image resolution process a
host/path combo got. Then we return the error with the highest priority,
which is much more likely to be the most relevant error to the user.

The ranking of priority then is (from lowest to highest):
- Underlying transport errors (TLS, TCP, etc)
- 404 errors
- Other 4XX/5XX errors
- Manifest rejection (due to max size exceeded)

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-26 11:33:50 +01:00
Justin Chadwell
a3404ac422 docker: add missing info log for 4XX/5XX responses
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-24 15:28:37 +01:00
Jin Dong
83ff030e4f Change http.Header copy to builtin Clone
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2023-07-08 05:25:47 -07:00
Jin Dong
cdb153ec92 Resolve docker.NewResolver race condition
Signed-off-by: Jin Dong <djdongjin95@gmail.com>
2023-07-08 05:25:47 -07:00
Cardy.Tang
50376ec9ff remotes: allow FetchByDigest client to pass mediatype as header
accept mediatype header is required for some layers, add additional field for client.

Signed-off-by: Cardy.Tang <zuniorone@gmail.com>
2023-06-27 16:36:01 +08:00
Kay Yan
f4ad68d5f2 fix-push-image-rate
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
2023-06-15 19:08:03 +00:00
Sebastiaan van Stijn
577696f608
replace some basic uses of fmt.Sprintf()
Really tiny gains here, and doesn't significantly impact readability:

    BenchmarkSprintf
    BenchmarkSprintf-10    11528700     91.59 ns/op   32 B/op  1 allocs/op
    BenchmarkConcat
    BenchmarkConcat-10    100000000     11.76 ns/op    0 B/op  0 allocs/op

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-10 13:24:43 +02:00
Sebastiaan van Stijn
52997ea983
remotes/docker: ResolverOptions: fix deprecation comments
Go deprecation comments must be formatted to have an empty comment line before
them. Fix the formatting to make sure linters and editors detect that these
are deprecated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-01 09:28:12 +02:00
Samuel Karp
c60ba138b6
Merge pull request #8502 from mstmdev/fix-typos 2023-05-16 08:41:02 -07:00
mstmdev
cdaa4025e9 Fix some typos
Signed-off-by: Pan Yibo <mstmdev@gmail.com>
2023-05-16 10:12:50 +08:00
Kohei Tokunaga
6e2c915a44
Bump up golangci-lint to v1.52.2
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-09 15:07:55 +09:00
Paweł Gronowski
dfc7590d5a
remotes/docker: Add MountedFrom and Exists push status
This makes it possible to check whether content didn't actually need to
be pushed to the remote registry and was cross-repo mounted or already
existed.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-04-07 10:38:37 +02:00
Paweł Gronowski
dd3eedf3c3 labels: Add LabelDistributionSource
Add a public const for "containerd.io/distribution.source" in `labels`
package and replace hardcoded usages.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-03-08 10:01:58 +01:00
Fu Wei
8cb00f45c9
Merge pull request #8143 from mxpv/log
Add Fields type alias to log package
2023-02-21 10:22:23 +08:00
Maksym Pavlenko
06e085c8b5 Add Fields type alias to log package
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-02-20 17:29:08 -08:00
Akihiro Suda
8bf975b4fa
lint: silence "type HostFileConfig is unused (unused)"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-02-16 03:50:22 +09:00
Fish-pro
76a62e52ae Use http constants instead of string
Signed-off-by: Fish-pro <zechun.chen@daocloud.io>
2023-02-10 14:01:48 +08:00
Justin Chadwell
9f6058d029 pushWriter: correctly propagate errors
In the refactor from 926b9c72f61b5be6bf8d952512f1d0932fbaf898, the error
handling was substantially reworked, and changed the types of errors
returned.

Notably, in the case of a network error, instead of propogating the
error through to return from pushWriter.Write (as previously), it would
be propagated through to pushWriter.Commit - however, this is too late,
since we've already closed the io.Pipe by the time we would have reached
this function. Therefore, we get the generic error message  "io:
read/write on closed pipe" for *every network error*.

This patch corrects this behavior to ensure that the correct error
object is always returned as early as possible, by checking the error
result after writing and detecting a closed pipe.

Additionally, we do some additional hardening - specifically we prevent
falling through when resetting the content or detecting errors, and
update the tests to explicitly check for the ErrReset message.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 11:37:41 +00:00
Akihiro Suda
4adf3fb3af
Merge pull request #7906 from Iceber/use_label_uncompressed
Use the const labels.LabelUncompressed
2023-01-04 01:04:20 +09:00
Iceber Gu
778e8f2af4 Use the const labels.LabelUncompressed
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2023-01-03 18:29:21 +08:00
Wei Fu
6b7e237fc7 chore: use go fix to cleanup old +build buildtag
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-12-29 14:25:14 +08:00
Derek McGowan
ba243ffaaf
Merge pull request #7521 from iyear/fix-defer
Fix incorrect defer usage and refactor judgement
2022-12-27 15:50:41 -08:00
Hsing-Yu (David) Chen
d5010a12df docs: Authorizer.Authorize could return ErrUnexpectedStatus
Signed-off-by: Hsing-Yu (David) Chen <davidhsingyuchen@gmail.com>
2022-12-08 13:24:48 -08:00
Sebastiaan van Stijn
f0ce2f6d36
remotes/docker: remove deprecated NewAuthorizer alias
This was deprecated since containerd v1.3.0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-20 22:27:21 +01:00
Maksym Pavlenko
14a08721cc Remove OpenTelementry imports from remotes/docker package
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-11-15 17:32:26 -08:00
Luca Comellini
b829e6f2d3
Bump OpenTelemetry to v1.11.1
Signed-off-by: Luca Comellini <luca.com@gmail.com>
2022-11-10 22:15:10 -08:00
Swagat Bora
7def13dde3 Add a thin wrapper around otel Span object
Signed-off-by: Swagat Bora <sbora@amazon.com>
2022-11-11 01:28:27 +00:00
Derek McGowan
31bb8fef7e
Merge pull request #7460 from AkihiroSuda/fetch-by-digest
remotes: add `FetcherByDigest` for fetching blobs without foreknown descriptors (useful for general-purpose CAS)
2022-11-04 15:20:29 -07:00
Kazuyoshi Kato
e306680f44
Merge pull request #7607 from jess-sol/main
Support default hosts.toml configuration
2022-11-04 10:22:00 -07:00
Akihiro Suda
dfd6a3aa6e
remotes: add FetcherByDigest for fetching blobs without foreknown descriptors
Fetching blobs without foreknown descriptors is useful for using a registry as a general-purpose CAS.

Related: `oras blob fetch` (ORAS v0.15.0)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-11-04 11:41:07 +09:00
Jess
f0bf9e7f8b Support default hosts.toml configuration
Add support for an optional `default` registry hosts config, should no
other hosts config match.

Signed-off-by: Jess <jess@ros.io>
2022-11-03 16:26:51 -06:00