Commit Graph

196 Commits

Author SHA1 Message Date
Derek McGowan
6f027e38a8
Remove redundant build tags
Remove build tags which are already implied by the name of the file.
Ensures build tags are used consistently

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-08-05 22:27:46 -07:00
Phil Estes
6389fc79f0
Merge pull request #5770 from wzshiming/fix/push-docker-remote-close
remotes/docker/pusher.go: Fix missing Close() on push to docker remote
2021-07-21 10:06:41 -04:00
Shiming Zhang
53ec1abec2 remotes/docker/pusher.go: Fix missing Close()
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2021-07-21 11:37:08 +08:00
Shiming Zhang
67d07fe5cf remotes/docker/fetcher.go: Fix missing Close()
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2021-07-21 11:36:23 +08:00
coryb
894b6ae39b Fix missing Body.Close() calls on push to docker remote
Discovered this while using HTTP tracing via OpenTelemetry inside of
buildkitd, where the trace spans were not being reported for the
registry PUT http requests.  The spans are only reported on the Close
for the Body, after adding these Close calls, the spans are reported as
expected.

Signed-off-by: coryb <cbennett@netflix.com>
2021-07-11 08:14:57 -07:00
Derek McGowan
a7ad6b3be5
Add support for registry host path override
Adds support for mirrors which are non-compliant with the
OCI distribution specification but have previously mirrored
content with a namespace prefix after the API root `/v2`.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-07-02 09:48:27 -07:00
Derek McGowan
95c708572f
Update documenation for OCI distribution 1.0
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-07-01 17:24:08 -07:00
Ethan Chen
50ad4b96c4 Fix incorrect UA used for registry authentication
Previously, containerd uses Go's default UA "Go-http-client/1.1" while authenticating with registry.
This commit changes it to the pattern like "containerd/v1.5.2" which is used for all other requests.

Signed-off-by: Ethan Chen <pov@mahou-shoujo.moe>
2021-05-26 09:45:54 +08:00
Derek McGowan
dbccd0d357
Merge pull request #5275 from haslersn/fix-4531
Try next mirror in case of non-404 errors, too
2021-05-19 07:57:57 -07:00
Derek McGowan
b1d4140a22
Update docker resolver to authorize redirects
Allows redirects to be authorized if authorization is provided
for the redirected to host. The authorization will always go
to the redirect and never to the referrer.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-05-14 12:16:24 -07:00
ktock
ab1654d0e2 Fix PushHandler cannot push image that contains duplicated blobs
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-04-20 14:00:53 +09:00
Sebastian Hasler
c0e2f4b5d2 Try next mirror in case of non-404 errors, too
Signed-off-by: Sebastian Hasler <sebastian.hasler@gmx.net>
2021-04-15 12:04:07 +02:00
Mike Brown
d21fe46250 adds log for each failed host and status not found on host
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-04-14 13:23:22 -05:00
Aaron Lehmann
4c1fa57194 remotes/docker: Only return "already exists" on push when the upload was successful
The `(dockerPusher).Push` method uses a `StatusTracker` to check if an
upload already happened, before repeating the upload. However, there is
no provision for failure handling. If a PUT request returns an error,
the `StatusTracker` will still see the upload as if it happened
successfully. Add a status boolean so that only successful uploads
short-circuit `Push`.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-04-06 19:45:24 -07:00
Derek McGowan
261c107ffc
Merge pull request #5278 from mxpv/toml
Migrate TOML to github.com/pelletier/go-toml
2021-04-01 21:24:52 -07:00
Phil Estes
7ae0a60fb5
Add OCI ref.name to unique key in remotes handler
This allows a pusher to be used for more than one tag without creating a
new resolver/pusher. The current implementation checks the ref key
tracker status based on type and hash and will skip the push even if the
repository reference is unique.

Signed-off-by: Phil Estes <estesp@amazon.com>
2021-04-01 14:20:30 -04:00
Maksym Pavlenko
5ada2f74a7 Keep host order as defined in TOML file
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-04-01 09:29:16 -07:00
Maksym Pavlenko
6866b36ab6 Add workaround to keep docker hosts structs private
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-30 22:20:14 -07:00
Maksym Pavlenko
9e19a29847 Fix hosts test on Windows
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-27 16:21:52 -07:00
Maksym Pavlenko
d56b49c13d Rewrite Docker hosts parser
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-27 15:39:46 -07:00
Wei Fu
f8c2f04756 remotes/ctr: allow to limit max concurrent uploads like downloads
Also add flags for push/pull subcommand to limit max concurrent
uploads/downloads.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-03-25 14:37:02 +08:00
Maksym Pavlenko
22ef69d77d Support HTTP debug in ctr
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-22 09:32:34 -07:00
Phil Estes
2061227b56
Merge pull request #4949 from hs0210/work
Add unit test for func in remotes/docker/pusher.go
2021-03-19 13:11:46 -04:00
Phil Estes
b395f2f9a9
Merge pull request #5160 from MadhavJivrajani/master
Use net.IP.IsLoopback() to match loopback addresses
2021-03-17 09:37:17 -04:00
Phil Estes
a1138182d5
Merge pull request #5180 from dmcgowan/lint-enforce-comments
Fix exported comments enforcer in CI
2021-03-15 10:50:06 -04:00
Phil Estes
bd4f468c62
Merge pull request #5186 from cpuguy83/fix_docker_cert_loading
Fix docker style cert loading.
2021-03-15 10:36:42 -04:00
Madhav Jivrajani
8f863afd3a Use net.IP.IsLoopback() to match loopback addresses
- changed the `MatchLocalhost` function in remotes/docker/registry.go
	- Make use of SplitHostPort to split host and port number
- Added additional tests for modified code in remotes/docker/registry_test.go
- Note: this does not handle mathcing of IP addresses in octal, decimal or hex format or a mix of these.

Fixes: #5129

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2021-03-15 09:57:01 +05:30
Phil Estes
f6a9942733
Better error when handling a descriptor of size zero
Signed-off-by: Phil Estes <estesp@amazon.com>
2021-03-13 22:48:55 -05:00
Brian Goff
1fd99e24a2 Fix docker style cert loading.
The certs dir parsing was skipping over files instead of reading them,
as such the certs would never load.

It was also stating the file name rather than the full path for cert
pairs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-03-12 22:42:20 +00:00
Derek McGowan
35eeb24a17
Fix exported comments enforcer in CI
Add comments where missing and fix incorrect comments

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-03-12 08:47:05 -08:00
Ilya Dmitrichenko
d1b7784357
Use ErrUnexpectedStatus more consistently
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2021-03-11 14:37:59 +00:00
Ilya Dmitrichenko
a11d785784
Include URL and method in ErrUnexpectedStatus
This should help with debugging expected responses.

Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2021-03-11 14:28:46 +00:00
Hu Shuai
1d26595cf3 Add unit test for func in remotes/docker/pusher.go
Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2021-01-19 16:24:27 +08:00
Paul "TBBle" Hampson
785ac3ab71 Ensure request.header is never nil
Header.Clone() will return `nil` if called on a nil object.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2021-01-13 05:46:50 +11:00
Phil Estes
9db6aa625d
Merge pull request #4302 from songjiayang/update-docker-resolver
Refactor dockerResolver with resolveDockerBase helper
2021-01-12 11:15:06 -05:00
Shengjing Zhu
5988bfc1ef docs: Various typo found by codespell
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2020-12-22 13:22:16 +08:00
Akihiro Suda
318e34b983
Merge pull request #4855 from tonistiigi/push-concurrent-access
docker: avoid concurrent map access panic
2020-12-16 22:35:33 +09:00
Tonis Tiigi
4dfec7fa01 pusher: add missing authentication support for requests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-15 22:26:59 -08:00
Tonis Tiigi
bf323c5bdd docker: avoid concurrent map access panic
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-15 22:26:26 -08:00
Tonis Tiigi
f601887a3c docker: don’t hide pusher response error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-10 23:19:08 -08:00
Sebastiaan van Stijn
b086062cfd
httpReadSeeker: fix typo in error-message
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-29 23:07:43 +01:00
Maksym Pavlenko
c59d1cd5b0 Fix linter issues
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2020-10-07 15:42:01 -07:00
ktock
4b882eb93a Export repository scope helper functions
`docker.Authorizer` requires library clients to configure scope via context.
It is helpful for the clients to use the helper (currently private) functions
for generating scope string and to use that function with the combination of
other scope-related ones (e.g. `docker.WithScope`).

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2020-10-06 10:49:01 +09:00
songjiayang
5867c88832 Refactor dockerResolver to make code simple
1.add resolveDockerBase helper
2.dockerBase header copy with header.Clone()

Signed-off-by: songjiayang <songjiayang@jd.com>
2020-09-14 11:38:05 +08:00
Ilya Dmitrichenko
2de55060ee
Log unexpected responses
This accomplishes a few long-standing TODO items, but also helps users
in showing exact registry error messages

Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2020-09-03 14:52:11 +01:00
Derek McGowan
a7b2304f69
Merge pull request #4445 from tonistiigi/auth-refactor
docker: split private token helper functions to reusable pkg
2020-08-25 12:25:23 -07:00
Tonis Tiigi
b5185eae6c auth: return token structs from fetcher functions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-04 12:59:06 -07:00
Akihiro Suda
0f08a55d6b
Merge pull request #4413 from dmcgowan/registry-proxy-ns
Add namespace query parameter for registry proxying
2020-08-04 01:58:52 +09:00
Derek McGowan
adeba792f1
Add namespace query parameter for registry proxying
Proxy registries are designed to serve content from upstreams.
However, the proxy hostname will usually not match the hostname
of the upstream, requiring the proxy to only use a single
upstream or use its own pattern matching to determine the upstream.
To solve this issue, the client will pass along the namespace which
is being used for the request, allowing mirrors to easily map
to multiple upstreams. This query parameter can safely be ignored
if multiple upstreams are not supported.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2020-08-03 09:32:44 -07:00
Tonis Tiigi
957bcb3dff docker: split private token helper functions to reusable pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 22:50:46 -07:00