Commit Graph

131 Commits

Author SHA1 Message Date
Paul "TBBle" Hampson
785ac3ab71 Ensure request.header is never nil
Header.Clone() will return `nil` if called on a nil object.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2021-01-13 05:46:50 +11:00
Phil Estes
9db6aa625d
Merge pull request #4302 from songjiayang/update-docker-resolver
Refactor dockerResolver with resolveDockerBase helper
2021-01-12 11:15:06 -05:00
Shengjing Zhu
5988bfc1ef docs: Various typo found by codespell
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2020-12-22 13:22:16 +08:00
Akihiro Suda
318e34b983
Merge pull request #4855 from tonistiigi/push-concurrent-access
docker: avoid concurrent map access panic
2020-12-16 22:35:33 +09:00
Tonis Tiigi
4dfec7fa01 pusher: add missing authentication support for requests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-15 22:26:59 -08:00
Tonis Tiigi
bf323c5bdd docker: avoid concurrent map access panic
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-15 22:26:26 -08:00
Tonis Tiigi
f601887a3c docker: don’t hide pusher response error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-10 23:19:08 -08:00
Sebastiaan van Stijn
b086062cfd
httpReadSeeker: fix typo in error-message
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-29 23:07:43 +01:00
Maksym Pavlenko
c59d1cd5b0 Fix linter issues
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2020-10-07 15:42:01 -07:00
ktock
4b882eb93a Export repository scope helper functions
`docker.Authorizer` requires library clients to configure scope via context.
It is helpful for the clients to use the helper (currently private) functions
for generating scope string and to use that function with the combination of
other scope-related ones (e.g. `docker.WithScope`).

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2020-10-06 10:49:01 +09:00
songjiayang
5867c88832 Refactor dockerResolver to make code simple
1.add resolveDockerBase helper
2.dockerBase header copy with header.Clone()

Signed-off-by: songjiayang <songjiayang@jd.com>
2020-09-14 11:38:05 +08:00
Ilya Dmitrichenko
2de55060ee
Log unexpected responses
This accomplishes a few long-standing TODO items, but also helps users
in showing exact registry error messages

Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2020-09-03 14:52:11 +01:00
Derek McGowan
a7b2304f69
Merge pull request #4445 from tonistiigi/auth-refactor
docker: split private token helper functions to reusable pkg
2020-08-25 12:25:23 -07:00
Tonis Tiigi
b5185eae6c auth: return token structs from fetcher functions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-04 12:59:06 -07:00
Akihiro Suda
0f08a55d6b
Merge pull request #4413 from dmcgowan/registry-proxy-ns
Add namespace query parameter for registry proxying
2020-08-04 01:58:52 +09:00
Derek McGowan
adeba792f1
Add namespace query parameter for registry proxying
Proxy registries are designed to serve content from upstreams.
However, the proxy hostname will usually not match the hostname
of the upstream, requiring the proxy to only use a single
upstream or use its own pattern matching to determine the upstream.
To solve this issue, the client will pass along the namespace which
is being used for the request, allowing mirrors to easily map
to multiple upstreams. This query parameter can safely be ignored
if multiple upstreams are not supported.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2020-08-03 09:32:44 -07:00
Tonis Tiigi
957bcb3dff docker: split private token helper functions to reusable pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 22:50:46 -07:00
Tonis Tiigi
03b5a053ea remotes: mark GetTokenScopes public
Authorizer interface can’t be really implemented because
scopes are passed in on a side channel via private value in context.


Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-29 23:01:36 -07:00
Wei Fu
f821b77151
Merge pull request #4272 from dmcgowan/update-registry-host-config
Registry config header support and fixes
2020-07-03 18:47:46 +08:00
Derek McGowan
3dd8242a67
Add host specific headers
Allows configuring headers per registry host

Signed-off-by: Derek McGowan <derek@mcg.dev>
2020-06-01 18:27:41 -07:00
Hu Shuai
230cf6deda Add unit test for func in remotes/docker/handler.go
Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-05-28 16:54:33 +08:00
Derek McGowan
84619ee998
Fix configurations with no server provided
When a server is specified at the top level, there is a bug
that prevents the keys from being checked properly.
When no server is provided, the server attempts to parse
with an empty host, leaving partial values and a defaulted
skip verify configuration.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2020-05-19 19:16:50 -07:00
Derek McGowan
06b0cd45ba
Fix nil pointer errors
Signed-off-by: Derek McGowan <derek@mcg.dev>
2020-05-19 19:16:39 -07:00
Gaurav Singh
2325182529 docker: fix data race on err
Signed-off-by: Gaurav Singh <gaurav1086@gmail.com>
2020-05-17 09:20:38 -04:00
Sebastiaan van Stijn
1b9640496e
ConfigureHosts: remove deprecated DualStack option
The `DualStack` option was deprecated in Go 1.12, and is now enabled by default
(through commit github.com/golang/go@efc185029bf770894defe63cec2c72a4c84b2ee9).

> The Dialer.DualStack field is now meaningless and documented as deprecated.
>
> To disable fallback, set FallbackDelay to a negative value.

The default `FallbackDelay` is 300ms; to make this more explicit, this patch
sets `FallbackDelay` to the default value.

Note that Docker Hub currently does not support IPv6 (DNS for registry-1.docker.io
has no AAAA records, so we should not hit the 300ms delay).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-10 12:31:58 +02:00
Sebastiaan van Stijn
dc92ad6520
Replace errors.Cause() with errors.Is()
Dependencies may be switching to use the new `%w` formatting
option to wrap errors; switching to use `errors.Is()` makes
sure that we are still able to unwrap the error and detect the
underlying cause.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-08 14:36:45 +02:00
Derek McGowan
067aba732e
Add test for default setup for host configuration
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-04-06 14:39:49 -07:00
Wei Fu
d9a1c3f9e4
bugfix: add default host config if not set
If there is not specific host config, like ctr does, the resolver will
fail to get host path. And this patch is to add default host config if
needs.

And default config host config should have all caps for pull and push.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2020-04-06 14:38:52 -07:00
Maksym Pavlenko
9ba5ea232c
Merge pull request #4138 from dmcgowan/registry-configuration-tools
Registry configuration package
2020-04-02 10:13:30 -07:00
Derek McGowan
17b6050d20
Add Docker resolver configuration package
Add configuration toml file format and configuration
function to configure registry hosts from a directory
based configuration. Compatible with Docker registry
certificate loading.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-03-31 22:52:10 -07:00
Hu Shuai
c263c5b146 Add unit test for func in remotes/docker/handler.go
Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-04-01 11:41:01 +08:00
Hu Shuai
c8d9c6f2a6 Add unit test for func in remotes/docker/handler.go
Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-03-31 11:29:34 +08:00
Sebastiaan van Stijn
f2edc6f164
vendor: update gotest.tools v3.0.2
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-28 17:47:20 +01:00
Josh Dolitsky
d8a0d29c23 Set octet-stream content-type on put request
Signed-off-by: Josh Dolitsky <393494+jdolitsky@users.noreply.github.com>
2020-02-12 11:39:33 -06:00
Alex Price
a022c21819 Improve host fallback behaviour in docker remote
This commit improves the fallback behaviour when resolving and
fetching images with multiple hosts. If an error is encountered
when resolving and fetching images, and more than one host is being
used, we will try the same operation on the next host. The error
from the first host is preserved so that if all hosts fail, we can
display the error from the first host.

fixes #3850

Signed-off-by: Alex Price <aprice@atlassian.com>
2019-12-13 11:42:32 +11:00
Derek McGowan
08517e5864
Allow empty scope authorization
Registries may allow using token authorization without
explicitly setting the scope. This may cover use cases where
no scope is required for an endpoint or the registry is only
covering authentication using the token. This aligns with the
oauth2 spec which specifies the scope as optional.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-12-12 11:05:48 -08:00
Evan Cordell
7177af84ac Allow 202 response code for commit
Quay returns this status code when pushing

Signed-off-by: Evan Cordell <cordell.evan@gmail.com>
2019-10-14 09:21:31 -04:00
Michael Crosby
901bcb2231 Add distribution subpkgs to core
Ref: #3554

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-10-07 13:38:12 -04:00
Derek McGowan
394db03f15
Fix all media types in Accept header to match RFC
Fixes the media type to align with Accept HTTP header
RFC which specifies glob syntax */*

See https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-26 14:48:42 -07:00
Brian Goff
e84a84a5a9 Add function to set custom auth scope in context
Currently auth.docker.io uses a custom auth scope for (docker) plugins
`repository(plugin):<repo>:<perms>`.
This makes it impossible to use containerd distribution tooling to fetch
plugins without also supplying a totally custom authorizer.

This changes allows clients to set the correct scope on the context.
It's a little bit nasty but "works".

I'm also a bit suspect of some a couple of these unexported context
functrions. Before the primary one used `contextWithRepositoryScope`
overwrites any scope value and there is another one that appends the
scope value.
With this change they both append...

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-09-18 11:29:16 -07:00
Michael Crosby
f06e605f1a
Merge pull request #3515 from fuweid/me-remove-comment-line
remotes: remove useless line
2019-08-09 09:21:36 -04:00
Wei Fu
282b19efd2 remotes: remove useless line
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-08-09 17:15:09 +08:00
ethan
0a3769eec6 pusher.go: error message typo correction
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-07 10:09:23 +08:00
Jintao Zhang
4b4ce4a26c remotes: remove unnecessary string formatting
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-08-05 02:16:57 +08:00
Derek McGowan
c965a6c4da
Prevent push by tag for sub-manifests
When pushing a manifest list, all manifests should be pushed by digest
and only the final manifest pushed by tag. The Pusher was preventing
this by mistakenly disallowing objects to contain a digest. When objects
have a digest, only push tags associated with that digest.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-31 15:19:44 -07:00
Derek McGowan
518be1cb07
Fix bug in setting request body
Go documentation says
`Use of GetBody still requires setting Body`.
This change ensures the body is always set in
addition to GetBody. This fixes a bug where
sometimes the body is nil.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-18 11:21:51 -07:00
Derek McGowan
0b29c9c371
Update resolver to handle endpoint configuration
Adds support for registry mirrors
Adds support for multiple pull endpoints
Adds capabilities to limit trust in public mirrors
Fixes user agent header missing


Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-16 11:28:23 -07:00
Konstantin Maksimov
3d3dbc8fbf Handle RequestTimeout and TooManyRequests
Retry 5 times in case of StatusRequestTimeout StatusTooManyRequests
This fixes the issue #2680 "Make content fetch retry more robust"

Signed-off-by: Konstantin Maksimov <kmaksimov@gmail.com>
2019-07-01 19:19:40 +03:00
zhangyue
12a14c4424 fix: polish log to make more clear
Signed-off-by: zhangyue <zy675793960@yeah.net>
2019-06-20 20:46:26 +08:00
Wei Fu
dd7c0aabcc remotes: support cross-repo-push
With distribution source label in content store, select the longest
common prefix components as condidate mount blob source and try to push
with mount blob.

Fix #2964

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-06-13 09:51:46 +08:00