Commit Graph

104 Commits

Author SHA1 Message Date
msg555
ee902afa5f Compute manifest metadata when not provided.
This closes #3238

Signed-off-by: msg555 <msg555@gmail.com>
2019-05-01 00:53:16 -04:00
Davor Kapsa
eded188f4f Fix misspells
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 20:38:24 +02:00
Jared Cordasco
fc167f9578 Correct PusherFunc helper to match Pusher intf
Signed-off-by: Jared Cordasco <jcordasc@coglib.com>
2019-04-12 09:12:55 -05:00
Michael Crosby
de1da8be32 Update handling of registry errors
Update to #3109

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-04 14:51:03 -04:00
Michael Crosby
9bc231532d
Merge pull request #3109 from bainsy88/issue_3076
Add code to return errors from registries
2019-04-04 13:23:37 -04:00
Jack Baines
d15832a4c2 Address review comments
-Fix whitespace on imports
-Fix test case naming

Signed-off-by: Jack Baines <jack.baines@uk.ibm.com>
2019-03-21 15:59:49 +00:00
Jack Baines
908b771086 Add code to return message field of returned registry errors
Docker registries return errors in a know format so this change now checks for these
errors and returns the message field. If the error is not in the expected format fall
back to the original behaviour.

https://github.com/containerd/containerd/issues/3076

Signed-off-by: Jack Baines <jack.baines@uk.ibm.com>
2019-03-19 21:05:36 +00:00
Wei Fu
506b815483 remotes: add distribution labels to blob data
We can use cross repository push feature to reuse the existing blobs in
the same registry. Before make push fast, we know where the blob comes
from.

Use the `containerd.io/distribution.source. = [,]` as label format. For
example, the blob is downloaded by the docker.io/library/busybox:latest
and the label will be

    containerd.io/distribution.source.docker.io = library/busybox

If the blob is shared by different repos in the same registry, the repo
name will be appended, like:

    containerd.io/distribution.source.docker.io = library/busybox,x/y

NOTE:
1. no need to apply for legacy docker image schema1.
2. the concurrent fetch actions might miss some repo names in label, but
it is ok.
3. it is optional. no need to add label if the engine only uses images
not push.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-03-12 13:42:54 +08:00
Derek McGowan
dc09ed1e1a
Add image handler wrapper
Gives clients more control of the pull process, allowing
the client to operate on a descriptor after it has been
pulled. This could be useful for filtering output or
tracking children before they dispatched to. This can
also be used to call custom unpackers to have visibility
into a pulled config in parallel to the downloads.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-02-12 16:34:06 -08:00
Lantao Liu
d7ed403072 Add image pull concurrency limit.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-01-29 10:27:05 -08:00
Wei Fu
132ee9b826 fix: linter issue
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.

close: #2945

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-01-23 22:54:51 +08:00
Wei Fu
5137fc6f11 bugfix: support application/octet-stream during pull
Even though application/octet-stream issue has been fixed in docker,
there exists lots of images which contains the invalid mediatype.

In order to pull those images, containerd client side modifies the
manifest content before insert/update image reference.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-12-29 13:58:36 +08:00
Eric Lin
2db0c6c262 Explicitly stating utf-8 when fetching oauth token
Fixes #2870

Signed-off-by: Eric Lin <linxiulei@gmail.com>
2018-12-12 12:16:27 +08:00
Derek McGowan
b8631c750c
Merge pull request #2751 from Charliekenney23/add-useragent-resolver-option
Add custom headers resolver option
2018-11-20 10:51:10 -08:00
Tonis Tiigi
a2e8ab9ff7 remotes: avoid panic with default resolver conf
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-19 17:33:59 -08:00
Charles Kenney
068880ff16 Add custom headers option to dockerResolver
Signed-off-by: Charles Kenney <charlesc.kenney@gmail.com>
2018-10-31 19:42:42 -04:00
Charles Kenney
7c85d873a0 fix typo in ResolverOptions.Credentials documentation
Signed-off-by: Charles Kenney <charlesc.kenney@gmail.com>
2018-10-31 11:06:28 -04:00
Gábor Lipták
4a0c40664a
Correct redundant nil check
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2018-10-19 16:50:17 -04:00
Michael Crosby
5993d096ff
Merge pull request #2691 from fuweid/bugfix_cache_empty_label
bugfix: cache empty layer for docker schema1 image
2018-10-15 15:43:23 -04:00
Derek McGowan
26506e9e23
Update empty layer label
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-10-15 11:28:45 -07:00
Derek McGowan
a6198b7692
Update resolver code
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-28 14:32:23 -07:00
Wei Fu
57d79e1b49 bugfix: cache empty layer for docker schema1 image
containerd should cache empty label for docker schema1 image.

if not, the original empty layer will be non-empty layer and the image
config will be changed too. in this case, the image ID will be changed.

check the blob empty label to avoid changing image ID when repull docker
schema1 image.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-09-28 15:45:59 +08:00
nashasha1
7f03ad6579 Fix typos
Signed-off-by: nashasha1 <a4012017@sina.com>
2018-09-07 16:59:42 +08:00
Wei Fu
9db21deb5d bugfix: avoid to re-calculate blob state for schema1
Use containerd.io/uncompressed label to avoid to re-calculate blob
diffID.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-09-06 09:14:56 +08:00
Derek McGowan
9edcfcc1cb
Add platform match comparer interface
Adds a new platform interface for matching and comparing platforms.
This new interface allows both filtering and ordering of platforms
to support running multiple platform and choosing the best platform.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-27 00:11:46 -07:00
Phil Estes
c55b9636f7
Merge pull request #2263 from alibaba/fix_no_gzip
fixbug: blob for schemav1 could be uncompressed
2018-07-30 18:49:28 -04:00
Lantao Liu
c8a8a49584 Support specifying host in resolver.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-07-09 18:26:02 -07:00
Vincent Demeester
832b05ae67
Update tests to use gotest.tools angel
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-08 21:02:01 +02:00
frank yang
046536cfb1 fixbug: blob for schemav1 could be uncompressed
Signed-off-by: frank yang <yyb196@gmail.com>
2018-06-08 11:50:54 +08:00
Akihiro Suda
703c25e452 fix schema1 fetchBlob()
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-07 17:51:41 +09:00
Bin Du
9b865d86a9 docker/pusher: handle location string containing path and query
Signed-off-by: Bin Du <bindu@microsoft.com>
2018-06-05 19:25:19 +00:00
Akihiro Suda
d88de4a34f content: change Writer/ReaderAt to take OCI
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.

No OCI descriptor field is written to the store.

No change on gRPC API.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-01 11:51:43 +09:00
Derek McGowan
59740d8985
Fix invalid length bug with some registries
Ensures that the client can handle cases where the
registry ignores content length.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-26 23:41:56 -07:00
Stephen J Day
27a14f1a9d
remotes/docker: protect secret with a mutex
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-04-05 15:16:38 -07:00
Derek McGowan
5304ef294b
Add writer open helper to handle unavailable refs
Updates blob writer helper to use new open and ensure
unavailable errors are always handled.
Removes duplication of unavailable handling code.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-03-21 16:30:22 -07:00
Derek McGowan
a0b818e093
Merge pull request #2200 from jessvalarezo/multiarch-pulls
allow content to be pulled for specific platform(s), all platforms
2018-03-14 14:46:30 -07:00
Jess Valarezo
c3cf3d7822 allow content to be pulled for specific platform(s), all platforms
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-03-12 17:31:42 -07:00
ruediger-maass
373f1e5612 Adds handling of 401 for POST /v2/token without authentication
This fix adds support for image registries that expect authentication for POST /v2/token such as used by the GET. E.g., JFrog Artifactory y has been observed to respond with a 401 (Unauthorized) in that case. Adding 401 in addition to the current handling of 405 and 404 in the resolver solves the authentication problem. Finally, this enables image pulls also for Artifactory.

Signed-off-by: Ruediger Maass <ruediger.maass@de.ibm.com>
2018-02-28 02:32:44 +00:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Daniel Nephin
3279acca82 Check timestamps in snapshot storage test suite
also use t.Helper()
convert assertions to canonical

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Daniel Nephin
ef48a0268e Migrate to gotestyourself/assert
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Derek McGowan
944a9ade37
Update fetch handling
Fix issue where manifest content must always be fetched
even if it is already fully downloaded or shared locally.
Simplify children label setting and platform filtering.
Prevent getting a fetcher when content shared locally.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-02-09 14:33:14 -08:00
Derek McGowan
424c0fb59d
Merge pull request #2075 from hinshun/converter-mediatypes
Allow specifying media type for a converted schema1 manifest for compatibility with distribution
2018-02-06 10:58:16 -08:00
Edgar Lee
e53cdc9a49
Refactor client push into helper function
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2018-01-31 17:13:54 -08:00
Edgar Lee
d2f073c6f5
Add converter options to convert schema1 manifest to docker schema2 manifest
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2018-01-31 17:13:12 -08:00
Michael Crosby
d179c61231 Revert "Use jsoniteer for faster json encoding/decoding"
This reverts commit 4233b87b89.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-29 15:47:48 -05:00
Michael Crosby
4233b87b89 Use jsoniteer for faster json encoding/decoding
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-26 16:32:55 -05:00
Darren Stahl
f404331e90 ctr: print error when pull auth fails
Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 16:14:09 -08:00
Daniel Nephin
49fffe8ec7 Less verbose tests
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-12-14 11:00:40 -05:00
Stephen J Day
ebf80ab943
remotes: remove unused hints
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-12-05 14:32:10 -08:00