Commit Graph

82 Commits

Author SHA1 Message Date
nashasha1
7f03ad6579 Fix typos
Signed-off-by: nashasha1 <a4012017@sina.com>
2018-09-07 16:59:42 +08:00
Wei Fu
9db21deb5d bugfix: avoid to re-calculate blob state for schema1
Use containerd.io/uncompressed label to avoid to re-calculate blob
diffID.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-09-06 09:14:56 +08:00
Derek McGowan
9edcfcc1cb
Add platform match comparer interface
Adds a new platform interface for matching and comparing platforms.
This new interface allows both filtering and ordering of platforms
to support running multiple platform and choosing the best platform.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-27 00:11:46 -07:00
Phil Estes
c55b9636f7
Merge pull request #2263 from alibaba/fix_no_gzip
fixbug: blob for schemav1 could be uncompressed
2018-07-30 18:49:28 -04:00
Lantao Liu
c8a8a49584 Support specifying host in resolver.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-07-09 18:26:02 -07:00
Vincent Demeester
832b05ae67
Update tests to use gotest.tools angel
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-08 21:02:01 +02:00
frank yang
046536cfb1 fixbug: blob for schemav1 could be uncompressed
Signed-off-by: frank yang <yyb196@gmail.com>
2018-06-08 11:50:54 +08:00
Akihiro Suda
703c25e452 fix schema1 fetchBlob()
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-07 17:51:41 +09:00
Bin Du
9b865d86a9 docker/pusher: handle location string containing path and query
Signed-off-by: Bin Du <bindu@microsoft.com>
2018-06-05 19:25:19 +00:00
Akihiro Suda
d88de4a34f content: change Writer/ReaderAt to take OCI
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.

No OCI descriptor field is written to the store.

No change on gRPC API.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-01 11:51:43 +09:00
Derek McGowan
59740d8985
Fix invalid length bug with some registries
Ensures that the client can handle cases where the
registry ignores content length.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-26 23:41:56 -07:00
Stephen J Day
27a14f1a9d
remotes/docker: protect secret with a mutex
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-04-05 15:16:38 -07:00
Derek McGowan
5304ef294b
Add writer open helper to handle unavailable refs
Updates blob writer helper to use new open and ensure
unavailable errors are always handled.
Removes duplication of unavailable handling code.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-03-21 16:30:22 -07:00
Derek McGowan
a0b818e093
Merge pull request #2200 from jessvalarezo/multiarch-pulls
allow content to be pulled for specific platform(s), all platforms
2018-03-14 14:46:30 -07:00
Jess Valarezo
c3cf3d7822 allow content to be pulled for specific platform(s), all platforms
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-03-12 17:31:42 -07:00
ruediger-maass
373f1e5612 Adds handling of 401 for POST /v2/token without authentication
This fix adds support for image registries that expect authentication for POST /v2/token such as used by the GET. E.g., JFrog Artifactory y has been observed to respond with a 401 (Unauthorized) in that case. Adding 401 in addition to the current handling of 405 and 404 in the resolver solves the authentication problem. Finally, this enables image pulls also for Artifactory.

Signed-off-by: Ruediger Maass <ruediger.maass@de.ibm.com>
2018-02-28 02:32:44 +00:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Daniel Nephin
3279acca82 Check timestamps in snapshot storage test suite
also use t.Helper()
convert assertions to canonical

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Daniel Nephin
ef48a0268e Migrate to gotestyourself/assert
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Derek McGowan
944a9ade37
Update fetch handling
Fix issue where manifest content must always be fetched
even if it is already fully downloaded or shared locally.
Simplify children label setting and platform filtering.
Prevent getting a fetcher when content shared locally.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-02-09 14:33:14 -08:00
Derek McGowan
424c0fb59d
Merge pull request #2075 from hinshun/converter-mediatypes
Allow specifying media type for a converted schema1 manifest for compatibility with distribution
2018-02-06 10:58:16 -08:00
Edgar Lee
e53cdc9a49
Refactor client push into helper function
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2018-01-31 17:13:54 -08:00
Edgar Lee
d2f073c6f5
Add converter options to convert schema1 manifest to docker schema2 manifest
Signed-off-by: Edgar Lee <edgar.lee@docker.com>
2018-01-31 17:13:12 -08:00
Michael Crosby
d179c61231 Revert "Use jsoniteer for faster json encoding/decoding"
This reverts commit 4233b87b89.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-29 15:47:48 -05:00
Michael Crosby
4233b87b89 Use jsoniteer for faster json encoding/decoding
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-26 16:32:55 -05:00
Darren Stahl
f404331e90 ctr: print error when pull auth fails
Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 16:14:09 -08:00
Daniel Nephin
49fffe8ec7 Less verbose tests
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-12-14 11:00:40 -05:00
Stephen J Day
ebf80ab943
remotes: remove unused hints
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-12-05 14:32:10 -08:00
Akihiro Suda
63401970c7 importer: refactor
- Use lease API (previoisly, GC was not supported)
- Refactored interfaces for ease of future Docker v1 importer support

For usage, please refer to `ctr images import --help`.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-05 12:48:32 +09:00
Derek McGowan
b674c64827
Support unknown size from schema 1 manifests
Schema1 manifests did not set a size in the digest for the blobs,
breaking the expectations of the update http seeking reader. Now
the http seeker has been updated to support unknown size as a
value of negative 1 and the schema1 puller sets the unknown size
accordingly.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-21 15:45:09 -08:00
Daniel Nephin
f74862a0dd Add structcheck, unused, and varcheck linters.
Warn on unused and dead code

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-21 11:14:37 -05:00
Phil Estes
77cb722e1c
Merge pull request #1775 from dmcgowan/fix-push-different-tags
Update docker pusher check tag
2017-11-17 08:21:08 -05:00
Derek McGowan
56a35d5cb9
Update docker pusher check tag
Currently pushing a new tag to a manifest which already
exists in the registry skips the tag push because it
only checks that the manifest exists. This updates the
logic to instead check if the tag exists and is at the
same digest.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-16 17:29:29 -08:00
Stephen J Day
682151b166 remotes/docker: implement seekable http requests
To support resumable download, the fetcher for a remote must implement
`io.Seeker`. If implemented the `content.Copy` function will detect the
seeker and begin from where the download was terminated by a previous
attempt.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:13:06 -05:00
Daniel Nephin
2e7f7318cc Normalize 'already exists' errors
and fix some error messages where they were wrong or redundant

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 16:46:27 -05:00
Derek McGowan
dce27d8c62
Remove client use of gc root label
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-07 12:54:22 -08:00
Stephen J Day
8eee47cf82
remotes/docker/schema1: back off on locked ref
To allow concurrent pull of images of the v1 persuasion, we need to
backoff when multiple pullers are trying to operate on the same
resource. The back off logic is ported to v1 pull to match the behavior
for other images.

A little randomness is also added to the backoff to prevent thundering
herd and to reduce expected recovery time.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-30 15:40:54 -07:00
Michael Crosby
5a386446e3 Merge pull request #1653 from dmcgowan/fix-pull-race
gc: fix pull race
2017-10-18 10:10:32 -04:00
Derek McGowan
b798d87bd4
Ensure manifests are marked as root during pull
For schema1 mark blobs as roots and remove labels
once referenced by the created manifest.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-17 16:26:52 -07:00
Brian Goff
6eb56a2ec9 Use limited reader for some ReadAll cases.
Prevents a server from sending a large response causing containerd to
allocate too much RAM and potentially OOM.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-17 09:06:16 -04:00
Derek McGowan
7884707c2f
Add reference labels to snapshots and content
Ensure all snapshots and content are referenced on commit and
protected from cleanup.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 10:42:47 -07:00
Michael Crosby
451421b615 Comment more packages to pass go lint
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-02 13:54:56 -04:00
Allen Sun
4700968ff3 update docs and comments
Signed-off-by: Allen Sun <shlallen1990@gmail.com>
2017-09-25 14:48:28 +08:00
Stephen Day
709d34f882 Merge pull request #1537 from allencloud/add-reponse-body-close
add a response body close
2017-09-21 18:32:56 -07:00
Allen Sun
c6d55954e0 add a response body close
Signed-off-by: Allen Sun <shlallen1990@gmail.com>
2017-09-22 09:25:31 +08:00
Stephen J Day
13771fba83
images, remotes: add support for foreign layers
Add support for downloading layers with external URLs and
foreign/non-distributable mediatypes. This ensures that encountered
windows images are downloaded correctly. We still need to filter out the
extra windows resources when pulling linux, but this is a step towards
correctly supporting multi-platform images.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-20 17:06:13 -07:00
Akihiro Suda
2f08032924 remotes/docker: add scope (registry:foo/bar:pull)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-07 05:10:48 +00:00
Akihiro Suda
aaae706067 remotes/docker: allow 404 for POST /v2/token
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-07 04:58:44 +00:00
Kenfe-Mickaël Laventure
e1eeb0e0a2 Merge pull request #1475 from dmcgowan/content-commit-context
Add context to content commit
2017-09-06 11:04:31 -07:00
Derek McGowan
9613acb2ed
Add context to content commit
Content commit is updated to take in a context, allowing
content to be committed within the same context the writer
was in. This is useful when commit may be able to use more
context to complete the action rather than creating its own.
An example of this being useful is for the metadata implementation
of content, having a context allows tests to fully create
content in one database transaction by making use of the context.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-09-06 10:19:12 -07:00