Commit Graph

83 Commits

Author SHA1 Message Date
Derek McGowan
5e2d7efd82
Use a single custom annotation for export
Remove annotation prefix and add multiple index records
for manifests with multiple image names. This makes the
custom annotation more consistent with the OCI image
annotation. Additionally, ensure the OCI image annotation
always represents the tag (partial image name) as recommended
by the specification. The containerd image name annotation
will always contain the full image name.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-05-21 17:21:26 -07:00
Wei Fu
e61f7f4913
Merge pull request #3172 from dmcgowan/export-docker-compatibility
Update image export to support Docker format
2019-05-17 14:30:54 +08:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Derek McGowan
4754d2aeee
Update image export to support Docker format
Add manifest.json file which is used by Docker
to import images.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-04-04 15:23:06 -07:00
Sebastiaan van Stijn
525802f9ce
Use distribution's reference.ParseDockerRef
Using the cri utility caused other project to have
containerd/cri as a dependency, only for this utility.

The new `reference.ParseDockerRef` function does the
same (other than having a different name).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-02-07 13:13:25 +01:00
Lantao Liu
d7ed403072 Add image pull concurrency limit.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-01-29 10:27:05 -08:00
Phil Estes
b444d7d507
Handle additional cleanups from prior PR
Update comment and streamline getting the default platform spec

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2019-01-16 13:07:29 -05:00
kadisi
40267382c5 fix ctr image export not found error
Signed-off-by: Jie Zhang <iamkadisi@163.com>
2018-12-17 08:52:28 +08:00
Evan Hazlett
6f2f4e4343 checkpoint: add copts to checkpoint; save snapshotter to annotation
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:48:16 +00:00
Evan Hazlett
147208061c add image name and runtime name media types; remove task operation on checkpoint
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:47:17 +00:00
Evan Hazlett
45c700a955 refactor checkpoint and restore to client
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:45:22 +00:00
Gábor Lipták
9d01160902
Correct spelling
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2018-10-19 17:06:43 -04:00
Phil Estes
90b7b88e10
Merge pull request #2696 from dmcgowan/import-docker-tar-test
Content store fix and import integration test
2018-10-03 09:57:25 +02:00
Derek McGowan
b6107dca86
Add import integration test
Move tar creation test utilities to separate package
Test all supported formats for import

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-10-01 11:47:10 -07:00
Derek McGowan
ef8498bf79
Update manifest resolution for non-index manifests
Avoid filtering manifests by platform when a non-index
manifest is provided to the Manifest function.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-25 17:50:06 -07:00
Derek McGowan
da6d29033c
Clean up error messages
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-17 14:41:43 -07:00
Derek McGowan
a62be324b7
Unify docker and oci importer
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-17 14:41:43 -07:00
Derek McGowan
bce20b75da
Simplify docker importer
Support any layout and rely on manifest.json to reference blobs

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-14 14:21:27 -07:00
Derek McGowan
9e6db71954
Add docker importer
Update ctr to support all formats by default

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-14 14:21:26 -07:00
Derek McGowan
f57c5cdefb
Refactor image importer
Allow customization of reference creation.
Add option for digest references.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-14 14:21:26 -07:00
Derek McGowan
3d1082e06d
Add a manifest filter limiting the number of matches
Adds a manifest filter for pulling which ensures only one
manifest from a manifest list is pulled even when multiple matches.
Removes unused filter platform list.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-27 17:03:55 -07:00
Derek McGowan
9edcfcc1cb
Add platform match comparer interface
Adds a new platform interface for matching and comparing platforms.
This new interface allows both filtering and ordering of platforms
to support running multiple platform and choosing the best platform.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-27 00:11:46 -07:00
Tonis Tiigi
82e7204a3c oci-exporter: do not filter current platform on export
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-13 17:08:01 -07:00
Tonis Tiigi
53fe31d6de images: provide better error for manifest list match error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-20 13:43:27 -07:00
Vincent Demeester
832b05ae67
Update tests to use gotest.tools angel
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-08 21:02:01 +02:00
Akihiro Suda
d88de4a34f content: change Writer/ReaderAt to take OCI
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.

No OCI descriptor field is written to the store.

No change on gRPC API.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-01 11:51:43 +09:00
Michael Wan
7fd6d5e2be fix words misspell
Signed-off-by: Michael Wan <zirenwan@gmail.com>
2018-04-23 00:09:42 -04:00
Jess Valarezo
c3cf3d7822 allow content to be pulled for specific platform(s), all platforms
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-03-12 17:31:42 -07:00
Jess Valarezo
ac8008437a update platforms Parse to return platform spec, decouple matcher
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-02-27 14:21:49 -08:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Daniel Nephin
3279acca82 Check timestamps in snapshot storage test suite
also use t.Helper()
convert assertions to canonical

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Daniel Nephin
ef48a0268e Migrate to gotestyourself/assert
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Michael Crosby
5471ba9445
Merge pull request #2117 from dmcgowan/update-fetch-handlers
content: support manifest labeling on completed content ingests on the client
2018-02-12 11:08:14 -05:00
Derek McGowan
944a9ade37
Update fetch handling
Fix issue where manifest content must always be fetched
even if it is already fully downloaded or shared locally.
Simplify children label setting and platform filtering.
Prevent getting a fetcher when content shared locally.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-02-09 14:33:14 -08:00
Akihiro Suda
ee70879202 *: use narrower content interfaces
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-09 19:52:52 +09:00
Michael Crosby
d179c61231 Revert "Use jsoniteer for faster json encoding/decoding"
This reverts commit 4233b87b89.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-29 15:47:48 -05:00
Michael Crosby
4233b87b89 Use jsoniteer for faster json encoding/decoding
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-26 16:32:55 -05:00
Darren Stahl
95a0b3af95 Refactor checking for compressed diff type
Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 15:25:17 -08:00
Daniel Nephin
184bc25629 Add unconvert linter
This linter checks for unnecessary type convertions.

Some convertions are whitelisted because their type is different
on 32bit platforms

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-01-09 17:36:44 -05:00
Derek McGowan
40c67fdf78
Merge pull request #1880 from AkihiroSuda/refactor-importer
importer: refactor and fix GC
2017-12-04 20:56:26 -08:00
Akihiro Suda
63401970c7 importer: refactor
- Use lease API (previoisly, GC was not supported)
- Refactored interfaces for ease of future Docker v1 importer support

For usage, please refer to `ctr images import --help`.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-05 12:48:32 +09:00
Jess Valarezo
41ac9fac8e update images error
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-12-01 15:14:55 -08:00
Jess Valarezo
2b1595124f update image error messages
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-12-01 13:54:26 -08:00
Derek McGowan
3f1a61f76a
Add synchronous image delete
Synchronous image delete provides an option image delete to wait
until the next garbage collection deletes after an image is removed
before returning success to the caller.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-20 17:08:35 -08:00
Michael Crosby
a19fd6ed6e Add checkpoint media types to handler
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-17 11:26:28 -04:00
Michael Crosby
0ae9d8fabf Merge pull request #1567 from stevvooe/image-check-function
images: support checking status of image content
2017-10-04 11:40:43 -04:00
Stephen Day
f6d88927d5 Merge pull request #1583 from stevvooe/image-storage-validation
metadata: validation and testing of image store
2017-10-03 16:48:38 -07:00
Stephen J Day
698b6d15f7
metadata: validation and testing of image store
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-03 16:34:00 -07:00
Stephen J Day
c555df54c0
images: support checking status of image content
The `Check` function returns information about an image's content components
over a content provider. From this information, one can tell which content is
required, present or missing to run an image.

The utility can be demonstrated with the `check` command:

```console
$ ctr images check
REF                               TYPE                                                      DIGEST                                                                  STATUS            SIZE
docker.io/library/alpine:latest   application/vnd.docker.distribution.manifest.list.v2+json sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d incomplete (1/2)  1.5 KiB/1.9 MiB
docker.io/library/postgres:latest application/vnd.docker.distribution.manifest.v2+json      sha256:2f8080b9910a8b4f38ff5a55a82e77cb43d88bdbb16d723c71d18493590832e9 complete (13/13)  99.3 MiB/99.3 MiB
docker.io/library/redis:alpine    application/vnd.docker.distribution.manifest.v2+json      sha256:e633cded055a94202e4ccccb8125b7f383cd6ee56527ab890db643383a2647dd incomplete (6/7)  8.1 MiB/10.0 MiB
docker.io/library/ubuntu:latest   application/vnd.docker.distribution.manifest.list.v2+json sha256:60f835698ea19e8d9d3a59e68fb96fb35bc43e745941cb2ea9eaf4ba3029ed8a unavailable (0/?) 0.0 B/?
docker.io/trollin/busybox:latest  application/vnd.docker.distribution.manifest.list.v2+json sha256:54a6424f7a2d5f4f27b3d69e5f9f2bc25fe9087f0449d3cb4215db349f77feae complete (2/2)    699.9 KiB/699.9 KiB
```

The above shows us that we have two incomplete images and one that is
unavailable. The incomplete images are those that we know the complete
size of all content but some are missing. "Unavailable" means that the
check could not get enough information about the image to get its full
size.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-03 15:19:22 -07:00
Phil Estes
1c7a042e04 Merge pull request #1566 from stevvooe/split-children
images: provide separated children function for images
2017-10-03 13:46:46 -04:00