The interface that combines both content.InfoProvider and
content.Provider was duplicated in multiple places - create one directly
in `content` package and use it instead.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Persist manifest/indexes distribution source labels as annotations in
the index.json. This could allow the importer to fetch the missing blobs
from the external repository.
These can't really be persisted directly in blob descriptors because
that would alter the digests.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Allow importing/exporting archives which doesn't have all the referenced
blobs. This allows to export/import an image with only some of the
platforms available locally while still persisting the full index.
> The blobs directory MAY be missing referenced blobs, in which case the missing blobs SHOULD be fulfilled by an external blob store.
https://github.com/opencontainers/image-spec/blob/v1.0/image-layout.md#blobs
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The reference/docker package was a fork of github.com/distribution/distribution,
which could not easily be used as a direct dependency, as it brought many other
dependencies with it.
The "reference' package has now moved to a separate repository, which means
we can replace the local fork, and use the upstream implementation again.
The new module was extracted from the distribution repository at commit:
b9b19409cf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Several bits of code unmarshal image config JSON into an `ocispec.Image`, and then immediately create an `ocispec.Platform` out of it, but then discard the original image *and* miss several potential platform fields (most notably, `variant`).
Because `ocispec.Platform` is a strict subset of `ocispec.Image`, most of these can be updated to simply unmarshal the image config directly to `ocispec.Platform` instead, which allows these additional fields to be picked up appropriately.
We can use `tianon/raspbian` as a concrete reproducer to demonstrate.
Before:
```console
$ ctr content fetch docker.io/tianon/raspbian:bullseye-slim
...
$ ctr image ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/tianon/raspbian:bullseye-slim application/vnd.docker.distribution.manifest.v2+json sha256:66e96f8af40691b335acc54e5f69711584ef7f926597b339e7d12ab90cc394ce 28.6 MiB linux/arm/v7 -
```
(Note that the `PLATFORMS` column lists `linux/arm/v7` -- the image itself is actually `linux/arm/v6`, but one of these bits of code leads to only `linux/arm` being extracted from the image config, which `platforms.Normalize` then updates to an explicit `v7`.)
After:
```console
$ ctr image ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/tianon/raspbian:bullseye-slim application/vnd.docker.distribution.manifest.v2+json sha256:66e96f8af40691b335acc54e5f69711584ef7f926597b339e7d12ab90cc394ce 28.6 MiB linux/arm/v6 -
```
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
It was possible to still export the docker-compatible manifest.json
file, if a single platform image (as a standalone manifest) was
exported, even if the WithSkipDockerManifest option was explicitly set.
To resolve this, we remove all references to skipDockerManifest to,
adding it instead to the point-of-writing, simplifying the earlier logic
and making it clear exactly when this manifest file should be written.
Signed-off-by: Justin Chadwell <me@jedevc.com>
"SA1019: tar.TypeRegA has been deprecated since Go 1.11 and an alternative has been available since Go 1.1:
Use TypeReg instead. (staticcheck)"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:
```go
func BenchmarkSplit(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_ = strings.SplitN(s, "=", 2)[0]
}
}
}
func BenchmarkCut(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_, _, _ = strings.Cut(s, "=")
}
}
}
```
BenchmarkSplit
BenchmarkSplit-10 8244206 128.0 ns/op 128 B/op 4 allocs/op
BenchmarkCut
BenchmarkCut-10 54411998 21.80 ns/op 0 B/op 0 allocs/op
While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints; for cases where an specific number of items is expected, I used `strings.SplitN()`
with a suitable limit. This prevents (theoretical) unlimited splits.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
digest.Algorithm() and digest.Encoded() may panic for invalid digests.
Validate prior to calling those methods.
Signed-off-by: Samuel Karp <samuelkarp@google.com>
As reported, running import twice without using the compress import
option means that the content store will have existing layers during the
second import and the existing code hardcodes existing layer media type
to compressed. This fixes the issue by actually reading the header bytes
from the store and setting the media type appropriately.
Signed-off-by: Phil Estes <estesp@amazon.com>
The OCI image spec did a v1.0.2 security release for CVE-2021-41190, however
commit 09c9270fee, depends on MediaTypes that
have not yet been released by the OCI image-spec, so using current "main" instead.
full diff: 5ad6f50d62...693428a734
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This commit allows hook callbacks during image conversion.
This enbles the caller additional modification for each blob descriptor.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Currently uncompress converter only supports gzip. This commit fixes it to
support zstd as well.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
