The reference/docker package was a fork of github.com/distribution/distribution,
which could not easily be used as a direct dependency, as it brought many other
dependencies with it.
The "reference' package has now moved to a separate repository, which means
we can replace the local fork, and use the upstream implementation again.
The new module was extracted from the distribution repository at commit:
b9b19409cf
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Several bits of code unmarshal image config JSON into an `ocispec.Image`, and then immediately create an `ocispec.Platform` out of it, but then discard the original image *and* miss several potential platform fields (most notably, `variant`).
Because `ocispec.Platform` is a strict subset of `ocispec.Image`, most of these can be updated to simply unmarshal the image config directly to `ocispec.Platform` instead, which allows these additional fields to be picked up appropriately.
We can use `tianon/raspbian` as a concrete reproducer to demonstrate.
Before:
```console
$ ctr content fetch docker.io/tianon/raspbian:bullseye-slim
...
$ ctr image ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/tianon/raspbian:bullseye-slim application/vnd.docker.distribution.manifest.v2+json sha256:66e96f8af40691b335acc54e5f69711584ef7f926597b339e7d12ab90cc394ce 28.6 MiB linux/arm/v7 -
```
(Note that the `PLATFORMS` column lists `linux/arm/v7` -- the image itself is actually `linux/arm/v6`, but one of these bits of code leads to only `linux/arm` being extracted from the image config, which `platforms.Normalize` then updates to an explicit `v7`.)
After:
```console
$ ctr image ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/tianon/raspbian:bullseye-slim application/vnd.docker.distribution.manifest.v2+json sha256:66e96f8af40691b335acc54e5f69711584ef7f926597b339e7d12ab90cc394ce 28.6 MiB linux/arm/v6 -
```
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
It was possible to still export the docker-compatible manifest.json
file, if a single platform image (as a standalone manifest) was
exported, even if the WithSkipDockerManifest option was explicitly set.
To resolve this, we remove all references to skipDockerManifest to,
adding it instead to the point-of-writing, simplifying the earlier logic
and making it clear exactly when this manifest file should be written.
Signed-off-by: Justin Chadwell <me@jedevc.com>
"SA1019: tar.TypeRegA has been deprecated since Go 1.11 and an alternative has been available since Go 1.1:
Use TypeReg instead. (staticcheck)"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:
```go
func BenchmarkSplit(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_ = strings.SplitN(s, "=", 2)[0]
}
}
}
func BenchmarkCut(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_, _, _ = strings.Cut(s, "=")
}
}
}
```
BenchmarkSplit
BenchmarkSplit-10 8244206 128.0 ns/op 128 B/op 4 allocs/op
BenchmarkCut
BenchmarkCut-10 54411998 21.80 ns/op 0 B/op 0 allocs/op
While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints; for cases where an specific number of items is expected, I used `strings.SplitN()`
with a suitable limit. This prevents (theoretical) unlimited splits.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
digest.Algorithm() and digest.Encoded() may panic for invalid digests.
Validate prior to calling those methods.
Signed-off-by: Samuel Karp <samuelkarp@google.com>
As reported, running import twice without using the compress import
option means that the content store will have existing layers during the
second import and the existing code hardcodes existing layer media type
to compressed. This fixes the issue by actually reading the header bytes
from the store and setting the media type appropriately.
Signed-off-by: Phil Estes <estesp@amazon.com>
The OCI image spec did a v1.0.2 security release for CVE-2021-41190, however
commit 09c9270fee, depends on MediaTypes that
have not yet been released by the OCI image-spec, so using current "main" instead.
full diff: 5ad6f50d62...693428a734
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This commit allows hook callbacks during image conversion.
This enbles the caller additional modification for each blob descriptor.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Currently uncompress converter only supports gzip. This commit fixes it to
support zstd as well.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
During import, if an image does not match the host's platform,
it won't have any children labels set, which will result in the
Garbage Collector deleting its content later, resulting in an
unusable image. In this case, we should fail early.
This can still be bypassed by using ctr import --all-platforms.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
A previous commit made the Windows containerd/platforms.Default stricter
by requiring the OS Version to have a similar OS Version as the node's OS Version.
However, tar images (from docker save) do not have any OS Version information,
causing the containerd/import.Import's images.FilterPlatforms to filter out the image
entirely, which means that the images.SetChildrenLabels doesn't get to label
any children, which in turn will cause the Garbage Collector to remove content
related to the image.
This sets a default platform for the imported image if it's a Windows image which
doesn't have any OSVersion information, or if there's no platform information at
all.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
When several goroutines call uncompress converter in parallel, the ref name
conflicts each other. This leads to Writer method failing with Unavaliable error
without retry.
For solving this issue, OpenWriter helper should be used. This allows them to
retry in such situations.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Currently, `image.GetDiffID` cannot calculate DiffID of zstd layers because it
directly uses `compress/gzip` decompressor.
This commit fixes this issue by using the generic decompressor.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
