It was possible to still export the docker-compatible manifest.json
file, if a single platform image (as a standalone manifest) was
exported, even if the WithSkipDockerManifest option was explicitly set.
To resolve this, we remove all references to skipDockerManifest to,
adding it instead to the point-of-writing, simplifying the earlier logic
and making it clear exactly when this manifest file should be written.
Signed-off-by: Justin Chadwell <me@jedevc.com>
"SA1019: tar.TypeRegA has been deprecated since Go 1.11 and an alternative has been available since Go 1.1:
Use TypeReg instead. (staticcheck)"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:
```go
func BenchmarkSplit(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_ = strings.SplitN(s, "=", 2)[0]
}
}
}
func BenchmarkCut(b *testing.B) {
b.ReportAllocs()
data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
for i := 0; i < b.N; i++ {
for _, s := range data {
_, _, _ = strings.Cut(s, "=")
}
}
}
```
BenchmarkSplit
BenchmarkSplit-10 8244206 128.0 ns/op 128 B/op 4 allocs/op
BenchmarkCut
BenchmarkCut-10 54411998 21.80 ns/op 0 B/op 0 allocs/op
While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints; for cases where an specific number of items is expected, I used `strings.SplitN()`
with a suitable limit. This prevents (theoretical) unlimited splits.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
digest.Algorithm() and digest.Encoded() may panic for invalid digests.
Validate prior to calling those methods.
Signed-off-by: Samuel Karp <samuelkarp@google.com>
As reported, running import twice without using the compress import
option means that the content store will have existing layers during the
second import and the existing code hardcodes existing layer media type
to compressed. This fixes the issue by actually reading the header bytes
from the store and setting the media type appropriately.
Signed-off-by: Phil Estes <estesp@amazon.com>
The OCI image spec did a v1.0.2 security release for CVE-2021-41190, however
commit 09c9270fee, depends on MediaTypes that
have not yet been released by the OCI image-spec, so using current "main" instead.
full diff: 5ad6f50d62...693428a734
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This commit allows hook callbacks during image conversion.
This enbles the caller additional modification for each blob descriptor.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Currently uncompress converter only supports gzip. This commit fixes it to
support zstd as well.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
During import, if an image does not match the host's platform,
it won't have any children labels set, which will result in the
Garbage Collector deleting its content later, resulting in an
unusable image. In this case, we should fail early.
This can still be bypassed by using ctr import --all-platforms.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
A previous commit made the Windows containerd/platforms.Default stricter
by requiring the OS Version to have a similar OS Version as the node's OS Version.
However, tar images (from docker save) do not have any OS Version information,
causing the containerd/import.Import's images.FilterPlatforms to filter out the image
entirely, which means that the images.SetChildrenLabels doesn't get to label
any children, which in turn will cause the Garbage Collector to remove content
related to the image.
This sets a default platform for the imported image if it's a Windows image which
doesn't have any OSVersion information, or if there's no platform information at
all.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
When several goroutines call uncompress converter in parallel, the ref name
conflicts each other. This leads to Writer method failing with Unavaliable error
without retry.
For solving this issue, OpenWriter helper should be used. This allows them to
retry in such situations.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Currently, `image.GetDiffID` cannot calculate DiffID of zstd layers because it
directly uses `compress/gzip` decompressor.
This commit fixes this issue by using the generic decompressor.
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Go example:
```go
opts := []converter.Opt{
// convert Docker media types to OCI ones
converter.WithDocker2OCI(true),
// convert tar.gz layers to uncompressed tar layers
converter.WithLayerConvertFunc(uncompress.LayerConvertFunc),
}
srcRef := "example.com/foo:orig"
dstRef := "example.com/foo:converted"
dstImg, err = converter.Convert(ctx, client, dstRef, srcRef, opts...)
fmt.Println(dstImg.Target)
```
ctr example: `ctr images convert --oci --uncompress example.com/foo:orig example.com/foo:converted`
Go test: `go test -exec sudo -test.root -test.run TestConvert`
The implementation is from https://github.com/containerd/stargz-snapshotter/pull/224,
but eStargz-specific functions are not included in this PR.
eStargz converter can be specified by importing `estargz` package and using `WithLayerConvertFunc(estargz.LayerConvertFunc)` option.
This converter interface will be potentially useful for converting zstd and ocicrypt layers as well.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
zstd is a compression algorithm that has a very fast decoder, while
providing also good compression ratios. The fast decoder makes it
suitable for container images, as decompressing the tarballs is a very
expensive operation.
https://github.com/opencontainers/image-spec/pull/788 added support
for zstd to the OCI image specs.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Give control of the content labeling process for children to
the client. This allows the client to control the names
associated with the labels and filter out labels.
Signed-off-by: Derek McGowan <derek@mcg.dev>
This log message shows up in the client's logs. For any media type that
the client doesn't know about it will wind up with a warning log.
Downgrade this to debug since it is more of a development concern.
We encountered this trying to fetch Docker plugins which has a media
type for plugin configs.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Dependencies may be switching to use the new `%w` formatting
option to wrap errors; switching to use `errors.Is()` makes
sure that we are still able to unwrap the error and detect the
underlying cause.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
