WithModTimeUpperBound sets the upper bound value of the ModTime property
of the tar entry structs.
WithSourceDateEpoch now implies WithModTimeUpperBound too,
in addition to WithWhiteoutTime.
For moby/buildkit issue 3296
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Remove nolint-comments that weren't hit by linters, and remove the "structcheck"
and "varcheck" linters, as they have been deprecated:
WARN [runner] The linter 'structcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter. Replaced by unused.
WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter. Replaced by unused.
WARN [linters context] structcheck is disabled because of generics. You can track the evolution of the generics support by following the https://github.com/golangci/golangci-lint/issues/2649.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- fix "nolint" comments to be in the correct format (`//nolint:<linters>[,<linter>`
no leading space, required colon (`:`) and linters.
- remove "nolint" comments for errcheck, which is disabled in our config.
- remove "nolint" comments that were no longer needed (nolintlint).
- where known, add a comment describing why a "nolint" was applied.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes diff archives to be reproducible.
The value is expected to be passed from CLI applications via the $SOUCE_DATE_EPOCH env var.
See https://reproducible-builds.org/docs/source-date-epoch/
for the $SOURCE_DATE_EPOCH specification.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
It looks like this function was converting the time (`windows.NsecToTimespec()`),
only to convert it back (`windows.TimespecToNsec()`). This became clear when
moving the lines together:
```go
ctimespec := windows.NsecToTimespec(ctime.UnixNano())
c := windows.NsecToFiletime(windows.TimespecToNsec(ctimespec))
```
And looking at the Golang code, it looks like they're indeed the exact reverse:
```go
func TimespecToNsec(ts Timespec) int64 { return int64(ts.Sec)*1e9 + int64(ts.Nsec) }
func NsecToTimespec(nsec int64) (ts Timespec) {
ts.Sec = nsec / 1e9
ts.Nsec = nsec % 1e9
return
}
```
While modifying this code, also renaming the `e` variable to a more common `err`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When unpacking a TAR archive, containerd preserves file's owner:
https://github.com/containerd/containerd/blob/main/archive/tar.go#L384
In some cases this behavior is not desired. In current implementation we
avoid `Lchown` on Windows. Another case when this should be skipped is
when using native snapshotter on darwin and running as non-root user.
This PR extracts a generic option - `WithNoSameOwner` (same as
`tar --no-same-owner`) to skip `Lchown` when its not required.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
This code was copied from github.com/moby/moby/pkg/archive;
28842d3f09,
which got later simplified in
a5aed699cf
This patch aligns the containerd implementation with those changes, and uses
filepath.ToSlash() unconditionally on all platforms, as it's a no-op on platforms
that use a forward-slash; https://github.com/golang/go/blob/go1.19/src/path/filepath/path.go#L175-L183
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These functions were originally copied from the docker / moby repository in
4a7a8efc2d. Migrating these functions to use the
github.com/moby/sys/sequential module allows them being shared between moby,
docker/cli, and containerd, and to allow using them without importing all of sys
which also depends on hcsshim and more.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`ioutil` has been deprecated by golang. All the code in `ioutil` just
forwards functionality to code in either the `io` or `os` packages.
See https://github.com/golang/go/pull/51961 for more info.
Signed-off-by: Jeff Widman <jeff@jeffwidman.com>
This commit copies the fuzzers from the repository except for
containerd_import_structured_fuzzer.go.
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
Different tar(1) implementations default to different input and output
locations when none is specified. This can include tape devices like
/dev/st0 (on Linux) or /dev/sa0 (on FreeBSD), but may be overridden by
compilation options or environment variables. Using the f option with
the special value of - instructs tar(1) to read from stdin and write to
stdout instead of the default.
Signed-off-by: Samuel Karp <me@samuelkarp.com>
Calling link(2) with a symlink as the target will cause FreeBSD to
attempt to create a new hard link pointing to the target of the symlink
rather than a hardlink to the symlink itself. By contrast, Linux creates
a hardlink to the symlink.
Use linkat(2) instead, which accepts a flag controlling this behavior.
If linkat(2) is called with AT_SYMLINK_FOLLOW, it will behave the same
as link(2). If linkat(2) is called without AT_SYMLINK_FOLLOW, it will
behave the same as Linux's link(2) instead.
See FreeBSD's implementation of ln(1), which uses linkat(2) and controls
this behavior by way of the -P and -L flags:
3003117253/bin/ln/ln.c (L342-L343)
Signed-off-by: Samuel Karp <me@samuelkarp.com>
Before:
```
$ nerdctl pull gcr.io/kubeflow-images-public/tensorflow-1.14.0-notebook-cpu:v0.7.0
FATA[0026] failed to extract layer sha256:f8a604834d388fd3d04c26e4ed832b36b617ea8a4e0b1665b9199bd10cfcb171: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1818823128:
lchown /var/lib/containerd/tmpmounts/containerd-mount1818823128/usr/local/bin/docker-credential-gcr: invalid argument: unknown
```
After:
```
$ nerdctl pull gcr.io/kubeflow-images-public/tensorflow-1.14.0-notebook-cpu:v0.7.0
FATA[0027] failed to extract layer sha256:f8a604834d388fd3d04c26e4ed832b36b617ea8a4e0b1665b9199bd10cfcb171: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount3521205359:
failed to Lchown "/var/lib/containerd/tmpmounts/containerd-mount3521205359/usr/local/bin/docker-credential-gcr" for UID 205001, GID 5000:
lchown /var/lib/containerd/tmpmounts/containerd-mount3521205359/usr/local/bin/docker-credential-gcr: invalid argument
(Hint: try increasing the number of subordinate IDs in /etc/subuid and /etc/subgid): unknown
```
Related to moby/moby issue 43576 but for containerd
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
As a matter of fact, there are two frame formats defined by Zstandard: Zstandard frames and Skippable frames.
So we should probably support zstd algorithms with skippable frames.
See https://tools.ietf.org/id/draft-kucherawy-dispatch-zstd-00.html#rfc.section.2 for more details.
Signed-off-by: Da McGrady <dabkb@aol.com>
On FreeBSD + zfs, stat call seem to return garbage in RDev for regular
files & folders. The value returned is large enough not to fit into
`Devmajor` & `Devminor` fields of the tar header. Fortunately, these
fields are required just for special devices.
This change
* adds a check into `setHeaderForSpecialDevice` that the
input header represents a special device. If it's not the case, we
don't set the Devmajor & Devminor fields.
* fixes the nil check on `getxattr`: it never returns nils, but rather
an empty slice instead
Signed-off-by: Artem Khramov <akhramov@pm.me>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows. This issue also
affects Unix users who have “.” listed explicitly in their PATH and are running
“go get” outside of a module or with module mode disabled.
This issue is not limited to the go command itself, and can also affect binaries
that use `os.Command`, `os.LookPath`, etc.
From the related blogpost (ttps://blog.golang.org/path-security):
> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing
This patch replaces all uses of `os/exec` with `golang.org/x/sys/execabs`. While
some uses of `os/exec` should not be problematic (e.g. part of tests), it is
probably good to be consistent, in case code gets moved around.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove build tags which are already implied by the name of the file.
Ensures build tags are used consistently
Signed-off-by: Derek McGowan <derek@mcg.dev>
Move to single lchmod interface mirroring other implementations.
Separate logic for freebsd which supports symlink no follow flag.
Signed-off-by: Derek McGowan <derek@mcg.dev>
This parallels the implementation of windowsDiff.Apply, including
bouncing very briefly though archive.WriteDiff and then straight back
out into Windows-specific code.
It's mostly pulling existing mechanisms from non-Windows Compare or
Windows Apply, and highlights that there's probably a lot of scope for
refactoring on top of this.
Now the export-related integration tests pass CI on Windows.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
zstd is a compression algorithm that has a very fast decoder, while
providing also good compression ratios. The fast decoder makes it
suitable for container images, as decompressing the tarballs is a very
expensive operation.
https://github.com/opencontainers/image-spec/pull/788 added support
for zstd to the OCI image specs.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Specifically, all the functions above applyWindowsLayer are actually
used by the (generic) applyNaive code, while the functions below this
point are specific to applyWindowsLayer.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
applyFunc now takes an io.Reader instead of a tar.Reader because I'm
trying to mirror the API of the not-yet-exposed implementation of this
same behaviour in github.com/Microsoft/hcsshim/internal/ociwclayer,
with an eye to later moving to that implementation it is ever exposed.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
