This fixes the issue with the usage of the deprecated attribute.Any
function that original caused build issues.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This test will make sure there aren't any issues with multilayered
images during import. Keep in mind that in the case of multilayered
images, they have to be unpacked first in order to be usable.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
In linux platform, the shim server always listens on the socket before
the containerd task manager dial it. It is unlikely that containerd task
manager should handle reconnect because the shim can't restart. For this
case, the containerd task manager should fail fast if there is ENOENT or
ECONNREFUSED error.
And if the socket file is deleted during cleanup the exited task, it
maybe cause that containerd task manager takes long time to reload the
dead shim. For that task.v2 manager, the race case is like:
```
TaskService.Delete
TaskManager.Delete(runtime/v2/manager.go)
shim.delete(runtime/v2/shim.go)
shimv2api.Shutdown(runtime/v2/task/shim.pb.go)
<- containerd has been killed or restarted somehow
bundle.Delete
```
The shimv2api.Shutdown will cause that the shim deletes socket file
(containerd-shim-runc-v2 does). But the bundle is still there. During
reloading, the containerd will wait for the socket file appears again
in 100 seconds. It is not reasonable. The Reconnect should prevent this
case by fast fail.
Closes: #5648.
Fixes: #5597.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
During import, if an image does not match the host's platform,
it won't have any children labels set, which will result in the
Garbage Collector deleting its content later, resulting in an
unusable image. In this case, we should fail early.
This can still be bypassed by using ctr import --all-platforms.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
This picks a fix to properly handle images containing symlinks
inside which point to an unexisting file.
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Windows HostProcess containers can run containerized workloads on a Windows host.
These containers operate as normal processes but have access to the host network
namespace, storage, and devices when given the appropriate user privileges.
HostProcess containers support the ability to run as one of the following Windows
service accounts: LocalSystem, LocalService, NetworkService.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
The test sets container's Linux.SecurityContext.NamespaceOptions.Pid = NamespaceMode_CONTAINER,
which will ensure that the container keeps running even if the sandbox container dies. We do
not have that option on Windows.
Adds additional logging in the test, so it is easier to figure out which assertion failed.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
This version brings in some bug fixes to layer handling. The actual fix isn't
present in the diff as it's not used here, but the Windows shim is built from
the tag present in go.mod, so the fix will be in the Windows shim on a new release
of Containerd if this tag is in.
Signed-off-by: Daniel Canter <dcanter@microsoft.com>
With the introduction of Windows Server 2022, some images have been updated
to support WS2022 in their manifest list. This commit updates the test images
accordingly.
Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows. This issue also
affects Unix users who have “.” listed explicitly in their PATH and are running
“go get” outside of a module or with module mode disabled.
This issue is not limited to the go command itself, and can also affect binaries
that use `os.Command`, `os.LookPath`, etc.
From the related blogpost (ttps://blog.golang.org/path-security):
> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing
This patch replaces all uses of `os/exec` with `golang.org/x/sys/execabs`. While
some uses of `os/exec` should not be problematic (e.g. part of tests), it is
probably good to be consistent, in case code gets moved around.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/pelletier/go-toml/compare/v1.8.1...v1.9.3
- v1.9.3: Clarify license and comply with Apache 2.0
- v1.9.2: Add Encoder.CompactComments to omit extra new line
- v1.9.1: Fix empty trees line counting
v1.9.0
-------------------
The highlight of this version is that the whole toml.Tree structure has been made
public in a backward compatible way. This allows everyone using v1.x to fully
access the data and metadata in the tree to extend the library.
This is hopefully the last release in the v1.x track, as go-toml v2 is the main
focus of development.
What's new
- TOML 1.0.0-rc.3
- Improved default tag for durations
- Provide Tree and treeValue public aliases
- Expose MarshalOrder
- Value string representation public function
Fixed bugs
- Do not allow T-prefix on local dates
- toml.Unmarshaler supports leaf nodes
- Fix date lexer to only support 4-digit year
- Fix ToMap for tables in mixed-type arrays
- Fix ToMap for tables in nested mixed-type arrays
- Support literal multiline marshal
Performance
- Remove date regexp
- Remove underscore regexps
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- task.Kill() might fail in theory
- Giving a longer timeout may help us understand whether the failure is
a timing issue or not.
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
This brings in some cri api changes for cgroups, Windows pod sandbox security
context changes and some new fields for the Windows version of a privileged
container.
This also unfortunately bumps the prometheus client, grpc middleware, bolt
and klog :(
Signed-off-by: Daniel Canter <dcanter@microsoft.com>
Remove build tags which are already implied by the name of the file.
Ensures build tags are used consistently
Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Jayme Howard <g.prime@gmail.com>
Run `go mod tidy`
Signed-off-by: Jayme Howard <g.prime@gmail.com>
Follow correct procedure by running `make vendor`
Signed-off-by: Jayme Howard <g.prime@gmail.com>
In containerd 1.5.x, we introduced support for go modules by adding a
go.mod file in the root directory. This go.mod lists all the things
needed across the whole code base (with the exception of
integration/client which has its own go.mod). So when projects that
need to make calls to containerd API will pull in some code from
containerd/containerd, the `go mod` commands will add all the things
listed in the root go.mod to the projects go.mod file. This causes
some problems as the list of things needed to make a simple API call
is enormous. in effect, making a API call will pull everything that a
typical server needs as well as the root go.mod is all encompassing.
In general if we had smaller things folks could use, that will make it
easier by reducing the number of things that will end up in a consumers
go.mod file.
Now coming to a specific problem, the root containerd go.mod has various
k8s.io/* modules listed. Also kubernetes depends on containerd indirectly
via both moby/moby (working with docker maintainers seperately) and via
google/cadvisor. So when the kubernetes maintainers try to use latest
1.5.x containerd, they will see the kubernetes go.mod ending up depending
on the older version of kubernetes!
So if we can expose just the minimum things needed to make a client API
call then projects like cadvisor can adopt that instead of pulling in
the entire go.mod from containerd. Looking at the existing code in
cadvisor the minimum things needed would be the api/ directory from
containerd. Please see proof of concept here:
github.com/google/cadvisor/pull/2908
To enable that, in this PR, we add a go.mod file in api/ directory. we
split the Protobuild.yaml into two, one for just the things in api/
directory and the rest in the root directory. We adjust various targets
to build things correctly using `protobuild` and also ensure that we
end up with the same generated code as before as well. To ensure we
better take care of the various go.mod/go.sum files, we update the
existing `make vendor` and also add a new `make verify-vendor` that one
can run locally as well in the CI.
Ideally, we would have a `containerd/client` either as a standalone repo
or within `containerd/containerd` as a separate go module. but we will
start here to experiment with a standalone api go module first.
Also there are various follow ups we can do, for example @thaJeztah has
identified two tasks we could do after this PR lands:
github.com/containerd/containerd/pull/5716#discussion_r668821396
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
systemd uses SIGRTMIN+n signals, but containerd didn't support the signals
since Go's sys/unix doesn't support them.
This change introduces SIGRTMIN+n handling by utilizing moby/sys/signal.
Fixes#5402.
https://www.freedesktop.org/software/systemd/man/systemd.html#Signals
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
For Windows, the container image's OS version must closely match the host's OS version.
For this reason, we need to add the --os-version annotation in image manifest lists,
so the Windows nodes can pull the appropriate image from the list.
Previously, the docker manifest CLI did not have the capability to set the --os-version,
it, but it has been introduced in docker 20.10.0.
We're also adding busybox.exe in the image, so we can run Linux commands inside the
container, so the tests will be simpler.
When building Windows images, a docker buildx builder needs to be created and used. When
building Windows images with docker buildx, the flag --output=type=registry is required,
otherwise it cannot be referenced on a Linux node.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
If a container failed to start due to a bad command, the container could not be
recreated with a proper command in its stead. Adds a test that verifies this scenario.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
A previous commit introduced EnsureImageExists, which ensures that a particular image
already exists. It also deduplicates the image pulling code. Some tests missed this
update.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
Most of the tests creating and using Pod Sandboxes in the same way. We can create
a common function that will do that for us, so we can have less duplicated code,
and make it easier to add new tests in the future.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
There was a known issue regarding how the symlink files mounted as
volumes were being handled on Windows. This commit adds tests that
will check against those issue to ensure there won't be any
regressions.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
Currently, the cri-integration tests do not work on Windows due to various reasons.
One of the reasons is because all the tests are using Linux-specific images.
Previous commits refactored the image pulling / usage in the cri-integration tests,
making it easier to update, and easier to configure a custom registry to pull images
with Windows support.
For Windows runs, custom registries can be created, which will also contain Windows
images, and the cri-integration tests can be configured to use those registries by
specifying the "--repo-list" argument, a YAML file which will contain an alternative
mapping of the default registries. This is similar to how E2E tests are handled for
Windows runs in Kubernetes.
Some of the tests are Skipped, as they do not pass yet on Windows.
Windows does not collect inodes used stats, thus, the tests that were expecting non-zero
inodes stats were failing.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
This forces vendoring to only take dependencies of this repository to
be taken into account, effectively cutting the circular dependency (for
the vendored code), and to prevent depending on transitive dependencies
coming from older versions of containerd.
go mod does not allow using the main module as a local "replace" rule using
a path; see golang/go#45492 and golang/go#34417, so instead, an empty module
is used.
One change observed is that older versions containerd depended on an older
version of imgcrypt that had an "indirect" dependency on more current versions
of gopkg.in/yaml.v2 and prometheus/procfs.
For those, a temporary "indirect" dependency was added, until prometheus/client_golang
and kubernetes are updated.
from go mod graph (before):
github.com/containerd/imgcrypt@v1.0.4-0.20210301171431-0ae5c75f59ba gopkg.in/yaml.v2@v2.4.0
github.com/containerd/imgcrypt@v1.0.4-0.20210301171431-0ae5c75f59ba github.com/prometheus/procfs@v0.6.0
For some reason, some older versions of containerd are still taken into account,
causing satori/go.uuid to be added as "indirect" dependency, likely because some
modules have this dependency in their go.sum. This should likely disappear once
those plugins are updated to contain a current version of containerd.
git grep 'github.com/satori/go.uuid'
vendor/github.com/Microsoft/hcsshim/go.sum:github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
vendor/github.com/containerd/aufs/go.sum:github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
vendor/github.com/containerd/imgcrypt/go.sum:github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
vendor/github.com/containerd/nri/go.sum:github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
vendor/github.com/containerd/zfs/go.sum:github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We can check that the reported containers stats are not lower than it should
by using the resource-consumer image. This image contains stress / testlimit.exe
which can consume a specified amount of memory.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
Most of the tests are pulling and deleting the same test images, which
can be quite inefficient, especially on Windows nodes, where the images
are larger than the Linux ones (a nanoserver Container image is ~250MB
in size). We can instead pull them only once, and reuse them. This will
reduce the test run time on Windows considerably.
Additionally, some of the test images are currently hosted on dockerhub
(busybox image), which has introduced image ratelimiting in November 2020,
which means that test runners could potentially hit that limit faster with
the current implementation. This will reduce that risk.
Some tests are specifically deleting images, so we always have to ensure
that they are pulled.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
Currently, the cri-integration tests do not work on Windows due to various reasons.
One of the reasons is because all the tests are using Linux-specific images. This
commit refactors the image pulling / usage in the cri-integration tests, making it
easier to update, and easier to configure the a custom registry to pull those images
from.
For Windows runs, custom registries can be created, which will also contain Windows
images, and the cri-integration tests can be configured to use those registries by
specifying the "--image-list" argument, a TOML file which will contain an alternative
mapping of the default images.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
- Add a multi-arch image with linux/amd64 and linux/arm64 (limit to just
what we are running in CI)
- Bump versions to 2.0 as 1.0 is the single-arch image
- Use `docker buildx` instead of just docker, so we don't need to build
manifests by hand
- busybox now does not have `nogroup`, since the test needs it, switch
over from busybox to ubuntu for just the volume-ownership image
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
full diff: https://github.com/google/uuid/compare/v1.1.2...v1.2.0
I did not switch to us the `uuid.NewString()` function; we are not currently using
`uuid.New()`, probably because it can `panic` in case an invalid UUID is produced;
in stead we are using `uuid.NewRandom()`, which is equivalent, but returns an
error instead.
Changes:
- v1.2.0: Add NewString()
This release introduces the NewString() function which is the equivalent of
uuid.New().String().
- v1.1.5: Syntactic cleanup
There are no code changes. A missing period was add to a godoc comment and the
linter was told to not complain that the results of hash.Write() are ignored
(the function cannot fail)
- v1.1.4: Further error optimizations
Do not allocate memory for errors (it is only one word)
- v1.1.3: Optimize error reporting
Optimize length of time it takes to discover an input is bad by no longer using
fmt.Errorf, which is quite slow. It now uses a custom error type that formats
the string when the Error method is called rather than when generating the error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: https://github.com/containernetworking/plugins/compare/v0.8.6...v0.9.1
changes in vendored code:
- (in containernetworking/plugins): Fix race condition in GetCurrentNS
- (in containernetworking/cni): tighten up plugin-finding logic
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This parallels the implementation of windowsDiff.Apply, including
bouncing very briefly though archive.WriteDiff and then straight back
out into Windows-specific code.
It's mostly pulling existing mechanisms from non-Windows Compare or
Windows Apply, and highlights that there's probably a lot of scope for
refactoring on top of this.
Now the export-related integration tests pass CI on Windows.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
```bash
curl -s https://mirror.gcr.io//v2/library/busybox/tags/list | jq '[.tags ]'
[
[
"1.26.2",
"1.27.2",
"1.28",
"1.29",
"1.29.2",
"1.30",
"1.30.1",
"1.31",
"1.31.0",
"1.31.1",
"1.32.0"
]
]
```
The latest is gone. I think we should setup image in github container
registry for CI if possible.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Having this log should not be critical, and removing it allows
using this package without pulling in the explicit dependency
on k8s.io/klog.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
task.Checkpoint should check status of container before handle
checkpoint. If the container has been paused, task.Checkpoint should
handle checkpoint and leave it paused.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Signed-off-by: Baijia <baijia.wr@antfin.com>
Due to the lack of an existing /proc dir in the rootfs of busybox, there
seems to be a race between the mount of /proc and whether the root dir
is already readonly? May need investigation, but for now use an image
that has a /proc dir existing.
Signed-off-by: Phil Estes <estesp@amazon.com>
While the issue hasn't been fixed in the kernel yet, we can workaround
the issue by not using overlayfs snapshotter.
The newly added step runs all tests that match /TestCheckpoint/.
So, TestCRWithImagePath has been renamed to match the regexp.
Fixes#3930.
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
Now that they are using a multi-arch image, they should work on Windows
like they work elsewhere.
This also means non-AMD64 platforms do this test with their native
platform version, not the linux/amd64 platform version.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
This reduces the need to pull random images from docker.io, and should
greatly reduce the tendancy to hit their hourly rate-limit during
integration test runs.
TestImagePullSomePlatforms uses k8s.gcr.io/pause:3.2 so that it does not
see the content pulled by TestImagePullAllPlatforms. This image is
multi-arch, but not multi-os.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
task.Delete() will try to kill the task before delete it,
but once the task is killed, the TaskExit event will also
tigger another task.Delete() which would conflict with this
test task.Delete(). To avoid this conflict, kill the task instead
of Delete it, and let TaskExit trigger task.Delete().
Signed-off-by: lifupan <lifupan@gmail.com>
