This change adds a new workflow that builds the volume test images
and pushes them to a remote registry.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Following PR #6284, the Windows Periodic Workflow is prevented from
running on any repository outside of the original.
While this achieves the goal of preventing senseless Windows workflow
failures in contributors' forks, it makes running the workflow for
contributors (even manually) impossible.
This PR adds a separate workflow file which triggers the Windows
Integration workflow iff it is being run off of the original repository,
thus maintaining the upstream scheduling, while allowing contributors to
manually trigger the workflow on their forks if they so desire.
Signed-off-by: Nashwan Azhari <nazhari@cloudbasesolutions.com>
In the Windows CI's periodic runs the `azure/CLI` GitHub action library
is leveraged to run various Azure-related commands.
To avoid possible desyncing between the auth libraries of `azure/Login`
and `azure/CLI` as described
[here](https://github.com/Azure/cli/issues/56#issuecomment-958705517),
this patch neglects to pass an explicit `azcliversion` to `azure/CLI` in
order to have it default to the Azure CLI version set up by `azure/Login`.
Signed-off-by: Nashwan Azhari <nazhari@cloudbasesolutions.com>
This moves all the release builds into a Dockerfile which is a bit
cleaner for setting up our build environment.
Non-linux/amd64 builds are cross-compiled.
Currently onlinux linux/amd64, linux/arm64, and windows/amd64 are
supported, but is easy to add more, provided their is a cross-compile
toolchain available for it.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
go1.17.3 (released 2021-11-04) includes security fixes to the archive/zip and
debug/macho packages, as well as bug fixes to the compiler, linker, runtime, the
go command, the misc/wasm directory, and to the net/http and syscall packages.
See the Go 1.17.3 milestone on our issue tracker for details.
From the announcement e-mail:
[security] Go 1.17.3 and Go 1.16.10 are released
We have just released Go versions 1.17.3 and 1.16.10, minor point releases.
These minor releases include two security fixes following the security policy:
- archive/zip: don't panic on (*Reader).Open
Reader.Open (the API implementing io/fs.FS introduced in Go 1.16) can be made
to panic by an attacker providing either a crafted ZIP archive containing
completely invalid names or an empty filename argument.
Thank you to Colin Arnott, SiteHost and Noah Santschi-Cooney, Sourcegraph Code
Intelligence Team for reporting this issue. This is CVE-2021-41772 and Go issue
golang.org/issue/48085.
- debug/macho: invalid dynamic symbol table command can cause panic
Malformed binaries parsed using Open or OpenFat can cause a panic when calling
ImportedSymbols, due to an out-of-bounds slice operation.
Thanks to Burak Çarıkçı - Yunus Yıldırım (CT-Zer0 Crypttech) for reporting this
issue. This is CVE-2021-41771 and Go issue golang.org/issue/48990.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The latest version of AZ CLI breaks the windows-periodic workflow.
See:
https://github.com/Azure/cli/issues/56
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
go1.17.2 (released 2021-10-07) includes a security fix to the linker and misc/wasm
directory, as well as bug fixes to the compiler, the runtime, the go command, and
to the time and text/template packages. See the Go 1.17.2 milestone on our issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.17.2+label%3ACherryPickApproved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Allow overwriting the target tag to support mirror images from multiple
sources under our single namespace.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Some extra steps were added for WS2022 in accordance with
the published image on Azure:
- Install Container Feature & reboot VM
- Create NAT network
Temporarily we skip critest steps for WS2022 until all test images
are updated in that project.
Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
Go 1.17 go mod download step (used to handle the separate integration
go.mod) seems to do a lot more work/validation than prior Go releases,
requiring more time for integration runs.
Signed-off-by: Phil Estes <estesp@amazon.com>
go1.16.7 (released 2021-08-05) includes a security fix to the net/http/httputil
package, as well as bug fixes to the compiler, the linker, the runtime, the go
command, and the net/http package. See the Go 1.16.7 milestone on the issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.16.7+label%3ACherryPickApproved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For some reason the Linux CI runs end up using go 1.15.14 instead of 1.16.6 for
the Windows runs, or any of the other CI steps. Not sure if this is due to
the linter installing it's own version of go or something else.
Signed-off-by: Daniel Canter <dcanter@microsoft.com>
We no longer would need go 1.13.x for moby/containerd compatibility with
containerd moving to 1.16.x
Signed-off-by: Alakesh Haloi <alakeshh@amazon.com>
The current latest version of CRIU is 3.15 and soon will be released
3.16. If CRIU is installed from PPA it would always test with the
latest released version.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
- ensure that the root go.mod and the module specific go.mod have the
same `require` and `replace` directives for different dependencies.
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
In containerd 1.5.x, we introduced support for go modules by adding a
go.mod file in the root directory. This go.mod lists all the things
needed across the whole code base (with the exception of
integration/client which has its own go.mod). So when projects that
need to make calls to containerd API will pull in some code from
containerd/containerd, the `go mod` commands will add all the things
listed in the root go.mod to the projects go.mod file. This causes
some problems as the list of things needed to make a simple API call
is enormous. in effect, making a API call will pull everything that a
typical server needs as well as the root go.mod is all encompassing.
In general if we had smaller things folks could use, that will make it
easier by reducing the number of things that will end up in a consumers
go.mod file.
Now coming to a specific problem, the root containerd go.mod has various
k8s.io/* modules listed. Also kubernetes depends on containerd indirectly
via both moby/moby (working with docker maintainers seperately) and via
google/cadvisor. So when the kubernetes maintainers try to use latest
1.5.x containerd, they will see the kubernetes go.mod ending up depending
on the older version of kubernetes!
So if we can expose just the minimum things needed to make a client API
call then projects like cadvisor can adopt that instead of pulling in
the entire go.mod from containerd. Looking at the existing code in
cadvisor the minimum things needed would be the api/ directory from
containerd. Please see proof of concept here:
github.com/google/cadvisor/pull/2908
To enable that, in this PR, we add a go.mod file in api/ directory. we
split the Protobuild.yaml into two, one for just the things in api/
directory and the rest in the root directory. We adjust various targets
to build things correctly using `protobuild` and also ensure that we
end up with the same generated code as before as well. To ensure we
better take care of the various go.mod/go.sum files, we update the
existing `make vendor` and also add a new `make verify-vendor` that one
can run locally as well in the CI.
Ideally, we would have a `containerd/client` either as a standalone repo
or within `containerd/containerd` as a separate go module. but we will
start here to experiment with a standalone api go module first.
Also there are various follow ups we can do, for example @thaJeztah has
identified two tasks we could do after this PR lands:
github.com/containerd/containerd/pull/5716#discussion_r668821396
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
full diff: https://github.com/cpuguy83/go-md2man/compare/v2.0.0...v2.0.1
- Fix handling multiple definition descriptions
- Fix inline markup causing table cells to split
- Remove escaping tilde character (prevents tildes (`~`) from disappearing).
- Do not escape dash, underscore, and ampersand (prevents ampersands (`&`) from disappearing).
- Ignore unknown HTML tags to prevent noisy warnings
Note that this only updates the binaries we install. The vendor code also
includes go-md2man (as indirect dependency of urfave/cli). I don't think we use that
feature, so I did not add it to our go.mod
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `cri-in-userns` stage is for testing "CRI-in-UserNS", which should be used in conjunction with "Kubelet-in-UserNS":
https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2033-kubelet-in-userns-aka-rootless
This feature is mostly expected to be used for `kind` and `minikube`.
Requires Rootless Docker/Podman/nerdctl with cgroup v2 delegation: https://rootlesscontaine.rs/getting-started/common/cgroup2/
(Rootless Docker/Podman/nerdctl prepares the UserNS, so we do not need to create UserNS by ourselves)
Usage:
```
podman build --target cri-in-userns -t cri-in-userns -f contrib/Dockerfile.test .
podman run -it --rm --privileged cri-in-userns
```
The stage is tested on CI with Rootless Podman on Fedora 34 on Vagrant.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Due to a change in Go, the go.mod file cannot declare a version of Go
above the installed `go version`; until the default Go version in GitHub
actions virt environments is 1.16, we have to install 1.16 before
running the project checks now.
Signed-off-by: Phil Estes <estesp@amazon.com>
Using `-E` preserves environment variables, except for PATH, so PATH has to be
manually set to match the current environment.
I removed env-vars that were redundant (such as `GOPATH=$GOPATH`), which should
be handled by `-E`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
cri-tools is hardcoded to use images which are broken
within their registry. Disable the tests to unblock
CI until fixed.
Signed-off-by: Derek McGowan <derek@mcg.dev>
This allows us to dig more details out of test runs and maintain a
better history.
For this we can use `gotestsum`, which is a utility that wraps `go test`
so that it outputs test2json (go's format) and output junit (a format
more easily imported into other systems).
The PR makes it possible to override the Makefile's use of `go test` to
use any other command tto executet the test. For CI we'll use `gotestsum
--`, where `gotestsum` expects everything after the `--` to be flags for
`go test`.
We then use environment variables to configure `gotestsum` (e.g.
`GOTESTSUM_JUNITFILE` is an env var accepted by `gotestsum`).
For cri tests, the test suite supports outputing test results to a
directory, these are in junit format already. The file is not named
properly just because the code that creates it (in ginkgo) is not
configured well. We can fix that upstream to give us a better name...
until then I'm keeping those results in a separate dir.
A second workflow is also added so the test results can be summed up and
a report added to the workflow run. The 2nd workflow is required for
this since PR runs do not have access to do some of this due to safety
reasons
(https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Though we don't officially support Apple platform, we should
at least run unit tests to make sure things are not broken.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
The integration test times have slightly increased and are often
hitting the 25 minutes timeout. This increases to give more room
but still keeps it low enough to catch regressions in tests
causing longer than expected execution.
Signed-off-by: Derek McGowan <derek@mcg.dev>
While the issue hasn't been fixed in the kernel yet, we can workaround
the issue by not using overlayfs snapshotter.
The newly added step runs all tests that match /TestCheckpoint/.
So, TestCRWithImagePath has been renamed to match the regexp.
Fixes#3930.
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
To investigate issues like #4969, it would be helpful to understand
the status of the VM at the end.
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
Without a name the logs use a carriage return followed by the long
comment as the name of the job step which is messy when working with the
actions API/logs.
Signed-off-by: Phil Estes <estesp@amazon.com>
Since both cri-containerd and runC runtime are widely used, the relevent
information should include runC and CRI configuration when file bug.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
The CI host was probably updated recently and the permission bits of the
directory was changed.
Fix 5077
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
We have enough failures these days; getting timed out when tests are
almost done is the last thing we need :)
On avg. the Linux integration tests are taking 15-17 min, but sometimes
they end up at 20 or a bit over and get canceled. I've seen rare cases
where the Vagrant setup+build+test runs gets very close to 40 min as
well.
Signed-off-by: Phil Estes <estesp@amazon.com>
https://github.com/actions/setup-go/tree/v2.1.3#v2
The V2 offers:
- Adds GOBIN to the PATH
- Proxy Support
- stable input
- Bug Fixes (including issues around version matching and semver)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
download.fedoraproject.org gives HTTP 404 at times,
breaking the CI. Let's give it another chance.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `.github/ISSUE_TEMPLATE/config.yml` to clarify where is the right place (in the most cases) to ask questions.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
CI was timing out after 15 minutes on the crun tests; extending
the timeout to 20 minutes (we can make it shorter again if we know
the exact time it takes to run)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Noticed this in the CI output:
Requested golangci-lint 'v1.29', using 'v1.29.0', calculation took 7969ms
Installing golangci-lint v1.29.0...
Downloading https://github.com/golangci/golangci-lint/releases/download/v1.29.0/golangci-lint-1.29.0-darwin-amd64.tar.gz ...
Using nearly 8 seconds to convert v1.29 to v1.29.0 seems a bit long,
so hard-coding to the full version to speedup CI somewhat.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With container-selinux policy updated to 2.145+ (the default for Fedora 32+) we
can enable SELinux=Enforcing mode in the CI workflow and pass all integration
and CRI tests except one, see https://github.com/containerd/containerd/issues/4460,
which has been marked as skipped.
Tested locally with:
- SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-integration
- SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-cri
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
These checks had to be disabled to get the CRI merge completed.
Now these should be added back.
After CRI merge, more time for lint is needed on mac.
Signed-off-by: Derek McGowan <derek@mcg.dev>
Add installation scripts needed to generate CRI + CNI tar package on
release.
Update Github action release script to generate CRI release tarballs for
Linux and Windows.
Signed-off-by: Derek McGowan <derek@mcg.dev>
`vagrant up` will build and install containerd and all dependencies,
setting up proper SELinux contexts on the runc and containerd binaries.
The VM is configured to be SELinux Enforcing by default but this gets
changed during various CI passes via a matrix param to Disabled and
Permissive before running tests. I have an open PR to fix the
container-selinux policy for containerd at
https://github.com/containers/container-selinux/pull/98 which once
accepted we will want to update the CI matrix to use Enforcing mode
instead of Permissive.
All tests currently pass in SELinux permissive mode with containerd
configured with `enable_selinux=true`. To see which tests are failing
with SELinux enforcing and an already spun up VM:
`SELINUX=Enforcing vagrant up --provision-with=selinux,test-cri`
To test SELinux enforcing in a new VM:
`vagrant destroy -force; SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-cri`
The `selinux` shell provisioner, parameterized by the SELINUX envvar,
will configure the system as you would expect, with the side effect that
containerd is configured with `enable_selinux=true` via
`/etc/containerd/config.toml` for Permissive or Enforcing modes and
`enable_selinux=false` when SELINUX=Disabled.
Provided that virtualization is suported, this Vagrantfile and provisioners
make it easy to test containerd/cri for conformance under SELinux on
non-SELinux systems.
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
Recent changes removed the need for libseccomp-dev when building
containerd. The btrfs tools package is already installed on GH Actions
runners and was already a no-op so the whole step can be removed.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
This will ensure that we can always get the current Windows OS build
version, without being put into Windows 8 compatibility mode.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
We are going to deprecate shim v1 (#4365), but it is still early to
disable the tests for them
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
GH runners now have a systemd-run containerd running on the standard
socket, impacting the CRI test's expectation of our CI-built containerd
running there.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
The previous names were confusing because v1 may look like shim API v1 (io.containerd.runtime.v1.linux").
Also, v2 may look like cgroup v2.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
- always apt-get update before installing packages
- move to tagged official create_release action
The official GH create_release action now has support for body text from
file.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
[BeforeEach] [k8s.io] Security Context
/home/runner/work/containerd/containerd/src/github.com/kubernetes-sigs/cri-tools/pkg/framework/framework.go:50
W0624 12:26:28.532644 30569 util_unix.go:103] Using "/var/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///var/run/containerd/containerd.sock".
W0624 12:26:28.532700 30569 util_unix.go:103] Using "/var/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///var/run/containerd/containerd.sock".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This will check that the tag is signed and then checkout the tag, build
official binaries, sha256sum the tarball, and upload those assets to the
release, officially generating a release in GitHub from the signed tag.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
- Remove dependency on libcontainer/system
- Get rid of socat for port forwarding
- Roll docker/distribution back to latest (v2.7.1) release
Now that 901bcb2231 was merged in containerd,
we no longer depend on the ParseDockerRef utility from docker/distribution,
so we can safely roll back to the latest release for this dependency.
- vendor: kubernetes v1.18.2
Fix client watch reestablishment handling of client-side timeouts
- Add config flag to default empty seccomp profile
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
omit sudo when EUID eqto 0
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
omit sudo when EUID eqto 0
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
omit sudo when EUID eqto 0
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
use gosu to omit sudo in GA
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
use gosu to omit sudo in GA
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
set working-dir for <<Setup gosu>> step
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
fix job permissions
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
Integration tests were running with latest Go release rather than the
version used everywhere else. Also, we don't need to install protoc from
tarball and also apt-get the package for Ubuntu when used as a
dependency for criu build.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Also few changes to names of workflow jobs. Integration tests can get
canceled without full completing at 10 minutes. Make sure we only use
checkout v2 action.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
full diff: https://github.com/golang/go/compare/go1.13.6...go1.13.7
go1.13.7 (released 2020/01/28) includes two security fixes. One mitigates
the CVE-2020-0601 certificate verification bypass on Windows. The other affects
only 32-bit architectures.
https://github.com/golang/go/issues?q=milestone%3AGo1.13.7+label%3ACherryPickApproved
- X.509 certificate validation bypass on Windows 10
A Windows vulnerability allows attackers to spoof valid certificate chains when
the system root store is in use. These releases include a mitigation for Go
applications, but it’s strongly recommended that affected users install the
Windows security update to protect their system.
This issue is CVE-2020-0601 and Go issue golang.org/issue/36834.
- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
functions of golang.org/x/crypto/cryptobyte can lead to a panic.
The malformed certificate can be delivered via a crypto/tls connection to a
client, or to a server that accepts client certificates. net/http clients can
be made to crash by an HTTPS server, while net/http servers that accept client
certificates will recover the panic and are unaffected.
Thanks to Project Wycheproof for providing the test cases that led to the
discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of golang.org/x/crypto/cryptobyte.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
