This allows us to dig more details out of test runs and maintain a
better history.
For this we can use `gotestsum`, which is a utility that wraps `go test`
so that it outputs test2json (go's format) and output junit (a format
more easily imported into other systems).
The PR makes it possible to override the Makefile's use of `go test` to
use any other command tto executet the test. For CI we'll use `gotestsum
--`, where `gotestsum` expects everything after the `--` to be flags for
`go test`.
We then use environment variables to configure `gotestsum` (e.g.
`GOTESTSUM_JUNITFILE` is an env var accepted by `gotestsum`).
For cri tests, the test suite supports outputing test results to a
directory, these are in junit format already. The file is not named
properly just because the code that creates it (in ginkgo) is not
configured well. We can fix that upstream to give us a better name...
until then I'm keeping those results in a separate dir.
A second workflow is also added so the test results can be summed up and
a report added to the workflow run. The 2nd workflow is required for
this since PR runs do not have access to do some of this due to safety
reasons
(https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When integration tests are run under nested VM (for SELinux, Cgroupsv2
testing) they are regularly starting to push past 10 minutes, causing
`go test` to fatally kill the test run (default timeout is 10m).
Signed-off-by: Phil Estes <estesp@amazon.com>
With container-selinux policy updated to 2.145+ (the default for Fedora 32+) we
can enable SELinux=Enforcing mode in the CI workflow and pass all integration
and CRI tests except one, see https://github.com/containerd/containerd/issues/4460,
which has been marked as skipped.
Tested locally with:
- SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-integration
- SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-cri
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
`vagrant up` will build and install containerd and all dependencies,
setting up proper SELinux contexts on the runc and containerd binaries.
The VM is configured to be SELinux Enforcing by default but this gets
changed during various CI passes via a matrix param to Disabled and
Permissive before running tests. I have an open PR to fix the
container-selinux policy for containerd at
https://github.com/containers/container-selinux/pull/98 which once
accepted we will want to update the CI matrix to use Enforcing mode
instead of Permissive.
All tests currently pass in SELinux permissive mode with containerd
configured with `enable_selinux=true`. To see which tests are failing
with SELinux enforcing and an already spun up VM:
`SELINUX=Enforcing vagrant up --provision-with=selinux,test-cri`
To test SELinux enforcing in a new VM:
`vagrant destroy -force; SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-cri`
The `selinux` shell provisioner, parameterized by the SELINUX envvar,
will configure the system as you would expect, with the side effect that
containerd is configured with `enable_selinux=true` via
`/etc/containerd/config.toml` for Permissive or Enforcing modes and
`enable_selinux=false` when SELINUX=Disabled.
Provided that virtualization is suported, this Vagrantfile and provisioners
make it easy to test containerd/cri for conformance under SELinux on
non-SELinux systems.
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
