This allows us to dig more details out of test runs and maintain a
better history.
For this we can use `gotestsum`, which is a utility that wraps `go test`
so that it outputs test2json (go's format) and output junit (a format
more easily imported into other systems).
The PR makes it possible to override the Makefile's use of `go test` to
use any other command tto executet the test. For CI we'll use `gotestsum
--`, where `gotestsum` expects everything after the `--` to be flags for
`go test`.
We then use environment variables to configure `gotestsum` (e.g.
`GOTESTSUM_JUNITFILE` is an env var accepted by `gotestsum`).
For cri tests, the test suite supports outputing test results to a
directory, these are in junit format already. The file is not named
properly just because the code that creates it (in ginkgo) is not
configured well. We can fix that upstream to give us a better name...
until then I'm keeping those results in a separate dir.
A second workflow is also added so the test results can be summed up and
a report added to the workflow run. The 2nd workflow is required for
this since PR runs do not have access to do some of this due to safety
reasons
(https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Though we don't officially support Apple platform, we should
at least run unit tests to make sure things are not broken.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
The integration test times have slightly increased and are often
hitting the 25 minutes timeout. This increases to give more room
but still keeps it low enough to catch regressions in tests
causing longer than expected execution.
Signed-off-by: Derek McGowan <derek@mcg.dev>
While the issue hasn't been fixed in the kernel yet, we can workaround
the issue by not using overlayfs snapshotter.
The newly added step runs all tests that match /TestCheckpoint/.
So, TestCRWithImagePath has been renamed to match the regexp.
Fixes#3930.
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
To investigate issues like #4969, it would be helpful to understand
the status of the VM at the end.
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
Without a name the logs use a carriage return followed by the long
comment as the name of the job step which is messy when working with the
actions API/logs.
Signed-off-by: Phil Estes <estesp@amazon.com>
Since both cri-containerd and runC runtime are widely used, the relevent
information should include runC and CRI configuration when file bug.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
The CI host was probably updated recently and the permission bits of the
directory was changed.
Fix 5077
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
We have enough failures these days; getting timed out when tests are
almost done is the last thing we need :)
On avg. the Linux integration tests are taking 15-17 min, but sometimes
they end up at 20 or a bit over and get canceled. I've seen rare cases
where the Vagrant setup+build+test runs gets very close to 40 min as
well.
Signed-off-by: Phil Estes <estesp@amazon.com>
download.fedoraproject.org gives HTTP 404 at times,
breaking the CI. Let's give it another chance.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add `.github/ISSUE_TEMPLATE/config.yml` to clarify where is the right place (in the most cases) to ask questions.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
CI was timing out after 15 minutes on the crun tests; extending
the timeout to 20 minutes (we can make it shorter again if we know
the exact time it takes to run)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With container-selinux policy updated to 2.145+ (the default for Fedora 32+) we
can enable SELinux=Enforcing mode in the CI workflow and pass all integration
and CRI tests except one, see https://github.com/containerd/containerd/issues/4460,
which has been marked as skipped.
Tested locally with:
- SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-integration
- SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-cri
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
