github/containerd
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity

ci: upload junit formatted test results

Browse Source 
This allows us to dig more details out of test runs and maintain a
better history.

For this we can use `gotestsum`, which is a utility that wraps `go test`
so that it outputs test2json (go's format) and output junit (a format
more easily imported into other systems).

The PR makes it possible to override the Makefile's use of `go test` to
use any other command tto executet the test. For CI we'll use `gotestsum
--`, where `gotestsum` expects everything after the `--` to be flags for
`go test`.
We then use environment variables to configure `gotestsum` (e.g.
`GOTESTSUM_JUNITFILE` is an env var accepted by `gotestsum`).

For cri tests, the test suite supports outputing test results to a
directory, these are in junit format already. The file is not named
properly just because the code that creates it (in ginkgo) is not
configured well. We can fix that upstream to give us a better name...
until then I'm keeping those results in a separate dir.

A second workflow is also added so the test results can be summed up and
a report added to the workflow run. The 2nd workflow is required for
this since PR runs do not have access to do some of this due to safety
reasons
(https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This commit is contained in:
Brian Goff
2021-03-25 19:11:29 +00:00
parent 1c05317a47
commit af1e2af720
5 changed files with 165 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
Makefile
View File

@@ -120,6 +120,9 @@ BINARIES=$(addprefix bin/,$(COMMANDS))
TESTFLAGS ?= $(TESTFLAGS_RACE) $(EXTRA_TESTFLAGS)
TESTFLAGS_PARALLEL ?= 8
# Use this to replace `go test` with, for instance, `gotestsum`
GOTEST ?= go test
OUTPUTDIR = $(join $(ROOTDIR), _output)
CRIDIR=$(OUTPUTDIR)/cri
@@ -170,15 +173,15 @@ build: ## build the go packages
test: ## run tests, except integration tests and tests that require root
@echo "$(WHALE) $@"
@go test ${TESTFLAGS} ${PACKAGES}
@$(GOTEST) ${TESTFLAGS} ${PACKAGES}
root-test: ## run tests, except integration tests
@echo "$(WHALE) $@"
@go test ${TESTFLAGS} ${TEST_REQUIRES_ROOT_PACKAGES} -test.root
@$(GOTEST) ${TESTFLAGS} ${TEST_REQUIRES_ROOT_PACKAGES} -test.root
integration: ## run integration tests
@echo "$(WHALE) $@"
@cd "${ROOTDIR}/integration/client" && go mod download && go test -v ${TESTFLAGS} -test.root -parallel ${TESTFLAGS_PARALLEL} .
@cd "${ROOTDIR}/integration/client" && go mod download && $(GOTEST) -v ${TESTFLAGS} -test.root -parallel ${TESTFLAGS_PARALLEL} .
# TODO integrate cri integration bucket with coverage
bin/cri-integration.test: