Commit Graph

123 Commits

Author SHA1 Message Date
Adelina Tuvenie
e6538b8bce Add trap to cri-integration test script
The cri-integration.sh script sets errexit option. This does not
work properly on Bash in Windows, espectially when the script is
piped to something else ( tee in this case ). In this particular
case, the problem arises from the fact that if the script exits
prematurely, it will not get a chance to call test_teardown and
thus clean the remaining containerd process, thus the whole
command will hang indefinetly.

Adding a simple trap on EXIT to call test_teardown will easily
fix this.

Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
2021-08-10 13:37:05 +03:00
Sebastiaan van Stijn
42a28ad2ca
Update Go to 1.16.7
go1.16.7 (released 2021-08-05) includes a security fix to the net/http/httputil
package, as well as bug fixes to the compiler, the linker, the runtime, the go
command, and the net/http package. See the Go 1.16.7 milestone on the issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.7+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-07 18:07:50 +02:00
Maksym Pavlenko
10eab21a4d Cleanup CI
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-08-06 14:59:05 -07:00
Sebastiaan van Stijn
6c257552a7
scripts: declare ROOT closer to where it's used, and some DRY changes
This also prevents shellcheck from complaining about a possibly
undefined variable.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-03 17:27:03 +02:00
Sebastiaan van Stijn
dba0ef4eb5
scripts: add missing quotes, and minor linting issues
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-03 17:23:18 +02:00
Derek McGowan
2b9f8334c6
Merge pull request #5792 from dims/script-to-check-if-entries-in-go.mod-files-are-in-sync
Script to check if entries in go.mod files are in sync
2021-07-28 17:55:49 -07:00
Derek McGowan
0d93936507
Update protobuf install to use correct repository
The protobuf repository is under the protocolbuffers org

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-07-28 09:51:27 -07:00
Davanum Srinivas
43117cf919
Script to check if entries in go.mod files are in sync
- ensure that the root go.mod and the module specific go.mod have the
  same `require` and `replace` directives for different dependencies.

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-07-27 17:52:32 -04:00
Sebastiaan van Stijn
18d6cc1b06
update gotestsum to v1.7.0
Same as previous commit, but a release was tagged;

1a94380793...v1.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-26 13:13:17 +02:00
Sebastiaan van Stijn
d2b6d192db
Update cpuguy83/go-md2man binary to v2.0.1
full diff: https://github.com/cpuguy83/go-md2man/compare/v2.0.0...v2.0.1

- Fix handling multiple definition descriptions
- Fix inline markup causing table cells to split
- Remove escaping tilde character (prevents tildes (`~`) from disappearing).
- Do not escape dash, underscore, and ampersand (prevents ampersands (`&`) from disappearing).
- Ignore unknown HTML tags to prevent noisy warnings

Note that this only updates the binaries we install. The vendor code also
includes go-md2man (as indirect dependency of urfave/cli). I don't think we use that
feature, so I did not add it to our go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-21 12:21:03 +02:00
Phil Estes
b809212b18
Merge pull request #5750 from thaJeztah/cleanup_test_dockerfile
Refactor / optimize contrib/Dockerfile.test
2021-07-20 10:33:55 -04:00
Phil Estes
1292c50fe9
Merge pull request #5754 from thaJeztah/update_gotestsum
update gotestsum to current master
2021-07-19 14:01:16 -04:00
Sebastiaan van Stijn
603962bc81
update gotestsum to current master
removes golang.org/x/cryto dependency:
full diff: 012a85e34a...1a94380793

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-19 10:00:02 +02:00
Akihiro Suda
43e0594ae9
update runc binary to v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-18 20:18:14 +09:00
Sebastiaan van Stijn
546538971c
install-critools: make sure DESTDIR exists
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-17 14:00:58 +02:00
Akihiro Suda
e72046f86b
Update Go to 1.16.6
Release notes: https://golang.org/doc/devel/release#go1.16

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-13 14:05:37 +09:00
Maksym Pavlenko
7eceeb950b
Merge pull request #5571 from adelina-t/win_set_specific_go_version
Install specific golang version in Windows CI.
2021-07-02 14:54:58 -07:00
Akihiro Suda
28bb59c080 update runc binary to v1.0.0 GA
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-06-22 10:26:46 -04:00
Brian Goff
bfbebf02b7 Update gotestsum to add timestamps to junit output
The current release of gotestsum is missing timestamps in the junit
data, which makes it difficult to import in an external system later.

https://github.com/gotestyourself/gotestsum/commit/012a85e34a7ce5554057d512e55dcb
includes the necessary changes to add the timestamp for the test run to
the junit output.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-16 16:17:40 +00:00
Adelina Tuvenie
419d616fec Install specific golang version in Windows CI.
For Periodic Windows runs we installed the latest available golang version.
It seems 1.16.5 is creating problems with go.sum. We now introduce the
ability to install specific versions for required packages when preparing
the testing env.

Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
2021-06-07 15:17:02 +03:00
Claudiu Belu
2f870aa896 integration: Cleanup containerd on test teardown
On Windows, we were only killing the keepalive process, and the
containerd process would keep running.

keepalive and containerd have the same PGID, so we can use that information
to kill both of them.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-06-01 15:22:43 +00:00
Derek McGowan
334e74765e
Merge pull request #5165 from adelina-t/azure_ci_workflow
Add CI periodic Windows Jobs.
2021-05-28 08:12:25 -07:00
Akihiro Suda
599127f4df
update runc binary to v1.0.0-rc95
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-05-19 19:18:59 +09:00
Claudiu Belu
09a0c9471b tests: Adds support for Windows cri-integration tests
Currently, the cri-integration tests do not work on Windows due to various reasons.
One of the reasons is because all the tests are using Linux-specific images.
Previous commits refactored the image pulling / usage in the cri-integration tests,
making it easier to update, and easier to configure a custom registry to pull images
with Windows support.

For Windows runs, custom registries can be created, which will also contain Windows
images, and the cri-integration tests can be configured to use those registries by
specifying the "--repo-list" argument, a YAML file which will contain an alternative
mapping of the default registries. This is similar to how E2E tests are handled for
Windows runs in Kubernetes.

Some of the tests are Skipped, as they do not pass yet on Windows.

Windows does not collect inodes used stats, thus, the tests that were expecting non-zero
inodes stats were failing.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-05-14 11:27:35 +00:00
Mike Brown
3157392513 just release ctd-decoder not ctr-enc
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-05-13 15:56:45 -05:00
Davanum Srinivas
5c99f150ae
Update the runc binary used with containerd to runc v1.0.0-rc94
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-05-10 13:11:07 -04:00
Derek McGowan
1ecc98f1de
Merge pull request #5462 from mikebrow/correct-critools-release
update to new cri-tools make install
2021-05-07 13:05:13 -07:00
Mike Brown
c74a6c4194 update to new cri-tools make install
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-05-07 12:45:24 -05:00
Mike Brown
b56527cb7e update seccomp version
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-05-03 19:36:16 -05:00
Derek McGowan
7942ae68b1
Revert "Specify seccomp target arch for CC"
This reverts commit 969ec89493.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-04-29 15:05:54 -07:00
Derek McGowan
f0b4258904
Merge pull request #5429 from cpuguy83/fix_libseccomp_cc
Specify seccomp target arch for CC
2021-04-28 12:00:04 -07:00
Brian Goff
969ec89493 Specify seccomp target arch for CC
seccomp's build scripts require setting the target arch that will be
built when cross compiling.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-04-28 17:07:44 +00:00
Claudiu Belu
5847340a7d tests: Refactors container image usage
Currently, the cri-integration tests do not work on Windows due to various reasons.
One of the reasons is because all the tests are using Linux-specific images. This
commit refactors the image pulling / usage in the cri-integration tests, making it
easier to update, and easier to configure the a custom registry to pull those images
from.

For Windows runs, custom registries can be created, which will also contain Windows
images, and the cri-integration tests can be configured to use those registries by
specifying the "--image-list" argument, a TOML file which will contain an alternative
mapping of the default images.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-04-27 08:44:21 +00:00
Adelina Tuvenie
421fc6ea73 Add CI periodic Windows Jobs.
Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
2021-04-20 17:18:46 +03:00
Phil Estes
6ef0d383e3
Merge pull request #5271 from cpuguy83/ci_output_junit
ci: upload junit formatted test results
2021-04-15 13:20:35 -04:00
Davanum Srinivas
9ad087947d
Switch all our tests to version 2
Also warn when someone uses version 1

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-04-12 13:18:33 -04:00
Akihiro Suda
14f357b90f
CI: update crun to 0.19
Release notes: https://github.com/containers/crun/releases

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-12 17:05:11 +09:00
Phil Estes
b52727342c
Merge pull request #5308 from AkihiroSuda/add-imgcrypt-bin
cri-cni-release: add imgcrypt binaries (v1.1.0)
2021-04-07 15:43:38 -04:00
Mike Brown
da998c81eb move to gcr.io/k8s-staging-cri-tools test images
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-04-07 10:04:02 -05:00
Akihiro Suda
92ea98eda5
cri-cni-release: add imgcrypt binaries (v1.1.0)
Add `ctd-decoder` and `ctr-enc`.

Close issue 5265

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-07 17:38:42 +09:00
Brian Goff
af1e2af720 ci: upload junit formatted test results
This allows us to dig more details out of test runs and maintain a
better history.

For this we can use `gotestsum`, which is a utility that wraps `go test`
so that it outputs test2json (go's format) and output junit (a format
more easily imported into other systems).

The PR makes it possible to override the Makefile's use of `go test` to
use any other command tto executet the test. For CI we'll use `gotestsum
--`, where `gotestsum` expects everything after the `--` to be flags for
`go test`.
We then use environment variables to configure `gotestsum` (e.g.
`GOTESTSUM_JUNITFILE` is an env var accepted by `gotestsum`).

For cri tests, the test suite supports outputing test results to a
directory, these are in junit format already. The file is not named
properly just because the code that creates it (in ginkgo) is not
configured well. We can fix that upstream to give us a better name...
until then I'm keeping those results in a separate dir.

A second workflow is also added so the test results can be summed up and
a report added to the workflow run. The 2nd workflow is required for
this since PR runs do not have access to do some of this due to safety
reasons
(https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-03-31 17:00:49 +00:00
Mike Brown
548d984f27
Merge pull request #5132 from mikebrow/add-some-debug-out-for-dep-setups
adds a new install-deps in the makefile; and some refactoring of the scripts
2021-03-24 15:28:16 -05:00
Mike Brown
9ad87b9ba7 adds critools-version
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-03-24 10:58:40 -05:00
Phil Estes
a1138182d5
Merge pull request #5180 from dmcgowan/lint-enforce-comments
Fix exported comments enforcer in CI
2021-03-15 10:50:06 -04:00
Sebastiaan van Stijn
55a9bbc033
Prevent runc inheriting BUILDTAGS from containerd
Both runc and containerd use BUILDTAGS to customize go build-tags.

When building containerd with custom build-tags, runc inherited
those, causing the default to be overwritten, e.g.;

    make BUILDTAGS=no_btrfs cri-cni-release
    (in script/setup/install-runc)

    HEAD is now at 12644e61 VERSION: release 1.0.0~rc93
    make[1]: Entering directory '/tmp/tmp.ZJzc2KtI0A/runc'
    go build -trimpath "-mod=vendor" "-buildmode=pie"  -tags "no_btrfs" -ldflags "-X main.gitCommit="12644e614e25b05da6fd08a38ffa0cfe1903fdec" -X main.version=1.0.0-rc93 " -o runc .
                                                   ^^^^^^^^^^^^^^^^

This patch brings back the BUILDTAGS make-var in the runc-install
script, which fixates the buildtags to our defaults.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-12 18:37:18 +01:00
Derek McGowan
35eeb24a17
Fix exported comments enforcer in CI
Add comments where missing and fix incorrect comments

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-03-12 08:47:05 -08:00
Mike Brown
2205da64a6 clean up install-cni and install-critools
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-03-11 10:21:32 -06:00
Sebastiaan van Stijn
79a51cd16a
move runc version to a separate file for easier consumption
This moves the runc version to build to scripts/setup/runc-version,
which makes it easier for packagers to find the default version
to use.

The RUNC_VERSION environment variable can still be used to override
the version, which can be used (e.g.) to test against different versions
in our CI.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-10 14:28:29 +01:00
Sebastiaan van Stijn
8325ba5d36
Separate runc binary version from libcontainer version
Now that the dependency on runc (libcontaienr) code has been reduced
considerably, it is probbaly ok to cut the version dependency between
libcontainer and the runc binary that is supported.

This patch separates the runc binary version from the version of
libcontainer that is defined in go.mod, and updates the documentation
accordingly.

The RUNC_COMMIT variable in the install-runc script is renamed to
RUNC_VERSION to encourage using tagged versions, and the Dockerfile
in contrib is updated to allow building with a custom version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-10 13:41:12 +01:00
Sebastiaan van Stijn
b89a63a235
Remove references to apparmor and selinux buildtags for runc
From the runc v1.0.0-rc93 release notes:

> The "selinux" and "apparmor" buildtags have been removed, and now all runc
> builds will have SELinux and AppArmor support enabled. Note that "seccomp"
> is still optional (though we very highly recommend you enable it).

Also adding a note about kmem support.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-10 12:16:54 +01:00