Commit Graph

190 Commits

Author SHA1 Message Date
Gabriel Adrian Samfira
4ccf287da7 Set CONTAINERD_ROOT in Windows cri-integration
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-03 07:44:44 +02:00
Derek McGowan
665877a5bf Move volume images from gcr to ghcr
With the ghcr images now built and working, switch over to
use these new images and update the default name.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-02 10:29:44 -08:00
Gabriel Adrian Samfira
323a62d7b9 Add permissions
Limit the scope of GITHUB_TOKEN to only have write access to packages
and read access to metadata. By default it seems to be granted access
equal to that of the github.actor that triggered the workflow, which
may include access to more than the workflow needs.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-02 18:28:51 +02:00
Gabriel Adrian Samfira
c6d26f0d37 Authenticate against ghcr.io
This change adds a login step that authenticates the runner to
ghcr.io. This allows whomever triggers the action to use github
packages as a destination for the container images.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-02 17:41:54 +02:00
Derek McGowan
591d7097e7 Merge pull request #6290 from gabriel-samfira/add-test-image-build-action
Integration: Add image build workflow
2021-11-30 13:51:24 -08:00
Gabriel Adrian Samfira
7ccd733d25 Add image build workflow
This change adds a new workflow that builds the volume test images
and pushes them to a remote registry.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-11-25 09:26:17 +00:00
Nashwan Azhari
0a25bc1aac Integration: Separate Windows Periodic Tests workflow trigger.
Following PR #6284, the Windows Periodic Workflow is prevented from
running on any repository outside of the original.

While this achieves the goal of preventing senseless Windows workflow
failures in contributors' forks, it makes running the workflow for
contributors (even manually) impossible.

This PR adds a separate workflow file which triggers the Windows
Integration workflow iff it is being run off of the original repository,
thus maintaining the upstream scheduling, while allowing contributors to
manually trigger the workflow on their forks if they so desire.

Signed-off-by: Nashwan Azhari <nazhari@cloudbasesolutions.com>
2021-11-24 18:18:14 +02:00
Phil Estes
0a284fcdbc Merge pull request #6284 from jonyhy96/feat-ci-skip-fork
feat: skip ci for fork
2021-11-23 23:20:37 -05:00
haoyun
507bfc91ba feat: skip ci for fork
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-11-24 10:24:13 +08:00
Phil Estes
72384a6531 Merge pull request #6268 from aznashwan/windows-ci-autodetect-az-cli
Integration: Remove explicit version passing to `azure/CLI`.
2021-11-23 12:44:35 -05:00
Nashwan Azhari
cb1359fa8b Integration: Remove explicit version passing to azure/CLI.
In the Windows CI's periodic runs the `azure/CLI` GitHub action library
is leveraged to run various Azure-related commands.

To avoid possible desyncing between the auth libraries of `azure/Login`
and `azure/CLI` as described
[here](https://github.com/Azure/cli/issues/56#issuecomment-958705517),
this patch neglects to pass an explicit `azcliversion` to `azure/CLI` in
order to have it default to the Azure CLI version set up by `azure/Login`.

Signed-off-by: Nashwan Azhari <nazhari@cloudbasesolutions.com>
2021-11-22 20:15:36 +02:00
Derek McGowan
5df9ec3665 Update release name to use consistent format
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-11-18 13:48:47 -08:00
Brian Goff
be84932e11 Rename release dockerfile to omit distro name
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-11-11 20:06:49 +00:00
Tonis Tiigi
28126fd605 release: improve dockerfile
Removes amd64 specifics and hardcoded arm64 package pulls.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-11 20:06:47 +00:00
Brian Goff
27d7c50384 Add arm64 to releases
This moves all the release builds into a Dockerfile which is a bit
cleaner for setting up our build environment.

Non-linux/amd64 builds are cross-compiled.
Currently onlinux linux/amd64, linux/arm64, and windows/amd64 are
supported, but is easy to add more, provided their is a cross-compile
toolchain available for it.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-11-11 20:00:34 +00:00
Sebastiaan van Stijn
869ccc01c1 Update Go to 1.17.3
go1.17.3 (released 2021-11-04) includes security fixes to the archive/zip and
debug/macho packages, as well as bug fixes to the compiler, linker, runtime, the
go command, the misc/wasm directory, and to the net/http and syscall packages.
See the Go 1.17.3 milestone on our issue tracker for details.

From the announcement e-mail:

[security] Go 1.17.3 and Go 1.16.10 are released

We have just released Go versions 1.17.3 and 1.16.10, minor point releases.
These minor releases include two security fixes following the security policy:

- archive/zip: don't panic on (*Reader).Open
  Reader.Open (the API implementing io/fs.FS introduced in Go 1.16) can be made
  to panic by an attacker providing either a crafted ZIP archive containing
  completely invalid names or an empty filename argument.
  Thank you to Colin Arnott, SiteHost and Noah Santschi-Cooney, Sourcegraph Code
  Intelligence Team for reporting this issue. This is CVE-2021-41772 and Go issue
  golang.org/issue/48085.
- debug/macho: invalid dynamic symbol table command can cause panic
  Malformed binaries parsed using Open or OpenFat can cause a panic when calling
  ImportedSymbols, due to an out-of-bounds slice operation.
  Thanks to Burak Çarıkçı - Yunus Yıldırım (CT-Zer0 Crypttech) for reporting this
  issue. This is CVE-2021-41771 and Go issue golang.org/issue/48990.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-11-05 12:46:33 +01:00
Gabriel Adrian Samfira
548579d0e2 Pin az CLI version
The latest version of AZ CLI breaks the windows-periodic workflow.
See:

https://github.com/Azure/cli/issues/56

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-11-04 15:56:27 +02:00
Phil Estes
4ed3c524cf Update test timeout based on recent cancellations
Signed-off-by: Phil Estes <estesp@amazon.com>
2021-10-08 16:10:55 -07:00
Sebastiaan van Stijn
de1a39bf32 Update Go to 1.17.2
go1.17.2 (released 2021-10-07) includes a security fix to the linker and misc/wasm
directory, as well as bug fixes to the compiler, the runtime, the go command, and
to the time and text/template packages. See the Go 1.17.2 milestone on our issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.2+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-08 15:26:34 +02:00
Derek McGowan
67b8831461 Update mirror images to take target image name
Allow overwriting the target tag to support mirror images from multiple
sources under our single namespace.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-09-28 14:08:46 -07:00
Maksym Pavlenko
483e23bdcd Merge pull request #6036 from dmcgowan/add-mirror-action
Add github action to mirror image
2021-09-21 16:06:33 -07:00
Derek McGowan
407d606941 Add github action to mirror image
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-09-21 13:23:51 -07:00
Kazuyoshi Kato
97df73004e gha: collect Vagrant VMs' IP addresses
To investigate #5748, this change collects Vagrant VMs' IP addresses.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-09-21 11:40:24 -07:00
Phil Estes
73dbbf5bfa Update to Go 1.17.1
Signed-off-by: Phil Estes <estesp@amazon.com>
2021-09-14 09:12:00 -04:00
Adelina Tuvenie
aeea697d84 Add WS2022 support in Windows Periodic tests
Some extra steps were added for WS2022 in accordance with
the published image on Azure:
- Install Container Feature & reboot VM
- Create NAT network

Temporarily we skip critest steps for WS2022 until all test images
are updated in that project.

Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
2021-09-07 15:42:58 +03:00
Phil Estes
736222dd87 Merge pull request #5897 from kzys/upgrade-golangci-lint
script: update golangci-lint from v1.38.0 and v1.36.0 to v1.42.0
2021-08-20 11:59:28 -04:00
Kazuyoshi Kato
4dd5ca70fb script: update golangci-lint from v1.38.0 and v1.36.0 to v1.42.0
golint has been deprecated and replaced by revive since v1.41.0.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-08-19 16:27:16 -07:00
Phil Estes
9fe7bc938b Bump integration timeout to 35 min
Go 1.17 go mod download step (used to handle the separate integration
go.mod) seems to do a lot more work/validation than prior Go releases,
requiring more time for integration runs.

Signed-off-by: Phil Estes <estesp@amazon.com>
2021-08-19 10:18:17 -04:00
Phil Estes
f8602c3725 Update to Go 1.17
Signed-off-by: Phil Estes <estesp@amazon.com>
2021-08-17 12:27:39 -04:00
Michael Crosby
7f0be5edd8 Merge pull request #5839 from dcantah/ci-weirdness
Fix Linux CI Linter using Go 1.15.14
2021-08-07 21:43:56 -04:00
Sebastiaan van Stijn
42a28ad2ca Update Go to 1.16.7
go1.16.7 (released 2021-08-05) includes a security fix to the net/http/httputil
package, as well as bug fixes to the compiler, the linker, the runtime, the go
command, and the net/http package. See the Go 1.16.7 milestone on the issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.16.7+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-07 18:07:50 +02:00
Fu Wei
fedd657ea0 Merge pull request #5838 from mxpv/ci
Cleanup CI
2021-08-07 11:33:14 +08:00
Daniel Canter
534685f955 Fix Linux CI Linter using go 1.15.14
For some reason the Linux CI runs end up using go 1.15.14 instead of 1.16.6 for
the Windows runs, or any of the other CI steps. Not sure if this is due to
the linter installing it's own version of go or something else.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2021-08-06 16:22:42 -07:00
Maksym Pavlenko
10eab21a4d Cleanup CI
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-08-06 14:59:05 -07:00
Alakesh Haloi
10824eaf2e remove go 1.13 from containerd
We no longer would need go 1.13.x for moby/containerd compatibility with
containerd moving to 1.16.x

Signed-off-by: Alakesh Haloi <alakeshh@amazon.com>
2021-08-04 11:58:53 -07:00
Radostin Stoyanov
d1e868c832 ci: install criu from PPA
The current latest version of CRIU is 3.15 and soon will be released
3.16. If CRIU is installed from PPA it would always test with the
latest released version.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-07-29 07:03:59 +01:00
Derek McGowan
5d3e8fd2c0 Merge pull request #5144 from devholic/release-non-cni-version
Update release workflow to upload binary without CNI
2021-07-28 23:03:19 -07:00
Derek McGowan
a3e431ad41 Merge pull request #5725 from adelina-t/enable_critest
Enable critest on Windows Periodic Jobs
2021-07-28 23:01:45 -07:00
Sunghoon Kang
d62d6c11dc Split release steps into multiple tasks
Signed-off-by: Sunghoon Kang <hoon@linecorp.com>
2021-07-28 18:42:54 +09:00
Sunghoon Kang
b4807122d2 Update release workflow to upload binary without CNI
Unlike [containerd/cri](https://github.com/containerd/cri) releases
(https://storage.googleapis.com/cri-containerd-release),
current release workflow doesn't publish binaries without CNI.

Signed-off-by: Sunghoon Kang <hoon@linecorp.com>
2021-07-28 16:56:37 +09:00
Davanum Srinivas
43117cf919 Script to check if entries in go.mod files are in sync
- ensure that the root go.mod and the module specific go.mod have the
  same `require` and `replace` directives for different dependencies.

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-07-27 17:52:32 -04:00
Adelina Tuvenie
23bc3db914 Enable critest on Windows
Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
2021-07-27 16:35:38 +03:00
Davanum Srinivas
494b940f14 Introduce a new go module - containerd/api for use in standalone clients
In containerd 1.5.x, we introduced support for go modules by adding a
go.mod file in the root directory. This go.mod lists all the things
needed across the whole code base (with the exception of
integration/client which has its own go.mod). So when projects that
need to make calls to containerd API will pull in some code from
containerd/containerd, the `go mod` commands will add all the things
listed in the root go.mod to the projects go.mod file. This causes
some problems as the list of things needed to make a simple API call
is enormous. in effect, making a API call will pull everything that a
typical server needs as well as the root go.mod is all encompassing.
In general if we had smaller things folks could use, that will make it
easier by reducing the number of things that will end up in a consumers
go.mod file.

Now coming to a specific problem, the root containerd go.mod has various
k8s.io/* modules listed. Also kubernetes depends on containerd indirectly
via both moby/moby (working with docker maintainers seperately) and via
google/cadvisor. So when the kubernetes maintainers try to use latest
1.5.x containerd, they will see the kubernetes go.mod ending up depending
on the older version of kubernetes!

So if we can expose just the minimum things needed to make a client API
call then projects like cadvisor can adopt that instead of pulling in
the entire go.mod from containerd. Looking at the existing code in
cadvisor the minimum things needed would be the api/ directory from
containerd. Please see proof of concept here:
github.com/google/cadvisor/pull/2908

To enable that, in this PR, we add a go.mod file in api/ directory. we
split the Protobuild.yaml into two, one for just the things in api/
directory and the rest in the root directory. We adjust various targets
to build things correctly using `protobuild` and also ensure that we
end up with the same generated code as before as well. To ensure we
better take care of the various go.mod/go.sum files, we update the
existing `make vendor` and also add a new `make verify-vendor` that one
can run locally as well in the CI.

Ideally, we would have a `containerd/client` either as a standalone repo
or within `containerd/containerd` as a separate go module. but we will
start here to experiment with a standalone api go module first.

Also there are various follow ups we can do, for example @thaJeztah has
identified two tasks we could do after this PR lands:

github.com/containerd/containerd/pull/5716#discussion_r668821396

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-07-27 07:34:59 -04:00
Sebastiaan van Stijn
d2b6d192db Update cpuguy83/go-md2man binary to v2.0.1
full diff: https://github.com/cpuguy83/go-md2man/compare/v2.0.0...v2.0.1

- Fix handling multiple definition descriptions
- Fix inline markup causing table cells to split
- Remove escaping tilde character (prevents tildes (`~`) from disappearing).
- Do not escape dash, underscore, and ampersand (prevents ampersands (`&`) from disappearing).
- Ignore unknown HTML tags to prevent noisy warnings

Note that this only updates the binaries we install. The vendor code also
includes go-md2man (as indirect dependency of urfave/cli). I don't think we use that
feature, so I did not add it to our go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-21 12:21:03 +02:00
Akihiro Suda
e72046f86b Update Go to 1.16.6
Release notes: https://golang.org/doc/devel/release#go1.16

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-13 14:05:37 +09:00
Derek McGowan
9fa7d15077 Merge pull request #5682 from kzys/gha-fork
gha: make release workflow work in forks
2021-07-09 11:04:00 -07:00
Akihiro Suda
aefabe5462 Dockerfile.test: add "cri-in-userns" (aka rootless) test stage
The `cri-in-userns` stage is for testing "CRI-in-UserNS", which should be used in conjunction with "Kubelet-in-UserNS":
https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2033-kubelet-in-userns-aka-rootless

This feature is mostly expected to be used for `kind` and `minikube`.

Requires Rootless Docker/Podman/nerdctl with cgroup v2 delegation: https://rootlesscontaine.rs/getting-started/common/cgroup2/
(Rootless Docker/Podman/nerdctl prepares the UserNS, so we do not need to create UserNS by ourselves)

Usage:
```
podman build --target cri-in-userns -t cri-in-userns -f contrib/Dockerfile.test .
podman run -it --rm --privileged cri-in-userns
```

The stage is tested on CI with Rootless Podman on Fedora 34 on Vagrant.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-09 14:50:04 +09:00
Kazuyoshi Kato
4c6e4a06ff gha: make release workflow work in forks
Fixes #5098.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-07-02 16:16:02 -07:00
Phil Estes
394f864750 Re-enable criu in main integration runs
Ubuntu kernel updates claim that the criu+overlayfs bug is finally fixed
in released kernels.

Signed-off-by: Phil Estes <estesp@amazon.com>
2021-06-16 17:46:27 -04:00
Phil Estes
3ab9744336 Add proper Go version before project checks
Due to a change in Go, the go.mod file cannot declare a version of Go
above the installed `go version`; until the default Go version in GitHub
actions virt environments is 1.16, we have to install 1.16 before
running the project checks now.

Signed-off-by: Phil Estes <estesp@amazon.com>
2021-06-10 10:48:25 -04:00