containerd/.github/workflows/ci.yml
Davanum Srinivas 494b940f14
Introduce a new go module - containerd/api for use in standalone clients
In containerd 1.5.x, we introduced support for go modules by adding a
go.mod file in the root directory. This go.mod lists all the things
needed across the whole code base (with the exception of
integration/client which has its own go.mod). So when projects that
need to make calls to containerd API will pull in some code from
containerd/containerd, the `go mod` commands will add all the things
listed in the root go.mod to the projects go.mod file. This causes
some problems as the list of things needed to make a simple API call
is enormous. in effect, making a API call will pull everything that a
typical server needs as well as the root go.mod is all encompassing.
In general if we had smaller things folks could use, that will make it
easier by reducing the number of things that will end up in a consumers
go.mod file.

Now coming to a specific problem, the root containerd go.mod has various
k8s.io/* modules listed. Also kubernetes depends on containerd indirectly
via both moby/moby (working with docker maintainers seperately) and via
google/cadvisor. So when the kubernetes maintainers try to use latest
1.5.x containerd, they will see the kubernetes go.mod ending up depending
on the older version of kubernetes!

So if we can expose just the minimum things needed to make a client API
call then projects like cadvisor can adopt that instead of pulling in
the entire go.mod from containerd. Looking at the existing code in
cadvisor the minimum things needed would be the api/ directory from
containerd. Please see proof of concept here:
github.com/google/cadvisor/pull/2908

To enable that, in this PR, we add a go.mod file in api/ directory. we
split the Protobuild.yaml into two, one for just the things in api/
directory and the rest in the root directory. We adjust various targets
to build things correctly using `protobuild` and also ensure that we
end up with the same generated code as before as well. To ensure we
better take care of the various go.mod/go.sum files, we update the
existing `make vendor` and also add a new `make verify-vendor` that one
can run locally as well in the CI.

Ideally, we would have a `containerd/client` either as a standalone repo
or within `containerd/containerd` as a separate go module. but we will
start here to experiment with a standalone api go module first.

Also there are various follow ups we can do, for example @thaJeztah has
identified two tasks we could do after this PR lands:

github.com/containerd/containerd/pull/5716#discussion_r668821396

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-07-27 07:34:59 -04:00

616 lines
18 KiB
YAML

name: CI
on:
push:
branches:
- main
- master
- 'release/**'
pull_request:
branches:
- main
- master
- 'release/**'
jobs:
#
# golangci-lint
#
linters:
name: Linters
runs-on: ${{ matrix.os }}
timeout-minutes: 10
strategy:
matrix:
go-version: [1.16.6]
os: [ubuntu-18.04, macos-10.15, windows-2019]
steps:
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- name: Set env
shell: bash
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- uses: golangci/golangci-lint-action@v2
with:
version: v1.36.0
working-directory: src/github.com/containerd/containerd
args: --timeout=5m
#
# Project checks
#
project:
name: Project Checks
runs-on: ubuntu-18.04
timeout-minutes: 5
steps:
- uses: actions/setup-go@v2
with:
go-version: '1.16.6'
- shell: bash
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
fetch-depth: 100
- uses: containerd/project-checks@v1
with:
working-directory: src/github.com/containerd/containerd
- name: verify go modules and vendor directory
run: |
make verify-vendor
working-directory: src/github.com/containerd/containerd
#
# Protobuf checks
#
protos:
name: Protobuf
runs-on: ubuntu-18.04
timeout-minutes: 5
defaults:
run:
working-directory: src/github.com/containerd/containerd
steps:
- uses: actions/setup-go@v2
with:
go-version: '1.16.6'
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- name: Set env
shell: bash
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "GO111MODULE=off" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- name: Install protobuf
run: |
sudo -E PATH=$PATH script/setup/install-protobuf
sudo chmod +x /usr/local/bin/protoc
sudo chmod og+rx /usr/local/include/google /usr/local/include/google/protobuf /usr/local/include/google/protobuf/compiler
sudo chmod -R og+r /usr/local/include/google/protobuf/
protoc --version
- run: script/setup/install-dev-tools
- run: make proto-fmt
- run: make check-protos check-api-descriptors
man:
name: Manpages
runs-on: ubuntu-18.04
timeout-minutes: 5
steps:
- uses: actions/setup-go@v2
with:
go-version: '1.16.6'
- name: Set env
shell: bash
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- run: GO111MODULE=on go get github.com/cpuguy83/go-md2man/v2@v2.0.1
- run: make man
working-directory: src/github.com/containerd/containerd
# Make sure binaries compile with other platforms
crossbuild:
name: Crossbuild Binaries
needs: [project, linters, protos, man]
runs-on: ubuntu-18.04
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
include:
- goos: linux
goarch: arm64
- goos: linux
goarch: arm
goarm: "7"
- goos: linux
goarch: arm
goarm: "5"
- goos: freebsd
goarch: amd64
- goos: freebsd
goarch: arm64
- goos: windows
goarch: arm
goarm: "7"
steps:
- uses: actions/setup-go@v2
with:
go-version: '1.16.6'
- name: Set env
shell: bash
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- run: |
set -e -x
packages=""
platform="${{matrix.goos}}/${{matrix.goarch}}"
if [ -n "${{matrix.goarm}}" ]; then
platform+="/v${{matrix.goarm}}"
fi
case "${platform}" in
linux/arm/v5)
packages+=" crossbuild-essential-armel"
echo "CGO_ENABLED=1" >> $GITHUB_ENV
echo "CC=arm-linux-gnueabi-gcc" >> $GITHUB_ENV
;;
linux/arm/v7)
packages+=" crossbuild-essential-armhf"
echo "CGO_ENABLED=1" >> $GITHUB_ENV
echo "CC=arm-linux-gnueabihf-gcc" >> $GITHUB_ENV
;;
linux/arm64)
packages+=" crossbuild-essential-arm64"
echo "CGO_ENABLED=1" >> $GITHUB_ENV
echo "CC=aarch64-linux-gnu-gcc" >> $GITHUB_ENV
;;
windows/arm/v7)
echo "CGO_ENABLED=0" >> $GITHUB_ENV
;;
esac
if [ -n "${packages}" ]; then
sudo apt-get update && sudo apt-get install -y ${packages}
fi
name: install deps
- name: Build
working-directory: src/github.com/containerd/containerd
env:
GOOS: ${{matrix.goos}}
GOARCH: ${{matrix.goarch}}
GOARM: ${{matrix.goarm}}
run: |
make build
make binaries
#
# Build containerd binaries
#
binaries:
name: Binaries
runs-on: ${{ matrix.os }}
timeout-minutes: 10
needs: [project, linters, protos, man]
strategy:
matrix:
os: [ubuntu-18.04, macos-10.15, windows-2019]
go-version: ['1.16.6']
include:
# Go 1.13.x is still used by Docker/Moby
- go-version: '1.13.x'
os: ubuntu-18.04
steps:
- uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go-version }}
- name: Set env
shell: bash
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- name: Make
run: |
make build
make binaries
working-directory: src/github.com/containerd/containerd
#
# Integration and CRI tests
#
integration-windows:
name: Windows Integration
runs-on: windows-2019
timeout-minutes: 30
needs: [project, linters, protos, man]
env:
GOTEST: gotestsum --
defaults:
run:
shell: bash
working-directory: src/github.com/containerd/containerd
steps:
- uses: actions/setup-go@v2
with:
go-version: '1.16.6'
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- uses: actions/checkout@v2
with:
repository: Microsoft/hcsshim
path: src/github.com/Microsoft/hcsshim
- name: Set env
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
echo "${{ github.workspace }}/src/github.com/containerd/containerd/bin" >> $GITHUB_PATH
- run: script/setup/install-dev-tools
- name: Binaries
env:
CGO_ENABLED: 1
run: |
set -o xtrace
mingw32-make.exe binaries
bindir="$(pwd)"
SHIM_COMMIT=$(grep 'Microsoft/hcsshim ' go.mod | awk '{print $2}')
cd ../../Microsoft/hcsshim
git fetch --tags origin "${SHIM_COMMIT}"
git checkout "${SHIM_COMMIT}"
GO111MODULE=on go build -mod=vendor -o "${bindir}/integration/client/containerd-shim-runhcs-v1.exe" ./cmd/containerd-shim-runhcs-v1
- run: script/setup/install-gotestsum
- name: Tests
env:
CGO_ENABLED: 1
GOTESTSUM_JUNITFILE: ${{github.workspace}}/test-unit-root.xml
run: mingw32-make.exe test root-test
- name: Integration 1
env:
CGO_ENABLED: 1
GOTESTSUM_JUNITFILE: ${{github.workspace}}/test-integration-serial-junit.xml
run: mingw32-make.exe integration
# Run the integration suite a second time. See discussion in github.com/containerd/containerd/pull/1759
- name: Integration 2
env:
TESTFLAGS_PARALLEL: 1
CGO_ENABLED: 1
GOTESTSUM_JUNITFILE: ${{github.workspace}}/test-integration-parallel-junit.xml
run: mingw32-make.exe integration
- uses: actions/upload-artifact@v2
if: always()
with:
name: TestResults Windows
path: |
${{github.workspace}}/*-junit.xml
integration-linux:
name: Linux Integration
runs-on: ubuntu-18.04
timeout-minutes: 30
needs: [project, linters, protos, man]
strategy:
fail-fast: false
matrix:
runtime: [io.containerd.runtime.v1.linux, io.containerd.runc.v1, io.containerd.runc.v2]
runc: [runc, crun]
exclude:
- runtime: io.containerd.runc.v1
runc: crun
- runtime: io.containerd.runtime.v1.linux
runc: crun
env:
GOTEST: gotestsum --
steps:
- uses: actions/setup-go@v2
with:
go-version: '1.16.6'
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- name: Set env
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- name: Install containerd dependencies
env:
RUNC_FLAVOR: ${{ matrix.runc }}
run: |
sudo apt-get install -y gperf
sudo -E PATH=$PATH script/setup/install-seccomp
sudo -E PATH=$PATH script/setup/install-runc
sudo -E PATH=$PATH script/setup/install-cni
sudo -E PATH=$PATH script/setup/install-critools
working-directory: src/github.com/containerd/containerd
- name: Install criu
run: |
sudo apt-get install -y \
libprotobuf-dev \
libprotobuf-c-dev \
protobuf-c-compiler \
protobuf-compiler \
python-protobuf \
libnl-3-dev \
libnet-dev \
libcap-dev \
python-future
wget https://github.com/checkpoint-restore/criu/archive/v3.13.tar.gz -O criu.tar.gz
tar -zxf criu.tar.gz
cd criu-3.13
sudo make install-criu
- name: Install containerd
env:
CGO_ENABLED: 1
run: |
make binaries
sudo -E PATH=$PATH make install
working-directory: src/github.com/containerd/containerd
- run: sudo -E PATH=$PATH script/setup/install-gotestsum
working-directory: src/github.com/containerd/containerd
- name: Tests
env:
GOPROXY: direct
GOTESTSUM_JUNITFILE: ${{github.workspace}}/test-unit-root-junit.xml
run: |
make test
sudo -E PATH=$PATH make root-test
working-directory: src/github.com/containerd/containerd
- name: Integration 1
env:
GOPROXY: direct
TEST_RUNTIME: ${{ matrix.runtime }}
RUNC_FLAVOR: ${{ matrix.runc }}
GOTESTSUM_JUNITFILE: ${{github.workspace}}/test-integration-serial-junit.xml
run: |
extraflags=""
[ "${RUNC_FLAVOR}" == "crun" ] && {
extraflags="EXTRA_TESTFLAGS=-no-criu";
}
sudo -E PATH=$PATH make integration ${extraflags} TESTFLAGS_RACE=-race
working-directory: src/github.com/containerd/containerd
# Run the integration suite a second time. See discussion in github.com/containerd/containerd/pull/1759
- name: Integration 2
env:
GOPROXY: direct
TEST_RUNTIME: ${{ matrix.runtime }}
RUNC_FLAVOR: ${{ matrix.runc }}
GOTESTSUM_JUNITFILE: ${{github.workspace}}/test-integration-parallel-junit.xml
run: |
extraflags=""
[ "${RUNC_FLAVOR}" == "crun" ] && {
extraflags="EXTRA_TESTFLAGS=-no-criu";
}
sudo -E PATH=$PATH TESTFLAGS_PARALLEL=1 make integration ${extraflags}
working-directory: src/github.com/containerd/containerd
- name: CRI Integration Test
env:
TEST_RUNTIME: ${{ matrix.runtime }}
run: |
CONTAINERD_RUNTIME=$TEST_RUNTIME make cri-integration
working-directory: src/github.com/containerd/containerd
- name: cri-tools critest
env:
TEST_RUNTIME: ${{ matrix.runtime }}
run: |
BDIR="$(mktemp -d -p $PWD)"
mkdir -p ${BDIR}/{root,state}
cat > ${BDIR}/config.toml <<EOF
version = 2
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "${TEST_RUNTIME}"
EOF
sudo ls /etc/cni/net.d
sudo -E PATH=$PATH /usr/local/bin/containerd -a ${BDIR}/c.sock --config ${BDIR}/config.toml --root ${BDIR}/root --state ${BDIR}/state --log-level debug &> ${BDIR}/containerd-cri.log &
sudo -E PATH=$PATH /usr/local/bin/ctr -a ${BDIR}/c.sock version
sudo -E PATH=$PATH critest --report-dir "${{github.workspace}}/critestreport" --runtime-endpoint=unix:///${BDIR}/c.sock --parallel=8
TEST_RC=$?
test $TEST_RC -ne 0 && cat ${BDIR}/containerd-cri.log
sudo pkill containerd
sudo -E rm -rf ${BDIR}
test $TEST_RC -eq 0 || /bin/false
# Log the status of this VM to investigate issues like
# https://github.com/containerd/containerd/issues/4969
- name: Host Status
if: always()
run: |
set -x
mount
df
losetup -l
- uses: actions/upload-artifact@v2
if: always()
with:
name: TestResults ${{ matrix.runtime }} ${{matrix.runc}}
path: |
*-junit.xml
${{github.workspace}}/critestreport/*.xml
tests-mac-os:
name: MacOS unit tests
runs-on: macos-10.15
timeout-minutes: 10
needs: [project, linters, protos, man]
env:
GOTEST: gotestsum --
steps:
- uses: actions/setup-go@v2
with:
go-version: '1.16.6'
- uses: actions/checkout@v2
with:
path: src/github.com/containerd/containerd
- name: Set env
run: |
echo "GOPATH=${{ github.workspace }}" >> $GITHUB_ENV
echo "${{ github.workspace }}/bin" >> $GITHUB_PATH
- run: sudo -E PATH=$PATH script/setup/install-gotestsum
working-directory: src/github.com/containerd/containerd
- name: Tests
env:
GOPROXY: direct
GOTESTSUM_JUNITFILE: "${{ github.workspace }}/macos-test-junit.xml"
run: |
make test
working-directory: src/github.com/containerd/containerd
- uses: actions/upload-artifact@v2
if: always()
with:
name: TestResults MacOS
path: |
*-junit.xml
cgroup2:
name: CGroupsV2 - SELinux enforced
# nested virtualization is only available on macOS hosts
runs-on: macos-10.15
timeout-minutes: 45
needs: [project, linters, protos, man]
strategy:
matrix:
# Currently crun is disabled to decrease CI flakiness.
# We can enable crun again when we get a better CI infra.
runc: [runc]
env:
GOTEST: gotestsum --
steps:
- uses: actions/checkout@v2
- name: "Cache ~/.vagrant.d/boxes"
uses: actions/cache@v2
with:
path: ~/.vagrant.d/boxes
key: vagrant-${{ hashFiles('Vagrantfile*') }}
- name: Vagrant start
run: |
# Retry if it fails (download.fedoraproject.org returns 404 sometimes)
vagrant up || vagrant up
- name: Integration
env:
RUNC_FLAVOR: ${{ matrix.runc }}
SELINUX: Enforcing
GOTESTSUM_JUNITFILE: /tmp/test-integration-junit.xml
run: vagrant up --provision-with=selinux,install-runc,install-gotestsum,test-integration
- name: CRI test
env:
RUNC_FLAVOR: ${{ matrix.runc }}
SELINUX: Enforcing
REPORT_DIR: /tmp/critestreport
run: vagrant up --provision-with=selinux,install-runc,install-gotestsum,test-cri
- name: Get test reports
if: always()
run: |
set -e
vagrant plugin install vagrant-vbguest
vagrant plugin install vagrant-scp
vagrant scp :/tmp/test-integration-junit.xml "${{ github.workspace }}/"
vagrant scp :/tmp/critestreport "${{ github.workspace }}/critestreport"
- uses: actions/upload-artifact@v2
if: always()
with:
name: TestResults cgroup2 ${{ matrix.runtime }} ${{matrix.runc}}
path: |
${{github.workspace}}/*-junit.xml
${{github.workspace}}/critestreport/*
cgroup2-misc:
name: CGroupsV2 - rootless CRI test
# nested virtualization is only available on macOS hosts
runs-on: macos-10.15
timeout-minutes: 45
needs: [project, linters, protos, man]
steps:
- uses: actions/checkout@v2
- name: "Cache ~/.vagrant.d/boxes"
uses: actions/cache@v2
with:
path: ~/.vagrant.d/boxes
key: vagrant-${{ hashFiles('Vagrantfile*') }}
- name: Vagrant start
run: |
# Retry if it fails (download.fedoraproject.org returns 404 sometimes)
vagrant up || vagrant up
# slow, so separated from the regular cgroup2 task
- name: CRI-in-UserNS test with Rootless Podman
run: |
vagrant up --provision-with=install-rootless-podman
# Execute rootless podman to create the UserNS env
vagrant ssh -- podman build --target cri-in-userns -t cri-in-userns -f /vagrant/contrib/Dockerfile.test /vagrant
vagrant ssh -- podman run --rm --privileged cri-in-userns