Commit Graph

186 Commits

Author SHA1 Message Date
zounengren
7854e0bffe bump continuity and console version that remove pkg/errors
Signed-off-by: Zou Nengren <zouyee1989@gmail.com>
2021-09-21 21:56:18 +08:00
Derek McGowan
3e72e335fe
Use github images for integration tests
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-09-20 13:35:13 -07:00
wllenyj
6d961f9673 CI: Switch to available latest images
`mirror.gcr.io/library/busybox:1.32` is unavailable.

```console
$ curl -s https://mirror.gcr.io//v2/library/busybox/tags/list | jq '.tags'
[
      "1.26.2",
      "1.27.2",
      "1.28",
      "1.29",
      "1.29.2",
      "1.29.3",
      "1.30",
      "1.30.1",
      "1.31",
      "1.31.0",
      "1.31.1",
      "1.32.0"
]
```

Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
2021-09-14 11:59:19 +08:00
Akihiro Suda
3d734d0345
Merge pull request #5976 from ukontainer/pr-continuity
vendor: update continuity for darwin support
2021-09-11 20:32:48 +09:00
Fu Wei
d9f921e4f0
Merge pull request #5906 from thaJeztah/replace_os_exec 2021-09-11 10:38:53 +08:00
Fu Wei
8766f2118c
Merge pull request #5972 from zouyee/labels 2021-09-11 09:42:43 +08:00
Hajime Tazaki
8ff8b1b823 vendor: update continuity for darwin support
This picks a fix to properly handle images containing symlinks
inside which point to an unexisting file.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
2021-09-11 08:32:37 +09:00
zounengren
16cd6ed137 Additional integration tests for pulling image with labels
Signed-off-by: Zou Nengren <zouyee1989@gmail.com>
2021-09-10 20:16:57 +08:00
Michael Crosby
d040ca3845
Merge pull request #5861 from kzys/restart-monitor
integration: investigate TestRestartMonitor's failure
2021-09-09 10:56:40 -04:00
Daniel Canter
1dd0d59b73 go.mod: Update hcsshim to v0.8.21
This version brings in some bug fixes to layer handling. The actual fix isn't
present in the diff as it's not used here, but the Windows shim is built from
the tag present in go.mod, so the fix will be in the Windows shim on a new release
of Containerd if this tag is in.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2021-08-31 15:09:35 -07:00
Adelina Tuvenie
6d3d34b85d Update Pause image in tests & config
With the introduction of Windows Server 2022, some images have been updated
to support WS2022 in their manifest list. This commit updates the test images
accordingly.

Signed-off-by: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
2021-08-31 19:42:57 +03:00
Sebastiaan van Stijn
2ac9968401
replace uses of os/exec with golang.org/x/sys/execabs
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows. This issue also
affects Unix users who have “.” listed explicitly in their PATH and are running
“go get” outside of a module or with module mode disabled.

This issue is not limited to the go command itself, and can also affect binaries
that use `os.Command`, `os.LookPath`, etc.

From the related blogpost (ttps://blog.golang.org/path-security):

> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing

This patch replaces all uses of `os/exec` with `golang.org/x/sys/execabs`. While
some uses of `os/exec` should not be problematic (e.g. part of tests), it is
probably good to be consistent, in case code gets moved around.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-25 18:11:09 +02:00
Fu Wei
ebe8f8c6be
Merge pull request #5899 from AkihiroSuda/runc-v1.0.2
bump up runc v1.0.2
2021-08-24 23:04:36 +08:00
Fu Wei
6fa9588531
Merge pull request #5903 from AkihiroSuda/gofmt117
Run `go fmt` with Go 1.17
2021-08-24 23:01:41 +08:00
Akihiro Suda
e185167677
go.mod: update runc to v1.0.2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-08-23 23:24:45 +09:00
Sebastiaan van Stijn
4292969106
go.mod: github.com/pelletier/go-toml v1.9.3
full diff: https://github.com/pelletier/go-toml/compare/v1.8.1...v1.9.3

- v1.9.3: Clarify license and comply with Apache 2.0
- v1.9.2: Add Encoder.CompactComments to omit extra new line
- v1.9.1: Fix empty trees line counting

v1.9.0
-------------------

The highlight of this version is that the whole toml.Tree structure has been made
public in a backward compatible way. This allows everyone using v1.x to fully
access the data and metadata in the tree to extend the library.

This is hopefully the last release in the v1.x track, as go-toml v2 is the main
focus of development.

What's new

- TOML 1.0.0-rc.3
- Improved default tag for durations
- Provide Tree and treeValue public aliases
- Expose MarshalOrder
- Value string representation public function

Fixed bugs

- Do not allow T-prefix on local dates
- toml.Unmarshaler supports leaf nodes
- Fix date lexer to only support 4-digit year
- Fix ToMap for tables in mixed-type arrays
- Fix ToMap for tables in nested mixed-type arrays
- Support literal multiline marshal

Performance

- Remove date regexp
- Remove underscore regexps

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-23 15:38:29 +02:00
Akihiro Suda
d3aa7ee9f0
Run go fmt with Go 1.17
The new `go fmt` adds `//go:build` lines (https://golang.org/doc/go1.17#tools).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-08-22 09:31:50 +09:00
Kazuyoshi Kato
f7b1ceb9f6 integration: investigate TestRestartMonitor's failure
- task.Kill() might fail in theory
- Giving a longer timeout may help us understand whether the failure is
  a timing issue or not.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-08-19 11:38:53 -07:00
Kazuyoshi Kato
acb81bbdaf integration: fix TestContainerPids
task.Pids returns the task's processes, but the order is not guaranteed.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-08-19 09:56:22 -07:00
Daniel Canter
e634f04d8c go.mod: update kubernetes to v1.22.0
This brings in some cri api changes for cgroups, Windows pod sandbox security
context changes and some new fields for the Windows version of a privileged
container.

This also unfortunately bumps the prometheus client, grpc middleware, bolt
and klog :(

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2021-08-09 16:16:54 -04:00
Derek McGowan
6f027e38a8
Remove redundant build tags
Remove build tags which are already implied by the name of the file.
Ensures build tags are used consistently

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-08-05 22:27:46 -07:00
Alakesh Haloi
3c5424454b [otel-tracing] vendor and go modules changes
This tracks all the vendor file changes and changes to go modules due
to OpenTelemetry support

Signed-off-by: Alakesh Haloi <alakeshh@amazon.com>
2021-08-04 14:25:12 -07:00
Jayme Howard
ce437864b2 mergo: Upgrade to 0.3.12 to fix panic
Signed-off-by: Jayme Howard <g.prime@gmail.com>

Run `go mod tidy`

Signed-off-by: Jayme Howard <g.prime@gmail.com>

Follow correct procedure by running `make vendor`

Signed-off-by: Jayme Howard <g.prime@gmail.com>
2021-07-29 12:20:09 -05:00
Derek McGowan
caf9e256b7
Merge pull request #5693 from kzys/sigrtmin
Support SIGRTMIN+n signals
2021-07-27 11:58:57 -07:00
Derek McGowan
6d31cdda50
Merge pull request #5716 from dims/add-new-go-module-for-api-directory
Introduce a new go module - containerd/api for use in standalone clients
2021-07-27 09:37:10 -07:00
Davanum Srinivas
494b940f14
Introduce a new go module - containerd/api for use in standalone clients
In containerd 1.5.x, we introduced support for go modules by adding a
go.mod file in the root directory. This go.mod lists all the things
needed across the whole code base (with the exception of
integration/client which has its own go.mod). So when projects that
need to make calls to containerd API will pull in some code from
containerd/containerd, the `go mod` commands will add all the things
listed in the root go.mod to the projects go.mod file. This causes
some problems as the list of things needed to make a simple API call
is enormous. in effect, making a API call will pull everything that a
typical server needs as well as the root go.mod is all encompassing.
In general if we had smaller things folks could use, that will make it
easier by reducing the number of things that will end up in a consumers
go.mod file.

Now coming to a specific problem, the root containerd go.mod has various
k8s.io/* modules listed. Also kubernetes depends on containerd indirectly
via both moby/moby (working with docker maintainers seperately) and via
google/cadvisor. So when the kubernetes maintainers try to use latest
1.5.x containerd, they will see the kubernetes go.mod ending up depending
on the older version of kubernetes!

So if we can expose just the minimum things needed to make a client API
call then projects like cadvisor can adopt that instead of pulling in
the entire go.mod from containerd. Looking at the existing code in
cadvisor the minimum things needed would be the api/ directory from
containerd. Please see proof of concept here:
github.com/google/cadvisor/pull/2908

To enable that, in this PR, we add a go.mod file in api/ directory. we
split the Protobuild.yaml into two, one for just the things in api/
directory and the rest in the root directory. We adjust various targets
to build things correctly using `protobuild` and also ensure that we
end up with the same generated code as before as well. To ensure we
better take care of the various go.mod/go.sum files, we update the
existing `make vendor` and also add a new `make verify-vendor` that one
can run locally as well in the CI.

Ideally, we would have a `containerd/client` either as a standalone repo
or within `containerd/containerd` as a separate go module. but we will
start here to experiment with a standalone api go module first.

Also there are various follow ups we can do, for example @thaJeztah has
identified two tasks we could do after this PR lands:

github.com/containerd/containerd/pull/5716#discussion_r668821396

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-07-27 07:34:59 -04:00
Derek McGowan
b5b5dcc273
Merge pull request #5162 from claudiubelu/integration-test-images
integration: Windows volume-copy-up images
2021-07-26 17:29:40 -07:00
Kazuyoshi Kato
1d3d08026d Support SIGRTMIN+n signals
systemd uses SIGRTMIN+n signals, but containerd didn't support the signals
since Go's sys/unix doesn't support them.

This change introduces SIGRTMIN+n handling by utilizing moby/sys/signal.

Fixes #5402.

https://www.freedesktop.org/software/systemd/man/systemd.html#Signals

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-07-26 09:36:43 -07:00
Kazuyoshi Kato
00d52bb159 integration: log all processes to investigate the test failure
The test is failing on
https://github.com/containerd/containerd/runs/3129202049#step:10:43.

I'd like to know whether the order of the array is the cause, or
the result is completely off.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-07-21 16:54:04 -07:00
Akihiro Suda
a12c7bd1cf
go.mod: runc v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-18 20:39:34 +09:00
Derek McGowan
18321f539c
Move loop check to before sleep
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-07-13 12:07:21 -07:00
Derek McGowan
2e8a572dfd
Add timestamp to flaky restart monitor test
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-07-13 12:07:21 -07:00
Akihiro Suda
55fd2ab5d6
integration/client: go mod tidy
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-13 14:36:02 +09:00
Maksym Pavlenko
d38c186d9d
Merge pull request #5578 from claudiubelu/integration/failed-container
test integration: Adds a test that restarts a failed container
2021-07-09 12:08:20 -07:00
Phil Estes
cf600abecc
Merge pull request #5619 from mikebrow/cri-add-v1-proxy-alpha
[CRI] move up to CRI v1 and support v1alpha in parallel
2021-07-09 14:07:24 -04:00
Claudiu Belu
16deba098c integration: Windows volume-copy-up images
For Windows, the container image's OS version must closely match the host's OS version.
For this reason, we need to add the --os-version annotation in image manifest lists,
so the Windows nodes can pull the appropriate image from the list.

Previously, the docker manifest CLI did not have the capability to set the --os-version,
it, but it has been introduced in docker 20.10.0.

We're also adding busybox.exe in the image, so we can run Linux commands inside the
container, so the tests will be simpler.

When building Windows images, a docker buildx builder needs to be created and used. When
building Windows images with docker buildx, the flag --output=type=registry is required,
otherwise it cannot be referenced on a Linux node.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-06-30 11:19:54 +00:00
Mike Brown
a5c417ac06 move up to CRI v1 and support v1alpha in parallel
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-06-28 09:34:12 -05:00
Claudiu Belu
04ab4418fd test integration: Adds a test that restarts a failed container
If a container failed to start due to a bad command, the container could not be
recreated with a proper command in its stead. Adds a test that verifies this scenario.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-06-25 10:33:38 +00:00
Akihiro Suda
e1f28659bc
Merge pull request #5287 from claudiubelu/cri-integration-cleanup
Cri integration cleanup
2021-06-25 12:42:33 +09:00
Davanum Srinivas
d9694b2976
Sync integration/go.mod with root go.mod
the root go.mod no longer pins these two imports to specific versions

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-06-21 16:32:07 -04:00
ktock
cd33c007ce CI: Switch to available latest images
`mirror.gcr.io/library/busybox:1.32.0` is unavailable.

```console
$ curl -s https://mirror.gcr.io//v2/library/busybox/tags/list | jq '.tags'
[
  "1.26.2",
  "1.27.2",
  "1.28",
  "1.29",
  "1.29.2",
  "1.29.3",
  "1.30",
  "1.30.1",
  "1.31",
  "1.31.0",
  "1.31.1",
  "1.32",
  "latest"
]
```

`mirror.gcr.io/library/alpine:3.13` is unavailable.

```console
$ curl -s https://mirror.gcr.io//v2/library/alpine/tags/list | jq '.tags'
[
  "20200428",
  "3",
  "3.10",
  "3.11",
  "3.12",
  "3.12.0",
  "3.12.1",
  "3.13.5",
  "3.3",
  "3.4",
  "3.6",
  "3.7",
  "3.8",
  "3.9",
  "latest"
]
```

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-06-21 14:54:50 +09:00
Davanum Srinivas
01585595a5
update integration/client go.sum
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-06-07 09:12:41 -04:00
Claudiu Belu
cabe677369 tests: Use EnsureImageExists for image pulling
A previous commit introduced EnsureImageExists, which ensures that a particular image
already exists. It also deduplicates the image pulling code. Some tests missed this
update.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-06-03 16:02:42 +00:00
Claudiu Belu
a78bdf22df tests: Refactors PodSandbox creation
Most of the tests creating and using Pod Sandboxes in the same way. We can create
a common function that will do that for us, so we can have less duplicated code,
and make it easier to add new tests in the future.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-06-03 16:02:38 +00:00
Claudiu Belu
93e268f625 tests: Symlink volume tests
There was a known issue regarding how the symlink files mounted as
volumes were being handled on Windows. This commit adds tests that
will check against those issue to ensure there won't be any
regressions.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-06-03 13:26:22 +00:00
Ma Xinjian
e1c845df50 Fix small typo
Signed-off-by: Ma Xinjian <maxj.fnst@fujitsu.com>
2021-05-24 11:49:55 +08:00
Priyanka Saggu
ec28288f45
bump runc version to v1.0.0-rc95
Signed-off-by: Priyanka Saggu <priyankasaggu11929@gmail.com>
2021-05-19 17:24:56 +05:30
Phil Estes
a152049d7d
Merge pull request #5163 from claudiubelu/test-windows
tests: Adds support for Windows cri-integration tests
2021-05-17 09:24:31 -04:00
Priyanka Saggu
b6a2517491
bump hcsshim version to v0.8.17
Signed-off-by: Priyanka Saggu <priyankasaggu11929@gmail.com>
2021-05-16 01:38:43 +05:30
Claudiu Belu
09a0c9471b tests: Adds support for Windows cri-integration tests
Currently, the cri-integration tests do not work on Windows due to various reasons.
One of the reasons is because all the tests are using Linux-specific images.
Previous commits refactored the image pulling / usage in the cri-integration tests,
making it easier to update, and easier to configure a custom registry to pull images
with Windows support.

For Windows runs, custom registries can be created, which will also contain Windows
images, and the cri-integration tests can be configured to use those registries by
specifying the "--repo-list" argument, a YAML file which will contain an alternative
mapping of the default registries. This is similar to how E2E tests are handled for
Windows runs in Kubernetes.

Some of the tests are Skipped, as they do not pass yet on Windows.

Windows does not collect inodes used stats, thus, the tests that were expecting non-zero
inodes stats were failing.

Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
2021-05-14 11:27:35 +00:00