Commit Graph

10422 Commits

Author SHA1 Message Date
Derek McGowan
04e57d71b2
Seperate shim manager and task service
Create new shim manager interface and deprecate older
shim manager interface.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-13 10:37:12 -08:00
Kazuyoshi Kato
44b28b61ff medatada: make namespaces' deletion error less cryptic
The error message was unnecessary cryptic. `snapshot-[name]` notation
was only used here and hard to understand.

Instead it should say `snapshots on "..." snapshotter`.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-13 09:28:24 -08:00
Eric Ernst
20419feaac cri, sandbox: pass sandbox resource details if available, applicable
CRI API has been updated to include a an optional `resources` field in the
LinuxPodSandboxConfig field, as part of the RunPodSandbox request.

Having sandbox level resource details at sandbox creation time will have
large benefits for sandboxed runtimes. In the case of Kata Containers,
for example, this'll allow for better support of SW/HW architectures
which don't allow for CPU/memory hotplug, and it'll allow for better
queue sizing for virtio devices associated with the sandbox (in the VM
case).

If this sandbox resource information is provided as part of the run
sandbox request, let's introduce a pattern where we will update the
pause container's runtiem spec to include this information in the
annotations field.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-12-13 08:41:41 -08:00
Phil Estes
6e9e759553
Merge pull request #6225 from jonyhy96/feat-blot-open-timeout
feat: add timeout for bolt open
2021-12-13 11:22:28 -05:00
Wei Fu
9e9ee66bfd integration: deflake TestContainerdRestart
The CRI-plugin will setup watcher for each container after
StartContainer or RunPodSandbox. It will cleanup task(container/sandbox)
if received the exit event from watcher.

The original test design is to `Delete` sandbox container to get
NOT_READY state and expect to receive NotFound error. It depends on that
CRI-plugin cleanups container after `Delete` API. If not, the shim will
be cleanup and test code will receive `ttrpc: closed: unknown` or other
unknown error. It is flaky.

In this patch, the test will only send the kill signal and wait for the
exit event. When sandbox exits, the state will and must be NOT_READY.

```plain
// test fail log

=== RUN TestContainerdRestart
restart_test.go:92: Make sure no sandbox is running before test
restart_test.go:97: Start test sandboxes and containers
common.go:115: Image "k8s.gcr.io/pause:3.6" already exists, not pulling.
common.go:115: Image "k8s.gcr.io/pause:3.6" already exists, not pulling.
restart_test.go:139:
Error Trace: restart_test.go:139
Error: Should be true
Test: TestContainerdRestart
Messages: delete should return not found error but returned failed to delete task: ttrpc: closed: unknown
--- FAIL: TestContainerdRestart (4.25s)

// containerd log

&TaskExit{ContainerID:4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184,ID:4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184,Pid:28430,ExitStatus:137,ExitedAt:2021-12-12 07:56:01.400753012 +0000 UTC,XXX_unrecognized:[],}"
time="2021-12-12T07:56:01.401120516Z" level=debug msg="event forwarded" ns=k8s.io topic=/tasks/exit type=containerd.events.TaskExit
time="2021-12-12T07:56:01.418934208Z" level=debug msg="event forwarded" ns=k8s.io topic=/tasks/delete type=containerd.events.TaskDelete
time="2021-12-12T07:56:01.419192910Z" level=info msg="shim disconnected" id=4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184
time="2021-12-12T07:56:01.419235911Z" level=warning msg="cleaning up after shim disconnected" id=4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184 namespace=k8s.io
time="2021-12-12T07:56:01.419247711Z" level=info msg="cleaning up dead shim"
time="2021-12-12T07:56:01.419235311Z" level=error msg="failed sending message on channel" error="write unix /run/containerd/s/18afde7fcde70236eb31b9f43f3bd92af1dc1186583c501aa1396255f87f95d4->@: write: broken pipe"
time="2021-12-12T07:56:01.419354712Z" level=debug msg="failed to delete task" error="ttrpc: closed" id=4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184
```

CI Link: `https://pipelines.actions.githubusercontent.com/G4SighzWVVZ6vsyiz7FFMFjLjRzveJHseEnVyibkSq87Cl2x4O/_apis/pipelines/1/runs/9501/signedlogcontent/76?urlExpires=2021-12-12T08%3A42%3A08.0765750Z&urlSigningMethod=HMACV1&urlSignature=pH93isMSFdZUo1ndnZynJpZbPGrEyvt12MO03fgUU7I%3D`

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-12-13 23:43:26 +08:00
haoyun
dd26d3d092 feat: support custom timeout for blot open
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 17:02:37 +08:00
haoyun
c0d07094be feat: Errorf usage
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 14:31:53 +08:00
Akihiro Suda
7fa44fc98f
Merge pull request #6333 from thaJeztah/bump_go_1.17.4
Update Go to 1.17.5
2021-12-13 13:18:08 +09:00
Sebastiaan van Stijn
0e472420bf
Update Go to 1.17.5
go1.17.5 (released 2021-12-09) includes security fixes to the syscall and net/http
packages. See the Go 1.17.5 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.5+label%3ACherryPickApproved

Update Go to 1.17.4
--------------------

go1.17.4 (released 2021-12-02) includes fixes to the compiler, linker, runtime,
and the go/types, net/http, and time packages. See the Go 1.17.4 milestone on
the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.4+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-12 01:25:02 +01:00
Derek McGowan
569042e6df
Prepare release notes for v1.6.0-beta.4
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-10 15:35:28 -08:00
Derek McGowan
4236f6b225
Merge pull request #6332 from thaJeztah/consistent_import_alias
cri/server: use consistent alias for pkg/ioutil
2021-12-10 15:22:56 -08:00
Derek McGowan
1057ed3857
Merge pull request #6364 from dmcgowan/disable-restart-monitor-test-windows
Disable restart monitor test in Windows
2021-12-10 14:38:20 -08:00
Derek McGowan
552a27081c
Disable restart monitor test in Windows
Skip this test until this error can be evaluated and the appropriate
test fix or environment configuration can be determined.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-10 11:43:23 -08:00
Maksym Pavlenko
a95a8b8fff
Merge pull request #6338 from kzys/windows-fail-fast
Don't fail-fast on Windows integration tests
2021-12-10 09:52:03 -08:00
Sebastiaan van Stijn
b7f673790f
OCI: Mount (accessible) host devices in privileged rootless containers
Allow rootless containers with privileged to mount devices that are accessible
(ignore permission errors in rootless mode).

This patch updates oci.getDevices() to ignore access denied errors on sub-
directories and files within the given path if the container is running with
userns enabled.

Note that these errors are _only_ ignored on paths _under_ the specified path,
and not the path itself, so if `HostDevices()` is used, and `/dev` itself is
not accessible, or `WithDevices()` is used to specify a device that is not
accessible, an error is still produced.

Tests were added, which includes a temporary workaround for compatibility
with Go 1.16 (we could decide to skip these tests on Go 1.16 instead).

To verify the patch in a container:

    docker run --rm -v $(pwd):/go/src/github.com/containerd/containerd -w /go/src/github.com/containerd/containerd golang:1.17 sh -c 'go test -v -run TestHostDevices ./oci'
    === RUN   TestHostDevicesOSReadDirFailure
    --- PASS: TestHostDevicesOSReadDirFailure (0.00s)
    === RUN   TestHostDevicesOSReadDirFailureInUserNS
    --- PASS: TestHostDevicesOSReadDirFailureInUserNS (0.00s)
    === RUN   TestHostDevicesDeviceFromPathFailure
    --- PASS: TestHostDevicesDeviceFromPathFailure (0.00s)
    === RUN   TestHostDevicesDeviceFromPathFailureInUserNS
    --- PASS: TestHostDevicesDeviceFromPathFailureInUserNS (0.00s)
    === RUN   TestHostDevicesAllValid
    --- PASS: TestHostDevicesAllValid (0.00s)
    PASS
    ok  	github.com/containerd/containerd/oci	0.006s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-10 12:16:59 +01:00
Sebastiaan van Stijn
7f70ff9672
oci.getDevices(): move "non-dir, non '/dev/console'" case into switch
This makes it slightly clearer that these are all part of the same logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-10 12:16:57 +01:00
Maksym Pavlenko
95b83fa54f
Merge pull request #6355 from dmcgowan/integration-client-restart-daemon-output
Update restart monitor test to output daemon logs on failure
2021-12-09 13:51:01 -08:00
Derek McGowan
2c96d5b067
Run windows parallel integration test as short
This prevents tests which spawn daemons from running at
the same time as the first integration test.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-09 11:45:38 -08:00
Derek McGowan
807213fd32
Update restart monitor test to output daemon logs on failure
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-09 11:45:37 -08:00
Phil Estes
26d356d09d
Merge pull request #6263 from thaJeztah/bump_oci_specs
go.mod: update image-spec to latest (v1.0.3-dev)
2021-12-09 13:26:20 -05:00
Michael Crosby
9b0303913f
only relabel cri managed host mounts
Co-authored-by: Samuel Karp <skarp@amazon.com>
Signed-off-by: Michael Crosby <michael@thepasture.io>
Signed-off-by: Samuel Karp <skarp@amazon.com>
2021-12-09 09:53:47 -08:00
Sebastiaan van Stijn
90cdc6c9a6
images/converter: remove deprecated types
Removes the, now obsolete structs/types:

- DualIndex
- DualManifest
- ObjectWithMediaType

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-09 17:50:12 +01:00
Sebastiaan van Stijn
fc8138468f
go.mod: update image-spec to latest (v1.0.3-dev)
The OCI image spec did a v1.0.2 security release for CVE-2021-41190, however
commit 09c9270fee, depends on MediaTypes that
have not yet been released by the OCI image-spec, so using current "main" instead.

full diff: 5ad6f50d62...693428a734

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-09 17:50:09 +01:00
Sebastiaan van Stijn
2d3009038c
cri/server: use consistent alias for pkg/ioutil
Consistently use cioutil to prevent it being confused for Golang's ioutil.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-09 17:47:22 +01:00
Phil Estes
a04656c1dd
Merge pull request #6353 from mxpv/gids
Skip WithAdditionalGIDs on Darwin
2021-12-08 11:57:25 -05:00
Derek McGowan
f9cfadcdd2
Merge pull request #6303 from jsturtevant/fix-rg-timestamp
Use RFC3339 format on Windows periodic resource group creation so cleanup works
2021-12-08 08:41:35 -08:00
Alakesh Haloi
dc13bcd51e Enable lazy init for ext4 with devicemapper
Add file system options for config file, so that user can use
non-default file system parameters for the fs type of choice
Using file system options in config file overwrites the default
options already being used.

Signed-off-by: Alakesh Haloi <alakeshh@amazon.com>
2021-12-07 13:32:32 -08:00
James Sturtevant
68dabdcc69 Use RFC3339 format so rg cleanup works
Signed-off-by: James Sturtevant <jstur@microsoft.com>
2021-12-07 11:04:36 -08:00
Nashwan Azhari
6f9d80907e Remove Windows integration testing for 2004.
Considering Windows 2004's EoL on the 14th of December, 2021,
this PR removes all periodic integration testing for 2004.

Signed-off-by: Nashwan Azhari <nazhari@cloudbasesolutions.com>
2021-12-07 19:49:33 +02:00
Gabriel Adrian Samfira
77a321a073 Replace find with native Go code
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-07 11:04:38 +00:00
Gabriel Adrian Samfira
22dc60e059 Enable TestVolumeOwnership on Windows
This change enables the TestVolumeOwnership on Windows. The test
assumes that the volume-ownership image is built on Windows, thus
ensuring that Windows file security info (ACLs and ownership info)
are attached to the C:\volumes\test_dir path.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-07 10:47:13 +00:00
Maksym Pavlenko
441bcb56d7 Skip WithAdditionalGIDs on Darwin
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-12-06 14:26:43 -08:00
Kazuyoshi Kato
4c1e26cd5b Don't fail-fast on Windows integration tests
As like other integration tests, Windows integration tests should not
fail-fast. So developers can see whether an issue is platform-specific
or not.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-06 10:56:33 -08:00
Derek McGowan
753f95e23f
Merge pull request #6330 from AkihiroSuda/runc-1.0.3
update runc to v1.0.3
2021-12-06 10:54:53 -08:00
Derek McGowan
1c32c00fe7
Merge pull request #6329 from mxpv/mac
Diff apply native snapshotter on Darwin
2021-12-06 10:29:56 -08:00
ningmingxiao
54e95e6b88
fix Implicit memory aliasing in for loop
Signed-off-by: ningmingxiao <ning.mingxiao@zte.com.cn>
2021-12-06 21:36:31 +08:00
Akihiro Suda
c9d1e155c7
go.mod: github.com/opencontainers/runc v1.0.3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-12-06 15:12:22 +09:00
Akihiro Suda
44995d4831
update runc binary to v1.0.3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-12-06 15:06:39 +09:00
Maksym Pavlenko
416899fc8e Allow native snapshotter on Darwin
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-12-04 14:08:09 -08:00
Fu Wei
292b0c1561
Merge pull request #6328 from kzys/go-install
Do not use `go get` to install executables
2021-12-04 11:28:56 +08:00
Fu Wei
4af023480f
Merge pull request #6326 from mxpv/builtins 2021-12-04 08:33:03 +08:00
Kazuyoshi Kato
6b0e2414c8 Do not use go get to install executables
https://go.dev/doc/go-get-install-deprecation

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-03 14:56:33 -08:00
Maksym Pavlenko
d2f1dda0f1 Include runtime v2 in default builtins
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-12-03 11:30:03 -08:00
Maksym Pavlenko
d06bce4d3d
Merge pull request #6314 from dcantah/ws2022-ci
Add Windows Server 2022 CI runs
2021-12-03 10:51:00 -08:00
Phil Estes
d742453086
Merge pull request #6324 from gabriel-samfira/set-explicit-acl
Set explicit ACL on test files
2021-12-03 13:14:04 -05:00
Phil Estes
2c1f517811
Merge pull request #6311 from thaJeztah/update_test_image
integration: align tags of test images
2021-12-03 13:12:27 -05:00
Derek McGowan
0721777960
Merge pull request #6325 from gabriel-samfira/set-containerd-root
Set CONTAINERD_ROOT in Windows cri-integration
2021-12-03 09:07:04 -08:00
Phil Estes
e0e605cf39
Merge pull request #6312 from thaJeztah/export_DeviceFromPath
export oci.DeviceFromPath()
2021-12-03 11:47:31 -05:00
Gabriel Adrian Samfira
392f604eba
Use a single RUN command
Run test file creation in single RUN command.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-03 18:10:39 +02:00
Danielle Lancashire
533dd1c0ee fixup: check for swap accounting
Signed-off-by: Danielle Lancashire <dani@builds.terrible.systems>
2021-12-03 16:04:01 +01:00