Commit Graph

10081 Commits

Author SHA1 Message Date
Derek McGowan
0def0dbecb
Merge pull request #6275 from gabriel-samfira/enable-volume-ownership-test
Integration: Enable TestVolumeOwnership on Windows
2021-12-14 09:33:24 -08:00
Derek McGowan
d0da25cf54
Merge pull request #6296 from dmcgowan/prepare-1.6.0-beta.4
Prepare release notes for v1.6.0-beta.4
2021-12-14 08:05:27 -08:00
Phil Estes
330961c2d5
Merge pull request #6358 from jonyhy96/feat-error
refactor: functions for error log and error return
2021-12-14 10:16:54 -05:00
Fu Wei
d47fa40d1b
Merge pull request #6021 from dmcgowan/runc-shim-plugin 2021-12-14 10:19:23 +08:00
Derek McGowan
ac531108ab
Merge pull request #6155 from egernst/cri-update-for-sandbox-sizing
CRI update for sandbox sizing
2021-12-13 16:21:30 -08:00
Derek McGowan
7d268f7120
Merge pull request #6354 from kzys/ns-snapshot
medatada: make namespaces' deletion error less cryptic
2021-12-13 15:54:08 -08:00
Fabiano Fidêncio
f1c7993311 seutil: Fix setting the "container_kvm_t" label
The ability to handle KVM based runtimes with SELinux has been added as
part of d715d00906.

However, that commit introduced some logic to check whether the
"container_kvm_t" label would or not be present in the system, and while
the intentions were good, there's two major issues with the approach:
1. Inspecting "/etc/selinux/targeted/contexts/customizable_types" is not
   the way to go, as it doesn't list the "container_kvm_t" at all.
2. There's no need to check for the label, as if the label is invalid an
   "Invalid Label" error will be returned and that's it.

With those two in mind, let's simplify the logic behind setting the
"container_kvm_t" label, removing all the unnecessary code.

Here's an output of VMM process running, considering:
* The state before this patch:
  ```
  $ containerd --version
  containerd github.com/containerd/containerd v1.6.0-beta.3-88-g7fa44fc98 7fa44fc98f
  $ kubectl apply -f ~/simple-pod.yaml
  pod/nginx created
  $ ps -auxZ | grep cloud-hypervisor
  system_u:system_r:container_runtime_t:s0 root 609717 4.0  0.5 2987512 83588 ?    Sl   08:32   0:00 /usr/bin/cloud-hypervisor --api-socket /run/vc/vm/be9d5cbabf440510d58d89fc8a8e77c27e96ddc99709ecaf5ab94c6b6b0d4c89/clh-api.sock
  ```

* The state after this patch:
  ```
  $ containerd --version
  containerd github.com/containerd/containerd v1.6.0-beta.3-89-ga5f2113c9 a5f2113c9fc15b19b2c364caaedb99c22de4eb32
  $ kubectl apply -f ~/simple-pod.yaml
  pod/nginx created
  $ ps -auxZ | grep cloud-hypervisor
  system_u:system_r:container_kvm_t:s0:c638,c999 root 614842 14.0  0.5 2987512 83228 ? Sl 08:40   0:00 /usr/bin/cloud-hypervisor --api-socket /run/vc/vm/f8ff838afdbe0a546f6995fe9b08e0956d0d0cdfe749705d7ce4618695baa68c/clh-api.sock
  ```

Note, the tests were performed using the following configuration snippet:
```
[plugins]
  [plugins.cri]
    enable_selinux = true
    [plugins.cri.containerd]
      [plugins.cri.containerd.runtimes]
        [plugins.cri.containerd.runtimes.kata]
           runtime_type = "io.containerd.kata.v2"
           privileged_without_host_devices = true
```

And using the following pod yaml:
```
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  runtimeClassName: kata
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80
```

Fixes: #6371

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-12-14 00:09:17 +01:00
Derek McGowan
5bd38c6a10
Merge pull request #6373 from minbaev/add-list-stat
ListContainerStats to return stats response if ContainerStatsFilter is nil
2021-12-13 14:53:46 -08:00
Alexander Minbaev
c8a009d18c add-list-stat: return container list if filter is nil
Signed-off-by: Alexander Minbaev <alexander.minbaev@ibm.com>
2021-12-13 15:09:18 -06:00
Mike Brown
d977f77a50
Merge pull request #6369 from fuweid/fix_test_case
integration: deflake TestContainerdRestart
2021-12-13 13:54:27 -06:00
Derek McGowan
f83ab813d2
Use task plugin for runc shim
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-13 10:37:13 -08:00
Derek McGowan
04e57d71b2
Seperate shim manager and task service
Create new shim manager interface and deprecate older
shim manager interface.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-13 10:37:12 -08:00
Kazuyoshi Kato
44b28b61ff medatada: make namespaces' deletion error less cryptic
The error message was unnecessary cryptic. `snapshot-[name]` notation
was only used here and hard to understand.

Instead it should say `snapshots on "..." snapshotter`.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-13 09:28:24 -08:00
Eric Ernst
20419feaac cri, sandbox: pass sandbox resource details if available, applicable
CRI API has been updated to include a an optional `resources` field in the
LinuxPodSandboxConfig field, as part of the RunPodSandbox request.

Having sandbox level resource details at sandbox creation time will have
large benefits for sandboxed runtimes. In the case of Kata Containers,
for example, this'll allow for better support of SW/HW architectures
which don't allow for CPU/memory hotplug, and it'll allow for better
queue sizing for virtio devices associated with the sandbox (in the VM
case).

If this sandbox resource information is provided as part of the run
sandbox request, let's introduce a pattern where we will update the
pause container's runtiem spec to include this information in the
annotations field.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-12-13 08:41:41 -08:00
Phil Estes
6e9e759553
Merge pull request #6225 from jonyhy96/feat-blot-open-timeout
feat: add timeout for bolt open
2021-12-13 11:22:28 -05:00
Wei Fu
9e9ee66bfd integration: deflake TestContainerdRestart
The CRI-plugin will setup watcher for each container after
StartContainer or RunPodSandbox. It will cleanup task(container/sandbox)
if received the exit event from watcher.

The original test design is to `Delete` sandbox container to get
NOT_READY state and expect to receive NotFound error. It depends on that
CRI-plugin cleanups container after `Delete` API. If not, the shim will
be cleanup and test code will receive `ttrpc: closed: unknown` or other
unknown error. It is flaky.

In this patch, the test will only send the kill signal and wait for the
exit event. When sandbox exits, the state will and must be NOT_READY.

```plain
// test fail log

=== RUN TestContainerdRestart
restart_test.go:92: Make sure no sandbox is running before test
restart_test.go:97: Start test sandboxes and containers
common.go:115: Image "k8s.gcr.io/pause:3.6" already exists, not pulling.
common.go:115: Image "k8s.gcr.io/pause:3.6" already exists, not pulling.
restart_test.go:139:
Error Trace: restart_test.go:139
Error: Should be true
Test: TestContainerdRestart
Messages: delete should return not found error but returned failed to delete task: ttrpc: closed: unknown
--- FAIL: TestContainerdRestart (4.25s)

// containerd log

&TaskExit{ContainerID:4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184,ID:4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184,Pid:28430,ExitStatus:137,ExitedAt:2021-12-12 07:56:01.400753012 +0000 UTC,XXX_unrecognized:[],}"
time="2021-12-12T07:56:01.401120516Z" level=debug msg="event forwarded" ns=k8s.io topic=/tasks/exit type=containerd.events.TaskExit
time="2021-12-12T07:56:01.418934208Z" level=debug msg="event forwarded" ns=k8s.io topic=/tasks/delete type=containerd.events.TaskDelete
time="2021-12-12T07:56:01.419192910Z" level=info msg="shim disconnected" id=4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184
time="2021-12-12T07:56:01.419235911Z" level=warning msg="cleaning up after shim disconnected" id=4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184 namespace=k8s.io
time="2021-12-12T07:56:01.419247711Z" level=info msg="cleaning up dead shim"
time="2021-12-12T07:56:01.419235311Z" level=error msg="failed sending message on channel" error="write unix /run/containerd/s/18afde7fcde70236eb31b9f43f3bd92af1dc1186583c501aa1396255f87f95d4->@: write: broken pipe"
time="2021-12-12T07:56:01.419354712Z" level=debug msg="failed to delete task" error="ttrpc: closed" id=4b4c1d1d303c14a2cc759631d163f153ba8536e9ea6821744a509e4a17346184
```

CI Link: `https://pipelines.actions.githubusercontent.com/G4SighzWVVZ6vsyiz7FFMFjLjRzveJHseEnVyibkSq87Cl2x4O/_apis/pipelines/1/runs/9501/signedlogcontent/76?urlExpires=2021-12-12T08%3A42%3A08.0765750Z&urlSigningMethod=HMACV1&urlSignature=pH93isMSFdZUo1ndnZynJpZbPGrEyvt12MO03fgUU7I%3D`

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-12-13 23:43:26 +08:00
haoyun
dd26d3d092 feat: support custom timeout for blot open
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 17:02:37 +08:00
haoyun
c0d07094be feat: Errorf usage
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 14:31:53 +08:00
Akihiro Suda
7fa44fc98f
Merge pull request #6333 from thaJeztah/bump_go_1.17.4
Update Go to 1.17.5
2021-12-13 13:18:08 +09:00
Sebastiaan van Stijn
0e472420bf
Update Go to 1.17.5
go1.17.5 (released 2021-12-09) includes security fixes to the syscall and net/http
packages. See the Go 1.17.5 milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.5+label%3ACherryPickApproved

Update Go to 1.17.4
--------------------

go1.17.4 (released 2021-12-02) includes fixes to the compiler, linker, runtime,
and the go/types, net/http, and time packages. See the Go 1.17.4 milestone on
the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.17.4+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-12 01:25:02 +01:00
Derek McGowan
569042e6df
Prepare release notes for v1.6.0-beta.4
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-10 15:35:28 -08:00
Derek McGowan
4236f6b225
Merge pull request #6332 from thaJeztah/consistent_import_alias
cri/server: use consistent alias for pkg/ioutil
2021-12-10 15:22:56 -08:00
Derek McGowan
1057ed3857
Merge pull request #6364 from dmcgowan/disable-restart-monitor-test-windows
Disable restart monitor test in Windows
2021-12-10 14:38:20 -08:00
Derek McGowan
552a27081c
Disable restart monitor test in Windows
Skip this test until this error can be evaluated and the appropriate
test fix or environment configuration can be determined.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-10 11:43:23 -08:00
Maksym Pavlenko
a95a8b8fff
Merge pull request #6338 from kzys/windows-fail-fast
Don't fail-fast on Windows integration tests
2021-12-10 09:52:03 -08:00
Sebastiaan van Stijn
b7f673790f
OCI: Mount (accessible) host devices in privileged rootless containers
Allow rootless containers with privileged to mount devices that are accessible
(ignore permission errors in rootless mode).

This patch updates oci.getDevices() to ignore access denied errors on sub-
directories and files within the given path if the container is running with
userns enabled.

Note that these errors are _only_ ignored on paths _under_ the specified path,
and not the path itself, so if `HostDevices()` is used, and `/dev` itself is
not accessible, or `WithDevices()` is used to specify a device that is not
accessible, an error is still produced.

Tests were added, which includes a temporary workaround for compatibility
with Go 1.16 (we could decide to skip these tests on Go 1.16 instead).

To verify the patch in a container:

    docker run --rm -v $(pwd):/go/src/github.com/containerd/containerd -w /go/src/github.com/containerd/containerd golang:1.17 sh -c 'go test -v -run TestHostDevices ./oci'
    === RUN   TestHostDevicesOSReadDirFailure
    --- PASS: TestHostDevicesOSReadDirFailure (0.00s)
    === RUN   TestHostDevicesOSReadDirFailureInUserNS
    --- PASS: TestHostDevicesOSReadDirFailureInUserNS (0.00s)
    === RUN   TestHostDevicesDeviceFromPathFailure
    --- PASS: TestHostDevicesDeviceFromPathFailure (0.00s)
    === RUN   TestHostDevicesDeviceFromPathFailureInUserNS
    --- PASS: TestHostDevicesDeviceFromPathFailureInUserNS (0.00s)
    === RUN   TestHostDevicesAllValid
    --- PASS: TestHostDevicesAllValid (0.00s)
    PASS
    ok  	github.com/containerd/containerd/oci	0.006s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-10 12:16:59 +01:00
Sebastiaan van Stijn
7f70ff9672
oci.getDevices(): move "non-dir, non '/dev/console'" case into switch
This makes it slightly clearer that these are all part of the same logic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-10 12:16:57 +01:00
Maksym Pavlenko
95b83fa54f
Merge pull request #6355 from dmcgowan/integration-client-restart-daemon-output
Update restart monitor test to output daemon logs on failure
2021-12-09 13:51:01 -08:00
Derek McGowan
2c96d5b067
Run windows parallel integration test as short
This prevents tests which spawn daemons from running at
the same time as the first integration test.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-09 11:45:38 -08:00
Derek McGowan
807213fd32
Update restart monitor test to output daemon logs on failure
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-12-09 11:45:37 -08:00
Phil Estes
26d356d09d
Merge pull request #6263 from thaJeztah/bump_oci_specs
go.mod: update image-spec to latest (v1.0.3-dev)
2021-12-09 13:26:20 -05:00
Michael Crosby
9b0303913f
only relabel cri managed host mounts
Co-authored-by: Samuel Karp <skarp@amazon.com>
Signed-off-by: Michael Crosby <michael@thepasture.io>
Signed-off-by: Samuel Karp <skarp@amazon.com>
2021-12-09 09:53:47 -08:00
Sebastiaan van Stijn
90cdc6c9a6
images/converter: remove deprecated types
Removes the, now obsolete structs/types:

- DualIndex
- DualManifest
- ObjectWithMediaType

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-09 17:50:12 +01:00
Sebastiaan van Stijn
fc8138468f
go.mod: update image-spec to latest (v1.0.3-dev)
The OCI image spec did a v1.0.2 security release for CVE-2021-41190, however
commit 09c9270fee, depends on MediaTypes that
have not yet been released by the OCI image-spec, so using current "main" instead.

full diff: 5ad6f50d62...693428a734

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-09 17:50:09 +01:00
Sebastiaan van Stijn
2d3009038c
cri/server: use consistent alias for pkg/ioutil
Consistently use cioutil to prevent it being confused for Golang's ioutil.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-12-09 17:47:22 +01:00
Phil Estes
a04656c1dd
Merge pull request #6353 from mxpv/gids
Skip WithAdditionalGIDs on Darwin
2021-12-08 11:57:25 -05:00
Derek McGowan
f9cfadcdd2
Merge pull request #6303 from jsturtevant/fix-rg-timestamp
Use RFC3339 format on Windows periodic resource group creation so cleanup works
2021-12-08 08:41:35 -08:00
Alakesh Haloi
dc13bcd51e Enable lazy init for ext4 with devicemapper
Add file system options for config file, so that user can use
non-default file system parameters for the fs type of choice
Using file system options in config file overwrites the default
options already being used.

Signed-off-by: Alakesh Haloi <alakeshh@amazon.com>
2021-12-07 13:32:32 -08:00
James Sturtevant
68dabdcc69 Use RFC3339 format so rg cleanup works
Signed-off-by: James Sturtevant <jstur@microsoft.com>
2021-12-07 11:04:36 -08:00
Nashwan Azhari
6f9d80907e Remove Windows integration testing for 2004.
Considering Windows 2004's EoL on the 14th of December, 2021,
this PR removes all periodic integration testing for 2004.

Signed-off-by: Nashwan Azhari <nazhari@cloudbasesolutions.com>
2021-12-07 19:49:33 +02:00
Gabriel Adrian Samfira
77a321a073 Replace find with native Go code
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-07 11:04:38 +00:00
Gabriel Adrian Samfira
22dc60e059 Enable TestVolumeOwnership on Windows
This change enables the TestVolumeOwnership on Windows. The test
assumes that the volume-ownership image is built on Windows, thus
ensuring that Windows file security info (ACLs and ownership info)
are attached to the C:\volumes\test_dir path.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2021-12-07 10:47:13 +00:00
Maksym Pavlenko
441bcb56d7 Skip WithAdditionalGIDs on Darwin
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-12-06 14:26:43 -08:00
Kazuyoshi Kato
4c1e26cd5b Don't fail-fast on Windows integration tests
As like other integration tests, Windows integration tests should not
fail-fast. So developers can see whether an issue is platform-specific
or not.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-06 10:56:33 -08:00
Derek McGowan
753f95e23f
Merge pull request #6330 from AkihiroSuda/runc-1.0.3
update runc to v1.0.3
2021-12-06 10:54:53 -08:00
Derek McGowan
1c32c00fe7
Merge pull request #6329 from mxpv/mac
Diff apply native snapshotter on Darwin
2021-12-06 10:29:56 -08:00
Akihiro Suda
c9d1e155c7
go.mod: github.com/opencontainers/runc v1.0.3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-12-06 15:12:22 +09:00
Akihiro Suda
44995d4831
update runc binary to v1.0.3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-12-06 15:06:39 +09:00
Maksym Pavlenko
416899fc8e Allow native snapshotter on Darwin
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-12-04 14:08:09 -08:00
Fu Wei
292b0c1561
Merge pull request #6328 from kzys/go-install
Do not use `go get` to install executables
2021-12-04 11:28:56 +08:00