Commit Graph

205 Commits

Author SHA1 Message Date
Danny Canter
9a0331c477 maintenance: Remove WithWindowsNetworkNamespace from pkg/cri
Old TODO stating that pkg/cri/opts's `WithWindowsNetworkNamespace`
should be moved to the main containerd pkg was out of date as thats
already been done (well, to the /oci package). This just removes it
and swaps all uses of `WithWindowsNetworkNamespace` to the oci
packages impl.

Signed-off-by: Danny Canter <danny@dcantah.dev>
2022-10-23 06:45:32 -07:00
Mike Brown
3ce301ddee Merge pull request #7349 from thaJeztah/gofmt_119
clean-up "nolint" comments, remove unused ones, update golangci-lint
2022-10-17 10:50:24 -05:00
Samuel Karp
890398677e cri: PodSandboxStatus should tolerate missing task
Signed-off-by: Samuel Karp <samuelkarp@google.com>
2022-10-14 14:40:13 -07:00
Sebastiaan van Stijn
f9c80be1bb remove unneeded nolint-comments (nolintlint), disable deprecated linters
Remove nolint-comments that weren't hit by linters, and remove the "structcheck"
and "varcheck" linters, as they have been deprecated:

    WARN [runner] The linter 'structcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have abandoned the linter.  Replaced by unused.
    WARN [linters context] structcheck is disabled because of generics. You can track the evolution of the generics support by following the https://github.com/golangci/golangci-lint/issues/2649.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-12 14:41:01 +02:00
Sebastiaan van Stijn
29c7fc9520 clean-up "nolint" comments, remove unused ones
- fix "nolint" comments to be in the correct format (`//nolint:<linters>[,<linter>`
  no leading space, required colon (`:`) and linters.
- remove "nolint" comments for errcheck, which is disabled in our config.
- remove "nolint" comments that were no longer needed (nolintlint).
- where known, add a comment describing why a "nolint" was applied.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-12 14:40:59 +02:00
Sebastiaan van Stijn
d215725136 pkg/cri/(server|sbserver): criService.getTLSConfig() add TODO to verify nolint
This `//nolint`  was added in f5c7ac9272
to suppress warnings about the `NameToCertificate` function being deprecated:

    // Deprecated: NameToCertificate only allows associating a single certificate
    // with a given name. Leave that field nil to let the library select the first
    // compatible chain from Certificates.

Looking at that, it was deprecated in Go 1.14 through
eb93c684d4
(https://go-review.googlesource.com/c/go/+/205059), which describes:

    crypto/tls: select only compatible chains from Certificates

    Now that we have a full implementation of the logic to check certificate
    compatibility, we can let applications just list multiple chains in
    Certificates (for example, an RSA and an ECDSA one) and choose the most
    appropriate automatically.

    NameToCertificate only maps each name to one chain, so simply deprecate
    it, and while at it simplify its implementation by not stripping
    trailing dots from the SNI (which is specified not to have any, see RFC
    6066, Section 3) and by not supporting multi-level wildcards, which are
    not a thing in the WebPKI (and in crypto/x509).

We should at least have a comment describing why we are ignoring this, but preferably
review whether we should still use it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-12 14:40:11 +02:00
Qiutong Song
b41d6f40bb Update container with sandbox metadata after NetNS is created
Signed-off-by: Qiutong Song <songqt01@gmail.com>
2022-10-09 01:14:08 +00:00
Derek McGowan
1cc38f8df7 Merge pull request #5904 from qiutongs/ip-leakage-fix 2022-09-29 18:14:35 -07:00
Qiutong Song
4f4aad057d Persist container and sandbox if resource cleanup fails, like teardownPodNetwork
Signed-off-by: Qiutong Song <songqt01@gmail.com>
2022-09-27 14:38:41 +00:00
Maksym Pavlenko
39f7cd73e7 Merge pull request #7405 from kzys/cri-fuzz
Refactor CRI fuzzers
2022-09-22 16:55:27 -07:00
ruiwen-zhao
c6f571fc7d Set grpc code for unimplemented cri-api methods
Signed-off-by: ruiwen-zhao <ruiwen@google.com>
2022-09-22 07:24:48 +00:00
Kazuyoshi Kato
a37c64b20c Refactor CRI fuzzers
pkg/cri/sbserver/cri_fuzzer.go and pkg/cri/server/cri_fuzzer.go were
mostly the same.

This commit merges them together and move the unified fuzzer to
contrib/fuzz again to sort out dependencies. pkg/cri/ shouldn't consume
cmd/.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-09-19 22:14:11 +00:00
Iceber Gu
3cfde732e1 remotes/docker/config: Skipping TLS verification for localhost
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2022-09-13 17:40:23 +08:00
Abirdcfly
dcfaa30ba2 chore: remove duplicate word in comments
Signed-off-by: Abirdcfly Fu <fp544037857@gmail.com>
2022-08-29 13:05:32 +08:00
Samuel Karp
36d0cfd0fd Merge pull request #6517 from ruiwen-zhao/return-resource 2022-08-24 14:01:30 -07:00
ruiwen-zhao
b7b1200dd3 ContainerStatus to return container resources
Signed-off-by: ruiwen-zhao <ruiwen@google.com>
2022-08-24 19:08:06 +00:00
ruiwen-zhao
6e4b6830f1 Update CRI-API
Signed-off-by: ruiwen-zhao <ruiwen@google.com>
2022-08-10 03:55:51 +00:00
Maksym Pavlenko
ca3b9b50fe Run gofmt 1.19
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-08-04 18:18:33 -07:00
zounengren
d121efc6d8 replace with selinux label
Signed-off-by: zounengren <zouyee1989@gmail.com>
2022-07-24 20:11:16 +08:00
Jeff Widman
050cd58ce6 Drop deprecated ioutil
`ioutil` has been deprecated by golang. All the code in `ioutil` just
forwards functionality to code in either the `io` or `os` packages.

See https://github.com/golang/go/pull/51961 for more info.

Signed-off-by: Jeff Widman <jeff@jeffwidman.com>
2022-07-23 08:36:20 -07:00
Danielle Lancashire
3125f7e1a0 cri_stats: handle missing cpu stats
Signed-off-by: Danielle Lancashire <dani@builds.terrible.systems>
2022-07-22 12:10:24 +00:00
James Sturtevant
0d6881898e Refactor usageNanoCores be to used for all OSes
Signed-off-by: James Sturtevant <jstur@microsoft.com>
2022-07-19 16:49:08 -07:00
Daniel Canter
bcdc8468f8 Fix out of date comments for CRI store packages
All of the CRI store related packages all use the standard errdefs
errors now for if a key doesn't or already exists (ErrAlreadyExists,
ErrNotFound), but the comments for the methods still referenced
some unused package specific error definitions. This change just
updates the comments to reflect what errors are actually returned
and adds comments for some previously undocumented exported functions.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2022-07-11 13:57:39 -07:00
Derek McGowan
aee50aeac2 Merge pull request #7108 from fuweid/refactor-cri-api
pkg/cri: use marshal wrapper for version convertor
2022-06-29 13:58:15 -07:00
Wei Fu
c2703c08c9 pkg/cri: use marshal wrapper for version convertor
Use wrapper for ReopenContainerLog v1alpha proto.

Ref: #5619

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-06-29 22:20:47 +08:00
Kazuyoshi Kato
66cc0fc879 Copy FuzzCRI from cncf/cncf-fuzzing
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-06-27 22:54:25 +00:00
wllenyj
42a386c816 CRI: change the /dev/shm mount options in Sandbox.
All containers except the pause container, mount `/dev/shm" with flags
`nosuid,nodev,noexec`. So change mount options for pause container to
keep consistence.
This also helps to solve issues of failing to mount `/dev/shm` when
pod/container level user namespace is enabled.

Fixes: #6911

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Signed-off-by: Lei Wang <wllenyj@linux.alibaba.com>
2022-06-14 10:45:06 +08:00
wllenyj
a62a95789c CRI: remove default /dev/shm mount in Sandbox.
This's an optimization to get rid of redundant `/dev/shm" mounts for pause container.
In `oci.defaultMounts`, there is a default `/dev/shm` mount which is redundant for
pause container.

Fixes: #6911

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Signed-off-by: Lei Wang  <wllenyj@linux.alibaba.com>
2022-06-14 10:45:06 +08:00
Maksym Pavlenko
e71ffddb6b Merge pull request #7042 from samuelkarp/freebsd-unit-tests
Port (some) unit tests to FreeBSD
2022-06-10 15:05:52 -07:00
Kazuyoshi Kato
4ec6a379c0 Merge pull request #6918 from dcantah/windows-snapshotter-cleanup
Windows snapshotter touch ups and new functionality
2022-06-10 11:08:18 -07:00
Samuel Karp
42e019e634 cri/server: Disable tests on FreeBSD
The TestPodAnnotationPassthroughContainerSpec test and the
TestContainerAnnotationPassthroughContainerSpec test both depend on a
platform-specific implementation of criService.containerSpec, which is
unimplemented on FreeBSD.

The TestSandboxContainerSpec depends on a platform-specific
implementation oc criService.sandboxContainerSpec, which is
unimplemented on FreeBSD.

Signed-off-by: Samuel Karp <me@samuelkarp.com>
2022-06-09 18:54:10 -07:00
Shane Jennings
6190b0f04b Correct spelling mistake ("sanbdox" to "sandbox")
Signed-off-by: Shane Jennings <superzinbo@gmail.com>
2022-06-07 10:55:15 +01:00
Daniel Canter
44e12dc5d8 Windows snapshotter touch ups and new functionality
This change does a couple things to remove some cruft/unused functionality
in the Windows snapshotter, as well as add a way to specify the rootfs
size in bytes for a Windows container via a new field added in the CRI api in
k8s 1.24. Setting the rootfs/scratch volume size was assumed to be working
prior to this but turns out not to be the case.

Previously I'd added a change to pass any annotations in the containerd
snapshot form (containerd.io/snapshot/*) as labels for the containers
rootfs snapshot. This was added as a means for a client to be able to provide
containerd.io/snapshot/io.microsoft.container.storage.rootfs.size-gb as an
annotation and have that be translated to a label and ultimately set the
size for the scratch volume in Windows. However, this actually only worked if
interfacing with the CRI api directly (crictl) as Kubernetes itself will
fail to validate annotations that if split by "/" end up with > 2 parts,
which the snapshot labels will (containerd.io / snapshot / foobarbaz).

With this in mind, passing the annotations and filtering to
containerd.io/snapshot/* is moot, so I've removed this code in favor of
a new `snapshotterOpts()` function that will return platform specific
snapshotter options if ones exist. Now on Windows we can just check if
RootfsSizeInBytes is set on the WindowsContainerResources struct and
then return a snapshotter option that sets the right label.

So all in all this change:
- Gets rid of code to pass CRI annotations as labels down to snapshotters.

- Gets rid of the functionality to create a 1GB sized scratch disk if
the client provided a size < 20GB. This code is not used currently and
has a few logical shortcomings as it won't be able to create the disk
if a container is already running and using the same base layer. WCIFS
(driver that handles the unioning of windows container layers together)
holds open handles to some files that we need to delete to create the
1GB scratch disk is the underlying problem.

- Deprecates the containerd.io/snapshot/io.microsoft.container.storage.rootfs.size-gb
label in favor of a new containerd.io/snapshot/windows/rootfs.sizebytes label.
The previous label/annotation wasn't being used by us, and from a cursory
github search wasn't being used by anyone else either. Now that there is a CRI
field to specify the size, this should just be a field that users can set
on their pod specs and don't need to concern themselves with what it eventually
gets translated to, but non-CRI clients can still use the new label/deprecated
label as usual.

- Add test to cri integration suite to validate expanding the rootfs size.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2022-06-06 14:57:07 -07:00
Derek McGowan
c1bcabb454 Merge pull request from GHSA-5ffw-gxpp-mxpf
Limit the response size of ExecSync
2022-06-06 10:19:23 -07:00
Kazuyoshi Kato
40aa4f3f1b Implicitly discard the input to drain the reader
Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-06-06 09:57:13 -07:00
Phil Estes
2b661b890f Merge pull request #6899 from shuaichang/ISSUE6657-support-runtime-snapshotter
Support runtime level snapshotter for issue 6657
2022-06-03 10:04:53 +02:00
shuaichang
7b9f1d4058 Added support for runtime level snapshotter, issue 6657
Signed-off-by: shuaichang <shuai.chang@databricks.com>

Updated annotation name
2022-06-02 16:29:59 -07:00
Fu Wei
aa0aaa4947 Merge pull request #7009 from mikebrow/update-gocni 2022-06-02 11:09:46 +08:00
Mike Brown
e3b4d750db update go-cni/for cni update fixing plugins that don't respond with version
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2022-06-01 17:20:18 -05:00
Kazuyoshi Kato
c149e6c2ea Merge pull request #6996 from dcantah/hpc-validations
Add validations for Windows HostProcess CRI configs
2022-06-01 11:37:12 -07:00
Kazuyoshi Kato
49ca87d727 Limit the response size of ExecSync
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-05-31 22:21:35 +00:00
Daniel Canter
b5e1b8f619 Use t.Run for /pkg/cri tests
A majority of the tests in /pkg/cri are testing/validating multiple
things per test (generally spec or options validations). This flow
lends itself well to using *testing.T's Run method to run each thing
as a subtest so `go test` output can actually display which subtest
failed/passed.

Some of the tests in the packages in pkg/cri already did this, but
a bunch simply logged what sub-testcase was currently running without
invoking t.Run.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2022-05-29 18:32:09 -07:00
Daniel Canter
978ff393d2 Add validations for Windows HostProcess CRI configs
HostProcess containers require every container in the pod to be a
host process container and have the corresponding field set. The Kubelet
usually enforces this so we'd error before even getting here but we recently
found a bug in this logic so better to be safe than sorry.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2022-05-27 21:17:07 -07:00
AllenZMC
eaec6530d7 fix some confusing typos
Signed-off-by: AllenZMC <zhongming.chang@daocloud.io>
2022-05-17 23:53:36 +08:00
Akihiro Suda
42584167b7 Officially deprecate Schema 1
Schema 1 has been substantially deprecated since circa. 2017 in favor of Schema 2 introduced in Docker 1.10 (Feb 2016)
and its successor OCI Image Spec v1, but we have not officially deprecated Schema 1.

One of the reasons was that Quay did not support Schema 2 so far, but it is reported that Quay has been
supporting Schema 2 since Feb 2020 (moby/buildkit issue 409).

This PR deprecates pulling Schema 1 images but the feature will not be removed before containerd 2.0.
Pushing Schema 1 images was never implemented in containerd (and its consumers such as BuildKit).

Docker/Moby already disabled pushing Schema 1 images in Docker 20.10 (moby/moby PR 41295),
but Docker/Moby has not yet disabled pulling Schema 1 as containerd has not yet deprecated Schema 1.
(See the comments in moby/moby PR 42300.)
Docker/Moby is expected to disable pulling Schema 1 images in future after this deprecation.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-05-02 19:08:38 +09:00
Mike Brown
6b35307594 Merge pull request #5490 from askervin/5Bu_blockio
Support for cgroups blockio
2022-04-29 10:07:56 -05:00
Antti Kervinen
10576c298e cri: support blockio class in pod and container annotations
This patch adds support for a container annotation and two separate
pod annotations for controlling the blockio class of containers.

The container annotation can be used by a CRI client:
  "io.kubernetes.cri.blockio-class"

Pod annotations specify the blockio class in the K8s pod spec level:
  "blockio.resources.beta.kubernetes.io/pod"
  (pod-wide default for all containers within)

  "blockio.resources.beta.kubernetes.io/container.<container_name>"
  (container-specific overrides)

Correspondingly, this patch adds support for --blockio-class and
--blockio-config-file to ctr, too.

This implementation follows the resource class annotation pattern
introduced in RDT and merged in commit 893701220.

Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
2022-04-29 11:44:09 +03:00
Derek McGowan
6e0231f992 Merge pull request #6150 from fuweid/support-4984
feature: support image pull progress timeout
2022-04-26 12:15:09 -07:00
Wei Fu
00d102da9f feature: support image pull progress timeout
Kubelet sends the PullImage request without timeout, because the image size
is unknown and timeout is hard to defined. The pulling request might run
into 0B/s speed, if containerd can't receive any packet in that connection.
For this case, the containerd should cancel the PullImage request.

Although containerd provides ingester manager to track the progress of pulling
request, for example `ctr image pull` shows the console progress bar, it needs
more CPU resources to open/read the ingested files to get status.

In order to support progress timeout feature with lower overhead, this
patch uses http.RoundTripper wrapper to track active progress. That
wrapper will increase active-request number and return the
countingReadCloser wrapper for http.Response.Body. Each bytes-read
can be count and the active-request number will be descreased when the
countingReadCloser wrapper has been closed. For the progress tracker,
it can check the active-request number and bytes-read at intervals. If
there is no any progress, the progress tracker should cancel the
request.

NOTE: For each blob data, the containerd will make sure that the content
writer is opened before sending http request to the registry. Therefore, the
progress reporter can rely on the active-request number.

fixed: #4984

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-04-27 00:02:27 +08:00
Derek McGowan
3dbd6a2498 Merge pull request #6841 from kzys/proto-upgrade-6
Migrate off from github.com/gogo/protobuf
2022-04-25 15:12:51 -07:00