docker folks added NumCPU implementation for windows that
supported hot-plugging of CPUs. The implementation used the
GetProcessAffinityMask to be able to check which CPUs are
active as well.
3707a76921
The golang "runtime" package has also bene using GetProcessAffinityMask
since 1.6 beta1:
6410e67a1e
So we don't seem to need the sysinfo.NumCPU from docker/docker.
(Note that this is PR is an effort to get away from dependencies from
docker/docker)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
In https://github.com/kubernetes/kubernetes/pull/88372, we added the
ability to inject errors to the `FakeImageService`. Use this ability to
test the error paths executed by the `kubeGenericRuntimeManager` when
underlying `ImageService` calls fail.
I don't foresee this change having a huge impact, but it should set a
good precedent for test coverage, and should the failure case behavior
become more "interesting" or risky in the future, we already will have
the scaffolding in place with which we can expand the tests.
Unit test for updating container hugepage limit
Add warning message about ignoring case.
Update error handling about hugepage size requirements
Signed-off-by: sewon.oh <sewon.oh@samsung.com>
For Windows, CPU Requests ( Shares, Count and Maximum ) are mutually exclusive, however
Kubernetes sends them all anyway in the pod spec.
When using dockershim this is not an issue, as Docker checks for this specific situation
here: 1bd184a4c2/daemon/daemon_windows.go (L87-L106)
However, when using CRI-Containerd this pods fail to spawn with an error from hcsshim.
This PR intends to filter these values before they are sent to the CRI and not rely on the
runtime for it.
Related to: https://github.com/kubernetes/kubernetes/issues/84804
If Containerd is used on Windows, then we can also mount individual
files into containers (e.g.: termination-log files), which was not
possible with Docker.
Checks if the container runtime is containerd, and if it is, then also
mount the termination-log file.
add host file write for podIPs
update tests
remove import alias
update type check
update type check
remove import alias
update open api spec
add tests
update test
add tests
address review comments
update imports
remove todo and import alias
This starts ephemeral containers prior to init containers so that
ephemeral containers will still be started when init containers fail to
start.
Also improves tests and comments with review suggestions.
If kubelet never gets past sandbox creation (i.e., never attempted to
create containers for a pod), it should retry the sandbox creation on
failure, regardless of the restart policy of the pod.
If the container is not found, do not stop reading the log file
immediately but wait until we reach again EOF.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
it seems fsnotify can miss some read events, blocking the kubelet to
receive more data from the log file.
If we end up waiting for events with fsnotify, force a read from the
log file every second so that are sure to not miss new data for longer
than that.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
if the runtime is configured to rotate the log file, we might end up
watching the old fd where there are no more writes.
When a fsnotify event other than Write is received, reopen the log
file and recreate the watcher.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
1) Add suffix (`seconds` or `total`) to metric name
2) Switch Summary metric to Histogram metric (Summary metrics are not
supported completely by prometheus-to-sd and can't be aggregated.)
The messages for container lifecycle events are subtly inconsistent
and should be unified.
First, the field format for containers is hard to parse for a human,
so include the container name directly in the message for create
and start, and for kill remove the container runtime prefix.
Second, the pulling image event has inconsistent capitalization, fix
that to be sentence without punctuation.
Third, the kill container event was unnecessarily wordy and inconsistent
with the create and start events. Make the following changes:
* Use 'Stopping' instead of 'Killing' since kill is usually reserved for
when we decide to hard stop a container
* Send the event before we dispatch the prestop hook, since this is an
"in-progress" style event vs a "already completed" type event
* Remove the 'cri-o://' / 'docker://' prefix by printing the container
name instead of id (we already do that replacement at the lower level
to prevent high cardinality events)
* Use 'message' instead of 'reason' as the argument name since this is a
string for humans field, not a string for machines field
* Remove the hash values on the container spec changed event because no
human will ever be able to do anything with the hash value
* Use 'Stopping container %s(, explanation)?' form without periods to
follow event conventions
The end result is a more pleasant message for humans:
```
35m Normal Created Pod Created container
35m Normal Started Pod Started container
10m Normal Killing Pod Killing container cri-o://installer:Need to kill Pod
10m Normal Pulling Pod pulling image "registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-02-10-172026@sha256:3da5303d4384d24691721c1cf2333584ba60e8f82c9e782f593623ce8f83ddc5"
```
becomes
```
35m Normal Created Pod Created container installer
35m Normal Started Pod Started container installer
10m Normal Killing Pod Stopping container installer
10m Normal Pulling Pod Pulling image "registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-02-10-172026@sha256:3da5303d4384d24691721c1cf2333584ba60e8f82c9e782f593623ce8f83ddc5"
```
* value names are now purely random
* cleaning up leaked registry keys at Kubelet init
* fixing a small bug masking create errors
Signed-off-by: Jean Rouge <rougej+github@gmail.com>
This patch comprises the kubelet changes outlined in the GMSA KEP
(https://github.com/kubernetes/enhancements/blob/master/keps/sig-windows/20181221-windows-group-managed-service-accounts-for-container-identity.md)
to add GMSA support to Windows workloads.
More precisely, it includes the logic proposed in the KEP to resolve
which GMSA spec should be applied to which containers, and changes
`dockershim` to copy the relevant GMSA credential specs to Windows
registry values prior to creating the container, passing them down
to docker itself, and finally removing the values from the registry
afterwards; both these changes need to be activated with the `WindowsGMSA`
feature gate.
Includes unit tests.
Signed-off-by: Jean Rouge <rougej+github@gmail.com>
We've changed the Ephemeral Containers API, and container type will no
longer be required. Since this is the only feature using it, remove it.
This reverts commit ba6f31a6c6.
- Move from the old github.com/golang/glog to k8s.io/klog
- klog as explicit InitFlags() so we add them as necessary
- we update the other repositories that we vendor that made a similar
change from glog to klog
* github.com/kubernetes/repo-infra
* k8s.io/gengo/
* k8s.io/kube-openapi/
* github.com/google/cadvisor
- Entirely remove all references to glog
- Fix some tests by explicit InitFlags in their init() methods
Change-Id: I92db545ff36fcec83afe98f550c9e630098b3135
This vendor change was purely for the changes in docker to allow for
setting the Masked and Read-only paths.
See: moby/moby#36644
But because of the docker dep update it also needed cadvisor to be
updated and winterm due to changes in pkg/tlsconfig in docker
See: google/cadvisor#1967
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
should not event directly
**What this PR does / why we need it**:
should not event directly, using recordContainerEvent() to generate ref and deduplicate events instead.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
none
```
The requested Service Protocol is checked against the supported protocols of GCE Internal LB. The supported protocols are TCP and UDP.
SCTP is not supported by OpenStack LBaaS. If SCTP is requested in a Service with type=LoadBalancer, the request is rejected. Comment style is also corrected.
SCTP is not allowed for LoadBalancer Service and for HostPort. Kube-proxy can be configured not to start listening on the host port for SCTP: see the new SCTPUserSpaceNode parameter
changed the vendor github.com/nokia/sctp to github.com/ishidawataru/sctp. I.e. from now on we use the upstream version.
netexec.go compilation fixed. Various test cases fixed
SCTP related conformance tests removed. Netexec's pod definition and Dockerfile are updated to expose the new SCTP port(8082)
SCTP related e2e test cases are removed as the e2e test systems do not support SCTP
sctp related firewall config is removed from cluster/gce/util.sh. Variable name sctp_addr is corrected to sctpAddr in pkg/proxy/ipvs/proxier.go
cluster/gce/util.sh is copied from master
Automatic merge from submit-queue (batch tested with PRs 66177, 66185, 67136, 67157, 65065). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: reduce logging for backoff situations
xref https://bugzilla.redhat.com/show_bug.cgi?id=1555057#c6
Pods that are in `ImagePullBackOff` or `CrashLoopBackOff` currently generate a lot of logging at the `glog.Info()` level. This PR moves some of that logging to `V(3)` and avoids logging in situations where the `SyncPod` only fails because pod are in a BackOff error condition.
@derekwaynecarr @liggitt
This allows kubelets to stop the necessary work when the context has
been canceled (e.g., connection closed), and not leaking a goroutine
and inotify watcher waiting indefinitely.
Automatic merge from submit-queue (batch tested with PRs 64688, 64451, 64504, 64506, 56358). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cleanup some dead kubelet code
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Promote sysctl annotations to fields
#
**What this PR does / why we need it**:
Promoting experimental sysctl feature from annotations to API fields.
**Special notes for your reviewer**:
Following sysctl KEP: https://github.com/kubernetes/community/pull/2093
**Release note**:
```release-note
The Sysctls experimental feature has been promoted to beta (enabled by default via the `Sysctls` feature flag). PodSecurityPolicy and Pod objects now have fields for specifying and controlling sysctls. Alpha sysctl annotations will be ignored by 1.11+ kubelets. All alpha sysctl annotations in existing deployments must be converted to API fields to be effective.
```
**TODO**:
* [x] - Promote sysctl annotation in Pod spec
* [x] - Promote sysctl annotation in PodSecuritySpec spec
* [x] - Feature gate the sysctl
* [x] - Promote from alpha to beta
* [x] - docs PR - https://github.com/kubernetes/website/pull/8804
Automatic merge from submit-queue (batch tested with PRs 63252, 63160). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: logs: do not wait when following terminated container
Currently, a `kubectl logs -f` on a terminated container will output the logs, wait 5 seconds (`stateCheckPeriod`), then return. The 5 seconds delay should not occur as the container is terminated and unable to generate additional log messages.
This PR puts a check at the beginning of `waitLogs()` to avoid doing the wait when the container is not running.
@derekwaynecarr @smarterclayton
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Change seccomp annotation from "docker/default" to "runtime/default"
**What this PR does / why we need it**:
This PR changes seccomp annotation from "docker/default" to "runtime/default", so that it is can be applied to all kinds of container runtimes. This PR is a followup of [#1963](https://github.com/kubernetes/community/pull/1963).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#39845
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add ut for kuberuntime-gc
**What this PR does / why we need it**:
Add ut for kuberuntime-gc to cover more situations:
1) Add two uncovered cases to test sandbox-gc
(1) When there are more than one exited sandboxes,the older exited sandboxes without containers for existing pods should be garbage collected;
(2) Even though there are more than one exited sandboxes,the older exited sandboxes with containers for existing pods should not be garbage collected.
2) Add one uncovered case to test container-gc
(1) To cover the situation when allSourcesReady is set false;
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
"NONE"
```
Automatic merge from submit-queue (batch tested with PRs 62455, 62465, 62427, 62416, 62411). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kuberuntime: logs: reduce logging level on waitLogs msg
Lots of occurrences of this msg coming from `waitLogs()`:
```
E0411 13:17:04.589338 7645 logs.go:383] Container "4fbf541ed1900c4670216a6a1ecf752cd07ac430f5547c5497fbc4b78e564b78" is not running (state="CONTAINER_EXITED")
E0411 14:02:18.168502 7645 logs.go:383] Container "dba4c535666d05310889965418592727047320743a233e226e2266b399836150" is not running (state="CONTAINER_EXITED")
E0411 14:02:41.342645 7645 logs.go:383] Container "a946289b36fe3c375c29dce020005424f3b980237892253d42b8bd8bfb595756" is not running (state="CONTAINER_EXITED")
E0411 14:02:49.907317 7645 logs.go:383] Container "e1d6014330e7422c03ae6db501d4fb296a4501355517cb60e2f910f54741361d" is not running (state="CONTAINER_EXITED")
```
Added in https://github.com/kubernetes/kubernetes/pull/55140
This message prints whenever something is watching the log when the container dies.
The comment right after the error msg say "this is normal" and thus should not be logged at Error level.
@derekwaynecarr @feiskyer @Random-Liu
Automatic merge from submit-queue (batch tested with PRs 59027, 62333, 57661, 62086, 61584). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add UT case to cover the func legacyLogSymlink in legacy.go
**What this PR does / why we need it**:
Add UT case to cover the func legacyLogSymlink in legacy.go.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
"NONE"
```
Automatic merge from submit-queue (batch tested with PRs 60759, 60531, 60923, 60851, 58717). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not create dangling legacy symlink
Do not create dangling legacy symlink if the new symlink to container logs does not exist.
These dangling legacy symlink are later removed by kube runtime gc, so it's better if we do not
create them in the first place to avoid unnecessary work from kube runtime gc. This situation occurs when docker uses journald logging driver.
**What this PR does / why we need it**:
This PR fixes an issue where dangling symlink are being created.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None.
```
@derekwaynecarr @sjenning @dashpole @kubernetes/sig-node-pr-reviews
These dangling legacy symlink are removed by kube runtime gc, so it's better if we do not
create them in the first place to avoid unnecessary work from kube runtime gc.
Users must not be allowed to step outside the volume with subPath.
Therefore the final subPath directory must be "locked" somehow
and checked if it's inside volume.
On Windows, we lock the directories. On Linux, we bind-mount the final
subPath into /var/lib/kubelet/pods/<uid>/volume-subpaths/<container name>/<subPathName>,
it can't be changed to symlink user once it's bind-mounted.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: setup WindowsContainerResources for windows containers
**What this PR does / why we need it**:
This PR setups WindowsContainerResources for windows containers. It implements proposal here: https://github.com/kubernetes/community/pull/1510.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56734
**Special notes for your reviewer**:
**Release note**:
```release-note
WindowsContainerResources is set now for windows containers
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add FailedPostStartHook error message.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#54671
**Special notes for your reviewer**:
/cc @derekwaynecarr
cc @lovejoy @OJezu
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59463, 59719, 60181, 58283, 59966). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Set shared PID namespace mode based on PodSpec
**What this PR does / why we need it**: This PR enables pod process namespace sharing as an alpha feature, as described in [Shared PID Namespace Proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/pod-pid-namespace.md).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
WIP #1615
**Special notes for your reviewer**:
/assign @dchen1107
**Release note**:
```release-note
When the `PodShareProcessNamespace` alpha feature is enabled, setting `pod.Spec.ShareProcessNamespace` to `true` will cause a single process namespace to be shared between all containers in a pod.
```
Automatic merge from submit-queue (batch tested with PRs 60208, 60084, 60183, 59713, 60096). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use SeekStart, SeekCurrent, and SeekEnd repalace of deprecated constant
**What this PR does / why we need it**:
Use SeekStart, SeekCurrent, and SeekEnd repalace of deprecated constant.
'''
// Deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.
const (
SEEK_SET int = 0 // seek relative to the origin of the file
SEEK_CUR int = 1 // seek relative to the current offset
SEEK_END int = 2 // seek relative to the end
)
'''
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54191, 59374, 59824, 55032, 59906). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding per container stats for CRI runtimes
**What this PR does / why we need it**
This commit aims to collect per container log stats. The change was proposed as a part of #55905. The change includes change the log path from /var/pod/<pod uid>/containername_attempt.log to /var/pod/<pod uid>/containername/containername_attempt.log. The logs are collected by reusing volume package to collect metrics from the log path.
Fixes#55905
**Special notes for your reviewer:**
cc @Random-Liu
**Release note:**
```
Adding container log stats for CRI runtimes.
```
This commit aims to collect per container log stats. The
change was proposed as a part of #55905. The change includes
change of the log path from /var/pod/<pod uid>/containername_attempt.log
to /var/pod/<pod uid>/containername/containername_attempt.log.
The logs are collected by reusing volume package to collect
metrics from the log path.
Signed-off-by: abhi <abhi@docker.com>
Automatic merge from submit-queue (batch tested with PRs 59010, 59212, 59281, 59014, 59297). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve error returned when fetching container logs during pod termination
**What this PR does / why we need it**:
This change better handles fetching of logs when a container is in a crash loop backoff state. In cases where it is unable to fetch the logs, it gives a helpful error message back to a user who has requested logs of a container from a terminated pod. Rather than attempting to get logs for a container using an empty container ID, it returns a useful error message.
In cases where the container runtime gets an error, log the error but don't leak it back through the API to the user.
**Which issue(s) this PR fixes**:
Fixes#59296
**Release note**:
```release-note
NONE
```
This also incorporates the version string into the package name so
that incompatibile versions will fail to connect.
Arbitrary choices:
- The proto3 package name is runtime.v1alpha2. The proto compiler
normally translates this to a go package of "runtime_v1alpha2", but
I renamed it to "v1alpha2" for consistency with existing packages.
- kubelet/apis/cri is used as "internalapi". I left it alone and put the
public "runtimeapi" in kubelet/apis/cri/runtime.
Automatic merge from submit-queue (batch tested with PRs 58184, 59307, 58172). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add annotations to the device plugin API
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** : Related to #56649 but does not fix it
This adds the ability for the device plugins to annotate containers.
Product wise, this allows the NVIDIA device plugin to support CRI-O (which allows hooks through container annotations).
**Special notes for your reviewer**:
/area hw-accelerators
/cc @vishh @jiayingz @vikaschoudhary16
I'm wondering if it would make sense to fire a blank call to `newContainerAnnotations` at the start of the deviceplugin to get Annotations that are forbidden.
Current behavior is that any Annotations that conflicts with Kubelet will be overwritten by Kubelet.
**Release note**:
```release-note
NONE
```
This is part of the "Debug Containers" feature and is hidden behind
a feature gate. Debug containers have no stored spec, so this new
runtime label allows the kubelet to treat containers differently
without relying on spec.
Automatic merge from submit-queue (batch tested with PRs 58422, 58229, 58421, 58435, 58475). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: imagegc: exempt sandbox image
The image GC logic currently does not consider the sandbox image to be in-use by pods, since it isn't explicitly listed in the pod spec. However, it is trivially in-use if there are any pods running on the node.
This change adds logic to exempt the sandbox image from GC by always considering it as in-use.
**Reviewer Note**
I am changing `(m *kubeGenericRuntimeManager) GetImageRef` to return the ID always rather than the first tag if it exists. Seemed ok to me. Makes some error messages a little less readable in that the ID will be printed and not the tag. Just wanted to see what reviewers think about this.
@derekwaynecarr @dashpole
Automatic merge from submit-queue (batch tested with PRs 53631, 56960). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove unused code in UT files in pkg/
**What this PR does / why we need it**:
Remove unused code in UT files in pkg/ .
**Release note**:
```release-note
NONE
```
Credential provider is useful without the v1 API, move the only
dependency out so that we can more easily move credential provider to a
utility library in the future (other callers besides Kubelet may need to
load pull secrets like Docker).
Automatic merge from submit-queue (batch tested with PRs 57746, 57621, 56839, 57464). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
security_context_test.go(TestVerifyRunAsNonRoot): add more test cases
**What this PR does / why we need it**:
In #56503 we modified `VerifyRunAsNonRoot` function add add one more argument. As [was requested](https://github.com/kubernetes/kubernetes/pull/56503#discussion_r153870821) by @simo5, this change should have a unit test.
This PR adds this test and also some more to cover more execution paths.
**Release note**:
```release-note
NONE
```
PTAL @pweil- @liggitt
CC @simo5
Automatic merge from submit-queue (batch tested with PRs 56579, 55236, 56512, 56549, 56538). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
security_context_test.go(TestVerifyRunAsNonRoot): remove unused variables
**What this PR does / why we need it**:
This PR removed unused member and related variables from the test.
**Special notes for your reviewer**:
It's better to review this PR in the mode that ignore whitespace-related changes: https://github.com/kubernetes/kubernetes/pull/56579/files?w=1
**Release note**:
```release-note
NONE
```
CC @simo5
Automatic merge from submit-queue (batch tested with PRs 56401, 56506, 56551, 56298, 56581). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: include runtime error in event on CreatePodSandbox failure
Include the error from the runtime in the event message when CreatePodSandbox fails. The sandbox creation can fail for many reasons and including the error makes the situation easier to debug.
The event for failed PodSandboxStatus includes this information so there is already precedent for doing this.
xref https://bugzilla.redhat.com/show_bug.cgi?id=1506813
@eparis @derekwaynecarr @dchen1107 @vishh
/release-note-none
/sig node
Automatic merge from submit-queue (batch tested with PRs 55952, 49112, 55450, 56178, 56151). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix the wrong localhost seccomp path of CRI
**What this PR does / why we need it**:
Fix the wrong seccomp path comment.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55359
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix CRI localhost seccomp path in format localhost//profileRoot/profileName.
```
Automatic merge from submit-queue (batch tested with PRs 55938, 56055, 53385, 55796, 55922). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add partial CRI container log support.
For https://github.com/kubernetes/kubernetes/issues/44976.
New CRI log format:
```
TIMESTAMP STREAM TAG CONTENT
2016-10-06T00:17:09.669794202Z stdout P log content 1
2016-10-06T00:17:09.669794203Z stdout P log content 2
```
Although unlikely, if in the future we need more metadata in each line, we could extend TAG into multiple tags splitted by `:`.
@yujuhong @feiskyer @crassirostris @mrunalp @abhi @mikebrow
/cc @kubernetes/sig-node-api-reviews @kubernetes/sig-instrumentation-api-reviews
**Release note**:
```release-note
A new field is added to CRI container log format to support splitting a long log line into multiple lines.
```
Automatic merge from submit-queue (batch tested with PRs 55114, 52976, 54871, 55122, 55140). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make CRI logs parsing to a library
**What this PR does / why we need it**:
Make CRI logs parsing to a library.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55136
**Special notes for your reviewer**:
**Release note**:
```release-note
Add CRI log parsing library at pkg/kubelet/apis/cri/logs
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
pkg/api: extract Scheme/Registry/Codecs into pkg/api/legacyscheme
This serves as
- a preparation for the pkg/api->pkg/apis/core move
- and makes the dependency to the scheme explicit when vizualizing
left depenncies.
The later helps with our our efforts to split up the monolithic repo
into self-contained sub-repos, e.g. for kubectl, controller-manager
and kube-apiserver in the future.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Increases test coverage for kubelet/kuberuntime
What this PR does / why we need it:
Increases test coverage for kubelet/kuberuntime
#46123
Which issue this PR fixes:
#46123
/assign @feiskyer
Automatic merge from submit-queue (batch tested with PRs 54040, 52503). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Get fallback termination msg from docker when using journald log driver
**What this PR does / why we need it**:
When using the legacy docker container runtime and when a container has `terminationMessagePolicy=FallbackToLogsOnError` and when docker is configured with a log driver other than `json-log` (such as `journald`), the kubelet should not try to get the container's log from the json log file (since it's not there) but should instead ask docker for the logs.
**Which issue this PR fixes** fixes#52502
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed log fallback termination messages when using docker with journald log driver
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve codes which checks whether sandbox contains containers
Currently evictSandboxes() checks whether sandbox contains
containers, it traverses all the containers for every sandbox,
but when cluster has many containres, it wastes a lot of time.
It is better to use sets in this case.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52990, 53064, 52686, 52221, 53069). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CRI: Allow configuring stdout/stderr streams for Exec/Attach requests
Add stdout/stderr to exec and attach requests. Also check the request to
ensure it meets the requirements.
**Which issue this PR fixes**: fixes#44448
```release-note
CRI: Add stdout/stderr fields to Exec and Attach requests.
```
When using the legacy docker container runtime and when a container has
terminationMessagePolicy=FallbackToLogsOnError and when docker is
configured with a log driver other than json-log (such as journald),
the kubelet should not try to get the container's log from the
json log file (since it's not there) but should instead ask docker for
the logs.
Automatic merge from submit-queue (batch tested with PRs 50890, 52484, 52542, 52567, 50672). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Do not set message when terminationMessagePath not found
If terminationMessagePath is set to a file that does not exist, we should not log an error message and instead try falling back to logs (based on the user's request).
This also slightly simplifies the terminationMessagePath processing.
Seen in #50499
```release-note
If a container does not create a file at the `terminationMessagePath`, no message should be output about being unable to find the file.
```
Automatic merge from submit-queue (batch tested with PRs 52168, 48939, 51889, 52051, 50396). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Fix typo in kubelet kuberuntime container test
Changes "Expetected" to "Expected"
**What this PR does / why we need it**: Fixes a typo in a test
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51337, 47080, 52646, 52635, 52666). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Fix CRI container/imagefs stats.
`ContainerStats`, `ListContainerStats` and `ImageFsInfo` are returning `not implemented` error now.
This PR fixes it.
@yujuhong @feiskyer @yguo0905
Currently when evictSandboxes() checks whether sandbox contains
containers, it traverses all the containers for every sandbox,
but when cluster has many containres, it wastes a lot of time.
It is better to use sets in this case.
If terminationMessagePath is set to a file that does not exist, we
should not log an error message and instead try falling back to logs
(based on the user's request).
Automatic merge from submit-queue (batch tested with PRs 52442, 52247, 46542, 52363, 51781)
Make CPU manager release CPUs when Pod enters completed phase.
**What this PR does / why we need it**: When CPU manager is enabled, this PR releases allocated CPUs when container is not running and is non-restartable.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#52351
**Special notes for your reviewer**:
This bug is only reproduced for pods with `restartPolicy` = `Never` or `OnFailure`. The following output is from a 4 CPU node. This bug can be reproduced as long >= half the cores are requested.
pod1.yaml:
```
apiVersion: v1
kind: Pod
metadata:
name: test-pod1
spec:
containers:
- image: ubuntu
command: ["/bin/bash"]
args: ["-c", "sleep 5"]
name: test-container1
resources:
requests:
cpu: 2
memory: 100Mi
limits:
cpu: 2
memory: 100Mi
restartPolicy: "Never"
```
pod2.yaml:
```
apiVersion: v1
kind: Pod
metadata:
name: test-pod2
spec:
containers:
- image: ubuntu
command: ["/bin/bash"]
args: ["-c", "sleep 5"]
name: test-container1
resources:
requests:
cpu: 2
memory: 100Mi
limits:
cpu: 2
memory: 100Mi
restartPolicy: "Never"
```
Run a local Kubernetes cluster with CPU manager enabled.
```sh
KUBELET_FLAGS='--feature-gates=CPUManager=true --cpu-manager-policy=static --cpu-manager-reconcile-period=1s --kube-reserved=cpu=500m' ./hack/local-up-cluster.sh
```
_Before:_
Create `test-pod1` using pod1.yaml.
```
./cluster/kubectl.sh create -f pod1.yaml
```
Wait for the pod to complete and wait another 90 seconds (give enough time for GC to kick-in).
Create `test-pod2` using pod2.yaml.
```
./cluster/kubectl.sh create -f pod2.yaml
```
Get all pods in the cluster.
```
./cluster/kubectl.sh get pods -a
NAME READY STATUS RESTARTS AGE
test-pod1 0/1 Completed 0 1m
test-pod2 0/1 not enough cpus available to satisfy request 0 9s
```
_After:_
Create `test-pod1` using pod1.yaml.
```
./cluster/kubectl.sh create -f pod1.yaml
```
Wait for the pod to complete and wait another 90 seconds (give enough time for GC to kick-in).
Create `test-pod2` using pod2.yaml.
```
./cluster/kubectl.sh create -f pod2.yaml
```
Get all pods in the cluster.
```
./cluster/kubectl.sh get pods -a
NAME READY STATUS RESTARTS AGE
test-pod1 0/1 Completed 0 1m
test-pod2 0/1 Completed 0 9s
```
Automatic merge from submit-queue (batch tested with PRs 51186, 50350, 51751, 51645, 51837)
Wait for container cleanup before deletion
We should wait to delete pod API objects until the pod's containers have been cleaned up. See issue: #50268 for background.
This changes the kubelet container gc, which deletes containers belonging to pods considered "deleted".
It adds two conditions under which a pod is considered "deleted", allowing containers to be deleted:
Pods where deletionTimestamp is set, and containers are not running
Pods that are evicted
This PR also changes the function PodResourcesAreReclaimed by making it return false if containers still exist.
The eviction manager will wait for containers of previous evicted pod to be deleted before evicting another pod.
The status manager will wait for containers to be deleted before removing the pod API object.
/assign @vishh
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
Mount propagation in kubelet
Together with #45724 it implements mount propagation as proposed in https://github.com/kubernetes/community/pull/659
There is:
- New alpha annotation that allows user to explicitly set propagation mode for each `VolumeMount` in pod containers (to be replaced with real `VolumeMount.Propagation` field during beta) + validation + tests. "Private" is the default one (= no change to existing pods).
I know about proposal for real API fields for alpha feature in https://docs.google.com/document/d/1wuoSqHkeT51mQQ7dIFhUKrdi3-1wbKrNWeIL4cKb9zU/edit, but it seems it's not implemented yet. It would save me quite lot of code and ugly annotation.
- Updated CRI API to transport chosen propagation to Docker.
- New `kubelet --experimental-mount-propagation` option to enable the previous bullet without modifying types.go (worked around with changing `KubeletDeps`... not nice, but it's better than adding a parameter to `NewMainKubelet` and removing it in the next release...)
```release-note
kubelet has alpha support for mount propagation. It is disabled by default and it is there for testing only. This feature may be redesigned or even removed in a future release.
```
@derekwaynecarr @dchen1107 @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
Bugfix: Use local JSON log buffer in parseDockerJSONLog.
**What this PR does / why we need it**:
The issue described in #47800 is due to a race condition in `ReadLogs`: Because the JSON log buffer (`dockerJSONLog`) is package-scoped, any two goroutines modifying the buffer could race and overwrite the other's changes. In particular, one goroutine could unmarshal a JSON log line into the buffer, then another goroutine could `Reset()` the buffer, and the resulting `Stream` would be empty (`""`). This empty `Stream` is caught in a `case` block and raises an `unexpected stream type` error.
This PR creates a new buffer for each execution of `parseDockerJSONLog`, so each goroutine is guaranteed to have a local instance of the buffer.
**Which issue this PR fixes**: fixes#47800
**Release note**:
```release-note
Fixed an issue (#47800) where `kubectl logs -f` failed with `unexpected stream type ""`.
```
Automatic merge from submit-queue
When faild create pod sandbox record event.
I created pods because of the failure to create a sandbox, but there was no clear message telling me what was the failure, so I wanted to record an event when the sandbox was created.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51054, 51101, 50031, 51296, 51173)
Refactor kuberuntime test case with sets.String
**What this PR does / why we need it**:
change to make got and want use sets.String instead, since that is both safe and more clearly shows the intent.
ref: https://github.com/kubernetes/kubernetes/pull/50554
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/51396
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
