Automatic merge from submit-queue (batch tested with PRs 66850, 66902, 66779, 66864, 66912). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add methods to apimachinery to easy unit testing
When unit testing, you often want a selective scheme and codec factory. Rather than writing the vars and the init function and the error handling, you can simply do
`scheme, codecs := testing.SchemeForInstallOrDie(install.Install)`
@kubernetes/sig-api-machinery-misc
@sttts
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66850, 66902, 66779, 66864, 66912). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix unit test failure: TestNsenterExistsFile
**What this PR does / why we need it**:
Fix unit test failure: TestNsenterExistsFile
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66809
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```
At present, `log-file` location of glusterfs client mount is decided
by the plugin, however at times, users/admin want to configure log-file
of gluster fuse client process on their own in some setups. Even though they can pass log-file
mount option through storage class, before this patch glusterfs plugin
always discard it. This patch enable/give preference
to admin supplied log-file mount option if specified in storage class.
If the log-file mount option is incomplete or wrong, plugin fallback to the
location which is carved out by the combination of pvc and pod name.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 66877, 66066, 66851). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add namespace info for pod in log
**What this PR does / why we need it**:
add namespace info for pod in log
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix typo: scale dowm-> scale down
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64645, 66832). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Detect if GCE PD udev link is wrong and try to correct it
**What this PR does / why we need it**:
udev can miss scsi events and not update device links properly when disks are detached and reattached to the same node in a different order. This PR workarounds the issue by comparing the udev link name, which is the PD disk name, with the scsi disk serial number returned from scsi_id, and prevents mounting of the disk if the link is wrong. It will also run `udevadm trigger` on the disk to try to correct the bad link.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
This fix prevents a GCE PD volume from being mounted if the udev device link is stale and tries to correct the link.
```
Automatic merge from submit-queue (batch tested with PRs 65730, 66615, 66684, 66519, 66510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: add image-gc low/high validation check
Currently, there is no protection against setting the high watermark <= the low watermark for image GC
This PR adds a validation rule for that.
@smarterclayton
Automatic merge from submit-queue (batch tested with PRs 65730, 66615, 66684, 66519, 66510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update attach to use external objs
**Release note**:
```release-note
NONE
```
Updates attach command to use external versions.
cc @deads2k @soltysh
Automatic merge from submit-queue (batch tested with PRs 65730, 66615, 66684, 66519, 66510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Speed up HPA reaction to metric changes by removing scale up forbidden window
**What this PR does / why we need it**:
Speed up HPA reaction to metric changes by removing scale up forbidden window.
Scale up forbidden window was protecting HPA against making decision to scale up based on metrics gathered during pod initialisation (which may be invalid, for example pod may be using a lot of CPU despite not doing any "actual" work).
To avoid that negative effect only use per pod metrics from pods that are:
- ready (so metrics about them should be valid), or
- unready but creation and last readiness change timestamps are apart more than 10s (pods that have formerly been ready and so metrics are in at least some cases (pod becoming unready because of overload) very useful).
This PR only deals with scale up window. I'll send another PR dealing with scale down window soon.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
Speed up HPA reaction to metric changes by removing scale up forbidden window.
Scale up forbidden window was protecting HPA against making decision to scale up based on metrics gathered during pod initialisation (which may be invalid, for example pod may be using a lot of CPU despite not doing any "actual" work).
To avoid that negative effect only use per pod metrics from pods that are:
- ready (so metrics about them should be valid), or
- unready but creation and last readiness change timestamps are apart more than 10s (pods that have formerly been ready and so metrics are in at least some cases (pod becoming unready because of overload) very useful).
```
Automatic merge from submit-queue (batch tested with PRs 65730, 66615, 66684, 66519, 66510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add DynamicProvisioningScheduling support for EBS
**What this PR does / why we need it**:
This PR adds support for the DynamicProvisioningScheduling feature in EBS. With this in place, if VolumeBindingMode: WaitForFirstConsumer is specified in a EBS storageclass and DynamicProvisioningScheduling is enabled, EBS provisioner will use the selected node's LabelZoneFailureDomain as the zone to provision the EBS volume in.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
Related to #63232
Sample `describe pv` output with NodeAffinity populated:
```
~$ kubectl describe pv pvc-f9d2138b-7e3e-11e8-a4ea-064124617820
Name: pvc-f9d2138b-7e3e-11e8-a4ea-064124617820
Labels: failure-domain.beta.kubernetes.io/region=us-west-2
failure-domain.beta.kubernetes.io/zone=us-west-2a
Annotations: kubernetes.io/createdby=aws-ebs-dynamic-provisioner
pv.kubernetes.io/bound-by-controller=yes
pv.kubernetes.io/provisioned-by=kubernetes.io/aws-ebs
Finalizers: [kubernetes.io/pv-protection]
StorageClass: slow3
Status: Bound
Claim: default/pvc3
Reclaim Policy: Delete
Access Modes: RWO
Capacity: 6Gi
Node Affinity:
Required Terms:
Term 0: failure-domain.beta.kubernetes.io/zone in [us-west-2a]
failure-domain.beta.kubernetes.io/region in [us-west-2]
Message:
Source:
Type: AWSElasticBlockStore (a Persistent Disk resource in AWS)
VolumeID: aws://us-west-2a/vol-0fc1cdae7d10860f6
FSType: ext4
Partition: 0
ReadOnly: false
Events: <none>
```
**Release note**:
```release-note
none
```
/sig storage
/assign @msau42 @jsafrane
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
move feature gate checks inside IsCriticalPod
Currently `IsCriticalPod()` calls throughout the code are protected by `utilfeature.DefaultFeatureGate.Enabled(features.ExperimentalCriticalPodAnnotation)`.
However, with Pod Priority, this gate could be disabled which skips the priority check inside IsCriticalPod().
This PR moves the feature gate checking inside `IsCriticalPod()` and handles both situations properly.
@aveshagarwal @ravisantoshgudimetla @derekwaynecarr
/sig node
/sig scheduling
/king bug
Instead discard metric values for pods that are unready and have never
been ready (they may report misleading values, the original reason for
introducing scale up forbidden window).
Use per pod metric when pod is:
- Ready, or
- Not ready but creation timestamp and last readiness change are more
than 10s apart.
In the latter case we asume the pod was ready but later became unready.
We want to use metrics for such pods because sometimes such pods are
unready because they were getting too much load.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update exit code to 0 if patch not needed
**Release note**:
```release-note
The `kubectl patch` command no longer exits with exit code 1 when a redundant patch results in a no-op
```
The specific logic in the `patch` command that exited with code 1, was only doing so when there was no diff between an existing object and its patched counterpart. (In case of errors, we just return those, which eventually ends up exiting with code 1 anyway). This patch removes this block, as we should not be treating patch no-ops as errors.
Fixes https://github.com/kubernetes/kubernetes/issues/58212
cc @soltysh
Automatic merge from submit-queue (batch tested with PRs 65570, 65616). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Retry scheduling on StorageClass events
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56163
**Special notes for your reviewer**:
I have taken over #60006.
It's hard to test in e2e, because we cannot know reschedule of pod is triggered by which event (periodically service/node events will move pods to active queue too). ~~I'll add integration tests for this functionality after [this PR](https://github.com/kubernetes/kubernetes/pull/65296) get merged.~~ (already added)
**Release note**:
```release-note
NONE
```
Make annotations with newlines display a more consistent left edge, and indent the value
when the annotation is too long to give the value more space. Shorten the width of the
trimmed annotation to a value more consistent with our `-o wide` value.
Instead of putting the key and value flush with a `=` separator, make annotations closer
to fields than to labels by using `: ` as a separator.
Inline scripts may use newlines in these fields, and properly indenting makes the output more readable:
```
Command:
/bin/bash
-c
#!/bin/bash
echo "inline script should be indented"
```
Automatic merge from submit-queue (batch tested with PRs 66445, 66643, 60551). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve the output of `kubectl get events`
Events have long shown the most data of the core objects in their output, but that data is of varying use to a user. Following the principle that events are intended for the system to communicate information back to the user, and that Message is the primary human readable field, this commit alters the default columns to ensure event is shown with the most width given to the message, and all other fields organized by their relevance to the message.
1. Events are no longer sorted in the printer (this was a bug and was broken with paging and server side rendering)
2. Only the last seen, type, reason, kind, and message fields are shown by default, which makes the message prominent
3. Source, subobject, count, and first seen are only shown under `-o wide`
4. The duration fields were changed to be the more precise output introduced for job duration (2-3 sig figs)
5. Prioritized the column order for scanning - when, how important, what kind of error, what kind of object, and the message.
6. Trim trailing newlines on the message.
```release-note
Improved the output of `kubectl get events` to prioritize showing the message, and move some fields to `-o wide`.
```
```
$ kubectl get events --sort-by lastTimestamp
LAST SEEN TYPE REASON KIND MESSAGE
16m Normal SawCompletedJob CronJob Saw completed job: image-mirror-origin-v3.11-quay-1532581200
16m Normal SuccessfulDelete CronJob Deleted job image-mirror-origin-v3.11-quay-1532577600
14m Normal Scheduled Pod Successfully assigned 50c42204-9091-11e8-b2a1-0a58ac101869 to origin-ci-ig-n-fqfh
14m Normal Pulling Pod pulling image "docker-registry.default.svc:5000/ci/commenter:latest"
14m Normal Created Pod Created container
14m Normal Pulled Pod Successfully pulled image "docker-registry.default.svc:5000/ci/commenter:latest"
14m Normal Started Pod Started container
14m Normal SandboxChanged Pod Pod sandbox changed, it will be killed and re-created.
4m14s Normal ScaleDown Pod deleting pod for node scale down
4m14s Normal ScaleDown Pod deleting pod for node scale down
4m14s Normal ScaleDown Pod deleting pod for node scale down
4m14s Normal ScaleDown Pod deleting pod for node scale down
4m14s Normal ScaleDown Pod deleting pod for node scale down
4m14s Normal ScaleDown Pod deleting pod for node scale down
4m14s Normal ScaleDown Pod deleting pod for node scale down
4m13s Normal SuccessfulCreate ReplicationController Created pod: tide-30-hmncf
4m13s Normal Scheduled Pod Successfully assigned tide-30-hmncf to origin-ci-ig-n-x64l
4m12s Normal SuccessfulCreate ReplicationController Created pod: console-jenkins-operator-16-dd5k8
4m12s Normal SuccessfulCreate ReplicationController Created pod: sinker-23-scfmt
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
delete unused events
**What this PR does / why we need it**:
events (HostNetworkNotSupported, UndefinedShaper) is unused since #47058
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Default extensions/v1beta1 Deployment's ProgressDeadlineSeconds to MaxInt32
**What this PR does / why we need it**: Default values should be set in all API versions, because defaulting happens whenever a serialized version is read. When we switched to `apps/v1` as the storage version in `1.10` (#58854), `extensions/v1beta1` `DeploymentSpec.ProgressDeadlineSeconds` gets `apps/v1` default value (`600`) instead of being unset.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66135
**Special notes for your reviewer**: We need to cherrypick this fix to 1.10 and 1.11. Note that this fix will only help people who haven't upgraded to 1.10 or 1.11 when the storage version is changed.
@kubernetes/sig-apps-bugs
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66623, 66718). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cpumanager: validate topology in static policy
**What this PR does / why we need it**:
This patch adds a check for the static policy state validation. The check fails if the CPU topology obtained from cadvisor doesn't match with the current topology in the state file.
If the CPU topology has changed in a node, cpumanager static policy might try to assign non-present cores to containers.
For example in my test case, static policy had the default CPU set of `0-1,4-7`. Then kubelet was shut down and CPU 7 was offlined. After restarting the kubelet, CPU manager tries to assign the non-existent CPU 7 to containers which don't have exclusive allocations assigned to them:
Error response from daemon: Requested CPUs are not available - requested 0-1,4-7, available: 0-6)
This breaks the exclusivity, since the CPUs from the shared pool don't get assigned to non-exclusive containers, meaning that they can execute on the exclusive CPUs.
**Release note**:
```release-note
Added CPU Manager state validation in case of changed CPU topology.
```
Automatic merge from submit-queue (batch tested with PRs 66623, 66718). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
expose GC graph via debug handler
Many times when debugging GC problems, it's important to understand the state of the GC graph at a given point in time. This pull adds the ability to dump that graph in DOT format for later consumption. It does this by exposing an additional debug handler and allowing any controller init function to produce such a handler that is included under debug.
Sample full output
```
curl http://localhost:10252/debug/controllers/garbagecollector/graph
digraph full {
// Node definitions.
0 [
label="uid=8581a030-9043-11e8-ad4a-54e1ad486dd3
namespace=kube-system
Pod.v1/kube-dns-7b479ccbc6-qz468
"
group=""
version="v1"
kind="Pod"
namespace="kube-system"
name="kube-dns-7b479ccbc6-qz468"
uid="8581a030-9043-11e8-ad4a-54e1ad486dd3"
missing="false"
beingDeleted="false"
deletingDependents="false"
virtual="false"
];
1 [
label="uid=822052fc-9043-11e8-ad4a-54e1ad486dd3
namespace=kube-system
Deployment.v1.apps/kube-dns
"
group="apps"
version="v1"
kind="Deployment"
namespace="kube-system"
name="kube-dns"
uid="822052fc-9043-11e8-ad4a-54e1ad486dd3"
missing="false"
beingDeleted="false"
deletingDependents="false"
virtual="false"
];
2 [
label="uid=857bd8ac-9043-11e8-ad4a-54e1ad486dd3
namespace=kube-system
ReplicaSet.v1.apps/kube-dns-7b479ccbc6
"
group="apps"
version="v1"
kind="ReplicaSet"
namespace="kube-system"
name="kube-dns-7b479ccbc6"
uid="857bd8ac-9043-11e8-ad4a-54e1ad486dd3"
missing="false"
beingDeleted="false"
deletingDependents="false"
virtual="false"
];
// Edge definitions.
0 -> 2;
2 -> 1;
}
```
You can also select via UID and have all transitive dependencies output:
```
curl http://localhost:10252/debug/controllers/garbagecollector/graph?uid=8581a030-9043-11e8-ad4a-54e1ad486dd3
digraph full {
// Node definitions.
0 [
label="uid=822052fc-9043-11e8-ad4a-54e1ad486dd3
namespace=kube-system
Deployment.v1.apps/kube-dns
"
group="apps"
version="v1"
kind="Deployment"
namespace="kube-system"
name="kube-dns"
uid="822052fc-9043-11e8-ad4a-54e1ad486dd3"
missing="false"
beingDeleted="false"
deletingDependents="false"
virtual="false"
];
1 [
label="uid=8581a030-9043-11e8-ad4a-54e1ad486dd3
namespace=kube-system
Pod.v1/kube-dns-7b479ccbc6-qz468
"
group=""
version="v1"
kind="Pod"
namespace="kube-system"
name="kube-dns-7b479ccbc6-qz468"
uid="8581a030-9043-11e8-ad4a-54e1ad486dd3"
missing="false"
beingDeleted="false"
deletingDependents="false"
virtual="false"
];
2 [
label="uid=857bd8ac-9043-11e8-ad4a-54e1ad486dd3
namespace=kube-system
ReplicaSet.v1.apps/kube-dns-7b479ccbc6
"
group="apps"
version="v1"
kind="ReplicaSet"
namespace="kube-system"
name="kube-dns-7b479ccbc6"
uid="857bd8ac-9043-11e8-ad4a-54e1ad486dd3"
missing="false"
beingDeleted="false"
deletingDependents="false"
virtual="false"
];
// Edge definitions.
1 -> 2;
2 -> 0;
}
```
And with some sample rendering:
```
curl http://localhost:10252/debug/controllers/garbagecollector/graph | dot -T svg -o project.svg
```
produces
@kubernetes/sig-api-machinery-pr-reviews
/assign @caesarxuchao @liggitt
```release-note
NONE
```
