Automatic merge from submit-queue (batch tested with PRs 60699, 63780). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
e2e/storage: parameterize container images
**What this PR does / why we need it**:
The CSI integration test for hostpath was hard-coded to use the latest
stable release of the sidecar and hostpath container images. This
makes sense for regression testing of changes made in Kubernetes
itself, but the same test is also useful for testing the "canary"
images on quay.io before tagging them as a new release or for testing
locally produced images. Both is now possible via command line
parameters.
**Which issue(s) this PR fixes**:
Related-to: kubernetes-csi/docs#23
**Special notes for your reviewer**:
The commit message has usage instructions.
```release-note
NONE
```
/sig storage
Automatic merge from submit-queue (batch tested with PRs 63386, 64624, 62297, 64847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Let the garbage collector use json merge patch when SMP is not supported
**What this PR does / why we need it**:
Let garbage collector fallback to use json merge patch when strategic merge patch returns 415. This enables orphan delete on custom resources.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56348
**Special notes for your reviewer**:
This PR is developed based on https://github.com/kubernetes/kubernetes/pull/56595. Ref https://github.com/kubernetes/kubernetes/pull/56606 for more information.
**Release note**:
```release-note
Orphan delete is now supported for custom resources
```
/sig api-machinery
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Promote sysctl annotations to fields
#
**What this PR does / why we need it**:
Promoting experimental sysctl feature from annotations to API fields.
**Special notes for your reviewer**:
Following sysctl KEP: https://github.com/kubernetes/community/pull/2093
**Release note**:
```release-note
The Sysctls experimental feature has been promoted to beta (enabled by default via the `Sysctls` feature flag). PodSecurityPolicy and Pod objects now have fields for specifying and controlling sysctls. Alpha sysctl annotations will be ignored by 1.11+ kubelets. All alpha sysctl annotations in existing deployments must be converted to API fields to be effective.
```
**TODO**:
* [x] - Promote sysctl annotation in Pod spec
* [x] - Promote sysctl annotation in PodSecuritySpec spec
* [x] - Feature gate the sysctl
* [x] - Promote from alpha to beta
* [x] - docs PR - https://github.com/kubernetes/website/pull/8804
Automatic merge from submit-queue (batch tested with PRs 64009, 64780, 64354, 64727, 63650). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Never clean backoff in job controller
**What this PR does / why we need it**:
In https://github.com/kubernetes/kubernetes/pull/60985 I've added a mechanism which allows immediate job status update, unfortunately that broke the backoff logic seriously. I'm sorry for that. I've changed the `immediate` mechanism so that it NEVER cleans the backoff, but for the cases when we want fast status update it uses a zero backoff.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62382
**Special notes for your reviewer**:
/assign @janetkuo
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Volume topology aware dynamic provisioning: work based on new API
**What this PR does / why we need it**:
The PR has been split to 3 parts:
Part1: https://github.com/kubernetes/kubernetes/pull/63232 for basic scheduler and PV controller plumbing
Part2: https://github.com/kubernetes/kubernetes/pull/63233 for API change
and the PR itself includes work based on the API change:
- Dynamic provisioning allowed topologies scheduler work
- Update provisioning interface to be aware of selected node and topology
**Which issue(s) this PR fixes**
Feature: https://github.com/kubernetes/features/issues/561
Design: https://github.com/kubernetes/community/issues/2168
**Special notes for your reviewer**:
/sig storage
/sig scheduling
/assign @msau42 @jsafrane @saad-ali @bsalamat
@kubernetes/sig-storage-pr-reviews
@kubernetes/sig-scheduling-pr-reviews
**Release note**:
```release-note
Volume topology aware dynamic provisioning
```
Automatic merge from submit-queue (batch tested with PRs 63322, 64718, 64708, 64775, 64777). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove event handler to satisfy alpha tests
**What this PR does / why we need it**:
An original assumption of time out did not fix issue. The events look masked by lubelet flags so reducing test
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64578
/cc @msau42
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64344, 64709, 64717, 63631, 58647). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add gpu cluster upgrade test.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
Currently running GPUMasterUpgrade test should pass with gpu nodes but running GPUClusterUpgrade test will run into https://github.com/kubernetes/kubernetes/issues/63506
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Modify e2e tests to use priorityClass beta client version & switch priorityClass to beta
**What this PR does / why we need it**:
/cc @bsalamat @aveshagarwal @liggitt
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #57471
**Special notes for your reviewer**:
Once #57963 merges, this could go in.
**Release note**:
```release-note
Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix error handling in gc e2e test
**What this PR does / why we need it**:
Error messages were not being surfaced in log of GC e2e test, part of fixing #60463
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix test tag on dynamic config tests
The test accidentally got turned off when the NodeAlphaFeature tag was
added in #64125. This PR updates the tag to turn it back on.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bind mount subpath with same read/write settings as underlying volume
**What this PR does / why we need it**:
https://github.com/kubernetes/kubernetes/pull/63045 broke two scenarios:
* If volumeMount path already exists in container image, container runtime will try to chown the volume
* In SELinux system, we will try to set SELinux labels when starting the container
This fix makes it so that the subpath bind mount will inherit the read/write settings of the underlying volume mount. It does this by using the "bind,remount" mount options when doing the bind mount.
The underlying volume mount is ro when the volumeSource.readOnly flag is set. This is for persistent volume types like PVC, GCE PD, NFS, etc. When this is set, we won't try to configure SELinux labels. Also in this mode, subpaths have to already exist in the volume, we cannot make new directories on a read only volume.
When volumeMount.readOnly is set, the container runtime is in charge of making the volume in the container readOnly, but the underlying volume mount on the host can be writable. This can be set for any volume type, and is permanently set for atomic volume types like configmaps, secrets. In this case, SELinux labels will be applied before the container runtime makes the volume readOnly. And subpaths don't have to exist.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64120
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue for readOnly subpath mounts for SELinux systems and when the volume mountPath already existed in the container image.
```
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert "Remove rescheduler and corresponding tests from master"
Reverts kubernetes/kubernetes#64364
After discussing with @bsalamat on how DS controllers(ref: https://github.com/kubernetes/kubernetes/pull/63223#discussion_r192277527) cannot create pods if the cluster is at capacity and they have to rely on rescheduler for making some space, we thought it is better to
- Bring rescheduler back.
- Make rescheduler priority aware.
- If cluster is full and if **only** DS controller is not able to create pods, let rescheduler be run and let it evict some pods which have less priority.
- The DS controller pods will be scheduled now.
So, I am reverting this PR now. Step 2, 3 above are going to be in rescheduler.
/cc @bsalamat @aveshagarwal @k82cn
Please let me know your thoughts on this.
```release-note
Revert #64364 to resurrect rescheduler. More info https://github.com/kubernetes/kubernetes/issues/64725 :)
```
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add Makefile target to push fat manifest for multi-arch images
Signed-off-by: Nitesh Konkar <niteshkonkar@in.ibm.com>
**What this PR does / why we need it**:
This PR adds support to push fat manifest for multi-arch images. We need this
so that we can seamlessly pull and run the test images on multiple platforms.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 64481, 64569). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Instrument envelop transformer.
**What this PR does / why we need it**:
Add metrics for envelope transformer:
transformation_operation_count
transformation_failures_count
envelope_transformation_cache_misses_count
data_key_generation_latencies_microseconds
data_key_generation_failures_count
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Subpath env expansion alpha tests failing
**What this PR does / why we need it**:
The alpha tests which wait for events do not get events back in the wait time
Increased timeout to set to the framework default
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64578
**Special notes for your reviewer**:
@kubernetes/sig-storage-bugs
/cc @msau42
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64057, 63223, 64346, 64562, 64408). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Schedule DaemonSet Pods in scheduler.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
part of #59194
**Special notes for your reviewer**:
**Release note**:
```release-note
`ScheduleDaemonSetPods` is an alpha feature (since v1.11) that causes DaemonSet Pods
to be scheduler by default scheduler, instead of Daemonset controller. When it is enabled,
the `NodeAffinity` term (instead of `.spec.nodeName`) is added to the DaemonSet Pods;
this enables the default scheduler to bind the Pod to the target host. If node affinity
of DaemonSet Pod already exists, it will be replaced.
DaemonSet controller will only perform these operations when creating DaemonSet Pods;
and those operations will only modify the Pods of DaemonSet, no changes are made to the
`.spec.template` of DaemonSet.
```
Automatic merge from submit-queue (batch tested with PRs 63348, 63839, 63143, 64447, 64567). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add block volume support to internal provisioners.
**What this PR does / why we need it**:
Internal provisioners now create filesystem PVs when block PVs are requested. This leads to unbindable PVCs.
In this PR, volume plugins that support block volumes provision block PVs when block is requested. All the other provisioners return clear error in `kubectl describe pvc`:
```
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ProvisioningFailed 7s (x2 over 18s) persistentvolume-controller Failed to provision volume with StorageClass "standard": kubernetes.io/cinder does not support block volume provisioning
```
```release-note
NONE
```
cc @kubernetes/vmware for vsphere changes
cc @andyzhangx for Azure changes
/assign @copejon @mtanino
Automatic merge from submit-queue (batch tested with PRs 63348, 63839, 63143, 64447, 64567). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move pkg/scheduler/schedulercache -> pkg/scheduler/cache
**What this PR does / why we need it**:
Move pkg/scheduler/schedulercache -> pkg/scheduler/cache
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63813
**Special notes for your reviewer**:
In order to prevent name conflicts still rename the `cache` to `schedulercache`.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63445, 63820). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
DNS record scale test
**What this PR does / why we need it**:
Adds e2e scalability test for querying DNS with a scaled up number of records. Specifically, it creates ~~30 services per node~~ 10000 services, then queries the cluster DNS and validates the response. This relates to a graduation criteria listed in kubernetes/community#1956.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
none
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62460, 64480, 63774, 64540, 64337). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Modified regional PD test to fetch template name from GCE
**What this PR does / why we need it**: Previously, the regional PD failover e2e test assumes a specific relationship between the names of an instance group and its corresponding template. It turns out to not always hold true for different types of clusters. Instead, the test should fetch the correct template name by calling out to GCE.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #59988
Need to cherry pick this back to 1.10 along with #64223
**Release note**:
```release-note
NONE
```
/assign @saad-ali @wojtek-t
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove rescheduler and corresponding tests from master
**What this PR does / why we need it**:
This is to remove rescheduler from master branch as we are promoting priority and preemption to beta.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #57471
**Special notes for your reviewer**:
/cc @bsalamat @aveshagarwal
**Release note**:
```release-note
Remove rescheduler from master.
```
Automatic merge from submit-queue (batch tested with PRs 58920, 58327, 60577, 49388, 62306). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Dynamic env in subpath - Fixes Issue 48677
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48677
**Special notes for your reviewer**:
**Release note**:
```release-note
Adds the VolumeSubpathEnvExpansion alpha feature to support environment variable expansion
Sub-paths cannot be mounted with a dynamic volume mount name.
This fix provides environment variable expansion to sub paths
This reduces the need to manage symbolic linking within sidecar init containers to achieve the same goal
```
Automatic merge from submit-queue (batch tested with PRs 63328, 64316, 64444, 64449, 64453). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixing scheduling latency metrics
**What this PR does / why we need it**:
Allows to measure and to display scheduling latency metrics during tests. Provides new functionality of resetting scheduler latency metrics.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63493
**Special notes for your reviewer**:
E2eSchedulingLatency, SchedulingAlgorithmLatency, BindingLatency are now available
as subtypes of OperationLatency.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add metadata to kubelet eviction event annotations
**What this PR does / why we need it**:
Add annotations to kubelet eviction events. Annotations include
"offending_containers" : comma-seperated list of containers.
"offending_containers_usage": comma-seperated list of usage.
"starved_resource": v1.ResourceName of the starved resource
**Special notes for your reviewer**:
Adding annotations to events required changing the `EventRecorder` interface to add a `AnnotatedEventf` function, which can add annotations to an event.
**Release note**:
```release-note
NONE
```
/assign @dchen1107
cc @mwielgus @schylek @kgrygiel
Automatic merge from submit-queue (batch tested with PRs 64456, 64457, 64472). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
e2e node: mark pod cgroup test as [NodeConformance]
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64456, 64457, 64472). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix dynamic kubelet config tests
**What this PR does / why we need it**:
The default for dynamic kubelet config is now true, so if it is unset, assume it is enabled for testing.
**Release note**:
```release-note
NONE
```
/sig node
/kind bug
/priority critical-urgent
/assign @mtaufen
Automatic merge from submit-queue (batch tested with PRs 64392, 63885). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix bug with scheduler throughput variable pass-by-value
Because I keep forgetting go slice semantics :p
/cc @wojtek-t
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move Ceph server secret creation to common code.
The secret should be created only on one place.
**Release note**:
```release-note
NONE
```
@jeffvance @copejon @rootfs @msau42 PTAL
Automatic merge from submit-queue (batch tested with PRs 64399, 64324, 64404, 64406, 64396). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add daemonset when to getReplicasFromRuntimeObject when cleaning objects in e2e
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64359
**Special notes for your reviewer**:
/assign @dims
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64399, 64324, 64404, 64406, 64396). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
DaemonSet internals are still in extensions
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64384
**Special notes for your reviewer**:
/assign @jiayingz @vishh
**Release note**:
```release-note
NONE
```
Putting the command line argument handling into the central test
context seems like the better solution, in particular considering that
argument handling might get changed in the future to use Viper.
Automatic merge from submit-queue (batch tested with PRs 63859, 63979). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Drop reapers
/assign @deads2k @juanvallejo
**Release note**:
```release-note
kubectl delete does not use reapers for removing objects anymore, but relies on server-side GC entirely
```
Automatic merge from submit-queue (batch tested with PRs 63232, 64257, 64183, 64266, 64134). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Measure scheduler throughput in density test
This is a step towards exposing scheduler-related metrics on [perf-dash](http://perf-dash.k8s.io/).
This particular PR adds scheduler throughput computation and makes the results available in our test artifacts.
So if you do some experiments, you'll have some historical baseline data to compare against.
xref https://github.com/kubernetes/kubernetes/issues/63493
fyi - @wojtek-t @davidopp @bsalamat @misterikkit
cc @kubernetes/sig-scheduling-misc @kubernetes/sig-scalability-misc
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64175, 63893). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add colon separators to improve readability of test names
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61963, 64279, 64130, 64125, 64049). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add node-exclusive tags to tests in test/e2e_node
Original issue: #59001
Depends on: #64128
Add node-exclusive tags based on the [proposal](https://docs.google.com/document/d/1BdNVUGtYO6NDx10x_fueRh_DLT-SVdlPC_SsXjYCHOE/edit#)
Follow-up PRs will:
- Tag the tests in `test/e2e/common`
- Change the test job configurations to use the new tests
- Remove the unused, non-node-exclusive tags in `test/e2e_node`
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63417, 64249, 64242, 64128, 64275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
conformance testdata: remove tags and trim spaces
**What this PR does / why we need it**:
To detect whether conformance tests have been changed, the test
argument with the `ConformanceIt` is recorded to conformance.txt.
This change remove additional tags (e.g., [Feature:]) and trim spaces,
so that the detection is less-prone to noise.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64060, 63904, 64218, 64208, 64247). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove knob of equiv class in perf test
**What this PR does / why we need it**:
The ecache knob is controlled by feature gate now:
```go
// createSchedulerConfigurator create a configurator for scheduler with given informer factory and default name.
func createSchedulerConfigurator(
clientSet clientset.Interface,
informerFactory informers.SharedInformerFactory,
) scheduler.Configurator {
// Enable EnableEquivalenceClassCache for all integration tests.
utilfeature.DefaultFeatureGate.Set("EnableEquivalenceClassCache=true")
```
We don't need to pass a flag (and it is not used).
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64174, 64187, 64216, 63265, 64223). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Moving Regional PD e2e tests to regular test suites
**What this PR does / why we need it**: Regional PD e2e tests previously only run on alpha suites. Need to move them to the regular test suites
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #59988
**Special notes for your reviewer**: Will be cherry-picked to 1.10 once merged.
**Release note**:
```release-note
NONE
```
/sig gcp
/sig storage
/assign @msau42 @saad-ali
Automatic merge from submit-queue (batch tested with PRs 64174, 64187, 64216, 63265, 64223). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Extend dns configmap tests to include CoreDNS
**What this PR does / why we need it**:
This PR extends the DNS configmap e2e tests to include testing the CoreDNS ConfigMap.
The tests now test the equivalent `stubdomain`, `federation` and `upstreamnameserver` configuration of kube-dns for CoreDNS, when CoreDNS is the installed DNS server.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62865
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Setup fsGroup for local volumes correctly
**What this PR does / why we need it**:
This pr fixes fsGroup check in local volume in containerized kubelet. Except this, it also fixes fsGroup check when volume source is a normal directory whether kubelet is running on the host or in a container.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61741
**Special notes for your reviewer**:
Bind mounts are detected in `/proc/mounts`, but it does not contain root of mount for bind mounts. So `mount.GetMountRefsByDev()` cannot get all references if source is a normal directory. e.g.
```
# mkdir /tmp/src /mnt/dst
# mount --bind /tmp/src /tmp/src # required by local-volume-provisioner, see https://github.com/kubernetes-incubator/external-storage/pull/499
# mount --bind /tmp/src /mnt/dst
# grep -P 'src|dst' /proc/mounts
tmpfs /tmp/src tmpfs rw,nosuid,nodev,noatime,size=4194304k 0 0
tmpfs /mnt/dst tmpfs rw,nosuid,nodev,noatime,size=4194304k 0 0
# grep -P 'src|dst' /proc/self/mountinfo
234 409 0:42 /src /tmp/src rw,nosuid,nodev,noatime shared:30 - tmpfs tmpfs rw,size=4194304k
235 24 0:42 /src /mnt/dst rw,nosuid,nodev,noatime shared:30 - tmpfs tmpfs rw,size=4194304k
```
We need to compare root of mount and device in this case.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add dynamic config metrics
This PR exports config-releated metrics from the Kubelet.
The Guages for active, assigned, and last-known-good config can be used
to identify config versions and produce aggregate counts across several
nodes. The error-reporting Gauge can be used to determine whether a node
is experiencing a config-related error, and to prodouce an aggregate
count of nodes in an error state.
https://github.com/kubernetes/features/issues/281
```release-note
The Kubelet now exports metrics that report the assigned (node_config_assigned), last-known-good (node_config_last_known_good), and active (node_config_active) config sources, and a metric indicating whether the node is experiencing a config-related error (node_config_error). The config source metrics always report the value 1, and carry the node_config_name, node_config_uid, node_config_resource_version, and node_config_kubelet_key labels, which identify the config version. The error metric reports 1 if there is an error, 0 otherwise.
```
Automatic merge from submit-queue (batch tested with PRs 64013, 63896, 64139, 57527, 62102). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor test utils that deal with Kubelet metrics for clarity
I found these functions hard to understand, because the names did not
accurately reflect their behavior. For example, GetKubeletMetrics
assumed that all of the metrics passed in were measuring latency.
The caller of GetKubeletMetrics was implicitly making this assumption,
but it was not obvious at the call site.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62756, 63862, 61419, 64015, 64063). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Handle TERM signal to reduce pod terminating time.
**What this PR does / why we need it**:
Register signal handler for `TERM`.
For container, process is run as PID 1. PID 1 is special in linux, it ignore any signal with the default action. So process will not terminate on `SIGINT` or `SIGTERM` unless it is coded to do so.
By default, docker use `TERM` signal to termiante pods, it will wait 10 seconds before sending `KILL` signal.
```
$ docker run -d --rm --name test busybox sh -c 'while true; do sleep 1; done'
aa827df5d7bfffc5ca4fae2429d0b761a5a142c57ba3e1faa59b305c92f1c875
$ time docker stop test
test
real 0m10.408s
user 0m0.048s
sys 0m0.020s
```
It's better to register a exit handler for `TERM`, it reduces a lot of time in waiting pods to termiante.
```
$ docker run -d --rm --name test busybox sh -c 'trap exit TERM; while true; do sleep 1; done'
e331bff454dba8e45df6065c3bd2a928e1c41303aafdf88ede38def3e4e5781f
$ time docker stop test
test
real 0m0.690s
user 0m0.048s
sys 0m0.020s
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62756, 63862, 61419, 64015, 64063). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use apps/v1 Deployment/ReplicaSet in controller and kubectl
This updates the Deployment controller and integration/e2e tests to use apps/v1, as part of #55714.
This also requires updating any other components that use the `deployment/util` package, most notably `kubectl`. That means client versions 1.11 and above will only work with server versions 1.9 and above. This is well within our client-server version skew policy of +/-1 minor version.
However, this PR *only* updates the parts of `kubectl` that used `deployment/util`. So although kubectl now requires apps/v1, it still also depends on extensions/v1beta1. Migrating other parts of kubectl to apps/v1 is beyond the scope of this PR, which was just to change the Deployment controller and fix all the fallout.
```release-note
kubectl: This client version requires the `apps/v1` APIs, so it will not work against a cluster version older than v1.9.0. Note that kubectl only guarantees compatibility with clusters that are +/-1 minor version away.
```
To detect whether conformance tests have been changed, the test
argument with the `ConformanceIt` is recorded to conformance.txt.
This change remove additional tags (e.g., [Feature:]) and trim spaces,
so that the detection is less-prone to noise.
This change also updates conformance.txt with normalized names.
Automatic merge from submit-queue (batch tested with PRs 63283, 64032, 64159, 64126, 64098). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix running e2e tests with completed kube-system pods
As per @timothysc's suggestion, this PR is the same as PR #53222, but with the CLA signed and with a comment explaining why completed pods are skipped. Credit for finding this fix goes to @dlosev.
From the original PR:
> Currently e2e runner fails during BeforeSuite execution if there are pods in kube-system namespace in 'Completed' state.
>
> Exactly this issue was fixed previously in #37325 by adding new option "skipSucceeded" to ignore 'Completed' pods. Then, in 2e3cd93#diff-eb7b79470992813ea1905e96c298b47bL557 that option was removed and code was adjusted to ignore 'Completed' pods all the time. But in 0bf96a3 that code was accidentally removed as redundant.
>
> This fix restores removed code.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63283, 64032, 64159, 64126, 64098). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Stub out BackendService check in Ingress upgrade test.
/assign @MrHohn
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 63914, 63887, 64116, 64026, 62933). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix preemption tests that use PDB
**What this PR does / why we need it**:
Scheduler integration tests that test preemption in presence of PDB had an issue causing PDB status not getting updated. This PR fixes the issue.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig scheduling
xref/ #57057
This PR exports config-releated metrics from the Kubelet.
The Guages for active, assigned, and last-known-good config can be used
to identify config versions and produce aggregate counts across several
nodes. The error-reporting Gauge can be used to determine whether a node
is experiencing a config-related error, and to prodouce an aggregate
count of nodes in an error state.
Automatic merge from submit-queue (batch tested with PRs 62025, 63851, 64077, 63967, 63991). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
switch rbac to external
The overall trajectory of the project is towards external types. Having all helpers agree on the version they operate on makes life much easier. We've already written one RBAC controller (role aggregation) and more may follow. `v1` has been around for a while now and we know that any future changes have to reliably roundtrip through it. This pull switches all the core helpers over to use the external types.
@kubernetes/sig-auth-pr-reviews
```release-note
`kubectl auth reconcile` only works with rbac.v1
```
The following tests are not tagged because they are not really run in
node e2e suites:
* A subset of host path tests that required SSH access
- should evaluate whether the tests should be moved to test/e2e_node
* GCP volume tests
- should evaluate whether these tests should be moved out of the
common directly since they are not shared with the node e2e suite.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
correct test logging package stackdrvier -> stackdriver
**What this PR does / why we need it**: corrects the name of the stackdriver logging test package
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63881, 64046, 63409, 63402, 63221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubelet responds to ConfigMap mutations for dynamic Kubelet config
This PR makes dynamic Kubelet config easier to reason about by leaving less room for silent skew scenarios. The new behavior is as follows:
- ConfigMap does not exist: Kubelet reports error status due to missing source
- ConfigMap is created: Kubelet starts using it
- ConfigMap is updated: Kubelet respects the update (but we discourage this pattern, in favor of incrementally migrating to a new ConfigMap)
- ConfigMap is deleted: Kubelet keeps using the config (non-disruptive), but reports error status due to missing source
- ConfigMap is recreated: Kubelet respects any updates (but, again, we discourage this pattern)
This PR also makes a small change to the config checkpoint file tree structure, because ResourceVersion is now taken into account when saving checkpoints. The new structure is as follows:
```
- dir named by --dynamic-config-dir (root for managing dynamic config)
| - meta
| - assigned (encoded kubeletconfig/v1beta1.SerializedNodeConfigSource object, indicating the assigned config)
| - last-known-good (encoded kubeletconfig/v1beta1.SerializedNodeConfigSource object, indicating the last-known-good config)
| - checkpoints
| - uid1 (dir for versions of object identified by uid1)
| - resourceVersion1 (dir for unpacked files from resourceVersion1)
| - ...
| - ...
```
fixes: #61643
```release-note
The dynamic Kubelet config feature will now update config in the event of a ConfigMap mutation, which reduces the chance for silent config skew. Only name, namespace, and kubeletConfigKey may now be set in Node.Spec.ConfigSource.ConfigMap. The least disruptive pattern for config management is still to create a new ConfigMap and incrementally roll out a new Node.Spec.ConfigSource.
```
Automatic merge from submit-queue (batch tested with PRs 63881, 64046, 63409, 63402, 63221). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Disable the public cadvisor port by default
**What this PR does / why we need it**:
Quoting @tallclair in #56523:
> We should add the deprecation warning in 1.10 along with a release note, but not change the default. The notes should warn that the default will change in 1.11. We disable the flag by default in 1.11, and remove it entirely in 1.12 or 1.13.
> If you currently depend on the UI or the API, speak up! Going forward, the recommended way of taking advantage of those features will be to run cAdvisor as a DaemonSet.
Disabling the publicly-available cAdvisor port is beneficial for security, as you might not want to expose the UI with lots of information about what your system is doing. We already did this for all kubeadm deployments in v1.7, and haven't recieved any issues for that. This should be okay to do at this stage, as this flag was deprecated in v1.10. Given we need to support this flag for one more release (v1.11), it makes perfect sense to instead switch it off in preparation for v1.12 when we can delete it (see the [deprecation policy](https://kubernetes.io/docs/reference/deprecation-policy/#deprecating-a-flag-or-cli))
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #56523
**Special notes for your reviewer**:
I removed the e2e test that expects cAdvisor to be running, as we don't expect it to be anymore.
**Release note**:
```release-note
[action required] The formerly publicly-available cAdvisor web UI that the kubelet ran on port 4194 by default is now turned off by default. The flag configuring what port to run this UI on `--cadvisor-port` was deprecated in v1.10. Now the default is `--cadvisor-port=0`, in other words, to not run the web server. The recommended way to run cAdvisor if you still need it, is via a DaemonSet. The `--cadvisor-port` will be removed in v1.12
```
cc @kubernetes/sig-cluster-lifecycle-pr-reviews @kubernetes/sig-auth-pr-reviews @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Only override objects from informer when version has increased.
**What this PR does / why we need it**:
We don't want an informer resync to override assumed volumes if the version has not increased.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63467
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60012, 63692, 63977, 63960, 64008). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use Dial with context
**What this PR does / why we need it**:
`net/http/Transport.Dial` field is deprecated:
```go
// DialContext specifies the dial function for creating unencrypted TCP connections.
// If DialContext is nil (and the deprecated Dial below is also nil),
// then the transport dials using package net.
DialContext func(ctx context.Context, network, addr string) (net.Conn, error)
// Dial specifies the dial function for creating unencrypted TCP connections.
//
// Deprecated: Use DialContext instead, which allows the transport
// to cancel dials as soon as they are no longer needed.
// If both are set, DialContext takes priority.
Dial func(network, addr string) (net.Conn, error)
```
This PR switches all `Dial` usages to `DialContext`. Fixes#63455.
**Special notes for your reviewer**:
Also related: https://github.com/kubernetes/kubernetes/pull/59287https://github.com/kubernetes/kubernetes/pull/58532https://github.com/kubernetes/kubernetes/issues/815https://github.com/kubernetes/community/pull/1166https://github.com/kubernetes/kubernetes/pull/58677https://github.com/kubernetes/kubernetes/pull/57932
**Release note**:
```release-note
HTTP transport now uses `context.Context` to cancel dial operations. k8s.io/client-go/transport/Config struct has been updated to accept a function with a `context.Context` parameter. This is a breaking change if you use this field in your code.
```
/sig api-machinery
/kind enhancement
/cc @sttts
Automatic merge from submit-queue (batch tested with PRs 63969, 63902, 63689, 63973, 63978). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Allow for system metrics discovery in Custom Metrics - Stackdriver Adapter test
**What this PR does / why we need it**:
Allow for system metrics discovery in Custom Metrics - Stackdriver Adapter test
**Release note**:
```release-note
NONE
```
