Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)
Add basic install and mount flexvolumes e2e tests
fixes https://github.com/kubernetes/kubernetes/issues/47010
These two tests install a skeleton "dummy" flex driver, attachable and non-attachable respectively, then test that a pod can successfully use the flex driver. They are labeled disruptive because kubelet and controller-manager get restarted as part of the flex install. IMO it's important to keep this install procedure as part of the test to isolate any bugs with the startup plugin probe code.
There is a bit of an ugly dependency on cluster/gce/config-test.sh because --flex-volume-plugin-dir must be set to a dir that's readable from controller-manager container and writable by the flex e2e test. The default path is not writable on GCE masters with read-only root so I picked a location that looks okay.
In the "dummy" drivers I trick kubelet into thinking there is a mount point by doing "mount -t tmpfs none ${MNTPATH} >/dev/null 2>&1", hope that is okay.
I have only tested on GCE and theoretically they may work on AWS but I don't think there is a need to test on multiple cloudproviders.
-->
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)
increase the GC e2e test timeout
Fix https://github.com/kubernetes/kubernetes/issues/50047.
The root cause is #50046. See log analysis in #50047. For now, we just increase the timeout.
Automatic merge from submit-queue
Fix pointer bug in local volume e2e test
**What this PR does / why we need it**:
Fix pointer bug in local volume e2e test
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/50043
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48237, 50084, 50019, 50069, 50090)
Allow for some pods not to get scheduled in CA tests.
This will allow us to ignore long tail node creation or failure
to create some nodes when running scalability tests on kubemark.
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Moves left networking e2e tests to test/e2e/network
**What this PR does / why we need it**:
#48784 forgot to move some networking e2e tests. This PR moves them.
It also move the networking tests from within `test/e2e/common/networking.go` to `test/e2e/network/networking.go`
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Associated PR #48784
Umbrella issue #49161
**Special notes for your reviewer**:
/assign @wojtek-t @bowei
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49916, 50050)
Update images used in the node e2e benchmark tests
Ref: https://github.com/kubernetes/kubernetes/issues/42926
- Update the cosbeta image since the new version contains a 'du' command fix that affects Docker performance.
- Add the coreos and ubuntu image that run Docker 1.12.6 so that we will have more data to compare.
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 50000, 49954, 49943, 50018, 49607)
Add [sig-autoscaling] prefix to autoscaling e2e tests
**What this PR does / why we need it**:
Add [sig-autoscaling] prefix to autoscaling e2e tests
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Umbrella issue #49161
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50000, 49954, 49943, 50018, 49607)
Update the DeleteReplicaSet in rs_util.go to use server side reaper
**What this PR does / why we need it**:
fix#47832
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47832
**Special notes for your reviewer**:
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 50000, 49954, 49943, 50018, 49607)
Add [sig-scalability] prefix to scalability e2e tests
**What this PR does / why we need it**:
Add [sig-scalability] prefix to scalability e2e tests
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Umbrella issue #49161
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add ubuntu to gluster and nfs tests
**What this PR does / why we need it**:
Enable gluster and nfs tests for ubuntu distro
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50039
**Special notes for your reviewer**:
**Release note**:
/release-note-none
/sig storage
Automatic merge from submit-queue (batch tested with PRs 49990, 49997, 44278, 49936, 49891)
Allow mode in e2e-framework to gather metrics only from master
This should enable getting metrics for our 5k-node clusters.
cc @kubernetes/sig-scalability-misc @gmarek
Automatic merge from submit-queue (batch tested with PRs 49992, 48861, 49267, 49356, 49886)
remove unused function
**What this PR does / why we need it**:
remove unused function which is not used months ago.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49871, 49422, 49092, 49858, 48999)
Refactor logging e2e tests, add new checks
I split existing code into smaller files to simplify the review and future changes
Also, there are new tests for stackdriver logging:
- ingesting system logs from all nodes
- ingesting logs in json format
- ingesting logs in glog format
Automatic merge from submit-queue (batch tested with PRs 49898, 49897, 49919, 48860, 49491)
Fix usage a make(struct, len()) followed by append()
A couple of places in the code we allocate with make() but then use
append(), instead of copy() or direct assignment. This results in a
slice with len() zero elements at the front followed by the expected
data. The correct form for such usage is `make(struct, 0, len())`.
I found these by running:
```
$ git grep -EI -A7 'make\([^,]*, len\(' | grep 'append(' -B7 | grep -v vendor
```
And then manually looking through the results. I'm sure something better
could exist.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49898, 49897, 49919, 48860, 49491)
Add basic local volume provisioner e2e tests
**What this PR does / why we need it**:
Adds e2e tests to test local volume provisioner.
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubernetes/issues/48832
**Special notes for your reviewer**:
- bring up local volume provisioner using bootstrapper
- have provisioner create a volume by creating a directory under discovery path.
- check persistent volume is created
- make a claim on the PV, write some data then delete the claim. Verify volume is cleaned up.
**Release note**:
```release-note
```
@ianchakeres @msau42
Automatic merge from submit-queue (batch tested with PRs 49651, 49707, 49662, 47019, 49747)
Add support for `no_new_privs` via AllowPrivilegeEscalation
**What this PR does / why we need it**:
Implements kubernetes/community#639
Fixes#38417
Adds `AllowPrivilegeEscalation` and `DefaultAllowPrivilegeEscalation` to `PodSecurityPolicy`.
Adds `AllowPrivilegeEscalation` to container `SecurityContext`.
Adds the proposed behavior to `kuberuntime`, `dockershim`, and `rkt`. Adds a bunch of unit tests to ensure the desired default behavior and that when `DefaultAllowPrivilegeEscalation` is explicitly set.
Tests pass locally with docker and rkt runtimes. There are also a few integration tests with a `setuid` binary for sanity.
**Release note**:
```release-note
Adds AllowPrivilegeEscalation to control whether a process can gain more privileges than it's parent process
```
Automatic merge from submit-queue (batch tested with PRs 49651, 49707, 49662, 47019, 49747)
improve detectability of deleted pods
**What this PR does / why we need it**:
Adds comment to `waitForPodTerminatedInNamespace` to better explain how it's implemented.
~~It improves pod deletion detection in the e2e framework as follows:~~
~~1. the `waitForPodTerminatedInNamespace` func looks for pod.Status.Phase == _PodFailed_ or _PodSucceeded_ since both values imply that all containers have terminated.~~
~~2. the `waitForPodTerminatedInNamespace` func also ignores the pod's Reason if the passed-in `reason` parm is "". Reason is not really relevant to the pod being deleted or not, but if the caller passes a non-blank `reason` then it will be lower-cased, de-blanked and compared to the pod's Reason (also lower-cased and de-blanked). The idea is to make Reason checking more flexible and to prevent a pod from being considered running when all of its containers have terminated just because of a Reason mis-match.~~
Releated to pr [49597](https://github.com/kubernetes/kubernetes/pull/49597) and issue [49529](https://github.com/kubernetes/kubernetes/issues/49529).
**Release note**:
```release-note
NONE
```
A couple of places in the code we allocate with make() but then use
append(), instead of copy() or direct assignment. This results in a
slice with len() zero elements at the front followed by the expected
data. The correct form for such usage is `make(struct, 0, len())`.
I found these by running:
```
$ git grep -EI -A7 'make\([^,]*, len\(' | grep 'append(' -B7 | grep -v vendor
```
And then manually looking through the results. I'm sure something better
could exist.
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Add ext4 and xfs tests to GCE PD basic mount tests
**What this PR does / why we need it**:
Add ext4 and xfs to basic GCE PD mount tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49511
**Special notes for your reviewer**:
**Release note**:
/release-note-none
/sig storage
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Enable garbage collection of custom resources
Enhance the garbage collector to periodically refresh the resources it monitors (via discovery) to enable custom resource definition GC (addressing #44507 and reverting #47432).
This is a replacement for #46000.
/cc @lavalamp @deads2k @sttts @caesarxuchao
/ref https://github.com/kubernetes/kubernetes/pull/48065
```release-note
The garbage collector now supports custom APIs added via CustomeResourceDefinition or aggregated apiservers. Note that the garbage collector controller refreshes periodically, so there is a latency between when the API is added and when the garbage collector starts to manage it.
```
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Add a support for GKE regional clusters in e2e tests.
**What this PR does / why we need it**:
Add a support for GKE regional clusters in e2e tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Use the core client with version
**What this PR does / why we need it**:
Replace the **deprecated** `clientSet.Core()` with `clientSet.CoreV1()`.
**Which issue this PR fixes**: fixes#49535
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49581, 49652, 49681, 49688, 44655)
Re-enable federated ingress test that was disabled due to a federated service deletion bug.
The details of the bug is described in PR #44626. We believe this bug fixes the flakiness in this test and hence we are re-enabling this test to get some mileage on it. If it turns out to be a problem again we are going to revert this back.
**Release note**:
```release-note
NONE
```
/assign @csbell
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 49581, 49652, 49681, 49688, 44655)
Move the audit e2e test out of the node SIG
It was mistakenly moved to sig-node in https://github.com/kubernetes/kubernetes/pull/48910, but this is an apiserver feature, not a node feature.
/cc @crassirostris
Automatic merge from submit-queue (batch tested with PRs 49581, 49652, 49681, 49688, 44655)
Add sig-testing OWNERS_ALIASES
/sig testing
**What this PR does / why we need it**:
follow the sig-foo-{reviewers,approvers} convention
- rename test-infra-maintainers to sig-testing-approvers
- copy sig-testing-approvers to sig-testing-reviewers
- remove inviduals in test/OWNERS in favor of new aliases
as a result
- rmmh gets test/ approver privileges
- spiffxp gets hack/jenkins/ approver privileges
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49580
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49712, 49694, 49714, 49670, 49717)
Reduce GC e2e test flakiness
Increase GC wait timeout in a flaky e2e test. The test expects a GC
operation to be performed within 30s, while in practice the operation
often takes longer due to a delay between the enqueueing of the owner's
delete operation and the GC's actual processing of that event. Doubling
the time seems to stabilize the test. The test's assumptions can be
revisited, and the processing delay under load can be investigated in
the future.
Extracted from https://github.com/kubernetes/kubernetes/pull/47665 per https://github.com/kubernetes/kubernetes/pull/47665#issuecomment-318219099.
/cc @sttts @caesarxuchao @deads2k @kubernetes/sig-api-machinery-bugs
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45813, 49594, 49443, 49167, 47539)
Add node e2e tests for GKE environment
Ref: https://github.com/kubernetes/kubernetes/issues/46891
This PR adds node e2e tests for validating images used on GKE.
- We pass the `SYSTEM_SPEC_NAME` to the node e2e test process via the flag `--system-spec-name` so that we can skip the environment specific tests using `RunIfSystemSpecNameIs()`.
- Also added `SkipIfContainerRuntimeIs()` as the opposite of `RunIfContainerRuntimeIs()`.
**Release note**:
```
None
```
Enhance the garbage collector to periodically refresh the resources it
monitors (via discovery) to enable custom resource definition GC.
This implementation caches Unstructured structs for any kinds not
covered by a shared informer. The existing meta-only codec only supports
compiled types; an improved codec which supports arbitrary types could
be introduced to optimize caching to store only metadata for all
non-informer types.
Automatic merge from submit-queue (batch tested with PRs 49619, 49598, 47267, 49597, 49638)
Remove default binding of system:node role to system:nodes group
part of https://github.com/kubernetes/features/issues/279
deprecation of this automatic binding announced in 1.7 in https://github.com/kubernetes/kubernetes/pull/46076
```release-note
RBAC: the `system:node` role is no longer automatically granted to the `system:nodes` group in new clusters. It is recommended that nodes be authorized using the `Node` authorization mode instead. Installations that wish to continue giving all members of the `system:nodes` group the `system:node` role (which grants broad read access, including all secrets and configmaps) must create an installation-specific `ClusterRoleBinding`.
```
Automatic merge from submit-queue (batch tested with PRs 49619, 49598, 47267, 49597, 49638)
improve log for pod deletion poll loop
**What this PR does / why we need it**:
It improves some logging related to waiting for a pod to reach a passed-in condition. Specifically, related to issue [49529](https://github.com/kubernetes/kubernetes/issues/49529) where better logging may help to debug the root cause.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49619, 49598, 47267, 49597, 49638)
Flag support in kubectl plugins
Adds support to flags in `kubectl` plugins. Flags are declared in the plugin descriptor and are passed to plugins through env vars, similar to global flags (which already works).
Fixes https://github.com/kubernetes/kubernetes/issues/49122
**Release note**:
```release-note
Added flag support to kubectl plugins
```
PTAL @monopole @kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 47738, 49196, 48907, 48533, 48822)
Move e2e dependent images from kubernetes/kubernetes.github.io repo
**What this PR does / why we need it**:
Move e2e dependent images from kubernetes/kubernetes.github.io repo
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48530
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 49238, 49595, 43494, 47897, 48905)
Add apps/v1beta2.ReplicaSet
~Depends on #48746~ (merged)
~Depends on #49357~ (merged)
xref: #49135
```release-note
Add a new API object apps/v1beta2.ReplicaSet
```
Automatic merge from submit-queue (batch tested with PRs 49665, 49689, 49495, 49146, 48934)
Add Integration tests for Inter-Pod Affinity(AntiAffinity)
Signed-off-by: vikaschoudhary16 <choudharyvikas16@gmail.com>
**What this PR does / why we need it**:
Adds integration tests for inter-pod affinity(anti-affinity)
**Special notes for your reviewer**:
Once after @bsalamat 's #48847 gets merged, changes in this PR will be restructured according to the merged changes.
ref/ #48176#48847
@kubernetes/sig-scheduling-pr-reviews
@davidopp @k82cn
Increase GC wait timeout in a flaky e2e test. The test expects a GC
operation to be performed within 30s, while in practice the operation
often takes longer due to a delay between the enqueueing of the owner's
delete operation and the GC's actual processing of that event. Doubling
the time seems to stabilize the test. The test's assumptions can be
revisited, and the processing delay under load can be investigated in
the future.
Automatic merge from submit-queue (batch tested with PRs 46358, 49408)
[Federation] Updates to enable hpa controllers test in integration and e2e
Enables the apis on api server in both scenario.
Additional logic to enable and run the crud portion of objects in integration, for controllers which implement additional logic in reconcile.
**Special notes for your reviewer**:
This on top of an existing PR https://github.com/kubernetes/kubernetes/pull/45497.
The last 2 commits are reviewable here
@kubernetes/sig-federation-pr-reviews
cc @marun @perotinus
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 46913, 48910, 48858, 47160)
Add e2e test for readOnlyRootFilesystem containers
**What this PR does / why we need it**:
This PR adds node e2e test for readOnlyRootFilesystem containers.
**Which issue this PR fixes**
Part of #44118.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46913, 48910, 48858, 47160)
move sig-node related e2e tests to node subdir
I need help making sure I picked the right ones and/or didn't miss anything.
Potential additions include: `logging_soak.go`, `ssh.go`, `kubelet_perf.go`.
/cc @dchen1107 @vishh @tallclair @yujuhong @Random-Liu @abgworrall @dashpole @yguo0905
Automatic merge from submit-queue (batch tested with PRs 48976, 49474, 40050, 49426, 49430)
Use presence of kubeconfig file to toggle standalone mode
Fixes#40049
```release-note
The deprecated --api-servers flag has been removed. Use --kubeconfig to provide API server connection information instead. The --require-kubeconfig flag is now deprecated. The default kubeconfig path is also deprecated. Both --require-kubeconfig and the default kubeconfig path will be removed in Kubernetes v1.10.0.
```
/cc @kubernetes/sig-cluster-lifecycle-misc @kubernetes/sig-node-misc
Automatic merge from submit-queue (batch tested with PRs 48976, 49474, 40050, 49426, 49430)
Remove duplicated import and wrong alias name of api package
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes#48975
**Special notes for your reviewer**:
/assign @caesarxuchao
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48224, 45431, 45946, 48775, 49396)
Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0
Ref: https://github.com/kubernetes/kubernetes/issues/42926
`cos-dev-61-9759-0-0` contains a fix in Linux utility `du` that would affect the measurement of docker performance in kubelet. I'd like to update the benchmark to use the new image.
**Release note**:
```
None
```
/assign @tallclair
/cc @kewu1992 @abgworrall
follow the sig-foo-{reviewers,approvers} convention
- rename test-infra-maintainers to sig-testing-approvers
- copy sig-testing-approvers to sig-testing-reviewers
- remove inviduals in test/OWNERS in favor of new aliases
as a result
- rmmh gets test/ approver privileges
- spiffxp gets hack/jenkins/ approver privileges
Automatic merge from submit-queue (batch tested with PRs 49286, 49550)
Remove myself from a bunch of places
I am assigned in reviews which I never get to do. I prefer drive-bys whenever I can do them rather than the bot choosing myself in random, ends up being mere spam.
@smarterclayton please approve.
Automatic merge from submit-queue (batch tested with PRs 48846, 49483, 49341)
Add [sig-network] prefix to network e2e tests
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Associated PR #48784
Umbrella issue #49161
**Special notes for your reviewer**:
/assign @@wojtek-t @freehan
/cc @bowei
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Remove flags low-diskspace-threshold-mb and outofdisk-transition-frequency
issue: #48843
This removes two flags replaced by the eviction manager. These have been depreciated for two releases, which I believe correctly follows the kubernetes depreciation guidelines.
```release-note
Remove depreciated flags: --low-diskspace-threshold-mb and --outofdisk-transition-frequency, which are replaced by --eviction-hard
```
cc @mtaufen since I am changing kubelet flags
cc @vishh @derekwaynecarr
/sig node
Automatic merge from submit-queue (batch tested with PRs 49358, 49253)
Remove hostname label condition in SchedulerPredicates
**What this PR does / why we need it**:
```
validates that NodeSelector is respected if matching [Conformance]
validates that required NodeAffinity setting is respected if matching
```
The two tests above make the assumption that the node names are equal to the `kubernetes.io/hostname` labels. Unfortunately, this is not necessarily true all the time. For instance, when using the AWS Cloud Provider + Container Linux:
- The node name is set using the AWS SDK's `ec2.Instance.PrivateDnsName` and has the form `ip-10-0-35-57.ca-central-1.compute.internal` [[1](https://github.com/kubernetes/kubernetes/blob/v1.7.1/pkg/cloudprovider/providers/aws/aws.go#L3343-L3346)] [[2](https://raw.githubusercontent.com/aws/aws-sdk-go/master/service/ec2/api.go)]
- The node's hostname, however, is a simple call to `os.Hostname()`, itself reading `/proc/sys/kernel/hostname`, which contains what the AWS DHCP assigned to the instance, typically the hostname short-form: `ip-10-0-16-137`. [[1](https://github.com/kubernetes/kubernetes/blob/v1.7.1/pkg/util/node/node.go#L43-L54)]
Consequently, we are trying to assign a pod to a node having the following label: `kubernetes.io/hostname=ip-10-0-35-57.ca-central-1.compute.internal` (in addition to the randomly generated label), whereas the actual label on the node is `kubernetes.io/hostname=ip-10-0-35-57`.
Furthermore, this inaccurate `kubernetes.io/hostname=<nodename>` condition is actually useless given we already match over a random label, that was assigned to that node. Later, the test ensures that the scheduled pod was scheduled to the right node by comparing the pod's node name and the node name we expected the pod to be on:
```
framework.ExpectNoError(framework.WaitForPodNotPending(cs, ns, labelPodName))
labelPod, err := cs.Core().Pods(ns).Get(labelPodName, metav1.GetOptions{})
framework.ExpectNoError(err)
Expect(labelPod.Spec.NodeName).To(Equal(nodeName))
```
The `k8s.io/apimachinery/pkg/types/nodename` data structure actually [warns](55bee3ad21/staging/src/k8s.io/apimachinery/pkg/types/nodename.go (L40-L43)) about the fact that the node name might be different than the hostname on AWS.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Added sig-storage labels to upgrade tests and moved them to appropriate directory
**What this PR does / why we need it**: Adding necessary sig identifier for storage upgrade tests.
/release-note-none
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)
Fix bug in command retrying in kubemark
This should fix some of the flakes mentioned in https://github.com/kubernetes/kubernetes/issues/46195.
It's showing that all subsequent retries have failed if the first one failed due to `ret_val` not being reassigned on success.
@bskiba Thanks for noticing :)
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)
StatefulSet: Remove `pod.alpha.kubernetes.io/initialized` annotation.
The `pod.alpha.kubernetes.io/initialized` annotation was originally a tool for validating StatefulSet's ordered Pod creation guarantees during the feature's alpha phase.
If set to "false" on a given Pod, it would interrupt StatefulSet's normal behavior. In v1.5.0, the annotation was deprecated and the default became "true" as part of StatefulSet's graduation to beta.
The annotation is now ignored, meaning it cannot be used to interrupt StatefulSet Pod management.
```release-note
StatefulSet: The deprecated `pod.alpha.kubernetes.io/initialized` annotation for interrupting StatefulSet Pod management is now ignored. If you were setting it to `true` or leaving it unset, no action is required. However, if you were setting it to `false`, be aware that previously-dormant StatefulSets may become active after upgrading.
```
ref #41605
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)
Fix issues for local storage allocatable feature
This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.
This PR fixes issue #47809
Replaces use of --api-servers with --kubeconfig in Kubelet args across
the turnup scripts. In many cases this involves generating a kubeconfig
file for the Kubelet and placing it in the correct location on the node.
Automatic merge from submit-queue
Allow nodes to create evictions for its own pods in NodeRestriction admission controller
**What this PR does / why we need it**: This PR adds support for `pods/eviction` sub-resource to the NodeRestriction admission controller so it allows a node to evict pods bound to itself.
**Which issue this PR fixes**: fixes#48666
**Special notes for your reviewer**: The NodeRestriction already allows nodes to delete pods bound to itself, so allowing nodes to also delete pods via the Eviction API probably makes sense.
```release-note
NodeRestriction allows a node to evict pods bound to itself
```
Automatic merge from submit-queue (batch tested with PRs 49409, 49352, 49266, 48418)
[e2e] Also verify content returned by kube-proxy healthz url
**What this PR does / why we need it**: Enhance kube-proxy url test. This helps to detect the port collision case --- node-problem-detector also serves /healthz to return 200 ok. Verify the content to confirm /healthz is served by kube-proxy.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: From #49263
**Special notes for your reviewer**:
/assign @freehan @nicksardo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49326, 49394, 49346, 49379, 49399)
Update to version gate CRDs to 1.7 and greater
**What this PR does / why we need it**:
Allows e2e's to be tested on earlier version do to version check.
xref: #49313
**Release note**:
```
NONE
```
/cc @kubernetes/sig-api-machinery-bugs @kubernetes/sig-testing-bugs
Automatic merge from submit-queue (batch tested with PRs 49420, 49296, 49299, 49371, 46514)
Refactoring taint functions to reduce sprawl
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45060
**Special notes for your reviewer**:
@gmarek @timothysc @k82cn @jayunit100 - I moved some fn's to helpers and some to utils. LMK, if you are ok with this change.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add a new API version apps/v1beta2
xref: #49135
This PR adds a new API version `apps/v1beta2` which contains a copy (of types, conversions, and defaults) of `apps/v1beta1` StatefulSet, Deployment, and their subresources. Note that `apps/v1beta2` is still WIP and we will make breaking changes to it before releasing 1.8.
Moving core controllers (StatefulSet, Deployment, ReplicaSet, DaemonSet) to `apps/v1beta2` is the first step of moving them to `apps/v1` (GA).
This PR is a starting point for DaemonSet and ReplicaSet to move from `/extensions` to `/apps` and for Deployment and StatefulSet to make some breaking changes (e.g. new defaults and/or remove deprecated fields).
```release-note
Add a new API version apps/v1beta2
```
Automatic merge from submit-queue
Reduce hollow proxy mem/node
As likely expected, kubemark-scale failed to even start with n1-standard-8 nodes. Because 1/3rd of our hollow nodes didn't even get scheduled due to their requests:
```
I0720 17:45:08.139] Found only 3325 ready hollow-nodes while waiting for 5000.
I0720 17:45:20.435] 3326 hollow-nodes are reported as 'Running'
I0720 17:45:20.442] 1675 hollow-nodes are reported as NOT 'Running'
```
If we want to experiment with smaller nodes anyway, then this change is needed. Though we most likely will end up OOM'ing.
Explanation for new value:
We have 62.5 hollow-node / real-node
=> mem available per hollow node = 30GB / 62.5 = 480MB
minus 100MB (kubelet)
minus 20MB (npd)
=> 360MB for proxy should be = 100MB + 5000*(mem/node)
=> 50KB mem/node (with some slight slack)
cc @kubernetes/sig-scalability-misc
Automatic merge from submit-queue
Add an integration test library and some integration tests for scheduler
**What this PR does / why we need it**:
1. Add an integration test library (utils.go) for scheduler testing.
2. Cleaned up some of the tests in scheduler_test.go with the new integration test library.
3. Add priority_test.go with a couple of examples on how to test scheduler priority function in integration tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
ref/ #48176
@kubernetes/sig-scheduling-pr-reviews
@davidopp @k82cn @vikaschoudhary16
Automatic merge from submit-queue (batch tested with PRs 49328, 49285, 49307, 49127, 49163)
Cleanup storage e2e test names
**What this PR does / why we need it**:
Some test names had redundant [sig-storage] tags. Also, some tests still had [Volume] tag. This PR removes those tags.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Release note**:
```release-note
NONE
```
/release-note-none
/sig storage
Automatic merge from submit-queue (batch tested with PRs 49330, 49252, 49262, 49278, 49334)
Add project to pd delete node gcloud command
**What this PR does / why we need it**: Add `--project=` to `gcloud compute instances list` calls from `Pod Disks should be able to detach from a node which was deleted`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: Fixes #https://github.com/kubernetes/kubernetes/issues/49185
**Special notes for your reviewer**:
CC @kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 49330, 49252, 49262, 49278, 49334)
Enable garbage collector e2e tests
These tests are not running in pre-submit: see 753266cb7d/jobs/config.json (L9207)
Automatic merge from submit-queue
remove redundant param in e2e_node/remote
**What this PR does / why we need it**:
* remove redundant param in e2e_node/remote/remote.go
* fix a small typo
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
add e2e tests for the bootstrapsigner and tokencleaner controllers, integration testing for bootstrap token auth
**What this PR does / why we need it**:
Add e2e test for bootstrap signer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
Test Ubuntu image using GKE image spec on master
Ref: https://github.com/kubernetes/kubernetes/issues/46891
This PR changes the files referenced in test-infra for running Ubuntu image tests against GKE system spec on master.
The two properties files are shared by the tests against all k8s branches but the `SYSTEM_SPEC_NAME` is only available on master. This should be fine because the tests in the non master branches will just ignore the unknown env variable.
**Release note**:
```
None
```
/assign @yujuhong
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
Modular extensions for kube scheduler perf testing framework
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45973
**Special notes for your reviewer**:
It is not same as the existing one, the previous one has a single nodeaffinity key with multiple values. This one has multiple keys, values.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49107, 47177, 49234, 49224, 49227)
tighten quota controller interface
While debugging a quota performance problem, I had to chase some references deeper than necessary because the interfaces were overly broad. This tightens them.
```release-note
NONE
```
Automatic merge from submit-queue
Fix too extensive logging in Stackdriver Logging e2e tests
Since logging pod now includes a cache, printing it out makes build-log unusable.
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)
Add cos-dev-61-9733-0-0 to the benchmark tests
Ref: https://github.com/kubernetes/kubernetes/issues/42926
m60 has docker 1.13.1 while m61 has 17.03. This PR adds m61 to the benchmark tests so that we will have more data to compare.
PS: We will support fetching the latest image in an image family in the node e2e tests in the future.
**Release note**:
```
None
```
/assign @yujuhong
/cc @kewu1992 @abgworrall
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)
fixit: break sig-cluster-lifecycle tests into subpackage
this is part of fixit week. ref #49161
@kubernetes/sig-cluster-lifecycle-misc
Automatic merge from submit-queue
Add PriorityClass API object under new "scheduling" API group
**What this PR does / why we need it**: This PR is a part of a series of PRs to add pod priority to Kubernetes. This PR adds a new API group called "scheduling" with a new API object called "PriorityClass". PriorityClass maps the string value of priority to its integer value.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: Given the size of this PR, I will add the admission controller for the PriorityClass in a separate PR.
**Release note**:
```release-note
Add PriorityClass API object under new "scheduling" API group
```
ref/ #47604
ref/ #48646
Automatic merge from submit-queue (batch tested with PRs 47509, 46821, 45319, 49121, 49125)
Tolerate a missing MasterName (for GKE)
When testing, GKE created clusters don't provide a `MasterName`, so don't throw a warning and give up when that happens.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47509, 46821, 45319, 49121, 49125)
volume i/o tests for storage plugins
**What this PR does / why we need it**:
Addresses issues [25268](https://github.com/kubernetes/kubernetes/issues/25268) and [28367](https://github.com/kubernetes/kubernetes/issues/28367), though it may be weak re. the streaming i/o issue. @matchstick
**Special notes for your reviewer**:
This is a new file. Plugins other than NFS, GlusterFS, iSCSI, and Ceph-RBD code will need to be supported in a separate PR.
```release-note
NONE
```
Automatic merge from submit-queue
Bump e2e mounttest image version to 0.8
Reduce the number of image files required for e2e test run
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49143, 49211)
Add more logging to PD node delete test
**What this PR does / why we need it**: Adds more logging to `Pod Disks should be able to detach from a node which was deleted` test which started failing because `gcloud compute instances list` is failing with `exit status 1`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: https://github.com/kubernetes/kubernetes/issues/49185
**Special notes for your reviewer**:
**Release note**: none
Automatic merge from submit-queue (batch tested with PRs 49058, 49072, 49137, 49182, 49045)
*: remove --insecure-allow-any-token option
~Since the authenticator is still used in e2e tests, don't remove
the actual package. Maybe a follow up?~
edit: e2e and integration tests have been switched over to the tokenfile
authenticator instead.
```release-note
The --insecure-allow-any-token flag has been removed from kube-apiserver. Users of the flag should use impersonation headers instead for debugging.
```
closes#49031
cc @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 49116, 49095)
Move pkg/api/v1/ref -> client-go/tools/reference
`pkg/api/v1/ref` is the only remaining package copied from pkg/api/v1 to client-go via staging/copy.sh.
Automatic merge from submit-queue (batch tested with PRs 48043, 48200, 49139, 36238, 49130)
Implement equivalence cache by caching and re-using predicate result
The last part of #30844, I opened a new PR instead of overwrite the old one because we changed some basic assumption by allowing invalidating equivalence cache item by individual predicate.
The idea of this PR is based on discussion in https://github.com/kubernetes/kubernetes/issues/32024
- [x] Pods belong to same controllerRef considered to be equivalent
- [x] ` podFitsOnNode` will use cached predicate result if it's available
- [x] Equivalence cache will be updated when if a fresh new predicate is done
- [x] `factory.go` will invalid specific predicate cache(s) based on the object change
- [x] Since `schedule` and `bind` are async, we need to optimistically invalid affected cache(s) before `bind`
- [x] Fully unit test of affected files
- [x] e2e test to verify cache update/invalid workflow
- [x] performance test results
- [x] Some nits fixes related but expected to result in `needs-rebase` so they are split to: #36060#35968#37512
cc @wojtek-t @davidopp
Automatic merge from submit-queue (batch tested with PRs 49055, 49128, 49132, 49134, 49110)
Remove affinity annotations leftover
**What this PR does / why we need it**:
This is a further cleanup for affinity annotations, following #47869.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
ref: #47869
**Special notes for your reviewer**:
- I remove the commented test cases and just leave TODOs instead. I think converting these untestable test cases for now is not necessary. We can add new test cases in future.
- I remove the e2e test case `validates that embedding the JSON PodAffinity and PodAntiAffinity setting as a string in the annotation value work` because we have a test case `validates that InterPod Affinity and AntiAffinity is respected if matching` to test the same thing.
/cc @aveshagarwal @bsalamat @gyliu513 @k82cn @timothysc
**Release note**:
```release-note
NONE
```
# The first commit's message is:
Modular extensions for kube scheduler perf testing framework
# This is the 2nd commit message:
Modular extensions for kube scheduler perf testing framework
Automatic merge from submit-queue (batch tested with PRs 48914, 48535, 49099, 48935, 48871)
Adopt debian-base as baseimage
**What this PR does / why we need it**:
Based on discussion from - https://github.com/kubernetes/kubernetes/pull/44910/files#r125150263
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49169
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 48481, 48256)
Refactor: pkg/util into sub-pkgs
**What this PR does / why we need it**:
- move code in pkg/util into sub-pkgs
- delete some unused funcs
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#15634
**Special notes for your reviewer**:
This is the final work of #15634. It will close that issue.
/cc @thockin
**Release note**:
```release-note
NONE
```
Add PriorityClass to pkg/registry
Add PriorityClass to pkg/master/master.go
Add PriorityClass to import_know_versions.go
Update linted packages
minor fix
e2e and integration tests have been switched over to the tokenfile
authenticator instead.
```release-note
The --insecure-allow-any-token flag has been removed from kube-apiserver. Users of the flag should use impersonation headers instead for debugging.
```
Automatic merge from submit-queue (batch tested with PRs 46094, 48544, 48807, 49102, 44174)
Shared Informer Run blocks until all goroutines finish
**What this PR does / why we need it**:
Makes Shared Informer Run method block until all goroutines it spawned finish. See #45454.
**Which issue this PR fixes**
Fixes#45454
**Release note**:
```release-note
NONE
```
