LocalVolumeType tmpfs added
Added checks to ensure tha volume created during setup contains expected testFileContent
Refactored tests out to avoid code duplication
Two different tests are performed with tmpfs:
-serial write and read in two different pods
-write and read in two different pods mounted at the same time
Fixed local storage test failures by integrating https://github.com/kubernetes/kubernetes/pull/50177
Switched NodeExec to SSH
Automatic merge from submit-queue (batch tested with PRs 49904, 50484, 50214)
Adding support for internal IP for e2e tests
Currently IssueSSHComand in util.go only checks for External IP address
to ssh, this PR adds check for internal IP too.
Closes#50630
Automatic merge from submit-queue (batch tested with PRs 50094, 48966, 49478, 50593, 49140)
Migrate sig-auth e2e tests.
**What this PR does / why we need it:** This PR adds [sig-auth] prefix to
workload e2e tests in accord to requirements of adding a SIG dashboard
to testgrid. Refer PR #48781 for guidelines.
**Release note**:
```release-note
```
**What this PR does / why we need it:** This PR adds [sig-auth] prefix to
workload e2e tests in accord to requirements of adding a SIG dashboard
to testgrid. Refer PR #48781 for guidelines.
Automatic merge from submit-queue
Moved node condition filter into a predicates.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50360
**Release note**:
```release-note
A new predicates, named 'CheckNodeCondition', was added to replace node condition filter. 'NetworkUnavailable', 'OutOfDisk' and 'NotReady' maybe reported as a reason when failed to schedule pods.
```
Automatic merge from submit-queue (batch tested with PRs 49847, 49743, 49853, 50225, 50479)
Add node benchmark tests for cos-m60 with docker 1.12.6
Ref: https://github.com/kubernetes/kubernetes/issues/42926
This PR adds a benchmark tests against cos-m60 with docker 1.12.6 on http://node-perf-dash.k8s.io. This test is useful for docker validation -- we can compare the performance of different dockers on the same OS.
cos-m60 comes with docker 1.13.1 by default, so we need to use cloud-init to downgrade the version to 1.12.6.
**Release note**:
```
None
```
/assign @dchen1107
Automatic merge from submit-queue (batch tested with PRs 50485, 49951, 50508, 50511, 50506)
Pass config to external Kubemark cluster in e2e tests
When cluster autoscaler is used in kubemark tests,
pass default kubeconfig as external cluster config.
@shyamjvs @gmarek
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50485, 49951, 50508, 50511, 50506)
fix a typo
**What this PR does / why we need it**:
fix a small typo
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
verions->versions
**Special notes for your reviewer**:
**Release note**:
NONE
```release-note
```NONE
Automatic merge from submit-queue (batch tested with PRs 50485, 49951, 50508, 50511, 50506)
Multiarch nonewprivs test image
**What this PR does / why we need it**:
This PR is for converting nonewprivs image which pushed very recently part of https://github.com/kubernetes/kubernetes/pull/47019.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#50498
**Special notes for your reviewer**:
**Release note**:
```NONE```
Automatic merge from submit-queue (batch tested with PRs 50537, 49699, 50160, 49025, 50205)
When not using a CloudProvider, set both InternalIP and ExternalIP on Nodes
#36095 changed all of the cloudproviders to set both InternalIP and ExternalIP on Nodes, but the non-cloudprovider fallback code now only sets InternalIP.
This causes the test "should be able to create a functioning NodePort service" in test/e2e/service.go to fail on cloud-provider-less clusters, because (with LegacyHostIP gone), it now will only try to work with ExternalIPs, and will fail if the node has only an InternalIP.
There isn't much other code that assumes that ExternalIP will always be set (there's something in pkg/master/master.go, but I don't know what it's doing, so maybe it's only useful in the case where InternalIP != ExternalIP anyway). But given that several of the cloudproviders (mesos, ovirt, rackspace) now explicitly set both InternalIP and ExternalIP to the same value always, it seemed right to do that in the fallback case too.
@deads2k FYI
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
code format in master_utils.go
**What this PR does / why we need it**:
code format
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #N/A
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
move logs to kubectl/util
Move `pkg/util/logs` to `pkg/kubectl/util/logs` per https://github.com/kubernetes/kubernetes/issues/48209#issuecomment-311730681
This will make kubeadm, kubefed, gke-certificates-controller and e2e have dependency on kubectl, which should be fine.
partially addresses: kubernetes/community#598
```release-note
NONE
```
/assign @apelisse @monopole
Automatic merge from submit-queue
Remove deprecated ESIPP beta annotations
**What this PR does / why we need it**:
Remove deprecated ESIPP beta annotations.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50187
**Special notes for your reviewer**:
/assign @MrHohn
/sig network
**Release note**:
```release-note
Beta annotations `service.beta.kubernetes.io/external-traffic` and `service.beta.kubernetes.io/healthcheck-nodeport` have been removed. Please use fields `service.spec.externalTrafficPolicy` and `service.spec.healthCheckNodePort` instead.
```
Automatic merge from submit-queue
Migrate to controller references helpers in meta/v1
**What this PR does / why we need it**:
This is a follow up for #48319 that migrates all method usages to new methods in meta/v1.
**Special notes for your reviewer**:
Looking at each commit individually might be easier.
**Release note**:
```release-note
NONE
```
/sig api-machinery
/kind cleanup
Automatic merge from submit-queue
Add Cluster Autoscaler scalability test suite
This suite is intended for manually testing Cluster Autoscaler on large clusters. It isn't supposed to be run automatically (at least for now).
It can be run on Kubemark (with #50440) with the following setup:
- start Kubemark with NUM_NODES=1 (as we require there to be exactly 1 replica per hollow-node replication controller in this setup)
- set kubemark-master machine type manually to appropriate type for the Kubemark cluster size. Maximum Kubemark cluster size reached in test run is defined by maxNodes constant, so for maxNodes=1000, please upgrade to n1-standard-32. Adjust if modifying maxNodes.
- start Cluster Autoscaler pod in the external cluster using image built from version with Kubemark cloud provider (release pending)
- for grabbing metrics from ClusterAutoscaler (with #50382), add "--include-cluster-autoscaler=true" parameter in addition to regular flags for gathering components' metrics/resource usage during e2e tests
cc @bskiba
Automatic merge from submit-queue (batch tested with PRs 45186, 50440)
Add functionality needed by Cluster Autoscaler to Kubemark Provider.
Make adding nodes asynchronous. Add method for getting target
size of node group. Add method for getting node group for node.
Factor out some common code.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45186, 50440)
Retry fed-svc creation on diff NodePort during e2e tests
**What this PR does / why we need it**:
Currently in federated end2end tests, the creation of services are
done with a randomize NodePort selection take is causing e2e test
flakes if the creation of a federated service failed if the port is
not available.
Now the util.CreateService(...) function is retrying to create the
service on different nodePort in case of error. The method retry until
success or all possible NodePorts have been tested and also failed.
**Which issue this PR fixes**
fixes#44018
Make adding nodes asynchronous. Add method for getting target
size of node group. Add method for getting node group for node.
Factor out some common code.
Automatic merge from submit-queue (batch tested with PRs 50386, 50374, 50444, 50382)
Add grabbing Cluster Autoscaler metrics in e2e tests
This adds:
- collecting metrics from Cluster Autoscaler before & after e2e test run
- --include-cluster-autoscaler opt-in flag
- passing external cluster client to MetricsGrabber (required for Kubemark setup, as Cluster Autoscaler doesn't run on master in this case)
Most types now have valid rest mappings because
NewDefaultRESTMapperFromScheme no longer ignores certain import
paths. Thus we can no longer use the lack of a valid REST mapping
as an indicator for when to use kindWhiteList. Thus kindWhiteList
now serves as a whitelist for all kinds and not just those that
formally had no mapping. This does mean that we could whitelist
kinds due to a name conflict, but that is unlikely as names such as
GetOptions are not appropriate for new objects.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 49725, 50367, 50391, 48857, 50181)
Add e2e test for privileged containers
**What this PR does / why we need it**:
This PR adds node e2e test for privileged containers.
**Which issue this PR fixes**
Part of #44118.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @Random-Liu
Automatic merge from submit-queue (batch tested with PRs 49642, 50335, 50390, 49283, 46582)
Improve GC discovery sync performance
Improve GC discovery sync performance by only syncing when discovered
resource diffs are detected. Before, the GC worker pool was shut down
and monitors resynced unconditionally every sync period, leading to
significant processing delays causing test flakes where otherwise
reasonable GC timeouts were being exceeded.
Related to https://github.com/kubernetes/kubernetes/issues/49966.
/cc @kubernetes/sig-api-machinery-bugs
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49642, 50335, 50390, 49283, 46582)
Add rbac.authorization.k8s.io/v1
xref https://github.com/kubernetes/features/issues/2
Promotes the rbac.authorization.k8s.io/v1beta1 API to v1 with no changes
```release-note
The `rbac.authorization.k8s.io/v1beta1` API has been promoted to `rbac.authorization.k8s.io/v1` with no changes.
The `rbac.authorization.k8s.io/v1alpha1` version is deprecated and will be removed in a future release.
```
Automatic merge from submit-queue (batch tested with PRs 50300, 50328, 50368, 50370, 50372)
Reduce hollow-kubelet cpu request
Fixes https://github.com/kubernetes/kubernetes/issues/50366
This should make kubemark-500 fit in 6 nodes again. Checked that it should be enough.
cc @kubernetes/sig-scalability-misc
Automatic merge from submit-queue (batch tested with PRs 50418, 49830, 49206, 49061, 49912)
add LocalZone into gce.conf and refactor gce cloud provider configura…
The main goal of this PR is to make gce cloud provider able to run locally.
1. added a LocalZone parameter into gce.conf.
2. refactor `newGCECloud` to avoid contacting metadata server if configuration is already available.
```release-note
None
```
Automatic merge from submit-queue
remove apps/v1beta2 defaulting codes for obj.Spec.Selector and obj.Labels
**What this PR does / why we need it**:
This PR removes defaulting codes for `obj.Spec.Selector`. Currently, `obj.Spec.Selector.MatchLabels` is set to `obj.Spec.Template.Labels` if `obj.Spec.Template.Labels != nil && obj.Spec.Selector == nil`. We should not perform this defaulting operation as controllers selectors are immutable.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50339
**Special notes for your reviewer**:
This PR removes defaulting codes for `apps/v1beta2` only. The defaulting codes for validation will be removed in another PR.
**Release note**:
```NONE
```
Automatic merge from submit-queue
VSphere cloud provider code refactoring
The current PR tracks the vSphere Cloud Provider code refactoring which includes the following changes.
- VCLib Package - A framework used by vSphere cloud provider for managing the vSphere entities. VCLib package mainly does the following:
- Volume management on datastore (Create/Delete)
- Volume management on Virtual Machines (Attach/Detach)
- Storage Policy Management
- vSphere Cloud Provider changes to implement the cloud provider interfaces by calling into VCLib package.
- Modifications to e2e tests to accomodate the latest design changes.
@divyenpatel @rohitjogvmw @luomiao
```release-note
vSphere cloud provider: vSphere cloud provider code refactoring
```
Automatic merge from submit-queue (batch tested with PRs 50016, 49583, 49930, 46254, 50337)
Alpha Dynamic Kubelet Configuration
Feature: https://github.com/kubernetes/features/issues/281
This proposal contains the alpha implementation of the Dynamic Kubelet Configuration feature proposed in ~#29459~ [community/contributors/design-proposals/dynamic-kubelet-configuration.md](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md).
Please note:
- ~The proposal doc is not yet up to date with this implementation, there are some subtle differences and some more significant ones. I will update the proposal doc to match by tomorrow afternoon.~
- ~This obviously needs more tests. I plan to write several O(soon). Since it's alpha and feature-gated, I'm decoupling this review from the review of the tests.~ I've beefed up the unit tests, though there is still plenty of testing to be done.
- ~I'm temporarily holding off on updating the generated docs, api specs, etc, for the sake of my reviewers 😄~ these files now live in a separate commit; the first commit is the one to review.
/cc @dchen1107 @vishh @bgrant0607 @thockin @derekwaynecarr
```release-note
Adds (alpha feature) the ability to dynamically configure Kubelets by enabling the DynamicKubeletConfig feature gate, posting a ConfigMap to the API server, and setting the spec.configSource field on Node objects. See the proposal at https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md for details.
```
Automatic merge from submit-queue (batch tested with PRs 50016, 49583, 49930, 46254, 50337)
Remove scheduledjobs
This is a prerequisite for promoting CronJobs to beta.
**Release note**:
```release-note
Remove deprecated ScheduledJobs endpoints, use CronJobs instead.
```
Automatic merge from submit-queue (batch tested with PRs 50016, 49583, 49930, 46254, 50337)
[Federation] Make the hpa scale time window configurable
This PR is on top of open pr https://github.com/kubernetes/kubernetes/pull/45993.
Please review only the last commit in this PR.
This adds a config param to controller manager, the value of which gets passed to hpa adapter via sync controller.
This is needed to reduce the overall time limit of the hpa scaling window to much lesser (then the default 2 mins) to get e2e tests run faster. Please see the comment on the newly added parameter.
**Special notes for your reviewer**:
@kubernetes/sig-federation-pr-reviews
@quinton-hoole
@marun to please validate the mechanism used to pass a parameter from cmd line to adapter.
**Release note**:
```
federation-controller-manager gets a new flag --hpa-scale-forbidden-window.
This flag is used to configure the duration used by federation hpa controller to determine if it can move max and/or min replicas
around (or not), of a cluster local hpa object, by comparing current time with the last scaled time of that cluster local hpa.
Lower value will result in faster response to scalibility conditions achieved by cluster local hpas on local replicas, but too low
a value can result in thrashing. Higher values will result in slower response to scalibility conditions on local replicas.
```
Pods associated with the test JobTemplate should use a zero
TerminationGracePeriodSeconds to ensure they're deleted immediately.
This should improve test timing assumption consistency.
Automatic merge from submit-queue
Support exec/attach/portforward in `kubectl proxy`
Use the UpgradeAwareProxy shared code in kubectl proxy. Provide a separate transport for those requests that does not have HTTP/2 enabled. Refactor the code to be a bit cleaner in places and to better separate changes.
Fixes#32026
```release-note
`kubectl proxy` will now correctly handle the `exec`, `attach`, and `portforward` commands. You must pass `--disable-filter` to the command in order to allow these endpoints.
```
Improve GC discovery sync performance by only syncing when discovered
resource diffs are detected. Before, the GC worker pool was shut down
and monitors resynced unconditionally every sync period, leading to
significant processing delays causing test flakes where otherwise
reasonable GC timeouts were being exceeded.
Related to https://github.com/kubernetes/kubernetes/issues/49966.
Automatic merge from submit-queue (batch tested with PRs 50173, 50324, 50288, 50263, 50333)
Add blank import for node tests
The node tests weren't being run because the weren't imported in the test/e2e/e2e_test.go file.
Thanks to @abgworrall for sounding the alarm (he noticed [sig-node] wasn't in the test results)!
/assign @yujuhong
/cc @abgworrall
Automatic merge from submit-queue
Fix local storage test failures
**What this PR does / why we need it**:
Fixed a few issues:
- CI environment on GCE cannot resolve node names, need to use IPs. Use a different SSH wrapper that will get the IPs from the node object.
- Use hostdir instead of containerdir now that commands are executed directly on the host, instead of through a container.
- Get the PVC object again after it is bound so that it has the PV name.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50128
**Release note**:
NONE
/release-note-none
/sig storage
Automatic merge from submit-queue
Add waitForFailure for e2e test framework
**What this PR does / why we need it**:
Add waitForFailure for e2e test framework, this could reduce the reliance on logs.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Part of #44118. Refer https://github.com/kubernetes/kubernetes/pull/48858#discussion_r128331726
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Deprecate Deployment .spec.rollbackTo field
~Depends on #48746~ (merged)
xref: #46934, #49135
1. Deprecate Deployment field `.spec.rollbackTo` in `extensions/v1beta1` and `apps/v1beta1`, and remove the same field and `/rollback` endpoint from `apps/v1beta2` Deployment.
1. Add an annotation `deprecated.deployment.rollback.to` in `apps/v1beta2` for conversion to/from other versions.
Note: `apps/v1beta2` is new in 1.8 (and WIP), so it is okay to make breaking changes to it.
```release-note
Deprecate Deployment .spec.rollbackTo field
```
Currently, in federated end2end tests, the creation of services are
done with a randomize NodePort selection. It causing e2e test
flakes if the creation of a federated service failed if the port is
not available.
Now the util.CreateService(...) function is re trying to create the
service on different nodePort in an error case. The method retries until
success or 10 creation retry with other random NodePorts.
If never the service has not been created properly on one of the
federated cluster, a Service shards cleanup is executed before retrying
again the federated service creation.
fixes#44018
Automatic merge from submit-queue
Add a simple cloud provider for e2e tests on kubemark
**What this PR does / why we need it**:
Adds a simplified cloud provider for kubemark. This enables us to add and
remove nodes and operate on nodegroups while running tests on kubemark.
This is needed to run scalability tests for cluster autoscaler on kubemark.
See https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/proposals/kubemark_integration.md
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Add e2e test for cronjob chained removal
This is test proving https://github.com/kubernetes/kubernetes/pull/44058 works with cronjobs. This will fail until the aforementioned PR merges.
@caesarxuchao ptal
Automatic merge from submit-queue (batch tested with PRs 50208, 50259, 49702, 50267, 48986)
Move ownership of proxy test to sig-network directory
```release-note
None
```
1. Deprecate `.spec.rollbackTo` field in extensions/v1beta1 and
apps/v1beta1 Deployments
2. Remove the same field from apps/v1beta2 Deployment, and remove
its rollback subresource and endpoint
Automatic merge from submit-queue
StatefulSet scale subresource
**What this PR does / why we need it**: This PR implements scale subresource for StatefulSet.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46005
**Special notes for your reviewer**:
**Release note**:
```release-note
StatefulSet uses scale subresource when scaling in accord with ReplicationController, ReplicaSet, and Deployment implementations.
```
**Feature Checklist**:
- [x] Introduce Registry interface for storage purpose
- [x] Introduce `ScaleREST New(), Get() and Update()` utility functions
- [x] Create a `ScaleREST` object at `NewREST()` and return it
- [x] Enable scale subresource by adding `/scale` field to the storage map
**Testing Checklist**:
- Unit testing
- [x] Modify `newStorage()` to call `NewStorage()`, and change all unit tests accordingly
- [x] Add unit tests for `ScaleREST Get() and Update()` utility functions
- [x] Add missing unit test for `ShortNames`
- Manual testing
- [x] Verify existence of the subresource using `kubectl proxy` command
- [x] Modify the subresource using `curl` via `POST`
- e2e testing
- [x] Add e2e tests using `RESTClient`
Automatic merge from submit-queue
Federated Job controller implementation
Note that job re-balance is not there yet as it's difficult to honor job deadline
requires #35945 and 35943
fixes#34261
@quinton-hoole @nikhiljindal @deepak-vij
**Release note**:
```release-note
Federated Job feature. It is now possible to create a Federated Job
that is automatically deployed to one or more federated clusters
(as Jobs in those clusters). Job parallelism and completions are
spread across clusters according to cluster selection and weighting
preferences. Federated Job status reflects the aggregate status
across all underlying cluster Jobs.
```
Automatic merge from submit-queue (batch tested with PRs 49524, 46760, 50206, 50166, 49603)
Handled taints on node in batch.
**What this PR does / why we need it**:
Enhanced helpers to handled taints on node in batch.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49522
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 49885, 49751, 49441, 49952, 49945)
Rename e2e sig framework files
**What this PR does / why we need it**:
make files be consistent across all sig e2e tests dir.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Umbrella issue #49161
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50087, 39587, 50042, 50241, 49914)
Add node e2e test for Docker's shared PID namespace
Ref: https://github.com/kubernetes/kubernetes/issues/42926
This PR adds a simple test for the shared PID namespace that's enabled when Docker is 1.13.1+.
/sig node
/area node-e2e
/assign @yujuhong
**Release note**:
```
None
```
This is needed for cluster autoscaler e2e test to
run on kubemark. We need the ability to add and
remove nodes and operate on nodegroups. Kubemark
does not provide this at the moment.
Automatic merge from submit-queue (batch tested with PRs 50091, 50231, 50238, 50236, 50243)
Fix storage tests for multizone test configuration.
**What this PR does / why we need it**:
This PR modifies "[sig-storage] Volumes PD should be mountable with (ext3|ext4)" tests to schedule pods in zone, where PD is created.
This is to make the test work in multizone environment.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 50091, 50231, 50238, 50236, 50243)
Move the sig-instrumentation test to a dedicated folder
Move the last remaining test to sig-instrumentation folder, also move "metrics" package to the "framework" folder
Related issue: https://github.com/kubernetes/kubernetes/issues/49161
/cc @xiangpengzhao @piosz
Automatic merge from submit-queue (batch tested with PRs 50091, 50231, 50238, 50236, 50243)
Modify e2e.go to arbitrarily pick one of zones we have nodes in for multizone tests.
**What this PR does / why we need it**:
When e2e runs in multizone configuration, zone config property can be empty.
This PR, in that case, overrides an empty value with arbitrarily chosen zone that we have nodes in.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 49855, 49915)
Let controllers ignore initialization timeout when creating pods
Partially address https://github.com/kubernetes/kubernetes/issues/48893#issuecomment-318540129.
This only updates the controllers that create pods with `GenerateName`.
The controllers ignore the timeout error when creating the pods, depending on how the initialization progress:
* If the initialization is successful in less than 5 mins, the controller will observe the creation via the informer. All is good.
* If the initialization fails, server will delete the pod, but the controller won't receive any event. The controller will not create new pod until the Creation expectation expires in 5 min.
* If the initialization takes too long (> 5 mins), the Creation expectation expires and the controller will create extra pods.
I'll send follow-up PRs to fix the latter two cases, e.g., by refactoring the sharedInformer.
Automatic merge from submit-queue
Fix typo in test/images/port-forward-tester/Makefile
**What this PR does / why we need it**: the image build fails due to this typo:
```console
$ make WHAT=port-forward-tester
./image-util.sh build port-forward-tester
Building image for port-forward-tester ARCH: ppc64le...
make[1]: Entering directory '[home]/src/k8s.io/kubernetes/test/images/port-forward-tester'
../image-util.sh bin
../image-util.sh: line 22: $2: unbound variable
```
Images already pushed.
**Release note**:
```release-note
NONE
```
/approve no-issue
/assign @mkumatag
Automatic merge from submit-queue (batch tested with PRs 48532, 50054, 50082)
Remove [k8s.io] tag and redundant [sig-storage] tags from volume tests
**What this PR does / why we need it**:
Removes redundant tags from storage e2e test names
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50178
**Release note**:
/release-note-none
Automatic merge from submit-queue (batch tested with PRs 48532, 50054, 50082)
Correcting two spelling mistakes
**What this PR does / why we need it**:
Correcting two spelling mistakes.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
NONE
```release-note
```
Automatic merge from submit-queue
Update OWNERS to correct members' handles
**What this PR does / why we need it**:
Fix some typos of members' handles as per https://github.com/kubernetes/kubernetes/issues/50048#issuecomment-319831957.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Associated with: #50048
**Special notes for your reviewer**:
/cc @madhusudancs @sebgoa @liggitt @saad-ali
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)
Add basic install and mount flexvolumes e2e tests
fixes https://github.com/kubernetes/kubernetes/issues/47010
These two tests install a skeleton "dummy" flex driver, attachable and non-attachable respectively, then test that a pod can successfully use the flex driver. They are labeled disruptive because kubelet and controller-manager get restarted as part of the flex install. IMO it's important to keep this install procedure as part of the test to isolate any bugs with the startup plugin probe code.
There is a bit of an ugly dependency on cluster/gce/config-test.sh because --flex-volume-plugin-dir must be set to a dir that's readable from controller-manager container and writable by the flex e2e test. The default path is not writable on GCE masters with read-only root so I picked a location that looks okay.
In the "dummy" drivers I trick kubelet into thinking there is a mount point by doing "mount -t tmpfs none ${MNTPATH} >/dev/null 2>&1", hope that is okay.
I have only tested on GCE and theoretically they may work on AWS but I don't think there is a need to test on multiple cloudproviders.
-->
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)
increase the GC e2e test timeout
Fix https://github.com/kubernetes/kubernetes/issues/50047.
The root cause is #50046. See log analysis in #50047. For now, we just increase the timeout.
Automatic merge from submit-queue
Fix pointer bug in local volume e2e test
**What this PR does / why we need it**:
Fix pointer bug in local volume e2e test
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/50043
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48237, 50084, 50019, 50069, 50090)
Allow for some pods not to get scheduled in CA tests.
This will allow us to ignore long tail node creation or failure
to create some nodes when running scalability tests on kubemark.
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Moves left networking e2e tests to test/e2e/network
**What this PR does / why we need it**:
#48784 forgot to move some networking e2e tests. This PR moves them.
It also move the networking tests from within `test/e2e/common/networking.go` to `test/e2e/network/networking.go`
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Associated PR #48784
Umbrella issue #49161
**Special notes for your reviewer**:
/assign @wojtek-t @bowei
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49916, 50050)
Update images used in the node e2e benchmark tests
Ref: https://github.com/kubernetes/kubernetes/issues/42926
- Update the cosbeta image since the new version contains a 'du' command fix that affects Docker performance.
- Add the coreos and ubuntu image that run Docker 1.12.6 so that we will have more data to compare.
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 50000, 49954, 49943, 50018, 49607)
Add [sig-autoscaling] prefix to autoscaling e2e tests
**What this PR does / why we need it**:
Add [sig-autoscaling] prefix to autoscaling e2e tests
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Umbrella issue #49161
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50000, 49954, 49943, 50018, 49607)
Update the DeleteReplicaSet in rs_util.go to use server side reaper
**What this PR does / why we need it**:
fix#47832
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47832
**Special notes for your reviewer**:
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 50000, 49954, 49943, 50018, 49607)
Add [sig-scalability] prefix to scalability e2e tests
**What this PR does / why we need it**:
Add [sig-scalability] prefix to scalability e2e tests
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Umbrella issue #49161
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add ubuntu to gluster and nfs tests
**What this PR does / why we need it**:
Enable gluster and nfs tests for ubuntu distro
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50039
**Special notes for your reviewer**:
**Release note**:
/release-note-none
/sig storage
Automatic merge from submit-queue (batch tested with PRs 49990, 49997, 44278, 49936, 49891)
Allow mode in e2e-framework to gather metrics only from master
This should enable getting metrics for our 5k-node clusters.
cc @kubernetes/sig-scalability-misc @gmarek
Automatic merge from submit-queue (batch tested with PRs 49992, 48861, 49267, 49356, 49886)
remove unused function
**What this PR does / why we need it**:
remove unused function which is not used months ago.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49871, 49422, 49092, 49858, 48999)
Refactor logging e2e tests, add new checks
I split existing code into smaller files to simplify the review and future changes
Also, there are new tests for stackdriver logging:
- ingesting system logs from all nodes
- ingesting logs in json format
- ingesting logs in glog format
Automatic merge from submit-queue (batch tested with PRs 49898, 49897, 49919, 48860, 49491)
Fix usage a make(struct, len()) followed by append()
A couple of places in the code we allocate with make() but then use
append(), instead of copy() or direct assignment. This results in a
slice with len() zero elements at the front followed by the expected
data. The correct form for such usage is `make(struct, 0, len())`.
I found these by running:
```
$ git grep -EI -A7 'make\([^,]*, len\(' | grep 'append(' -B7 | grep -v vendor
```
And then manually looking through the results. I'm sure something better
could exist.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49898, 49897, 49919, 48860, 49491)
Add basic local volume provisioner e2e tests
**What this PR does / why we need it**:
Adds e2e tests to test local volume provisioner.
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubernetes/issues/48832
**Special notes for your reviewer**:
- bring up local volume provisioner using bootstrapper
- have provisioner create a volume by creating a directory under discovery path.
- check persistent volume is created
- make a claim on the PV, write some data then delete the claim. Verify volume is cleaned up.
**Release note**:
```release-note
```
@ianchakeres @msau42
Automatic merge from submit-queue (batch tested with PRs 49651, 49707, 49662, 47019, 49747)
Add support for `no_new_privs` via AllowPrivilegeEscalation
**What this PR does / why we need it**:
Implements kubernetes/community#639
Fixes#38417
Adds `AllowPrivilegeEscalation` and `DefaultAllowPrivilegeEscalation` to `PodSecurityPolicy`.
Adds `AllowPrivilegeEscalation` to container `SecurityContext`.
Adds the proposed behavior to `kuberuntime`, `dockershim`, and `rkt`. Adds a bunch of unit tests to ensure the desired default behavior and that when `DefaultAllowPrivilegeEscalation` is explicitly set.
Tests pass locally with docker and rkt runtimes. There are also a few integration tests with a `setuid` binary for sanity.
**Release note**:
```release-note
Adds AllowPrivilegeEscalation to control whether a process can gain more privileges than it's parent process
```
Automatic merge from submit-queue (batch tested with PRs 49651, 49707, 49662, 47019, 49747)
improve detectability of deleted pods
**What this PR does / why we need it**:
Adds comment to `waitForPodTerminatedInNamespace` to better explain how it's implemented.
~~It improves pod deletion detection in the e2e framework as follows:~~
~~1. the `waitForPodTerminatedInNamespace` func looks for pod.Status.Phase == _PodFailed_ or _PodSucceeded_ since both values imply that all containers have terminated.~~
~~2. the `waitForPodTerminatedInNamespace` func also ignores the pod's Reason if the passed-in `reason` parm is "". Reason is not really relevant to the pod being deleted or not, but if the caller passes a non-blank `reason` then it will be lower-cased, de-blanked and compared to the pod's Reason (also lower-cased and de-blanked). The idea is to make Reason checking more flexible and to prevent a pod from being considered running when all of its containers have terminated just because of a Reason mis-match.~~
Releated to pr [49597](https://github.com/kubernetes/kubernetes/pull/49597) and issue [49529](https://github.com/kubernetes/kubernetes/issues/49529).
**Release note**:
```release-note
NONE
```
A couple of places in the code we allocate with make() but then use
append(), instead of copy() or direct assignment. This results in a
slice with len() zero elements at the front followed by the expected
data. The correct form for such usage is `make(struct, 0, len())`.
I found these by running:
```
$ git grep -EI -A7 'make\([^,]*, len\(' | grep 'append(' -B7 | grep -v vendor
```
And then manually looking through the results. I'm sure something better
could exist.
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Add ext4 and xfs tests to GCE PD basic mount tests
**What this PR does / why we need it**:
Add ext4 and xfs to basic GCE PD mount tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49511
**Special notes for your reviewer**:
**Release note**:
/release-note-none
/sig storage
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Enable garbage collection of custom resources
Enhance the garbage collector to periodically refresh the resources it monitors (via discovery) to enable custom resource definition GC (addressing #44507 and reverting #47432).
This is a replacement for #46000.
/cc @lavalamp @deads2k @sttts @caesarxuchao
/ref https://github.com/kubernetes/kubernetes/pull/48065
```release-note
The garbage collector now supports custom APIs added via CustomeResourceDefinition or aggregated apiservers. Note that the garbage collector controller refreshes periodically, so there is a latency between when the API is added and when the garbage collector starts to manage it.
```
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Add a support for GKE regional clusters in e2e tests.
**What this PR does / why we need it**:
Add a support for GKE regional clusters in e2e tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49538, 49708, 47665, 49750, 49528)
Use the core client with version
**What this PR does / why we need it**:
Replace the **deprecated** `clientSet.Core()` with `clientSet.CoreV1()`.
**Which issue this PR fixes**: fixes#49535
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49581, 49652, 49681, 49688, 44655)
Re-enable federated ingress test that was disabled due to a federated service deletion bug.
The details of the bug is described in PR #44626. We believe this bug fixes the flakiness in this test and hence we are re-enabling this test to get some mileage on it. If it turns out to be a problem again we are going to revert this back.
**Release note**:
```release-note
NONE
```
/assign @csbell
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 49581, 49652, 49681, 49688, 44655)
Move the audit e2e test out of the node SIG
It was mistakenly moved to sig-node in https://github.com/kubernetes/kubernetes/pull/48910, but this is an apiserver feature, not a node feature.
/cc @crassirostris
Automatic merge from submit-queue (batch tested with PRs 49581, 49652, 49681, 49688, 44655)
Add sig-testing OWNERS_ALIASES
/sig testing
**What this PR does / why we need it**:
follow the sig-foo-{reviewers,approvers} convention
- rename test-infra-maintainers to sig-testing-approvers
- copy sig-testing-approvers to sig-testing-reviewers
- remove inviduals in test/OWNERS in favor of new aliases
as a result
- rmmh gets test/ approver privileges
- spiffxp gets hack/jenkins/ approver privileges
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49580
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49712, 49694, 49714, 49670, 49717)
Reduce GC e2e test flakiness
Increase GC wait timeout in a flaky e2e test. The test expects a GC
operation to be performed within 30s, while in practice the operation
often takes longer due to a delay between the enqueueing of the owner's
delete operation and the GC's actual processing of that event. Doubling
the time seems to stabilize the test. The test's assumptions can be
revisited, and the processing delay under load can be investigated in
the future.
Extracted from https://github.com/kubernetes/kubernetes/pull/47665 per https://github.com/kubernetes/kubernetes/pull/47665#issuecomment-318219099.
/cc @sttts @caesarxuchao @deads2k @kubernetes/sig-api-machinery-bugs
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45813, 49594, 49443, 49167, 47539)
Add node e2e tests for GKE environment
Ref: https://github.com/kubernetes/kubernetes/issues/46891
This PR adds node e2e tests for validating images used on GKE.
- We pass the `SYSTEM_SPEC_NAME` to the node e2e test process via the flag `--system-spec-name` so that we can skip the environment specific tests using `RunIfSystemSpecNameIs()`.
- Also added `SkipIfContainerRuntimeIs()` as the opposite of `RunIfContainerRuntimeIs()`.
**Release note**:
```
None
```
Enhance the garbage collector to periodically refresh the resources it
monitors (via discovery) to enable custom resource definition GC.
This implementation caches Unstructured structs for any kinds not
covered by a shared informer. The existing meta-only codec only supports
compiled types; an improved codec which supports arbitrary types could
be introduced to optimize caching to store only metadata for all
non-informer types.
Automatic merge from submit-queue (batch tested with PRs 49619, 49598, 47267, 49597, 49638)
Remove default binding of system:node role to system:nodes group
part of https://github.com/kubernetes/features/issues/279
deprecation of this automatic binding announced in 1.7 in https://github.com/kubernetes/kubernetes/pull/46076
```release-note
RBAC: the `system:node` role is no longer automatically granted to the `system:nodes` group in new clusters. It is recommended that nodes be authorized using the `Node` authorization mode instead. Installations that wish to continue giving all members of the `system:nodes` group the `system:node` role (which grants broad read access, including all secrets and configmaps) must create an installation-specific `ClusterRoleBinding`.
```
Automatic merge from submit-queue (batch tested with PRs 49619, 49598, 47267, 49597, 49638)
improve log for pod deletion poll loop
**What this PR does / why we need it**:
It improves some logging related to waiting for a pod to reach a passed-in condition. Specifically, related to issue [49529](https://github.com/kubernetes/kubernetes/issues/49529) where better logging may help to debug the root cause.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49619, 49598, 47267, 49597, 49638)
Flag support in kubectl plugins
Adds support to flags in `kubectl` plugins. Flags are declared in the plugin descriptor and are passed to plugins through env vars, similar to global flags (which already works).
Fixes https://github.com/kubernetes/kubernetes/issues/49122
**Release note**:
```release-note
Added flag support to kubectl plugins
```
PTAL @monopole @kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 47738, 49196, 48907, 48533, 48822)
Move e2e dependent images from kubernetes/kubernetes.github.io repo
**What this PR does / why we need it**:
Move e2e dependent images from kubernetes/kubernetes.github.io repo
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48530
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 49238, 49595, 43494, 47897, 48905)
Add apps/v1beta2.ReplicaSet
~Depends on #48746~ (merged)
~Depends on #49357~ (merged)
xref: #49135
```release-note
Add a new API object apps/v1beta2.ReplicaSet
```
Automatic merge from submit-queue (batch tested with PRs 49665, 49689, 49495, 49146, 48934)
Add Integration tests for Inter-Pod Affinity(AntiAffinity)
Signed-off-by: vikaschoudhary16 <choudharyvikas16@gmail.com>
**What this PR does / why we need it**:
Adds integration tests for inter-pod affinity(anti-affinity)
**Special notes for your reviewer**:
Once after @bsalamat 's #48847 gets merged, changes in this PR will be restructured according to the merged changes.
ref/ #48176#48847
@kubernetes/sig-scheduling-pr-reviews
@davidopp @k82cn
Increase GC wait timeout in a flaky e2e test. The test expects a GC
operation to be performed within 30s, while in practice the operation
often takes longer due to a delay between the enqueueing of the owner's
delete operation and the GC's actual processing of that event. Doubling
the time seems to stabilize the test. The test's assumptions can be
revisited, and the processing delay under load can be investigated in
the future.
Automatic merge from submit-queue (batch tested with PRs 46358, 49408)
[Federation] Updates to enable hpa controllers test in integration and e2e
Enables the apis on api server in both scenario.
Additional logic to enable and run the crud portion of objects in integration, for controllers which implement additional logic in reconcile.
**Special notes for your reviewer**:
This on top of an existing PR https://github.com/kubernetes/kubernetes/pull/45497.
The last 2 commits are reviewable here
@kubernetes/sig-federation-pr-reviews
cc @marun @perotinus
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 46913, 48910, 48858, 47160)
Add e2e test for readOnlyRootFilesystem containers
**What this PR does / why we need it**:
This PR adds node e2e test for readOnlyRootFilesystem containers.
**Which issue this PR fixes**
Part of #44118.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46913, 48910, 48858, 47160)
move sig-node related e2e tests to node subdir
I need help making sure I picked the right ones and/or didn't miss anything.
Potential additions include: `logging_soak.go`, `ssh.go`, `kubelet_perf.go`.
/cc @dchen1107 @vishh @tallclair @yujuhong @Random-Liu @abgworrall @dashpole @yguo0905
Automatic merge from submit-queue (batch tested with PRs 48976, 49474, 40050, 49426, 49430)
Use presence of kubeconfig file to toggle standalone mode
Fixes#40049
```release-note
The deprecated --api-servers flag has been removed. Use --kubeconfig to provide API server connection information instead. The --require-kubeconfig flag is now deprecated. The default kubeconfig path is also deprecated. Both --require-kubeconfig and the default kubeconfig path will be removed in Kubernetes v1.10.0.
```
/cc @kubernetes/sig-cluster-lifecycle-misc @kubernetes/sig-node-misc
Automatic merge from submit-queue (batch tested with PRs 48976, 49474, 40050, 49426, 49430)
Remove duplicated import and wrong alias name of api package
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes#48975
**Special notes for your reviewer**:
/assign @caesarxuchao
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48224, 45431, 45946, 48775, 49396)
Update cos-dev image in benchmark tests to cos-dev-61-9759-0-0
Ref: https://github.com/kubernetes/kubernetes/issues/42926
`cos-dev-61-9759-0-0` contains a fix in Linux utility `du` that would affect the measurement of docker performance in kubelet. I'd like to update the benchmark to use the new image.
**Release note**:
```
None
```
/assign @tallclair
/cc @kewu1992 @abgworrall
follow the sig-foo-{reviewers,approvers} convention
- rename test-infra-maintainers to sig-testing-approvers
- copy sig-testing-approvers to sig-testing-reviewers
- remove inviduals in test/OWNERS in favor of new aliases
as a result
- rmmh gets test/ approver privileges
- spiffxp gets hack/jenkins/ approver privileges
Automatic merge from submit-queue (batch tested with PRs 49286, 49550)
Remove myself from a bunch of places
I am assigned in reviews which I never get to do. I prefer drive-bys whenever I can do them rather than the bot choosing myself in random, ends up being mere spam.
@smarterclayton please approve.
Automatic merge from submit-queue (batch tested with PRs 48846, 49483, 49341)
Add [sig-network] prefix to network e2e tests
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Associated PR #48784
Umbrella issue #49161
**Special notes for your reviewer**:
/assign @@wojtek-t @freehan
/cc @bowei
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Remove flags low-diskspace-threshold-mb and outofdisk-transition-frequency
issue: #48843
This removes two flags replaced by the eviction manager. These have been depreciated for two releases, which I believe correctly follows the kubernetes depreciation guidelines.
```release-note
Remove depreciated flags: --low-diskspace-threshold-mb and --outofdisk-transition-frequency, which are replaced by --eviction-hard
```
cc @mtaufen since I am changing kubelet flags
cc @vishh @derekwaynecarr
/sig node
Automatic merge from submit-queue (batch tested with PRs 49358, 49253)
Remove hostname label condition in SchedulerPredicates
**What this PR does / why we need it**:
```
validates that NodeSelector is respected if matching [Conformance]
validates that required NodeAffinity setting is respected if matching
```
The two tests above make the assumption that the node names are equal to the `kubernetes.io/hostname` labels. Unfortunately, this is not necessarily true all the time. For instance, when using the AWS Cloud Provider + Container Linux:
- The node name is set using the AWS SDK's `ec2.Instance.PrivateDnsName` and has the form `ip-10-0-35-57.ca-central-1.compute.internal` [[1](https://github.com/kubernetes/kubernetes/blob/v1.7.1/pkg/cloudprovider/providers/aws/aws.go#L3343-L3346)] [[2](https://raw.githubusercontent.com/aws/aws-sdk-go/master/service/ec2/api.go)]
- The node's hostname, however, is a simple call to `os.Hostname()`, itself reading `/proc/sys/kernel/hostname`, which contains what the AWS DHCP assigned to the instance, typically the hostname short-form: `ip-10-0-16-137`. [[1](https://github.com/kubernetes/kubernetes/blob/v1.7.1/pkg/util/node/node.go#L43-L54)]
Consequently, we are trying to assign a pod to a node having the following label: `kubernetes.io/hostname=ip-10-0-35-57.ca-central-1.compute.internal` (in addition to the randomly generated label), whereas the actual label on the node is `kubernetes.io/hostname=ip-10-0-35-57`.
Furthermore, this inaccurate `kubernetes.io/hostname=<nodename>` condition is actually useless given we already match over a random label, that was assigned to that node. Later, the test ensures that the scheduled pod was scheduled to the right node by comparing the pod's node name and the node name we expected the pod to be on:
```
framework.ExpectNoError(framework.WaitForPodNotPending(cs, ns, labelPodName))
labelPod, err := cs.Core().Pods(ns).Get(labelPodName, metav1.GetOptions{})
framework.ExpectNoError(err)
Expect(labelPod.Spec.NodeName).To(Equal(nodeName))
```
The `k8s.io/apimachinery/pkg/types/nodename` data structure actually [warns](55bee3ad21/staging/src/k8s.io/apimachinery/pkg/types/nodename.go (L40-L43)) about the fact that the node name might be different than the hostname on AWS.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Added sig-storage labels to upgrade tests and moved them to appropriate directory
**What this PR does / why we need it**: Adding necessary sig identifier for storage upgrade tests.
/release-note-none
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)
Fix bug in command retrying in kubemark
This should fix some of the flakes mentioned in https://github.com/kubernetes/kubernetes/issues/46195.
It's showing that all subsequent retries have failed if the first one failed due to `ret_val` not being reassigned on success.
@bskiba Thanks for noticing :)
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)
StatefulSet: Remove `pod.alpha.kubernetes.io/initialized` annotation.
The `pod.alpha.kubernetes.io/initialized` annotation was originally a tool for validating StatefulSet's ordered Pod creation guarantees during the feature's alpha phase.
If set to "false" on a given Pod, it would interrupt StatefulSet's normal behavior. In v1.5.0, the annotation was deprecated and the default became "true" as part of StatefulSet's graduation to beta.
The annotation is now ignored, meaning it cannot be used to interrupt StatefulSet Pod management.
```release-note
StatefulSet: The deprecated `pod.alpha.kubernetes.io/initialized` annotation for interrupting StatefulSet Pod management is now ignored. If you were setting it to `true` or leaving it unset, no action is required. However, if you were setting it to `false`, be aware that previously-dormant StatefulSets may become active after upgrading.
```
ref #41605
Automatic merge from submit-queue (batch tested with PRs 48636, 49088, 49251, 49417, 49494)
Fix issues for local storage allocatable feature
This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.
This PR fixes issue #47809
Replaces use of --api-servers with --kubeconfig in Kubelet args across
the turnup scripts. In many cases this involves generating a kubeconfig
file for the Kubelet and placing it in the correct location on the node.
Automatic merge from submit-queue
Allow nodes to create evictions for its own pods in NodeRestriction admission controller
**What this PR does / why we need it**: This PR adds support for `pods/eviction` sub-resource to the NodeRestriction admission controller so it allows a node to evict pods bound to itself.
**Which issue this PR fixes**: fixes#48666
**Special notes for your reviewer**: The NodeRestriction already allows nodes to delete pods bound to itself, so allowing nodes to also delete pods via the Eviction API probably makes sense.
```release-note
NodeRestriction allows a node to evict pods bound to itself
```
Automatic merge from submit-queue (batch tested with PRs 49409, 49352, 49266, 48418)
[e2e] Also verify content returned by kube-proxy healthz url
**What this PR does / why we need it**: Enhance kube-proxy url test. This helps to detect the port collision case --- node-problem-detector also serves /healthz to return 200 ok. Verify the content to confirm /healthz is served by kube-proxy.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: From #49263
**Special notes for your reviewer**:
/assign @freehan @nicksardo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49326, 49394, 49346, 49379, 49399)
Update to version gate CRDs to 1.7 and greater
**What this PR does / why we need it**:
Allows e2e's to be tested on earlier version do to version check.
xref: #49313
**Release note**:
```
NONE
```
/cc @kubernetes/sig-api-machinery-bugs @kubernetes/sig-testing-bugs
Automatic merge from submit-queue (batch tested with PRs 49420, 49296, 49299, 49371, 46514)
Refactoring taint functions to reduce sprawl
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45060
**Special notes for your reviewer**:
@gmarek @timothysc @k82cn @jayunit100 - I moved some fn's to helpers and some to utils. LMK, if you are ok with this change.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add a new API version apps/v1beta2
xref: #49135
This PR adds a new API version `apps/v1beta2` which contains a copy (of types, conversions, and defaults) of `apps/v1beta1` StatefulSet, Deployment, and their subresources. Note that `apps/v1beta2` is still WIP and we will make breaking changes to it before releasing 1.8.
Moving core controllers (StatefulSet, Deployment, ReplicaSet, DaemonSet) to `apps/v1beta2` is the first step of moving them to `apps/v1` (GA).
This PR is a starting point for DaemonSet and ReplicaSet to move from `/extensions` to `/apps` and for Deployment and StatefulSet to make some breaking changes (e.g. new defaults and/or remove deprecated fields).
```release-note
Add a new API version apps/v1beta2
```
Automatic merge from submit-queue
Reduce hollow proxy mem/node
As likely expected, kubemark-scale failed to even start with n1-standard-8 nodes. Because 1/3rd of our hollow nodes didn't even get scheduled due to their requests:
```
I0720 17:45:08.139] Found only 3325 ready hollow-nodes while waiting for 5000.
I0720 17:45:20.435] 3326 hollow-nodes are reported as 'Running'
I0720 17:45:20.442] 1675 hollow-nodes are reported as NOT 'Running'
```
If we want to experiment with smaller nodes anyway, then this change is needed. Though we most likely will end up OOM'ing.
Explanation for new value:
We have 62.5 hollow-node / real-node
=> mem available per hollow node = 30GB / 62.5 = 480MB
minus 100MB (kubelet)
minus 20MB (npd)
=> 360MB for proxy should be = 100MB + 5000*(mem/node)
=> 50KB mem/node (with some slight slack)
cc @kubernetes/sig-scalability-misc
Automatic merge from submit-queue
Add an integration test library and some integration tests for scheduler
**What this PR does / why we need it**:
1. Add an integration test library (utils.go) for scheduler testing.
2. Cleaned up some of the tests in scheduler_test.go with the new integration test library.
3. Add priority_test.go with a couple of examples on how to test scheduler priority function in integration tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
ref/ #48176
@kubernetes/sig-scheduling-pr-reviews
@davidopp @k82cn @vikaschoudhary16
Automatic merge from submit-queue (batch tested with PRs 49328, 49285, 49307, 49127, 49163)
Cleanup storage e2e test names
**What this PR does / why we need it**:
Some test names had redundant [sig-storage] tags. Also, some tests still had [Volume] tag. This PR removes those tags.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Release note**:
```release-note
NONE
```
/release-note-none
/sig storage
Automatic merge from submit-queue (batch tested with PRs 49330, 49252, 49262, 49278, 49334)
Add project to pd delete node gcloud command
**What this PR does / why we need it**: Add `--project=` to `gcloud compute instances list` calls from `Pod Disks should be able to detach from a node which was deleted`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: Fixes #https://github.com/kubernetes/kubernetes/issues/49185
**Special notes for your reviewer**:
CC @kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 49330, 49252, 49262, 49278, 49334)
Enable garbage collector e2e tests
These tests are not running in pre-submit: see 753266cb7d/jobs/config.json (L9207)
Automatic merge from submit-queue
remove redundant param in e2e_node/remote
**What this PR does / why we need it**:
* remove redundant param in e2e_node/remote/remote.go
* fix a small typo
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
add e2e tests for the bootstrapsigner and tokencleaner controllers, integration testing for bootstrap token auth
**What this PR does / why we need it**:
Add e2e test for bootstrap signer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
Test Ubuntu image using GKE image spec on master
Ref: https://github.com/kubernetes/kubernetes/issues/46891
This PR changes the files referenced in test-infra for running Ubuntu image tests against GKE system spec on master.
The two properties files are shared by the tests against all k8s branches but the `SYSTEM_SPEC_NAME` is only available on master. This should be fine because the tests in the non master branches will just ignore the unknown env variable.
**Release note**:
```
None
```
/assign @yujuhong
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
Modular extensions for kube scheduler perf testing framework
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45973
**Special notes for your reviewer**:
It is not same as the existing one, the previous one has a single nodeaffinity key with multiple values. This one has multiple keys, values.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49107, 47177, 49234, 49224, 49227)
tighten quota controller interface
While debugging a quota performance problem, I had to chase some references deeper than necessary because the interfaces were overly broad. This tightens them.
```release-note
NONE
```
Automatic merge from submit-queue
Fix too extensive logging in Stackdriver Logging e2e tests
Since logging pod now includes a cache, printing it out makes build-log unusable.
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)
Add cos-dev-61-9733-0-0 to the benchmark tests
Ref: https://github.com/kubernetes/kubernetes/issues/42926
m60 has docker 1.13.1 while m61 has 17.03. This PR adds m61 to the benchmark tests so that we will have more data to compare.
PS: We will support fetching the latest image in an image family in the node e2e tests in the future.
**Release note**:
```
None
```
/assign @yujuhong
/cc @kewu1992 @abgworrall
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)
fixit: break sig-cluster-lifecycle tests into subpackage
this is part of fixit week. ref #49161
@kubernetes/sig-cluster-lifecycle-misc
Automatic merge from submit-queue
Add PriorityClass API object under new "scheduling" API group
**What this PR does / why we need it**: This PR is a part of a series of PRs to add pod priority to Kubernetes. This PR adds a new API group called "scheduling" with a new API object called "PriorityClass". PriorityClass maps the string value of priority to its integer value.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: Given the size of this PR, I will add the admission controller for the PriorityClass in a separate PR.
**Release note**:
```release-note
Add PriorityClass API object under new "scheduling" API group
```
ref/ #47604
ref/ #48646
Automatic merge from submit-queue (batch tested with PRs 47509, 46821, 45319, 49121, 49125)
Tolerate a missing MasterName (for GKE)
When testing, GKE created clusters don't provide a `MasterName`, so don't throw a warning and give up when that happens.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47509, 46821, 45319, 49121, 49125)
volume i/o tests for storage plugins
**What this PR does / why we need it**:
Addresses issues [25268](https://github.com/kubernetes/kubernetes/issues/25268) and [28367](https://github.com/kubernetes/kubernetes/issues/28367), though it may be weak re. the streaming i/o issue. @matchstick
**Special notes for your reviewer**:
This is a new file. Plugins other than NFS, GlusterFS, iSCSI, and Ceph-RBD code will need to be supported in a separate PR.
```release-note
NONE
```
Automatic merge from submit-queue
Bump e2e mounttest image version to 0.8
Reduce the number of image files required for e2e test run
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49143, 49211)
Add more logging to PD node delete test
**What this PR does / why we need it**: Adds more logging to `Pod Disks should be able to detach from a node which was deleted` test which started failing because `gcloud compute instances list` is failing with `exit status 1`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: https://github.com/kubernetes/kubernetes/issues/49185
**Special notes for your reviewer**:
**Release note**: none
Automatic merge from submit-queue (batch tested with PRs 49058, 49072, 49137, 49182, 49045)
*: remove --insecure-allow-any-token option
~Since the authenticator is still used in e2e tests, don't remove
the actual package. Maybe a follow up?~
edit: e2e and integration tests have been switched over to the tokenfile
authenticator instead.
```release-note
The --insecure-allow-any-token flag has been removed from kube-apiserver. Users of the flag should use impersonation headers instead for debugging.
```
closes#49031
cc @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 49116, 49095)
Move pkg/api/v1/ref -> client-go/tools/reference
`pkg/api/v1/ref` is the only remaining package copied from pkg/api/v1 to client-go via staging/copy.sh.
Automatic merge from submit-queue (batch tested with PRs 48043, 48200, 49139, 36238, 49130)
Implement equivalence cache by caching and re-using predicate result
The last part of #30844, I opened a new PR instead of overwrite the old one because we changed some basic assumption by allowing invalidating equivalence cache item by individual predicate.
The idea of this PR is based on discussion in https://github.com/kubernetes/kubernetes/issues/32024
- [x] Pods belong to same controllerRef considered to be equivalent
- [x] ` podFitsOnNode` will use cached predicate result if it's available
- [x] Equivalence cache will be updated when if a fresh new predicate is done
- [x] `factory.go` will invalid specific predicate cache(s) based on the object change
- [x] Since `schedule` and `bind` are async, we need to optimistically invalid affected cache(s) before `bind`
- [x] Fully unit test of affected files
- [x] e2e test to verify cache update/invalid workflow
- [x] performance test results
- [x] Some nits fixes related but expected to result in `needs-rebase` so they are split to: #36060#35968#37512
cc @wojtek-t @davidopp
Automatic merge from submit-queue (batch tested with PRs 49055, 49128, 49132, 49134, 49110)
Remove affinity annotations leftover
**What this PR does / why we need it**:
This is a further cleanup for affinity annotations, following #47869.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
ref: #47869
**Special notes for your reviewer**:
- I remove the commented test cases and just leave TODOs instead. I think converting these untestable test cases for now is not necessary. We can add new test cases in future.
- I remove the e2e test case `validates that embedding the JSON PodAffinity and PodAntiAffinity setting as a string in the annotation value work` because we have a test case `validates that InterPod Affinity and AntiAffinity is respected if matching` to test the same thing.
/cc @aveshagarwal @bsalamat @gyliu513 @k82cn @timothysc
**Release note**:
```release-note
NONE
```
# The first commit's message is:
Modular extensions for kube scheduler perf testing framework
# This is the 2nd commit message:
Modular extensions for kube scheduler perf testing framework
Automatic merge from submit-queue (batch tested with PRs 48914, 48535, 49099, 48935, 48871)
Adopt debian-base as baseimage
**What this PR does / why we need it**:
Based on discussion from - https://github.com/kubernetes/kubernetes/pull/44910/files#r125150263
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49169
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 48481, 48256)
Refactor: pkg/util into sub-pkgs
**What this PR does / why we need it**:
- move code in pkg/util into sub-pkgs
- delete some unused funcs
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#15634
**Special notes for your reviewer**:
This is the final work of #15634. It will close that issue.
/cc @thockin
**Release note**:
```release-note
NONE
```
Add PriorityClass to pkg/registry
Add PriorityClass to pkg/master/master.go
Add PriorityClass to import_know_versions.go
Update linted packages
minor fix
e2e and integration tests have been switched over to the tokenfile
authenticator instead.
```release-note
The --insecure-allow-any-token flag has been removed from kube-apiserver. Users of the flag should use impersonation headers instead for debugging.
```
Automatic merge from submit-queue (batch tested with PRs 46094, 48544, 48807, 49102, 44174)
Shared Informer Run blocks until all goroutines finish
**What this PR does / why we need it**:
Makes Shared Informer Run method block until all goroutines it spawned finish. See #45454.
**Which issue this PR fixes**
Fixes#45454
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49019, 48919, 49040, 49018, 48874)
correcting spell mistake
**What this PR does / why we need it**:
Correcting spell mistake.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
NONE
```release-note
```
Automatic merge from submit-queue
Sanitize test names before using them as namespaces
**What this PR does / why we need it**: It sanitizes upgrade test names before creating namespaces.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48876
Automatic merge from submit-queue (batch tested with PRs 48231, 47377, 48797, 49020, 49033)
Update yaml and json with multi arch test images
**What this PR does / why we need it**:
This PR is for updating the yaml and json files under test/images folder with multi arch images
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 48494, 48733)
Move test-webserver from contrib/for-demos to kubernetes/test/images
**What this PR does / why we need it**:
This PR is for
- Moving the https://github.com/kubernetes/contrib/tree/master/for-demos/test-webserver to kubernetes/test/images - Refer https://github.com/kubernetes/contrib/pull/2544 for more information
- Multi architecture support for test-webserver image
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 48991, 48908)
Group every two services into one in load test
Ref https://github.com/kubernetes/kubernetes/issues/48938
Following from discussion with @bowei and @freehan .
This reduces #services to 8200 while keeping no. of backends same.
/cc @wojtek-t @gmarek
Automatic merge from submit-queue
Remove old, core/v1 specific constructs from RESTClient
Now that metav1 is abstracted from the APIs, RESTClient should also be agnostic to the core API.
* Remove `LabelSelectorParam` and `FieldSelectorParam` - use `VersionedParams` with `ListOptions`
* Remove `UintParam`
* Remove all legacy field selector logic from `VersionedParams` - ParameterCodec now handles that
* Remove special parameters (like `timeout`) which is no longer set by most clients
Automatic merge from submit-queue (batch tested with PRs 47417, 47638, 46930)
Added scheduler integration test owners.
**What this PR does / why we need it**:
Add OWNER file into scheduler integration test.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A
**Release note**:
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)
Support customized system spec in the node conformance test and create the GKE system spec
ref: https://github.com/kubernetes/kubernetes/issues/46891
- System specs are located in `test/e2e_node/system/specs`. Created one for validating GKE images in `test/e2e_node/system/specs/gke.yaml`.
- `--image-spec-name` can be used to specify a system spec in node e2e and conformance tests. This option maps to `SYSTEM_SPEC_NAME` in a test properties file, which is the user facing configuration. So, users can specify `SYSTEM_SPEC_NAME=gke` to run the image validation using the GKE system spec.
- If `SYSTEM_SPEC_NAME` is unspecified, the default spec (`system.DefaultSysSpec`) will be used.
- We can also use `make test-e2e-node SYSTEM_SPEC_NAME=gke` to run tests using GKE image spec.
**Release note**:
`None`
Automatic merge from submit-queue
Move api-machinery related e2e tests to a 'api-machinery' e2e test su…
…bdirectory.
**What this PR does / why we need it**:
Moves all e2e tests belonging to sig-api-machinery into a dedicated `test/e2e/apimachinery ` directory and updates the tests to use `SigDescribe` to prepend `[sig-api-machinery]` to the testnames of all apimachinery owned tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48578, 48895, 48958)
move sig-apps upgrade tests to its directory
**What this PR does / why we need it**: This PR moves sig-apps upgrade tests to its directory in accord to fixit requirements.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48839
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Azure PD (Managed/Blob)
This is exactly the same code as this [PR](https://github.com/kubernetes/kubernetes/pull/41950). It has a clean set of generated items. We created a separate PR to accelerate the accept/merge the PR
CC @colemickens
CC @brendandburns
**What this PR does / why we need it**:
1. Adds K8S support for Azure Managed Disks.
2. Adds support for dedicated blob disks (1:1 to storage account) in addition to shared blob disks (n:1 to storage account).
3. Automatically manages the underlying storage accounts. New storage accounts are created at 50% utilization. Max is 100 disks, 60 disks per storage account.
2. Addresses the current issues with Blob Disks:
..* Significantly faster attach process. Disks are now usually available for pods on nodes under 30 sec if formatted, under a min if not formatted.
..* Adds support to move disks between nodes.
..* Adds consistent attach/detach behavior, checks if the disk is leased/attached on a different node before attempting to attach to target nodes.
..* Fixes a random hang behavior on Azure VMs during mount/format (for both blob + managed disks).
..* Fixes a potential conflict by avoiding the use of disk names for mount paths. The new plugin uses hashed disk uri for mount path.
The existing AzureDisk is used as is. Additional "kind" property was added allowing the user to decide if the pd will be shared, dedicated or managed (Azure Managed Disks are used).
Due to the change in mounting paths, existing PDs need to be recreated as PV or PVCs on the new plugin.
Automatic merge from submit-queue (batch tested with PRs 48864, 48651, 47703)
Enable logexporter mechanism to dump logs from k8s nodes to GCS directly
Ref https://github.com/kubernetes/kubernetes/issues/48513
This adds support for logexporter from k8s side. Next I'll send a PR adding support from test-infra side.
/cc @kubernetes/sig-scalability-misc @kubernetes/test-infra-maintainers @fejta @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 48082, 48815, 48901, 48824)
Import kubectl tests in e2e_test.go so they start running.
```release-note
NONE
```
Automatic merge from submit-queue
add dockershim checkpoint node e2e test
Add a bunch of disruptive cases to test kubelet/dockershim's checkpoint work flow.
Some steps are quite hacky. Not sure if there is better ways to do things.
Automatic merge from submit-queue (batch tested with PRs 47999, 47890)
[Federation] Update namespace support to use the sync controller
This PR moves namespaces to use the sync controller.
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 47999, 47890)
LocalPV e2e test with two pods writing/reading to a shared file on one PVC
**What this PR does / why we need it**:
This PR adds an e2e test where one pod writes to a localPV and another reads from it. This e2e test is for the PersistentLocalVolumes alpha feature.
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubernetes/issues/48832
**Special notes for your reviewer**:
You need to setup the PersistentLocalVolumes feature-gate and NODE_LOCAL_SSDS environment variable.
For example:
```
KUBE_FEATURE_GATES="PersistentLocalVolumes=true" NODE_LOCAL_SSDS=1 go run hack/e2e.go -- -v --up
go run hack/e2e.go -- -v --test --test_args="--ginkgo.focus=\[Feature:LocalPersistentVolumes\]"
```
**Release note**:
```release-note
NONE
```
This PR fixes the following issues:
1. Use ResourceStorageScratch instead of ResourceStorage API to represent
local storage capacity
2. In eviction manager, use container manager instead of node provider
(kubelet) to retrieve the node capacity and reserved resources. Node
provider (kubelet) has a feature gate so that storagescratch information
may not be exposed if feature gate is not set. On the other hand,
container manager has all the capacity and allocatable resource
information.
Automatic merge from submit-queue (batch tested with PRs 48555, 48849)
GCE: Fix panic when service loadbalancer has static IP address
Fixes#48848
```release-note
Fix service controller crash loop when Service with GCP LoadBalancer uses static IP (#48848, @nicksardo)
```
Automatic merge from submit-queue
add [sig-apps] identifier to relevant upgrade tests
**What this PR does / why we need it**: This PR adds [sig-apps] identifier to relevant upgrade tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: xref #48839
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46738, 48827, 48831)
Moving disruption controller e2es to workload/
Based on #45301
Moving to track sig-apps in a single directory
cc @kubernetes/sig-contributor-experience-misc @kubernetes/sig-apps-misc @erictune @kow3ns @crimsonfaith91
Automatic merge from submit-queue
StatefulSet upgrade test - replicated database (mysql)
**What this PR does / why we need it**:
Adds a new upgrade test. The test creates a statefulset with a replicated mysql database. It populates the database and then continually polls the data while performing an upgrade.
Ultimately, this PR increases confidence of reliability during upgrades. It helps show that StatefulSets and Pod Disruption Budgets are doing what they're supposed to. Code to pay attention to this was added for #38336.
Also vendors in a golang mysql client driver, for use in the test.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48781, 48817, 48830, 48829, 48053)
Move kubectl e2e tests to their own directory and prefix the test nam…
```release-note
NONE
```
Automatic merge from submit-queue
Make storage e2e tests start with [sig-storage] instead of [k8s.io].
This makes understanding sig ownership from a test name very easy for
tools and humans.
- Use a SIGDescribe helper function that adds a [sig-storage] prefix instead of [k8s.io] for tests in storage/
- Move a test that should be in storage into storage.
- Make tests owned by multiple SIGs (configmap test) have [sig-storage] instead of [Volume] labels.
This means that all tests that sig-storage directly owns can be found with a simple regex.
/cc @kubernetes/sig-storage-pr-reviews
**What this PR does / why we need it**:
This will be used to make a testgrid dashboard for sig-storage.
**Release note**:
```release-note
NONE
```
Issue #48779
Automatic merge from submit-queue (batch tested with PRs 48594, 47042, 48801, 48641, 48243)
Prepare to introduce websockets for exec and portforward
Refactor the code in remotecommand to better represent the structure of
what is common between portforward and exec.
Ref #48633
Automatic merge from submit-queue (batch tested with PRs 48425, 41680, 48457, 48619, 48635)
[Federation] Remove redundant e2e
Now that federation of replicasets and deployments is implemented with the sync controller, the previous crud e2e duplicates coverage provided by the crudtester integration and e2e testing.
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 48405, 48742, 48748, 48571, 48482)
Remove unused parameter
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Move performance tests to test/e2e/scalability subdirectory
Following the move to make e2e tests more organized and accountable to the respective SIGs.
cc @kubernetes/sig-scalability-pr-reviews @wojtek-t @gmarek @grodrigues3
Automatic merge from submit-queue (batch tested with PRs 48698, 48712, 48516, 48734, 48735)
Name change: s/timstclair/tallclair/
I changed my name, and I'm migrating my user name to be consistent.
Automatic merge from submit-queue (batch tested with PRs 46865, 48661, 48598, 48658, 48614)
Move metrics_grabbert to test/e2e
cc @aleksandra-malinowska
Automatic merge from submit-queue
kube-apiserver: tests for aggregation and CRDs via delegation
In our integration tests we do not use the real kube-apiserver setup code, but mock our own. Here I use the actual `cmd/kube-apiserver/app.Run()` func with an testing etcd server. This can test the whole delegation chain of aggregator, apiextensions and kube-apiserver.
Automatic merge from submit-queue (batch tested with PRs 47232, 48625, 48613, 48567, 39173)
Include leaderelection in client-go;
Fix#39117
Fix https://github.com/kubernetes/client-go/issues/28
This PR:
* includes the leaderelection to the staging client-go
* to avoid conflict with golang's testing package, renames package /testing to /testutil, and renames cache/testing to cache/testframework
```release-note
client-go now includes the leaderelection package
```
Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)
remove dead code
This removes the dead code cruft since we stopped serving TPRs.
ref #48152
Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)
GCE: Use network project id for firewall/route mgmt and zone listing
- Introduces a new environment variable for plumbing the network project id which will be used for firewall and route management. fixes#48515
- onXPN is determined by metadata if config is not specified
- Split `if` conditions: fixes#48521
- Remove `getNetworkNameViaAPICall` which was used as a last resort for the `networkURL` (if empty) which was previously filled with the metadata network project & name.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Delete unused return
**What this PR does / why we need it**:
We do not use the function return, it's better not to write the return.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48374, 48524, 48519, 42548, 48615)
Add a node upgrade test for etcd statefulset with a PDB.
Tests for #38336
Automatic merge from submit-queue
Small fix for number of pods and nodes in test function
**What this PR does / why we need it**:
Small fix to have correct pods and nodes in test function.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Scheduler perf, nodes and pods number fix for 100 nodes 3k pods.
**Release note**:
```release-note
NONE
```
Now that federation of deployments is implemented with the sync
controller, the existing e2e coverage duplicates that provided by the
crudtester integration and e2e testing.
Now that replicaset federation is implemented with the sync
controller, the old replicaset-specific e2e tests duplicate coverage
provided by crudtester integration and e2e testing.
Automatic merge from submit-queue
Add ownership for the future of scheduler_perf and kubemark
**What this PR does / why we need it**:
The scheduler_perf project is cross-cutting with the other goals of the performance and scale initiatives, so, I've put together a list of interested parties who have been running, using, and contributing to it.
cc @kubernetes/sig-scheduling-pr-reviews @ravisantoshgudimetla @sjug
Automatic merge from submit-queue
Launch kubemark with an existing Kubemark master
In order to expand the use of kubemark, allow developers to use kubemark with a pre-existing Kubernetes cluster.
Ref issue #44393
Automatic merge from submit-queue
[e2e-ingress] Get node tag from instance under GKE
**What this PR does / why we need it**: Making ingress CI green again.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48167
**Special notes for your reviewer**:
/assign @nicksardo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47162, 48444, 48445)
Introducing a cluster-scoped resource in the wardle.k8s.io group.
**What this PR does / why we need it**:
This PR adds a cluster-scoped resource to the wardle.k8s.io group.
The cluster scoped resource has a field that indicates Flunder.Names that are disallowed.
The resource is going to be used by an admission plugin.
The admission plugin will list the cluster-scope resources and check against banned names.
**Special notes for your reviewer**:
Issue: #47868
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48480, 48353)
remove tpr api access
xref https://github.com/kubernetes/kubernetes/issues/48152
TPR tentacles go pretty deep. This gets us started by removing API access and we'll move down from there.
@kubernetes/sig-api-machinery-misc
@ironcladlou this should free up the GC implementation since TPRs will no longer be present and failing.
```release-note
Removing TPR api access per https://github.com/kubernetes/kubernetes/issues/48152
```
Automatic merge from submit-queue (batch tested with PRs 47043, 48448, 47515, 48446)
Fix secret/configmap/projected volume update tests to work for large clusters
Fixes https://github.com/kubernetes/kubernetes/issues/48359
/cc @kubernetes/sig-node-pr-reviews @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 46926, 48468)
Fix typo in cluster size autoscaling tests selector
This caused tests not to be run automatically
The cluster scoped resource has a field that indicates Flunder.Names that are disallowed.
The resource is going to be used by an admission plugin.
The admission plugin will list the cluster-scope resources and check against banned names.
Issue: #47868
Automatic merge from submit-queue (batch tested with PRs 47784, 47793, 48334, 48435, 48354)
Convert Stackdriver Logging load e2e tests to soak tests
Instead of loading logging mechanism for 10 minutes, load for 21 hours to detect regressions that require some time to build up.
Made possible by switching to pub/sub. Only merge after corresponding test suites have appropriate timeouts: https://github.com/kubernetes/test-infra/pull/3119
/cc @piosz @fgrzadkowski
Automatic merge from submit-queue (batch tested with PRs 47918, 47964, 48151, 47881, 48299)
Add ApiEndpoint support to GCE config.
**What this PR does / why we need it**:
Add the ability to change ApiEndpoint for GCE.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 48295, 48298, 47339, 44910, 48037)
Make Makefiles in `test/images/` compatible with multiple architectures
**What this PR does / why we need it**:
This PR is for making test images multi architecture for different platforms like amd64, arm, arm64, ppc64le
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #31331
**Special notes for your reviewer**:
- Actual tests need to be modified to use these images based on the architecture later.
- Not covering the cross building of docker images for `s390x` platform due to problem faced while running containers with `qemu-s390x-static`
- Will submit separate PR for `volume and pet` test images
- This PR depends on - https://github.com/kubernetes/ingress/pull/587
**Release note**:
```NONE```
Automatic merge from submit-queue (batch tested with PRs 46336, 47643)
Add node e2e tests for runAsUser
**What this PR does / why we need it**:
This PR adds node e2e tests for runAsUser.
**Which issue this PR fixes**
Part of #44118.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47850, 47835, 46197, 47250, 48284)
Do not fail on error when deleting ingress
Fixes#48239
If the api server or master is unavailable, the test should manually teardown load balancer resources.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47850, 47835, 46197, 47250, 48284)
Allocate clusterIP when change service type from ExternalName to ClusterIP
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#35354#46190
**Special notes for your reviewer**:
/cc @smarterclayton @thockin
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48214, 48154)
Adding a retry and traceroute to the master version checking
This is hitting a lot of connection refused errors in the e2e upgrade tests. We should make this more robust in case this is intermittent network errors. In the event of an error, attempt to log a traceroute to the master.
cc @kubernetes/sig-cluster-lifecycle-bugs @dchen1107
#47379
Automatic merge from submit-queue (batch tested with PRs 48168, 48199)
Fix some flakes in autoscaler e2e on gke
This PR should fix some of the flakes we found in e2e runs, while testing for 1.7 release:
- if one of the nodes is unschedulable in set up (causing set up to fail) we used to wait for wrong number of nodes in clean-up, adding unnecessary 20 minute wait to failing test
- we did not check for errors when creating RC in test, leading to tests failing later in hard to debug way (added retry loop and explicit test failure)
Automatic merge from submit-queue (batch tested with PRs 48004, 48205, 48130, 48207)
Add e2e tests for CA scale up when pending pod requests volume
Test verifying pending pods with PVC don't interfere with scale up, issue: kubernetes/autoscaler#22
Automatic merge from submit-queue
Changes node e2e tests to use the new Ubuntu image
ref: https://github.com/kubernetes/kubernetes/issues/46891
`ubuntu-docker10` and `ubuntu-docker12` images are deprecated in favor of the new one.
**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107
Automatic merge from submit-queue (batch tested with PRs 48118, 48159)
Ensures node becomes schedulable at the end of tests that delete nodes
**What this PR does / why we need it**: Further fixes the flakiness of "Pod Disk should be able to detach from a node which was deleted". When a node becomes ready but not schedulable, it was not included in the final node count.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48008
**Special notes for your reviewer**: Updated a similar test, "Pod Disk should be able to detach from a node whose api object was deleted", to use an "Expect" instead of a soft error because the test needs to guarantee that the environment is *completely* reset.
**Release note**:
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 45610, 47628)
Replace capacity with allocatable to calculate pod resource
It is not accurate to use capacity to do the calculation.
**What this PR does / why we need it**:
The currently cpu resource calculation for a pod in end2end test is incorrect.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes#47627
**Special notes for your reviewer**:
More details about capacity and allocatable:
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node-allocatable.md
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)
Pipe clusterID into gce_loadbalancer_external.go
**What this PR does / why we need it**: Small cleanup for GCE ELB codes.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48002
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Make big clusters work again after introduction of subnets
This PR does two things:
- make IP aliases automatically pick Node IP Range based on number of Nodes,
- fix logic for starting clusters >4095 Nodes that was broken by introduction of subnets,
cc @wojtek-t @shyamjvs
```release-note
Setting env var ENABLE_BIG_CLUSTER_SUBNETS=true will allow kube-up.sh to start clusters bigger that 4095 Nodes on GCE.
```
Ref https://github.com/kubernetes/kubernetes/issues/47344
Automatic merge from submit-queue (batch tested with PRs 48092, 47894, 47983)
Skip Deployment upgrade test on 1.5 and earlier.
The test relies on implementation details and would need a rewrite to work for older clusters.
xref #47685
Automatic merge from submit-queue (batch tested with PRs 48012, 47443, 47702, 47178)
Extending timeout waiting for delete node to become ready before the test ends.
**What this PR does / why we need it**: It seems to take longer than 5 minutes for the node to recover. Changing the timeout to 10 minutes.
This is an extension of PR #46746
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48008
/release-note-none
Automatic merge from submit-queue (batch tested with PRs 48074, 47971, 48044, 47514, 47647)
e2e: bump kubelet's resurce usage limit
We don't have per-OS image limits. Bumping these to more generous
numbers to not fail the tests.
Automatic merge from submit-queue (batch tested with PRs 47656, 47488)
Checked whether balanced Pods were created.
**What this PR does / why we need it**:
In #47382, it seems balanced Pods were not created; this PR will check whether balanced Pods were created, and show logs.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: partially fixes#47382
**Release note**:
```release-note-none
```
Automatic merge from submit-queue
Move the workload e2e tests to it's own package
**What this PR does / why we need it**:
Move the workload e2e tests to it's own package
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 47961, 46276)
Bumped Heapster to v1.4.0-beta.0
Heapster release candidate for Kubernetes 1.7
cc @dchen1107 @caesarxuchao
Automatic merge from submit-queue (batch tested with PRs 47650, 47936, 47939, 47986, 48006)
Remove e2e test that checked scheduler priority function for spreading of ReplicationController pods
**What this PR does / why we need it**:
Remove e2e test that checked scheduler priority function for spreading of ReplicationController pods.
The test is flaky in our test clusters, but passes in our local clusters. It looks like our e2e test clusters are not deterministic enough for checking all of the scheduler priority functions. Besides, this functionality is checked extensively by our unit tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47769
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47650, 47936, 47939, 47986, 48006)
[esipp-e2e] Change service port to avoid collision
**What this PR does / why we need it**: As https://github.com/kubernetes/kubernetes/issues/47745#issuecomment-310750499 indicates, changing service port in test to avoid collision.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47745
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Move e2e fromManifest funcs to manifest package
fixes https://github.com/kubernetes/kubernetes/issues/37007 again
The goal here is to remove e2e/framework's dependence on e2e/generated so that external projects can import e2e/framework without issue. I reorganize & try to make all the 'fromManifest' funcs consistent (to all use generated, return err) to prevent any reintroduction of generated into framework
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47993, 47892, 47591, 47469, 47845)
deprecate created-by annotation for cronjob
**What this PR does / why we need it**: This PR deprecates created-by annotation for cronjob. This is needed as we now have ControllerRef.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: xref #44407
**Special notes for your reviewer**: I will create 3 PRs to fix the issue as the annotation is used in various parts of the codebase: cronjob, pod drain, and e2e test framework. This is the first PR. Other PRs can be found here: #47471, #47475
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 47993, 47892, 47591, 47469, 47845)
Bump up npd version to v0.4.1
```
Bump up npd version to v0.4.1
```
Fixes#47219
Automatic merge from submit-queue (batch tested with PRs 47883, 47179, 46966, 47982, 47945)
Remove e2e test for least requested prioirty function
**What this PR does / why we need it**:
Deletes a flaky e2e test for "least requested" priority function of the scheduler. This priority function is tested more extensively in our unit tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47769
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47915, 47856, 44086, 47575, 47475)
deprecate created-by annotation for e2e test framework
**What this PR does / why we need it**: This PR deprecates created-by annotation for e2e test framework. This is needed as we now have ControllerRef.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: xref #44407
**Special notes for your reviewer**: This is the third PR for deprecating created-by annotation. Other PRs can be found here: #47469, #47471
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 47403, 46646, 46906, 46527, 46792)
[Federation] Convert the ReplicaSet controller to a sync controller.
See #45563 for previous discussion: that PR was split into two, with this one containing the actual conversion work and that one containing the implementation of the scheduling methods in the sync controller.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47403, 46646, 46906, 46527, 46792)
E2E:Delete unecessary check
**What this PR does / why we need it**:
Delete meaningless check
The deleted check is redundant.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47922, 47195, 47241, 47095, 47401)
Run cAdvisor on the same interface as kubelet
**What this PR does / why we need it**:
cAdvisor currently binds to all interfaces. Currently the only
solution is to use iptables to block access to the port. We
are better off making cAdvisor to bind to the interface that
kubelet uses for better security.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#11710
**Special notes for your reviewer**:
**Release note**:
```release-note
cAdvisor binds only to the interface that kubelet is running on instead of all interfaces.
```
Automatic merge from submit-queue
Remove initial resources e2e
This is just a cleanup PR. We won't continue initial resources effort. It is dropped in favor of VPA developed in kubernetes/autoscaler.
cc: @piosz @jszczepkowski @kgrygiel
Automatic merge from submit-queue (batch tested with PRs 47878, 47503, 47857)
restore working aggregator and avoid duplicate informers
Fixes https://github.com/kubernetes/kubernetes/issues/47866
This runs the informer all the way through and makes sure its started.
@lavalamp ptal
@kubernetes/sig-api-machinery-bugs
Automatic merge from submit-queue (batch tested with PRs 47851, 47824, 47858, 46099)
Revert "[Federation] Fix federated service reconcilation issue due to addition of External…"
Reverts kubernetes/kubernetes#45798
Reverting the temporary fix as the problem is fixed in #45869.
with that fix federation also can default ExternalTrafficLocalOnly if not set.
Issue: #45812
cc @MrHohn @madhusudancs @kubernetes/sig-federation-bugs
Automatic merge from submit-queue (batch tested with PRs 47851, 47824, 47858, 46099)
Revert 44714 manually
#44714 broke backward compatibility for old swagger spec that kubectl still uses. The decision on #47448 was to revert this change but the change was not automatically revertible. Here I semi-manually remove all references to UnixUserID and UnixGroupID and updated generated files accordingly.
Please wait for tests to pass then review that as there may still be tests that are failing.
Fixes#47448
Adding release note just because the original PR has a release note. If possible, we should remove both release notes as they cancel each other.
**Release note**: (removed by caesarxuchao)
UnixUserID and UnixGroupID is reverted back as int64 to keep backward compatibility.
Automatic merge from submit-queue
Add E2E tests for Azure internal loadbalancer support, fix an issue for public IP resource deletion.
**What this PR does / why we need it**:
- Add E2E tests for Azure internal loadbalancer support: https://github.com/kubernetes/kubernetes/pull/43510
- Fix an issue that public IP resource not get deleted when switching from external loadbalancer to internal static loadbalancer.
**Special notes for your reviewer**:
1. Add new Azure resource tag to Public IP resources to indicate kubernetes managed resources.
Currently we determine whether the public IP resource should be deleted by looking at LoadBalancerIp property on spec. In the scenario 'Switching from external loadbalancer to internal loadbalancer with static IP', that value might have been updated for internal loadbalancer. So here we're to add an explicit tag for kubernetes managed resources.
2. Merge cleanupPublicIP logic into cleanupLoadBalancer
**Release note**:
NONE
CC @brendandburns @colemickens
Automatic merge from submit-queue
Further reduce cluster-autoscaler e2e flakiness
Ref: https://github.com/kubernetes/autoscaler/issues/89
Add pdbs for additional kube-system pod, move adding
pdbs to separate function, as it will need to be reused
in new tests we're working on (ex. scale to 0).
Automatic merge from submit-queue
Added an e2e test timing HPA + CA scaling up from 1 to 8 pods and from 3 to >=4 clusters
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46847
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add new e2e test for cluster size autoscaler (evicting system pods)
This test verifies that cluster autoscaler drains nodes with system pods running if they have a PDB.
Automatic merge from submit-queue (batch tested with PRs 47726, 47693, 46909, 46812)
Plumb service resolver into webhook AC
This is the last piece of plumbing needed for https://github.com/kubernetes/features/issues/209
Automatic merge from submit-queue (batch tested with PRs 47726, 47693, 46909, 46812)
Additional e2e for StatefulSet Update
**What this PR does / why we need it**:
This PR adds additional e2e tests for StatefulSet update
fixes: #46942
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46327, 47166)
mark --network-plugin-dir deprecated for kubelet
**What this PR does / why we need it**:
**Which issue this PR fixes** : fixes#43967
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Fix flaky cluster-autoscaler e2e
Ref: https://github.com/kubernetes/autoscaler/issues/89
Add pdb to allow cluster-autoscaler to drain nodes with some kube-system components (turns out there can be enough of them to deny scale-down even with 5 healthy nodes). Increased scaleDownTimeout to take into account the time it will take to re-schedule pods running on broken node (this may reset scale-down timer).
Automatic merge from submit-queue (batch tested with PRs 47530, 47679)
Fix failing CassandraStatefulSet test in examples suite
Fix part of: https://github.com/kubernetes/kubernetes/issues/45677
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47530, 47679)
Use cos-stable-59-9460-64-0 instead of cos-beta-59-9460-20-0.
Remove dead code that has now moved to another repo as part of #47467
**Release note**:
```release-note
NONE
```
/sig node
- It contains a fix for ipaliasing.
- It contains a fix which decouples GPU driver installation from kernel
version.
Remove dead code that has now moved to another repo as part of #47467
Automatic merge from submit-queue
Fixed e2e test flake - ClusterDns - should create pod that uses dns
**What this PR does / why we need it**:
String replaced in this test for example pod yaml (dns-frontend-pod.yaml) file is incorrect.
**Which issue this PR fixes** :
fixes#45915
Internal attach/detach controller timers should be configurable and tests
should use much shorter values.
reconcilerSyncDuration is deliberately left out of TimerConfig because it's
the only one that's not a constant one, it's configurable by user.
Automatic merge from submit-queue
Add some debug info for deployment e2e testing
Add some debug info to printout all the ReplicaSets if there is no deployment object created, and add a enhancement to wait the pod to ready
**Release note**:
```
None
```
Automatic merge from submit-queue
Fix flaking cluster-autoscaler e2e
Ref: https://github.com/kubernetes/autoscaler/issues/89
Scale-down e2e require 5 healthy nodes to pass reliably, this PR should fix the flaking "should correctly scale down after a node is not needed and one node is broken" e2e.
Automatic merge from submit-queue (batch tested with PRs 47492, 47542, 46800, 47545, 45764)
delete dependent pods for rs when deleting deployments
Fix#44046, where user reported that the garbage collector didn't delete pods when a deployment was deleted with PropagationPolicy=Background.
Automatic merge from submit-queue (batch tested with PRs 47492, 47542, 46800, 47545, 45764)
separate group and version priority
Fixes https://github.com/kubernetes/kubernetes/issues/46322
This just modifies the API and does the minimal plumbing. I can extend this pull or do another to fix the priority problem.
Automatic merge from submit-queue
Don't test the debug /logs endpoint on GKE.
GKE will not enable the /logs endpoint in 1.7. I'd like this test to still test the other cluster level endpoints.
Automatic merge from submit-queue
Do not add unique label to DaemonSet
**What this PR does / why we need it**:
It's mainly for #46925. DaemonSet controller adds a unique label to DaemonSet, which is unexpected to federation.
The 1st commit addressed #46981 to construct history once and pass it around, so that we can avoid adding that unique label in DaemonSet in the 2nd commit. ~The 3rd commit just reverts the band-aid PR #47103.~
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46925, xref #46981
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47470, 47260, 47411, 46852, 46135)
Write reports for each upgrade test
Due to the way Ginkgo runs individual test cases and the level of coordination required for the upgrade tests, they were all run under a single Ginkgo test case. This PR generates and auxiliary report that break out the results of each upgrade test. This is accomplished by:
1) Wrapping `ginkgo.Fail` and `ginkgo.Skip` to get the actual failure or skip messages.
2) Recovering that info in the upgrade test to generate an auxiliary report.
I suggest reviewing commit by commit.
Sample report: https://storage.googleapis.com/krouseytestreports/logs/results/1/artifacts/junit_upgrades.xmlFixes: #47371
Automatic merge from submit-queue
test/kubemark/resources: configure custom etcd endpoints
We want to stress our own etcd cluster with Kubernetes
workloads, using kubemark e2e tests. This PR adds a new
environment variable 'ETCD_SERVERS' to configure custom
etcd endpoints.
/cc @xiang90 @hongchaodeng
Automatic merge from submit-queue (batch tested with PRs 47302, 47389, 47402, 47468, 47459)
Update to kube-addon-manager:v6.4-beta.2: kubectl v1.6.4 and refreshed base images
**What this PR does / why we need it**: refreshes base images for kube-addon-manager with fixes for CVE-2016-9841 and CVE-2016-9843.
x-ref https://github.com/kubernetes/kubernetes/issues/47386
**Special notes for your reviewer**: the updated images are not yet pushed, so tests will fail until that's done.
**Release note**:
```release-note
```
/assign @MrHohn
Automatic merge from submit-queue
Update GPU e2e tests.
* Use nvidia driver installer from external repo.
That installer decouples itself from COS image version (as long as the
image version is newer than cos-stable-59-9460-60-0).
A separate commit in the test-infra repo will update the cos version
used for this test to cos-stable-59-9460-60-0.
* Use cos-stable-59-9460-60-0 and newer installer for GPU node e2e tests.
This is to enable #47388.
This supercedes #47091.
**Release note**:
```release-note
NONE
```
/sig node
We want to stress our own etcd cluster with Kubernetes
workloads, using kubemark e2e tests. This PR adds a new
environment variable 'ETCD_SERVERS' to configure custom
etcd endpoints.
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
In 1.7, we add controller history to avoid unnecessary DaemonSet pod
restarts during pod adoption. We will not restart pods with matching
templateGeneration for backward compatibility, and will not restart pods
when template hash label matches current DaemonSet history, regardless
of templateGeneration.
Automatic merge from submit-queue (batch tested with PRs 47084, 46016, 46372)
Update adoption/release of DaemonSet controller history, and wait for history store sync
**What this PR does / why we need it**:
~Depends on #47075, so that DaemonSet controller can update history's controller ref. Ignore that commit when reviewing.~ (merged)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: #46981
**Special notes for your reviewer**: @kubernetes/sig-apps-bugs
**Release note**:
```release-note
NONE
```
That installer decouples itself from COS image version (as long as the
image version is newer than cos-stable-59-9460-60-0).
A separate commit in the test-infra repo will update the cos version
used for this test to cos-stable-59-9460-60-0.
Automatic merge from submit-queue (batch tested with PRs 46441, 43987, 46921, 46823, 47276)
Enable Node authorizer and NodeRestriction admission in kubemark
xref https://github.com/kubernetes/features/issues/279
We want to ensure scale testing covers use of the authorizer/admission pair that partitions nodes. This includes enabling the authorizer, which populates a graph of existing nodes and pods.
Kubemark is still running all nodes with a single credential, so a follow-up step is to generate unique credentials per node (or enable TLS bootstrapping) and remove the temporary rolebinding added in this PR so the node authorizer is the one authorizing each call by a hollow node.
Automatic merge from submit-queue
Shorten eviction tests, and increase test suite timeout
After #43590, the eviction manager is less aggressive when evicting pods. Because of that, many runs in the flaky suite time out.
To shorten the inode eviction test, I have lowered the eviction threshold.
To shorten the allocatable eviction test, I now set KubeReserved = NodeMemoryCapacity - 200Mb, so that any pod using 200Mb will be evicted. This shortens this test from 40 minutes, to 10 minutes.
While this should be enough to not hit the flaky suite timeout anymore, it is better to keep lower individual test timeouts than a lower suite timeout, since hitting the suite timeout means that even successful test runs are not reported.
/assign @Random-Liu @mtaufen
issue: #31362
Automatic merge from submit-queue
Change what is stored in DaemonSet history `.data`
**What this PR does / why we need it**:
In DaemonSet history `.data`, store a strategic merge patch that can be applied to restore a DaemonSet. Only PodSpecTemplate is saved.
This will become consistent with the data stored in StatefulSet history.
Before this fix, a serialized pod template is stored in `.data`; however, seriazlized pod template isn't a `runtime.RawExtension`, and caused problems when controllers try to patch the history's controller ref.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47008
**Special notes for your reviewer**: @kubernetes/sig-apps-bugs @erictune @kow3ns @kargakis @lukaszo @mengqiy
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Fix bad check in node e2e tests for GPUs.
When no nvidia device was attached, the -ne check had a syntax error:
sh: -ne: argument expected
This resulted in `Success` being echoed and the test passing incorrectly.
This was found while debugging issue #47216
/release-note-none
/sig node
/area node-e2e
/kind bug
When no nvidia device was attached, the -ne check had a syntax error:
sh: -ne: argument expected
This resulted in 'Success' being echoed and the test passing incorrectly.
This was found while debugging issue #47216
Automatic merge from submit-queue (batch tested with PRs 46750, 47141)
Speed up volume integration test
Partly solves https://github.com/kubernetes/kubernetes/issues/47129 .
On my local box:
before - 7m56.751s
after - 5m53.132s
So approx. 2m time saving. More saving will require refactoring of attach detach controller.
cc @mikedanese
Automatic merge from submit-queue (batch tested with PRs 36376, 47251)
client-go: GetOptions for dynamic client
Looks like `GetOptions` were forgotten in the dynamic client. Without them it's hard to write a dynamic initializer controller (useful for custom resources).
Automatic merge from submit-queue (batch tested with PRs 47113, 46665, 47189)
Improve the e2e node restart test
This commit includes the following two changes:
* Move pre-test checks (pods/nodes ready) to BeforeEach() so that it's
clear whether the test has run or not.
* Dumping logs for unready pods.
Automatic merge from submit-queue
Kubelet: rename cri package name to pkg/kubelet/apis/cri/v1alpha1/runtime
**What this PR does / why we need it**:
We have moved CRI from api/v1alpha1/runtime to apis/cri/v1alpha1, which changed the package name of CRI. This would cause a significant problem: old-versioned runtime (based on CRI in v1.6) doesn't work with latest kubelet v1.7, and vice versa.
This PR renames cri package name to `pkg/kubelet/apis/cri/v1alpha1/runtime` for fixing the problem.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes#47012
**Special notes for your reviewer**:
Should be included in v1.7.
**Release note**:
```release-note
CRI has been moved to package `pkg/kubelet/apis/cri/v1alpha1/runtime`.
```
Automatic merge from submit-queue
Move the nvidia installer to the beginning.
When the installer runs for the first time, it disables loadpin and restarts
the node. So, it is better to run it in the beginning so that we can avoid
redoing the later steps. One of the later steps include downloading a tar file
and untarring it. Doing that only once saves around 1m30s in test runtime for
the gci image.
/sig node
/area node-e2e
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46835, 46856)
Made tests that create Horizontal Pod Autoscaler delete it after they are done.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46847
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46835, 46856)
Made WaitForReplicas and EnsureDesiredReplicas use PollImmediate and improved logging.
**What this PR does / why we need it**: Most importantly, this results in better logging: timeout is logged at the level of the caller, not the helper function, helping debugging.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Allow pods to opt out of PodPreset mutation via an annotation on the pod
An annotation in the pod spec of the form:
podpreset.admission.kubernetes.io/PodPresetOptOut: "true"
Will cause the admission controller to skip manipulating the pod spec,
no matter the labelling.
This is an alternative implementation to pull #44163.
```release-note
Allow pods to opt out of PodPreset mutation via an annotation on the pod.
```
Automatic merge from submit-queue (batch tested with PRs 46885, 47197)
Fix e2e ns deletion message for flake analysis
**What this PR does / why we need it**:
Let's us know when pods have a missing deletion timestamp.
**Special notes for your reviewer**:
helps https://github.com/kubernetes/kubernetes/issues/47135
Automatic merge from submit-queue (batch tested with PRs 46885, 47197)
Let COS docker validation node test against gci-next-canary
**What this PR does / why we need it**:
This is for COS docker validation node test. We plan to use family gci-next-canary in container-vm-image-staging for future Docker upgration and validation.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47134
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cAdvisor currently binds to all interfaces. Currently the only
solution is to use iptables to block access to the port. We
are better off making cAdvisor to bind to the interface that
kubelet uses for better security.
Fixes#11710
Automatic merge from submit-queue (batch tested with PRs 47065, 47157, 47143)
Use actual hostname when creating network e2e test pod
**What this PR does / why we need it**:
This changes a e2e framework network test Pod use the actual hostname value to match the `kubernetes.io/hostname` label in it's `NodeSelector`. Currently it assumes the Node name will match that hostname label which is not true in all environments.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Fixescoreos/tectonic-installer#1018
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47065, 47157, 47143)
Removed a race condition from ResourceConsumer
**What this PR does / why we need it**: Without this PR there is a race condition in ResourceConsumer that sometimes results in communication to pods that might not exist anymore.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47127
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
When the installer runs for the first time, it disables loadpin and restarts
the node. So, it is better to run it in the beginning so that we can avoid
redoing the later steps. One of the later steps include downloading a tar file
and untarring it. Doing that only once saves around 1m30s in test runtime for
the gci image.
Automatic merge from submit-queue
Bump up npd version to v0.4.0
Fixes#47070.
Bump up npd version to [v0.4.0](https://github.com/kubernetes/node-problem-detector/releases/tag/v0.4.0).
```release-note
Bump up Node Problem Detector version to v0.4.0, which added support of parsing log from /dev/kmsg and ABRT.
```
/cc @dchen1107 @ajitak
Automatic merge from submit-queue
Bump external provisioner image to smaller version
The image is roughly half as big so this should improve speed/flakiness maybe
-->
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46979, 47078, 47138, 46916)
DeleteCollection should include uninitialized resources
Users who delete a collection expect all resources to be deleted, and
users can also delete an uninitialized resource. To preserve this
expectation, DeleteCollection selects all resources regardless of
initialization.
The namespace controller should list uninitialized resources in order to
gate cleanup of a namespace.
Fixes#47137
Automatic merge from submit-queue (batch tested with PRs 46979, 47078, 47138, 46916)
[federation][e2e] Fix cleanupServiceShardLoadBalancer
**What this PR does / why we need it**:
Fixes the issue mentioned in #46976
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46976
**Special notes for your reviewer**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45877, 46846, 46630, 46087, 47003)
update NetworkPolicy e2e test for v1 semantics
This makes the NetworkPolicy test at least correct for v1, although ideally we'll eventually add a few more tests... (So this covers about half of #46625.)
I've tested that this compiles, but not that it passes, since I don't have a v1-compatible NetworkPolicy implementation yet...
@caseydavenport @ozdanborne, maybe you're closer to having a testable plugin than I am?
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47024, 47050, 47086, 47081, 47013)
client-go: deprecate TPR example and add CRD example
/cc @nilebox
Part of https://github.com/kubernetes/kubernetes/issues/46702
Users who delete a collection expect all resources to be deleted, and
users can also delete an uninitialized resource. To preserve this
expectation, DeleteCollection selects all resources regardless of
initialization.
The namespace controller should list uninitialized resources in order to
gate cleanup of a namespace.
Automatic merge from submit-queue (batch tested with PRs 43005, 46660, 46385, 46991, 47103)
[gke-slow always fails] Defer DeleteGCEStaticIP before asserting error
From https://github.com/kubernetes/kubernetes/issues/46918.
I'm getting close to the root cause: During tests, CreateGCEStaticIP() in fact successfully created static IP, but the parser we wrote in test mistakenly think we failed, probably because the gcloud output format was changed recently (or not). I'm still looking into fixing that.
This PR defer the delete function before asserting the error so that we can stop consistently leaking static IP in every run.
/assign @krzyzacy @dchen1107
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43005, 46660, 46385, 46991, 47103)
add e2e node test for Pod hostAliases feature
**What this PR does / why we need it**: adds node e2e test for #45148
tests requested in https://github.com/kubernetes/kubernetes/issues/43632#issuecomment-298434125
**Release note**:
```release-note
NONE
```
@yujuhong @thockin
Automatic merge from submit-queue
Federation: create loadbalancer service in tests only if test depends on it
**What this PR does / why we need it**:
Creating LoadBalancer type of service for every test case is kind of expensive and time consuming to provision. So this PR changes the test cases to use LoadBalancer type services only when necessary.
**Which issue this PR fixes** (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes#47068
**Release note**:
```release-note
NONE
```
cc @kubernetes/sig-federation-pr-reviews
/assign @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 46977, 47005, 47018, 47061, 46809)
Directly grab map values instead of using loop-clause variables when setting up federated sync controller tests.
Go's loop-clause variables are allocated once and the items are copied to that variable while iterating through the loop. This means, these variables can't escape the scope since closures are bound to loop-clause variables whose value change during each iteration. Doing so would lead to undesired behavior. For more on this topic see: https://github.com/golang/go/wiki/CommonMistakes
So in order to workaround this problem in sync controller e2e tests, we iterate through the map and copy the map value to a variable inside the loop before using it in closures.
Fixes issue: #47059
**Release note**:
```release-note
NONE
```
/assign @marun @shashidharatd @perotinus
cc @csbell @nikhiljindal
/sig federation
Automatic merge from submit-queue (batch tested with PRs 46977, 47005, 47018, 47061, 46809)
Fix for cluster-autoscaler e2e failures
This may help with cluster-autoscaler e2e failing in setup if the tests are run before all machines in mig get fully ready.
Automatic merge from submit-queue (batch tested with PRs 46235, 44786, 46833, 46756, 46669)
implements StatefulSet update
**What this PR does / why we need it**:
1. Implements rolling update for StatefulSets
2. Implements controller history for StatefulSets.
3. Makes StatefulSet status reporting consistent with DaemonSet and ReplicaSet.
https://github.com/kubernetes/features/issues/188
**Special notes for your reviewer**:
**Release note**:
```release-note
Implements rolling update for StatefulSets. Updates can be performed using the RollingUpdate, Paritioned, or OnDelete strategies. OnDelete implements the manual behavior from 1.6. status now tracks
replicas, readyReplicas, currentReplicas, and updatedReplicas. The semantics of replicas is now consistent with DaemonSet and ReplicaSet, and readyReplicas has the semantics that replicas did prior to this release.
```
Automatic merge from submit-queue (batch tested with PRs 46235, 44786, 46833, 46756, 46669)
Fixed ResourceConsumer.CleanUp to properly clean up non-replication-controller resources and pods
**What this PR does / why we need it**: Without this fix CleanUp does not remove non-replication-controller resources and pods. This leads to pollution that in some cases inadvertently affects what is happening in AfterEachs before the namespace gets deleted.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46997, 47021)
Don't parse human-readable output from gcloud in tests
This is the reason `[k8s.io] Services should be able to change the type and ports of a service [Slow]` is currently failing on GKE e2e tests. For GKE jobs we run a prerelease version of gcloud, in which the default command output was changed.
gcloud's default output for commands is human readable, and is subject to change. Anything scripting against gcloud should always pass `--format=json|yaml|value(...)` so you get standardized output.
fixes: #46918
Automatic merge from submit-queue (batch tested with PRs 47083, 44115, 46881, 47082, 46577)
Add an e2e test for server side get
Print a better error from the response. Performs validation to ensure it
does not regress in alpha state.
This is tests and bug fixes for https://github.com/kubernetes/community/pull/363
@kubernetes/sig-api-machinery-pr-reviews
Implements history utilities for ControllerRevision in the controller/history package
StatefulSetStatus now has additional fields for consistency with DaemonSet and Deployment
StatefulSetStatus.Replicas now represents the current number of createdPods and StatefulSetStatus.ReadyReplicas is the current number of ready Pods
Automatic merge from submit-queue (batch tested with PRs 46718, 46828, 46988)
Update docs/ links to point to main site
**What this PR does / why we need it**:
This updates various links to either point to kubernetes.io or to the kubernetes/community repo instead of the legacy docs/ tree in k/k
Pre-requisite for #46813
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-docs-maintainers @chenopis @ahmetb @thockin
Go's loop-clause variables are allocated once and the items are copied
to that variable while iterating through the loop. This means, these
variables can't escape the scope since closures are bound to loop-clause
variables whose value change during each iteration. Doing so would lead
to undesired behavior. For more on this topic see:
https://github.com/golang/go/wiki/CommonMistakes
So in order to workaround this problem in sync controller e2e tests, we
iterate through the map and copy the map value to a variable inside the
loop before using it in closures.
Fixes issue: #47059
This commit includes the following two changes:
* Move pre-test checks (pods/nodes ready) to BeforeEach() so that it's
clear whether the test has run or not.
* Dumping logs for unready pods.
Automatic merge from submit-queue (batch tested with PRs 46897, 46899, 46864, 46854, 46875)
Wait for cloud-init to finish before starting tests.
This fixes#46889.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Avoid double printing output of gcloud commands in kubemark
Just noticed we were unnecessarily echoing the result again.
/cc @wojtek-t
Automatic merge from submit-queue
Add EmptyDir volume capacity isolation
This PR adds the support for isolating the emptyDir volume use. If user
sets a size limit for emptyDir volume, kubelet's eviction manager monitors its usage
and evict the pod if the usage exceeds the limit.
This feature is part of local storage capacity isolation and described in the proposal kubernetes/community#306
**Release note**:
```release-note
Alpha feature: allows users to set storage limit to isolate EmptyDir volumes. It enforces the limit by evicting pods that exceed their storage limits
```
Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)
Added Bind method to Scheduler Extender
- only one extender can support the bind method
- if an extender supports bind, scheduler delegates the pod binding to the extender
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#41235
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)
Multizone static pv test
**What this PR does / why we need it**:
Adds an e2e test for checking that pods get scheduled to the same zone as statically created PVs. This tests the PersistentVolumeLabel admission controller, which adds zone and region labels when PVs are created. As part of this, I also had to make changes to volume test utility code to pass in a zone parameter for creating PDs, and also had to add an argument to the e2e test program to accept a list of zones.
Fixes#46995
**Special notes for your reviewer**:
It's probably easier to review each commit separately.
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
[GCE] Support internal load balancers
**What this PR does / why we need it**:
Allows users to expose K8s services externally of the K8s cluster but within their GCP network.
Fixes#33483
**Important User Notes:**
- This is a beta feature. ILB could be enabled differently in the future.
- Requires nodes having version 1.7.0+ (ILB requires health checking and a health check endpoint on kube-proxy has just been exposed)
- This cannot be used for intra-cluster communication. Do not call the load balancer IP from a K8s node/pod.
- There is no reservation system for private IPs. You can specify a RFC 1918 address in `loadBalancerIP` field, but it could be lost to another VM or LB if service settings are modified.
- If you're running an ingress, your existing loadbalancer backend service must be using BalancingMode type `RATE` - not `UTILIZATION`.
- Option 1: With a 1.5.8+ or 1.6.4+ version master, delete all your ingresses, and re-create them.
- Option 2: Migrate to a new cluster running 1.7.0. Considering ILB requires nodes with 1.7.0, this isn't a bad idea.
- Option 3: Possible migration opportunity, but use at your own risk. More to come later.
**Reviewer Notes**:
Several files were renamed, so github thinks ~2k lines have changed. Review commits one-by-one to see the actual changes.
**Release note**:
```release-note
Support creation of GCP Internal Load Balancers from Service objects
```
Handle failure cases on startup gracefully to avoid causing cascading
errors and poor initialization in other components. Initial errors from
config load cause the initializer to pause and hold requests. Return
typed errors to better communicate failures to clients.
Add code to handle two specific cases - admin wants to bypass
initialization defaulting, and mirror pods (which want to bypass
initialization because the kubelet owns their lifecycle).
This PR adds two features:
1. add support for isolating the emptyDir volume use. If user
sets a size limit for emptyDir volume, kubelet's eviction manager
monitors its usage
and evict the pod if the usage exceeds the limit.
2. add support for isolating the local storage for container overlay. If
the container's overly usage exceeds the limit defined in container
spec, eviction manager will evict the pod.
An annotation in the pod spec of the form:
podpreset.admission.kubernetes.io/exclude: "true"
Will cause the admission controller to skip manipulating the pod spec,
no matter the labelling.
The annotation for a podpreset acting on a pod has also been slightly
modified to contain a podpreset prefix:
podpreset.admission.kubernetes.io/podpreset-{name} = resource version
Fixes#44161
Automatic merge from submit-queue (batch tested with PRs 45871, 46498, 46729, 46144, 46804)
PD e2e test: Ready node check now uses the most up-to-date node count.
Follow-up to PR #46746
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
Automatic merge from submit-queue
Respect PDBs during node upgrades and add test coverage to the ServiceTest upgrade test.
This is still a WIP... needs to be squashed at least, and I don't think it's currently passing until I increase the scale of the RC, but please have a look at the general outline. Thanks!
Fixes#38336
@kow3ns @bdbauer @krousey @erictune @maisem @davidopp
```
On GCE, node upgrades will now respect PodDisruptionBudgets, if present.
```
Automatic merge from submit-queue (batch tested with PRs 46681, 46786, 46264, 46680, 46805)
Enable Dialer on the Aggregator
Centralize the creation of the dialer during startup.
Have the dialer then passed in to both APIServer and Aggregator.
Aggregator the uses the dialer as its Transport base.
**What this PR does / why we need it**:Enables the Aggregator to use the Dialer/SSHTunneler to connect to the user-apiserver.
**Which issue this PR fixes** : fixes ##46679
**Special notes for your reviewer**:
**Release note**: None
Automatic merge from submit-queue
Implement Daemonset history
~Depends on #45867 (the 1st commit, ignore it when reviewing)~ (already merged)
Ref https://github.com/kubernetes/community/pull/527/ and https://github.com/kubernetes/community/pull/594
@kubernetes/sig-apps-api-reviews @kubernetes/sig-apps-pr-reviews @erictune @kow3ns @lukaszo @kargakis
---
TODOs:
- [x] API changes
- [x] (maybe) Remove rollback subresource if we decide to do client-side rollback
- [x] deployment controller
- [x] controller revision
- [x] owner ref (claim & adoption)
- [x] history reconstruct (put revision number, hash collision avoidance)
- [x] de-dup history and relabel pods
- [x] compare ds template with history
- [x] hash labels (put it in controller revision, pods, and maybe deployment)
- [x] clean up old history
- [x] Rename status.uniquifier when we reach consensus in #44774
- [x] e2e tests
- [x] unit tests
- [x] daemoncontroller_test.go
- [x] update_test.go
- [x] ~(maybe) storage_test.go // if we do server side rollback~
kubectl part is in #46144
---
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46620, 46732, 46773, 46772, 46725)
Fix AppArmor test for docker 1.13
... & better debugging.
The issue is that we run the pod containers in a shared PID namespace with docker 1.13, so PID 1 is no longer the container's root process. Since it's messy to get the container's root process, I switched to using `/proc/self` to read the apparmor profile. While this wouldn't catch a regression that caused only the init process to run with the wrong profile, I think it's a good approximation.
/cc @aulanov @Amey-D
Automatic merge from submit-queue (batch tested with PRs 46620, 46732, 46773, 46772, 46725)
Added missing documentation to NodeInstanceGroup.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Add initializer support to admission and uninitialized filtering to rest storage
Initializers are the opposite of finalizers - they allow API clients to react to object creation and populate fields prior to other clients seeing them.
High level description:
1. Add `metadata.initializers` field to all objects
2. By default, filter objects with > 0 initializers from LIST and WATCH to preserve legacy client behavior (known as partially-initialized objects)
3. Add an admission controller that populates .initializer values per type, and denies mutation of initializers except by certain privilege levels (you must have the `initialize` verb on a resource)
4. Allow partially-initialized objects to be viewed via LIST and WATCH for initializer types
5. When creating objects, the object is "held" by the server until the initializers list is empty
6. Allow some creators to bypass initialization (set initializers to `[]`), or to have the result returned immediately when the object is created.
The code here should be backwards compatible for all clients because they do not see partially initialized objects unless they GET the resource directly. The watch cache makes checking for partially initialized objects cheap. Some reflectors may need to change to ask for partially-initialized objects.
```release-note
Kubernetes resources, when the `Initializers` admission controller is enabled, can be initialized (defaulting or other additive functions) by other agents in the system prior to those resources being visible to other clients. An initialized resource is not visible to clients unless they request (for get, list, or watch) to see uninitialized resources with the `?includeUninitialized=true` query parameter. Once the initializers have completed the resource is then visible. Clients must have the the ability to perform the `initialize` action on a resource in order to modify it prior to initialization being completed.
```
Automatic merge from submit-queue
Add local storage (scratch space) allocatable support
This PR adds the support for allocatable local storage (scratch space).
This feature is only for root file system which is shared by kubernetes
componenets, users' containers and/or images. User could use
--kube-reserved flag to reserve the storage for kube system components.
If the allocatable storage for user's pods is used up, some pods will be
evicted to free the storage resource.
This feature is part of local storage capacity isolation and described in the proposal https://github.com/kubernetes/community/pull/306
**Release note**:
```release-note
This feature exposes local storage capacity for the primary partitions, and supports & enforces storage reservation in Node Allocatable
```
Automatic merge from submit-queue (batch tested with PRs 46239, 46627, 46346, 46388, 46524)
Dynamic webhook admission control plugin
Unit tests pass.
Needs plumbing:
* [ ] service resolver (depends on @wfender PR)
* [x] client cert (depends on ????)
* [ ] hook source (depends on @caesarxuchao PR)
Also at least one thing will need to be renamed after Chao's PR merges.
```release-note
Allow remote admission controllers to be dynamically added and removed by administrators. External admission controllers make an HTTP POST containing details of the requested action which the service can approve or reject.
```
Automatic merge from submit-queue (batch tested with PRs 46239, 46627, 46346, 46388, 46524)
move labels to components which own the APIs
During the apimachinery split in 1.6, we accidentally moved several label APIs into apimachinery. They don't belong there, since the individual APIs are not general machinery concerns, but instead are the concern of particular components: most commonly the kubelet. This pull moves the labels into their owning components and out of API machinery.
@kubernetes/sig-api-machinery-misc @kubernetes/api-reviewers @kubernetes/api-approvers
@derekwaynecarr since most of these are related to the kubelet
Add support for creating resources that are not immediately visible to
naive clients, but must first be initialized by one or more privileged
cluster agents. These controllers can mark the object as initialized,
allowing others to see them.
Permission to override initialization defaults or modify an initializing
object is limited per resource to a virtual subresource "RESOURCE/initialize"
via RBAC.
Initialization is currently alpha.
Automatic merge from submit-queue
[Flaky PR Test] Fix summary test
fixes issue: #46797
As we can see in the [example failure build log](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-node-kubelet/4319/build-log.txt), the summary containers are pinging google 100s of times a second. This causes the summary container to be killed occasionally, and fail the test. The summary containers are only supposed to ping every 10 seconds according to the current test. As it turns out, we were missing a semicolon, and were not sleeping between pings. For background, we ping google to generate network traffic, so that the summary test can validate network metrics.
This PR adds the semicolon to make the container sleep between calls, and decreases the sleep time from 10 seconds to 1 second, as 1 call / 10 seconds did not produce enough activity.
cc @kubernetes/kubernetes-build-cops @dchen1107
Automatic merge from submit-queue (batch tested with PRs 46648, 46500, 46238, 46668, 46557)
Add an e2e test for AdvancedAuditing
Enable a simple "advanced auditing" setup for e2e tests running on GCE, and add an e2e test that creates & deletes a pod, a secret, and verifies that they're audited.
Includes https://github.com/kubernetes/kubernetes/pull/46548
For https://github.com/kubernetes/features/issues/22
/cc @ericchiang @sttts @soltysh @ihmccreery
Automatic merge from submit-queue (batch tested with PRs 46648, 46500, 46238, 46668, 46557)
Support validating package versions in node conformance test
**What this PR does / why we need it**:
This PR adds a package validator in node conformance test for checking whether the locally installed packages meet the image spec.
**Special notes for your reviewer**:
The image spec for GKE (which has the package spec) will be in a separate PR. Then we will publish a new node conformance test image for GKE whose name should use the convention in https://github.com/kubernetes/kubernetes/issues/45760 and have `gke` in it.
**Release note**:
```
NONE
```
Respect PDBs during node upgrades and add test coverage to the
ServiceTest upgrade test. Modified that test so that we include pod
anti-affinity constraints and a PDB.
This PR adds the check for local storage request when admitting pods. If
the local storage request exceeds the available resource, pod will be
rejected.
This PR adds the support for allocatable local storage (scratch space).
This feature is only for root file system which is shared by kubernetes
componenets, users' containers and/or images. User could use
--kube-reserved flag to reserve the storage for kube system components.
If the allocatable storage for user's pods is used up, some pods will be
evicted to free the storage resource.
Automatic merge from submit-queue (batch tested with PRs 43505, 45168, 46439, 46677, 46623)
Add TPR to CRD migration helper.
This is a helper for migrating TPR data to CustomResource. It's rather hacky because it requires crossing apiserver boundaries, but doing it this way keeps the mess contained to the TPR code, which is scheduled for deletion anyway.
It's also not completely hands-free because making it resilient enough to be completely automated is too involved to be worth it for an alpha-to-beta migration, and would require investing significant effort to fix up soon-to-be-deleted TPR code. Instead, this feature will be documented as a best-effort helper whose results should be verified by hand.
The intended benefit of this over a totally manual process is that it should be possible to copy TPR data into a CRD without having to tear everything down in the middle. The process would look like this:
1. Upgrade to k8s 1.7. Nothing happens to your TPRs.
1. Create CRD with group/version and resource names that match the TPR. Still nothing happens to your TPRs, as the CRD is hidden by the overlapping TPR.
1. Delete the TPR. The TPR data is converted to CustomResource data, and the CRD begins serving at the same REST path.
Note that the old TPR data is left behind by this process, so watchers should not receive DELETE events. This also means the user can revert to the pre-migration state by recreating the TPR definition.
Ref. https://github.com/kubernetes/kubernetes/issues/45728
Centralize the creation of the dialer during startup.
Have the dialer then passed in to both APIServer and Aggregator.
Aggregator the sets the dialer on its Transport base.
This should allow the SSTunnel to be used but also allow the Aggregation
Auth to work with it.
Depending on Environment InsecureSkipTLSVerify *may* need to be set to
true.
Fixed as few tests to call CreateDialer as part of start-up.
Automatic merge from submit-queue (batch tested with PRs 46076, 43879, 44897, 46556, 46654)
Local storage plugin
**What this PR does / why we need it**:
Volume plugin implementation for local persistent volumes. Scheduler predicate will direct already-bound PVCs to the node that the local PV is at. PVC binding still happens independently.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Part of #43640
**Release note**:
```
Alpha feature: Local volume plugin allows local directories to be created and consumed as a Persistent Volume. These volumes have node affinity and pods will only be scheduled to the node that the volume is at.
```
Automatic merge from submit-queue
Node authorizer
This PR implements the authorization portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and kubernetes/features#279:
* Adds a new authorization mode (`Node`) that authorizes requests from nodes based on a graph of related pods,secrets,configmaps,pvcs, and pvs:
* Watches pods, adds edges (secret -> pod, configmap -> pod, pvc -> pod, pod -> node)
* Watches pvs, adds edges (secret -> pv, pv -> pvc)
* When both Node and RBAC authorization modes are enabled, the default RBAC binding that grants the `system:node` role to the `system:nodes` group is not automatically created.
* Tightens the `NodeRestriction` admission plugin to require identifiable nodes for requests from users in the `system:nodes` group.
This authorization mode is intended to be used in combination with the `NodeRestriction` admission plugin, which limits the pods and nodes a node may modify. To enable in combination with RBAC authorization and the NodeRestriction admission plugin:
* start the API server with `--authorization-mode=Node,RBAC --admission-control=...,NodeRestriction,...`
* start kubelets with TLS boostrapping or with client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>`
```release-note
kube-apiserver: a new authorization mode (`--authorization-mode=Node`) authorizes nodes to access secrets, configmaps, persistent volume claims and persistent volumes related to their pods.
* Nodes must use client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>` in order to be authorized by the node authorizer (the credentials obtained by the kubelet via TLS bootstrapping satisfy these requirements)
* When used in combination with the `RBAC` authorization mode (`--authorization-mode=Node,RBAC`), the `system:node` role is no longer automatically granted to the `system:nodes` group.
```
```release-note
RBAC: the automatic binding of the `system:node` role to the `system:nodes` group is deprecated and will not be created in future releases. It is recommended that nodes be authorized using the new `Node` authorization mode instead. Installations that wish to continue giving all members of the `system:nodes` group the `system:node` role (which grants broad read access, including all secrets and configmaps) must create an installation-specific ClusterRoleBinding.
```
Follow-up:
- [ ] enable e2e CI environment with admission and authorizer enabled (blocked by kubelet TLS bootstrapping enablement in https://github.com/kubernetes/kubernetes/pull/40760)
- [ ] optionally enable this authorizer and admission plugin in kubeadm
- [ ] optionally enable this authorizer and admission plugin in kube-up
Automatic merge from submit-queue
Switch gcloud compute copy-files to scp
gcloud is deprecating `gcloud compute copy-files` and switching to `gcloud compute scp`. Make the change before things start to break.
https://cloud.google.com/sdk/gcloud/reference/compute/copy-files
Warnings we get: `W0529 10:28:59.097] WARNING: `gcloud compute copy-files` is deprecated. Please use `gcloud compute scp` instead. Note that `gcloud compute scp` does not have recursive copy on by default. To turn on recursion, use the `--recurse` flag.`
/cc @jlowdermilk
Automatic merge from submit-queue
Support grabbing test suite metrics
**What this PR does / why we need it**:
Add support for grabbing metrics that cover the entire test suite's execution.
Update the "interesting" controller-manager metrics to match the
current names for the garbage collector, and add namespace controller
metrics to the list.
If you enable `--gather-suite-metrics-at-teardown`, the metrics file is written to a file with a name such as `MetricsForE2ESuite_2017-05-25T20:25:57Z.json` in the `--report-dir`. If you don't specify `--report-dir`, the metrics are written to the test log output.
I'd like to enable this for some of the `pull-*` CI jobs, which will require a separate PR to test-infra.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-testing-pr-reviews @smarterclayton @wojtek-t @gmarek @derekwaynecarr @timothysc
Automatic merge from submit-queue (batch tested with PRs 45534, 37212, 46613, 46350)
[e2e]Fix define redundant parameter
When timeout to reach HTTP service, redundant parameter make the
error is nil.
Automatic merge from submit-queue
no-snat test
This test checks that Pods can communicate with each other in the same cluster without SNAT.
I intend to create a job that runs this in small clusters (\~3 nodes) at a low frequency (\~once per day) so that we have a signal as we work on allowing multiple non-masquerade CIDRs to be configured (see [kubernetes-incubator/ip-masq-agent](https://github.com/kubernetes-incubator/ip-masq-agent), for example).
/cc @dnardo
Automatic merge from submit-queue
Added k82cn as one of scheduler approver.
According to the requirement of Approver at [community-membership.md](https://github.com/kubernetes/community/blob/master/community-membership.md), I meet the requirements as follow; so I'd like to add myself as an approver of scheduler.
* Reviewer of the codebase for at least 3 months
[k82cn]: [~3 months](6cc40678b6 )
* Primary reviewer for at least 10 substantial PRs to the codebase
[k82cn] Reviewed [40 PRs](https://github.com/issues?q=assignee%3Ak82cn+is%3Aclosed)
* Reviewed or merged at least 30 PRs to the codebase
[k82cn]: 71 merged PRs in kubernetes/kubernetes, and ~100 PRs in kuberentes at https://goo.gl/j2D1fR
As an approver,
* I agree to only approve familiar PRs
* I agree to be responsive to review/approve requests as per community expectations
* I agree to continue my reviewer work as per community expectations
* I agree to continue my contribution, e.g. PRs, mentor contributors
Automatic merge from submit-queue (batch tested with PRs 46407, 46457)
GCE - Refactor API for firewall and backend service creation
**What this PR does / why we need it**:
- Currently, firewall creation function actually instantiates the firewall object; this is inconsistent with the rest of GCE api calls. The API normally gets passed in an existing object.
- Necessary information for firewall creation, (`computeHostTags`,`nodeTags`,`networkURL`,`subnetworkURL`,`region`) were private to within the package. These now have public getters.
- Consumers might need to know whether the cluster is running on a cross-project network. A new `OnXPN` func will make that information available.
- Backend services for regions have been added. Global ones have been renamed to specify global.
- NamedPort management of instance groups has been changed from an `AddPortsToInstanceGroup` func (and missing complementary `Remove...`) to a single, simple `SetNamedPortsOfInstanceGroup`
- Addressed nitpick review comments of #45524
ILB needs the regional backend services and firewall refactor. The ingress controller needs the new `OnXPN` func to decide whether to create a firewall.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Move NetworkPolicy to v1
Move NetworkPolicy to v1
@kubernetes/sig-network-misc
**Release note**:
```release-note
NetworkPolicy has been moved from `extensions/v1beta1` to the new
`networking.k8s.io/v1` API group. The structure remains unchanged from
the beta1 API.
The `net.beta.kubernetes.io/network-policy` annotation on Namespaces
to opt in to isolation has been removed. Instead, isolation is now
determined at a per-pod level, with pods being isolated if there is
any NetworkPolicy whose spec.podSelector targets them. Pods that are
targeted by NetworkPolicies accept traffic that is accepted by any of
the NetworkPolicies (and nothing else), and pods that are not targeted
by any NetworkPolicy accept all traffic by default.
Action Required:
When upgrading to Kubernetes 1.7 (and a network plugin that supports
the new NetworkPolicy v1 semantics), to ensure full behavioral
compatibility with v1beta1:
1. In Namespaces that previously had the "DefaultDeny" annotation,
you can create equivalent v1 semantics by creating a
NetworkPolicy that matches all pods but does not allow any
traffic:
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: default-deny
spec:
podSelector:
This will ensure that pods that aren't match by any other
NetworkPolicy will continue to be fully-isolated, as they were
before.
2. In Namespaces that previously did not have the "DefaultDeny"
annotation, you should delete any existing NetworkPolicy
objects. These would have had no effect before, but with v1
semantics they might cause some traffic to be blocked that you
didn't intend to be blocked.
```
Automatic merge from submit-queue
Apply KubeProxyEndpointLagTimeout to ESIPP tests
Fixes#46533.
The previous construction of ESIPP tests is weird, so I redo it a bit.
A 30 seconds `KubeProxyEndpointLagTimeout` is introduced, as these tests ain't verifying performance, may be better to not make it too tight.
/assign @thockin
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46252, 45524, 46236, 46277, 46522)
Make GCE load-balancers create health checks for nodes
From #14661. Proposal on kubernetes/community#552. Fixes#46313.
Bullet points:
- Create nodes health check and firewall (for health checking) for non-OnlyLocal service.
- Create local traffic health check and firewall (for health checking) for OnlyLocal service.
- Version skew:
- Don't create nodes health check if any nodes has version < 1.7.0.
- Don't backfill nodes health check on existing LBs unless users explicitly trigger it.
**Release note**:
```release-note
GCE Cloud Provider: New created LoadBalancer type Service now have health checks for nodes by default.
An existing LoadBalancer will have health check attached to it when:
- Change Service.Spec.Type from LoadBalancer to others and flip it back.
- Any effective change on Service.Spec.ExternalTrafficPolicy.
```
Automatic merge from submit-queue (batch tested with PRs 45809, 46515, 46484, 46516, 45614)
Remove the reduplicated case judement
This patch remove the reduplicated case judgement
Automatic merge from submit-queue (batch tested with PRs 45809, 46515, 46484, 46516, 45614)
Fix incorrect printf format
**What this PR does / why we need it**: changes `%s` to `%d` for something that is actually an `int` (found via `make vet`).
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Update the "interesting" controller-manager metrics to match the
current names for the garbage collector, and add namespace controller
metrics to the list.
Automatic merge from submit-queue (batch tested with PRs 46429, 46308, 46395, 45867, 45492)
Controller history
**What this PR does / why we need it**:
Implements the ControllerRevision API object and clientset to allow for the implementation of StatefulSet update and DaemonSet history
```release-note
ControllerRevision type added for StatefulSet and DaemonSet history.
```
Automatic merge from submit-queue (batch tested with PRs 46429, 46308, 46395, 45867, 45492)
Summary Test looks at pods that have containers that restart.
Occasionally, the node can report extra containers that had been restarted through the summary API.
This test change tests a pod that restarts, and hopefully should allow us to reproduce and debug this behavior.
/assign @dchen1107
/release-note-none
Automatic merge from submit-queue (batch tested with PRs 46429, 46308, 46395, 45867, 45492)
Bump Go version to 1.8.3
This PR also removed this patched version of Go 1.8.1 which we used to use to workaround performance problem of Go 1.8.1.
Fix https://github.com/kubernetes/kubernetes/issues/45216
Ref #46391
@timothysc @bradfitz
Automatic merge from submit-queue (batch tested with PRs 46124, 46434, 46089, 45589, 46045)
Copy kubeconfig to kubemark master
This should save the effort of digging through jenkins agent and its container to get the kubeconfig.
Ideally we should have kubectl directly working on the kubemark master, but I'm facing some issues due to older version of kubectl present by default on the node.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45949, 46009, 46320, 46423, 46437)
e2e tests for storage policy support in Kubernetes
This PR covers e2e test cases for vSphere storage policy support in Kubernetes - #46176.
The following test scenario have been implemented.
- Specify only SPBM storage policy name.
- Verify if the disk is provisioned on a compatible datastore with max free space.
- Specify a storage policy name which is not defined on VC.
- Verify if PVC create errors out that no pbm profile with this policy is found.
- Specify both SPBM storage policy name and VSAN capabilities together.
- Verify if PVC create errors out that you can't use both SPBM policy name with VSAN capabilities. You can only specify one.
- Specify SPBM storage policy name with user specified datastore which is non-compatible.
- Verify if PVC create errors out that it can't provision a disk on a non-compatible datastore.
@jeffvance @divyenpatel
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 45949, 46009, 46320, 46423, 46437)
Unregister some metrics
delete some registered metrics since they are not observed
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45269, 46219, 45966)
Add overriding Stackdriver API endpoint
Allow using Stackdriver test endpoint.
Automatic merge from submit-queue (batch tested with PRs 45573, 46354, 46376, 46162, 46366)
GCE - Retrieve subnetwork name/url from gce.conf
**What this PR does / why we need it**:
Features like ILB require specifying the subnetwork if the network is type manual.
**Notes:**
The network URL can be [constructed](68e7e18698/pkg/cloudprovider/providers/gce/gce.go (L211-L217)) by fetching instance metadata; however, the subnetwork is not provided through this feature. Users must specify the subnetwork name/url through the gce.conf.
Although multiple subnets can exist in the same region for a network, the cloud provider will only use one subnet url for creating LBs.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45573, 46354, 46376, 46162, 46366)
Subresources are not included in apiserver prometheus metrics
Subresources are very often completely different code paths and errors
generated on those code paths are important to distinguish.
@kubernetes/sig-api-machinery-pr-reviews
```release-note
The Prometheus metrics for the kube-apiserver for tracking incoming API requests and latencies now return the `subresource` label for correctly attributing the type of API call.
```
Automatic merge from submit-queue (batch tested with PRs 45913, 46065, 46352, 46363, 46373)
Fix CheckPodsCondition to print out the correct podName
From a couple CIs (https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-serial/1114, https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-gci-qa-serial-master/2246, https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gke-pre-release/2187), all indicate we print out the wrong pod name in CheckPodsCondition for _"Pod XXX failed to be running and ready, or succeeded."_:
```
I0524 02:09:50.173] May 24 02:09:50.173: INFO: Waiting for pod heapster-v1.3.0-3806988011-kzkg6 in namespace 'kube-system' status to be 'running and ready, or succeeded'(found phase: "Running", readiness: false) (4m55.033881993s elapsed)
I0524 02:09:52.178] May 24 02:09:52.178: INFO: Waiting for pod heapster-v1.3.0-3806988011-kzkg6 in namespace 'kube-system' status to be 'running and ready, or succeeded'(found phase: "Running", readiness: false) (4m57.03848264s elapsed)
I0524 02:09:54.183] May 24 02:09:54.182: INFO: Waiting for pod heapster-v1.3.0-3806988011-kzkg6 in namespace 'kube-system' status to be 'running and ready, or succeeded'(found phase: "Running", readiness: false) (4m59.043463323s elapsed)
I0524 02:09:56.183] May 24 02:09:56.183: INFO: Pod fluentd-gcp-v2.0-6wf67 failed to be running and ready, or succeeded.
I0524 02:09:56.184] May 24 02:09:56.183: INFO: Wanted all 23 pods to be running and ready, or succeeded. Result: false. Pods: [heapster-v1.3.0-3806988011-kzkg6 kube-proxy-bootstrap-e2e-minion-group-bbwn rescheduler-v0.3.0-bootstrap-e2e-master monitoring-influxdb-grafana-v4-1q59k l7-default-backend-1044750973-zgxsc etcd-server-events-bootstrap-e2e-master kube-apiserver-bootstrap-e2e-master kube-proxy-bootstrap-e2e-minion-group-6nqb kube-proxy-bootstrap-e2e-minion-group-mzbz fluentd-gcp-v2.0-chd2x kube-dns-806549836-f8p46 fluentd-gcp-v2.0-44x97 kube-dns-autoscaler-2528518105-vlg8t fluentd-gcp-v2.0-p1h4b kube-controller-manager-bootstrap-e2e-master l7-lb-controller-v0.9.3-bootstrap-e2e-master kubernetes-dashboard-2917854236-tn3nx kube-dns-806549836-fq2fp kube-scheduler-bootstrap-e2e-master etcd-empty-dir-cleanup-bootstrap-e2e-master kube-addon-manager-bootstrap-e2e-master etcd-server-bootstrap-e2e-master fluentd-gcp-v2.0-6wf67]
I0524 02:09:56.184] May 24 02:09:56.183: INFO: At least one pod wasn't running and ready or succeeded at test start.
I0524 02:09:56.184] [AfterEach] [k8s.io] Restart [Disruptive]
```
Check the codes and found we always print out the last pod name, which is random. Pass the pod name into channel to fix.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45913, 46065, 46352, 46363, 46373)
Detect cohabitating resources in etcd storage test
**What this PR does / why we need it**:
This change updates the etcd storage path test to detect cohabitating resources by looking at their expected location in etcd. This was not detected in the past because the GVK check did not span across groups.
To limit noise from failures caused by multiple objects at the same location in etcd, the test now fails when different GVRs share the same expected path. Thus every object is expected to have a unique path.
@liggitt PTAL
Signed-off-by: Monis Khan <mkhan@redhat.com>
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46299, 46309, 46311, 46303, 46150)
Dont attach a GPU to ubuntu test machines for node e2e serial tests
This should fix flakes in the e2e_node serial suite.
@vishh I think this is what you were asking for...
/assign @vishh
Automatic merge from submit-queue (batch tested with PRs 46299, 46309, 46311, 46303, 46150)
Move docker validation test to separate project.
Docker validation test is leaking VMs because new docker version `DOCKER_VERSION=17.05.0-c` totally breaks the new gci image `GCE_IMAGES=gci-test-60-9579-0-0` with the `gci-docker-version` metadata specified.
The test successfully created the instance, but timed out when checking VM aliveness, and leaked the VM.
I've cleaned up all leaked VMs. This PR moves docker validation node e2e test into a separate project to not influencing other node e2e test.
@kewu1992 We should fix the docker automated validation test.
/cc @dchen1107 @yujuhong @abgworrall
Automatic merge from submit-queue (batch tested with PRs 42042, 46139, 46126, 46258, 46312)
Remove unused test properties
Issue: #42676
A separate serial memcg suite was created for the initial stages of re-enabling memcg notifications. Now that all e2e tests have memcg notifications enabled, this suite is no longer needed.
Automatic merge from submit-queue (batch tested with PRs 46149, 45897, 46293, 46296, 46194)
Chaosmonkey - Signal stop to tests and wait for done when disruption fails
**What this PR does / why we need it**:
Prevents tests from leaking resources because their Teardown was never called when test disruption fails.
**Which issue this PR fixes**
First problem of #45842
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46149, 45897, 46293, 46296, 46194)
GC: update required verbs for deletable resources, allow list of ignored resources to be customized
The garbage collector controller currently needs to list, watch, get,
patch, update, and delete resources. Update the criteria for
deletable resources to reflect this.
Also allow the list of resources the garbage collector controller should
ignore to be customizable, so downstream integrators can add their own
resources to the list, if necessary.
cc @caesarxuchao @deads2k @smarterclayton @mfojtik @liggitt @sttts @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue
Enable "kick the tires" support for Nvidia GPUs in COS
This PR provides an installation daemonset that will install Nvidia CUDA drivers on Google Container Optimized OS (COS).
User space libraries and debug utilities from the Nvidia driver installation are made available on the host in a special directory on the host -
* `/home/kubernetes/bin/nvidia/lib` for libraries
* `/home/kubernetes/bin/nvidia/bin` for debug utilities
Containers that run CUDA applications on COS are expected to consume the libraries and debug utilities (if necessary) from the host directories using `HostPath` volumes.
Note: This solution requires updating Pod Spec across distros. This is a known issue and will be addressed in the future. Until then CUDA workloads will not be portable.
This PR updates the COS base image version to m59. This is coupled with this PR for the following reasons:
1. Driver installation requires disabling a kernel feature in COS.
2. The kernel API for disabling this interface changed across COS versions
3. If the COS image update is not handled in this PR, then a subsequent COS image update will break GPU integration and will require an update to the installation scripts in this PR.
4. Instead of having to post `3` PRs, one each for adding the basic installer, updating COS to m59, and then updating the installer again, this PR combines all the changes to reduce review overhead and latency, and additional noise that will be created when GPU tests break.
**Try out this PR**
1. Get Quota for GPUs in any region
2. `export `KUBE_GCE_ZONE=<zone-with-gpus>` KUBE_NODE_OS_DISTRIBUTION=gci`
3. `NODE_ACCELERATORS="type=nvidia-tesla-k80,count=1" cluster/kube-up.sh`
4. `kubectl create -f cluster/gce/gci/nvidia-gpus/cos-installer-daemonset.yaml`
5. Run your CUDA app in a pod.
**Another option is to run a e2e manually to try out this PR**
1. Get Quota for GPUs in any region
2. export `KUBE_GCE_ZONE=<zone-with-gpus>` KUBE_NODE_OS_DISTRIBUTION=gci
3. `NODE_ACCELERATORS="type=nvidia-tesla-k80,count=1"`
4. `go run hack/e2e.go -- --up`
5. `hack/ginkgo-e2e.sh --ginkgo.focus="\[Feature:GPU\]"`
The e2e will install the drivers automatically using the daemonset and then run test workloads to validate driver integration.
TODO:
- [x] Update COS image version to m59 release.
- [x] Remove sleep from the install script and add it to the daemonset
- [x] Add an e2e that will run the daemonset and run a sample CUDA app on COS clusters.
- [x] Setup a test project with necessary quota to run GPU tests against HEAD to start with https://github.com/kubernetes/test-infra/pull/2759
- [x] Update node e2e serial configs to install nvidia drivers on COS by default
Automatic merge from submit-queue (batch tested with PRs 45587, 46286)
PDB Max Unavailable Field
Completes https://github.com/kubernetes/features/issues/285
```release-note
Adds a MaxUnavailable field to PodDisruptionBudget
```
Individual commits are self-contained; Last commit can be ignored because it is autogenerated code.
cc @kubernetes/sig-apps-api-reviews @kubernetes/sig-apps-pr-reviews
Allow the list of resources the garbage collector controller should
ignore to be customizable, so downstream integrators can add their own
resources to the list, if necessary.
Automatic merge from submit-queue (batch tested with PRs 45766, 46223)
Scheduler should use a shared informer, and fix broken watch behavior for cached watches
Can be used either from a true shared informer or a local shared
informer created just for the scheduler.
Fixes a bug in the cache watcher where we were returning the "current" object from a watch event, not the historic event. This means that we broke behavior when introducing the watch cache. This may have API implications for filtering watch consumers - but on the other hand, it prevents clients filtering from seeing objects outside of their watch correctly, which can lead to other subtle bugs.
```release-note
The behavior of some watch calls to the server when filtering on fields was incorrect. If watching objects with a filter, when an update was made that no longer matched the filter a DELETE event was correctly sent. However, the object that was returned by that delete was not the (correct) version before the update, but instead, the newer version. That meant the new object was not matched by the filter. This was a regression from behavior between cached watches on the server side and uncached watches, and thus broke downstream API clients.
```
Automatic merge from submit-queue (batch tested with PRs 46201, 45952, 45427, 46247, 46062)
Use shared informers in gc controller if possible
Modify the garbage collector controller to try to use shared informers for resources, if possible, to reduce the number of unique reflectors listing and watching the same thing.
cc @kubernetes/sig-api-machinery-pr-reviews @caesarxuchao @deads2k @liggitt @sttts @smarterclayton @timothysc @soltysh @kargakis @kubernetes/rh-cluster-infra @derekwaynecarr @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 38990, 45781, 46225, 44899, 43663)
Support parallel scaling on StatefulSets
Fixes#41255
```release-note
StatefulSets now include an alpha scaling feature accessible by setting the `spec.podManagementPolicy` field to `Parallel`. The controller will not wait for pods to be ready before adding the other pods, and will replace deleted pods as needed. Since parallel scaling creates pods out of order, you cannot depend on predictable membership changes within your set.
```
Automatic merge from submit-queue (batch tested with PRs 46133, 46211, 46224, 46205, 45910)
test/images/network-tester:bump rc/pod image version to 1.9
Current image version is 1.9，update the image version of the associated json file to 1.9
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46133, 46211, 46224, 46205, 45910)
Make CPU request for heapster in kubemark scale with the number of Nodes
This change updates the etcd storage path test to detect cohabitating
resources by looking at their expected location in etcd. This was not
detected in the past because the GVK check did not span across groups.
To limit noise from failures caused by multiple objects at the same
location in etcd, the test now fails when different GVRs share the same
expected path. Thus every object is expected to have a unique path.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Automatic merge from submit-queue
Add script to dump kubemark master logs
First step towards solving the issue https://github.com/kubernetes/kubernetes/issues/46109.
cc @kubernetes/test-infra-maintainers @wojtek-t @gmarek
Packaged the script as a docker container stored in gcr.io/google-containers
A daemonset deployment is included to make it easy to consume the installer
A cluster e2e has been added to test the installation daemonset along with verifying installation
by using a sample CUDA application.
Node e2e for GPUs updated to avoid running on nodes without GPU devices.
Signed-off-by: Vishnu kannan <vishnuk@google.com>
Previously, the scheduler created two separate list watchers. This
changes the scheduler to be able to leverage a shared informer, whether
passed in externally or spawned using the new in place method. This
removes the last use of a "special" informer in the codebase.
Allows someone wrapping the scheduler to use a shared informer if they
have more information avaliable.
Tokens controller previously needed a bit of extra help in order to be
safe for concurrent use. The new MutationCache allows it to keep a local
cache and still use a shared informer. The filtering event handler lets
it only see changes to secrets it cares about.
Automatic merge from submit-queue (batch tested with PRs 46014, 46152)
Updated test/test_owners.csv for federation test cases
To the best of my knowledge have updated the test owners for federation e2e test cases. PTAL and comment if any concern.
**Release note**:
```release-note
NONE
```
cc @kubernetes/sig-federation-pr-reviews @fejta
/assign @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 46033, 46122, 46053, 46018, 45981)
Command tree and exported env in kubectl plugins
This is part of `kubectl` plugins V1:
- Adds support to several env vars passing context information to the plugin. Plugins can make use of them to connect to the REST API, access global flags, get the path of the plugin caller (so that `kubectl` can be invoked) and so on. Exported env vars include
- `KUBECTL_PLUGINS_DESCRIPTOR_*`: the plugin descriptor fields
- `KUBECTL_PLUGINS_GLOBAL_FLAG_*`: one for each global flag, useful to access namespace, context, etc
- ~`KUBECTL_PLUGINS_REST_CLIENT_CONFIG_*`: one for most fields in `rest.Config` so that a REST client can be built.~
- `KUBECTL_PLUGINS_CALLER`: path to `kubectl`
- `KUBECTL_PLUGINS_CURRENT_NAMESPACE`: namespace in use
- Adds support for plugins as child of other plugins so that a tree of commands can be built (e.g. `kubectl myplugin list`, `kubectl myplugin add`, etc)
**Release note**:
```release-note
Added support to a hierarchy of kubectl plugins (a tree of plugins as children of other plugins).
Added exported env vars to kubectl plugins so that plugin developers have access to global flags, namespace, the plugin descriptor and the full path to the caller binary.
```
@kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 46033, 46122, 46053, 46018, 45981)
Log age of stats used for evictions during eviction tests
I recently added prometheus metrics for the age of the metrics used for evictions #43031. It would be nice to surface these during eviction tests, so I can better assess how old stats are, and whether or not the age of stats causes extra evictions.
This isnt super-high priority, and can be done after code-freeze, since it is a testing improvement. Feel free to take a look whenever either of you has time.
/assign @mtaufen
/assign @Random-Liu
Automatic merge from submit-queue (batch tested with PRs 45346, 45903, 45958, 46042, 45975)
ResourceQuota admission control injects registry
**What this PR does / why we need it**:
The `ResourceQuota` admission controller works with a registry that maps a GroupKind to an Evaluator. The registry used in the existing plug-in is not injectable, which makes usage of the ResourceQuota plug-in in other API server contexts difficult. This PR updates the code to support late injection of the registry via a plug-in initializer.
Automatic merge from submit-queue (batch tested with PRs 45996, 46121, 45707, 46011, 45564)
Fix waitForNPods in restart.go
From https://github.com/kubernetes/kubernetes/issues/45991#issuecomment-302292404.
Don't redefine `pods` so we can return real pod names instead of empty array.
/assign @dchen1107 @bowei
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add node e2e tests for hostNetwork
**What this PR does / why we need it**:
Add node e2e tests for hostNetwork.
**Which issue this PR fixes**
Part of #44118.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @Random-Liu @yujuhong
Automatic merge from submit-queue (batch tested with PRs 45950, 45968)
[Federation] Remove redundant e2e for secret and daemonset
Federation of daemonset and secret types is now implemented by the sync controller, and e2e testing for each type is provided via crud lifecycle e2e tests. This renders the legacy e2e tests for these types redundant, and this commit removes those tests.
The secret wait and delete functions required by the ingress e2e tests have been retained and moved to ingress.go.
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
Make real proxier in hollow-proxy optional (default=true)
Ref https://github.com/kubernetes/kubernetes/pull/45622
This allows using real proxier for hollow proxy, but we use the fake one by default.
cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
Add explicit image tag to cockroachdb example and test
@gyliu513
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
Add integration test for deployment
We have no integration test for Deployment currently. In this PR, add an integration test which covers an e2e test (create a new RollingUpdate deployment), add more replicas to the Deployment, and set minReadySeconds so that we can test maxUnavailable.
Plan to add more integration tests that cover e2e tests after this initial PR is merged.
@kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
[Federation] Add integration testing for cluster addition
This PR adds integration testing of the sync controller for cluster addition. This ensures coverage equivalency between the integration tests and the old controller unit tests, so those tests are removed by this PR.
Resolves#45257
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
Remove the deprecated --babysit-daemons kubelet flag
```release-note
Removes the deprecated kubelet flag --babysit-daemons
```
This flag has been deprecated for over a year (git blame says marked deprecated on March 1, 2016).
Relatively easy removal - nothing in the Kubelet relies on it anymore.
There was still some stuff in the provisioning scripts. It was easy to rip out, but in general we probably need to be more disciplined about updating the provisioning scripts at the same time that we initially mark things deprecated.
Automatic merge from submit-queue
Move all API related annotations into annotation_key_constants.go
Separate from #45869. See https://github.com/kubernetes/kubernetes/pull/45869#discussion_r116839411 for details.
This PR does nothing but move constants around :)
/assign @caesarxuchao
**Release note**:
```release-note
NONE
```
Federation of daemonset and secret types is now implemented by the
sync controller, and e2e testing for each type is provided via crud
lifecycle e2e tests. This renders the legacy e2e tests for these types
redundant, and this commit removes those tests.
Automatic merge from submit-queue (batch tested with PRs 44520, 45253, 45838, 44685, 45901)
Ensure ordering of using dynamic kubelet config and setting up tests.
This PR simply places the body of the eviction test within its own context. This ensures that the kubelet config is set before the pods are created, and that the kubelet config is reverted only after the pods are deleted.
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)
Apiregistration v1alpha1→v1beta1
Promoting apiregistration api from v1alpha1 to v1beta1.
API Registration is responsible for registering an API `Group`/`Version` with
another kubernetes like API server. The `APIService` holds information
about the other API server in `APIServiceSpec` type as well as general
`TypeMeta` and `ObjectMeta`. The `APIServiceSpec` type have the main
configuration needed to do the aggregation. Any request coming for
specified `Group`/`Version` will be directed to the service defined by
`ServiceReference` (on port 443) after validating the target using provided
`CABundle` or skipping validation if development flag `InsecureSkipTLSVerify`
is set. `Priority` is controlling the order of this API group in the overall
discovery document.
The return status is a set of conditions for this aggregation. Currently
there is only one condition named "Available", if true, it means the
api/server requests will be redirected to specified API server.
```release-note
API Registration is now in beta.
```
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
fix typo
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45860, 45119, 44525, 45625, 44403)
[Federation] Move annotations and related parsing code as common code
This PR moves some code, which was duplicate, around as common code.
Changes the names of structures used for annotations to common names.
s/FederatedReplicaSetPreferences/ReplicaAllocationPreferences/
s/ClusterReplicaSetPreferences/PerClusterPreferences/
This can be reused in job controller and hpa controller code.
**Special notes for your reviewer**:
@kubernetes/sig-federation-misc
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45408, 45355, 45528)
Admission options spits out admission control
**What this PR does / why we need it**:
This PR adds ApplyTo method to AdmissionOptions struct. The method creates and initialises admission control to the server configuration.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41331, 45591, 45600, 45176, 45658)
Move client/unversioned/remotecommand to client-go
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.
Module remotecommand relies on util/exec module which was copied to client-go/pkg/util
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.
Module remotecommand relies on util/exec module which will be copied to client-go/pkg/util
ApplyTo adds the admission chain to the server configuration the method lazily initializes a generic plugin
that is appended to the list of pluginInitializers.
apiserver.Config will hold an instance of SharedInformerFactory to ensure we only have once instance.
The field will be initialized in apisever.SecureServingOptions
Automatic merge from submit-queue
Reorganize kubelet tree so apis can be independently versioned
@yujuhong @lavalamp @thockin @bgrant0607
This is an example of how we might reorganize `pkg/kubelet` so the apis it exposes can be independently versioned. This would also provide a logical place to put the `KubeletConfiguration` type, which currently lives in `pkg/apis/componentconfig`; it could live in e.g. `pkg/kubelet/apis/config` instead.
Take a look when you have a chance and let me know what you think. The most significant change in this PR is reorganizing `pkg/kubelet/api` to `pkg/kubelet/apis`, the rest is pretty much updating import paths and `BUILD` files.
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Promotes Source IP preservation for Virtual IPs from Beta to GA
Fixes#33625. Feature issue: kubernetes/features#27.
Bullet points:
- Declare 2 fields (ExternalTraffic and HealthCheckNodePort) that mirror the ESIPP annotations.
- ESIPP alpha annotations will be ignored.
- Existing ESIPP beta annotations will still be fully supported.
- Allow promoting beta annotations to first class fields or reversely.
- Disallow setting invalid ExternalTraffic and HealthCheckNodePort on services. Default ExternalTraffic field for nodePort or loadBalancer type service to "Global" if not set.
**Release note**:
```release-note
Promotes Source IP preservation for Virtual IPs to GA.
Two api fields are defined correspondingly:
- Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
- Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
```
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Revert #45101 Mark PersistentVolumes as [Feature:Volumes]
**What this PR does / why we need it**:
Reverts #45101
`Feature` tag should only be used when a test/suite has dependencies not met by core CI. That is not the case for NFS backed PV tests.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45685, 45572, 45624, 45723, 45733)
Remove a test utility function that is redundant and kinda broken
Framework.WaitForAnEndpoint() has no timeout, so if something goes wrong and the endpoint doesn't get created, the test will hang forever. (This is happening for some reason sometimes in OpenShift right now, and when the CI system eventually times out and kills the VM, it loses the logs that would explain what failed.)
There's already another nearly-identical WaitForEndpoint() method that *does* take a timeout, so people can just use that instead.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45653, 45719, 45729, 45730, 44250)
Remove kubemark.sh as we don't use pod IP from it anymore
This has been pending for sometime now. We no longer seem to actually depend on the downwarp api for the pod IP (hollow-proxy for example now gets it using api call).
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45653, 45719, 45729, 45730, 44250)
Print pod startup latency metric as perfdata
Follows #45657
This should print pod startup latency in same format as api calls latencies.
cc @wojtek-t @gmarek
Automatic merge from submit-queue
Fix flag formatting errors in the node tests
There were three problems:
- Lack of a trailing space after prepending flags.
- Passing multiple flags in a string to --kubelet-flags seems to confuse
the flag parser; it stops parsing ALL flags as soon as it sees the
second kubelet flag. Fortunately, all instances of --kubelet-flags are
combined together, so we can just pass two of those.
- --feature-gates should be passed to the test framework, which then
forwards it to the kubelet, instead of using --kubelet-flags.
This hopefully fixes the dynamic config test failures on COS, which
started after #45602. (See: https://k8s-testgrid.appspot.com/google-node#kubelet-serial-gce-e2e)
Automatic merge from submit-queue
move from daemon_restart.go to framework/util.go
**What this PR does / why we need it**:
Moves the func `nodeExec` from daemon_restart.go to framework/util.go. This is the correct file for this func and is a more intuitive pkg for other callers to use. This is a small step of the larger effort of restructuring e2e tests to be more logically structured and easier for newcomers to understand.
```release-note
NONE
```
cc @timothysc @copejon
Automatic merge from submit-queue (batch tested with PRs 45691, 45667, 45698, 45715)
Make update_owners.py also emit a JSON sig-owners list.
This should experience less churn in general!
I'm going to make the triage page use this list to let sigs have individualized dashboards.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
detach the volume when pod is terminated
When pods are terminated we should detach the volume.
Fixes https://github.com/kubernetes/kubernetes/issues/45191
**Release note**:
```
Detach the volume when pods are terminated.
```
Automatic merge from submit-queue
Add properties file for cos-docker-validation test job
**What this PR does / why we need it**:
This is forked from test/e2e_node/jenkins/docker_validation/jenkins-validation.properties. It is used for COS docker validation test.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue
plumb stopch to post start hook index since many of them are starting go funcs
Many post-start hooks require a stop channel to properly terminate their go funcs.
@p0lyn0mial I think you need this for https://github.com/kubernetes/kubernetes/pull/45355 ptal.
@ncdc per request
@sttts can you review too since Andy is out?
There were three problems:
- Lack of a trailing space after prepending flags.
- Passing multiple flags in a string to --kubelet-flags seems to confuse
the flag parser; it stops parsing ALL flags as soon as it sees the
second kubelet flag. Fortunately, all instances of --kubelet-flags are
combined together, so we can just pass two of those.
- --feature-gates should be passed to the test framework, which then
forwards it to the kubelet, instead of using --kubelet-flags.
This hopefully fixes the dynamic config test failures on COS, which
started after #45602.
Automatic merge from submit-queue
small change to view more test info
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
small change to view more test info, think you very much
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45634, 45480)
Fix BY() format
**What this PR does / why we need it**:
i read other by(), just format, think you
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45569, 45602, 45604, 45478, 45550)
Minor bug fix in start-kubemark-master script
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45569, 45602, 45604, 45478, 45550)
Enable kernel memcg notification for node and cluster GCI/COS testing.
Sets --experimental-kernel-memcg-notification=true when running on the GCI/COS image. It sets this for master and nodes for cluster e2e tests, and for the node in node e2e tests.
Issue #42676
cc @dchen1107 @Random-Liu
Automatic merge from submit-queue
util.go: format for
**What this PR does / why we need it**:
format for.
delete redundant para.
make code clean.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43067, 45586, 45590, 38636, 45599)
Make SchedulerPredicates test more resiliant to recent Node restarts
cc @kubernetes/sig-scheduling-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43067, 45586, 45590, 38636, 45599)
Move rest of performance data gathered by tests to Summaries
cc @shyamjvs
Automatic merge from submit-queue (batch tested with PRs 43067, 45586, 45590, 38636, 45599)
Fix bug in hollow-node deletion in stop-kubemark script
Just noticed.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45453, 45307, 44987)
Migrate the docker client code from dockertools to dockershim
Move docker client code from dockertools to dockershim/libdocker. This includes
DockerInterface (renamed to Interface), FakeDockerClient, etc.
This is part of #43234
Automatic merge from submit-queue (batch tested with PRs 45304, 45006, 45527)
Fixed indent for some statefulset for e2e test.
**What this PR does / why we need it**:
Make sure the e2e test passed for statefulset.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45526
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45304, 45006, 45527)
Add node e2e tests for hostIPC
**What this PR does / why we need it**:
Add node e2e tests for hostIPC.
**Which issue this PR fixes**
Part of #44118.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @Random-Liu @yujuhong
Automatic merge from submit-queue (batch tested with PRs 44798, 45537, 45448, 45432)
Stream output of run-gcloud-compute-with-retries to stdout in realtime
Ref https://github.com/kubernetes/kubernetes/issues/40139#issuecomment-299894222 (3rd point)
This should help us get more info about timeouts during start-kubemark-master.sh.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45481, 45463)
ThirdPartyResource example: added watcher example, code cleanup
**NOTE**: This is a cleaned and updated version of PR https://github.com/kubernetes/kubernetes/pull/43027
**What this PR does / why we need it**:
An example of using go-client for watching on ThirdPartyResource events (create/update/delete).
Automatic merge from submit-queue (batch tested with PRs 45200, 45203)
Add .svc.cluster.local for service host
**What this PR does / why we need it**:
Running into DNS resolve problems, testing to see if this works better
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45508, 44258, 44126, 45441, 45320)
Removed 'default' row from test_owners.csv and Updated update_owners.py
**What this PR does / why we need it**:
Removes the 'default' row from test_owners.csv and the validation/update logic associated with it in update_owners.py.
The 'default' row is being removed because it results in too many issues being assigned to the default test owners when issues are automatically generated.
**Release note**:
```release-note
NONE
```
/assign
Automatic merge from submit-queue (batch tested with PRs 41903, 45311, 45474, 45472, 45501)
Removed old scheduler constructor.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A
**Release note**:
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)
Fix gcloud retries cmd to rightly capture return code
Fixes https://github.com/kubernetes/kubernetes/issues/45489
Seems like the issue was in doing this:
```
local var=$(cmd)
local ret_code=$? #This actually captures return code of "local" above and not "cmd"
```
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)
Separate healthz server from metrics server in kube-proxy
From #14661, proposal is on kubernetes/community#552.
Couple bullet points as in commit:
- /healthz will be served on 0.0.0.0:10256 by default.
- /metrics and /proxyMode will be served on port 10249 as before.
- Healthz handler will verify timestamp in iptables mode.
/assign @nicksardo @bowei @thockin
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
refactor names for the apiserver handling chain
The names and structure around the handling chain got a bit confused. This simplifies it back out into a single struct with three parts: overall handler, gorestful handler, pathrecording mux and makes the delegate wiring simpler
Automatic merge from submit-queue
Clean up petset
**What this PR does / why we need it**:
Rename legacy petset to statefulset.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
Use dedicated UnixUserID and UnixGroupID types
**What this PR does / why we need it**:
DRYs up type definitions by using the dedicated types in apimachinery
**Which issue this PR fixes**
#38120
**Release note**:
```release-note
UIDs and GIDs now use apimachinery types
```
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
remove mvPodUidDir test
**What this PR does / why we need it**:
Removes a [HostCleanup] test which deleted the pod's uid directory and checked to see that the host was cleaned up. This test consistently failed on gci (COS) due to OS differences related to handling mounting (containerized on COS) and rm'ing files being accessed. See #44592 for more details.
```release-note
NONE
```
cc. @msau42 @copejon
Automatic merge from submit-queue (batch tested with PRs 45364, 45369, 45323)
Fix namespace deletion log in e2e test framework.
**What this PR does / why we need it**:
Current log message for namespace deletion will show under wrong condition.
**Which issue this PR fixes**
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 45316, 45341)
Move hollow-node's initContainer from annotation to field
In 1.6, we have moved init containers spec from beta annotation to field. Updating this in kubemark hollow-node.
This should solve issues with setting of default fields (like ImagePullPolicy) in the container spec for older versions of apiservers too (as annotation->field change has been cherry picked for older releases too). We noticed this issue in https://github.com/kubernetes/kubernetes/issues/45290 where an old apiserver apparently wasn't setting default ImagePullPolicy for container spec inside annotations.
cc @wojtek-t @gmarek @gamename
Automatic merge from submit-queue
Deployment e2e fixes
* don't expect availability on Deployment creations
* switch to framework.Logf everywhere for consistency
* better use of Expect
Automatic merge from submit-queue
Fix golint verify error
I don't know why CI pass the hack/verify-golint.sh test.
But in my environment I get this:
> staging/src/k8s.io/client-go/util/workqueue/queue_test.go is in package workqueue_test, not workqueue
Errors from golint:
test/integration/apiserver/apiserver_test.go:102:13: should omit type string from declaration of var cascDel; it will be inferred from the right-hand side
Please fix the above errors. You can test via "golint" and commit the result.
!!! Error in hack/verify-golint.sh:98
Error in hack/verify-golint.sh:98. 'false' exited with status 1
Call stack:
1: hack/verify-golint.sh:98 main(...)
Exiting with status 1
This change fix this err in my environment.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45120, 45243)
skip integration test when run make bazel-test
we should opt for a seperate target for integration tests. This is breaking @deads2k who is trying to add an integration test in staging.
Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)
Move federated ingress e2e tests that don't pass to serial suite to avoid running it in presubmits.
This is due to a bug described here: https://github.com/kubernetes/ingress/issues/645.
Federated Ingress e2es fail when there is a high load which is pretty common in the presubmit environment when the PR activity is high. So we are disabling these tests to unblock federation presubmits.
**Release note**:
```release-note
NONE
```
/assign @csbell
cc @kubernetes/sig-federation-pr-reviews @nicksardo
Automatic merge from submit-queue
Add e2e network policy tests.
**What this PR does / why we need it**:
Add set of e2e tests for Network Policy. This has succesfully run againast a deployment using Calico as the network policy provider.
Specifically, adds a new e2e test file (/test/e2e/network_policy.go) which tests TCP connectivity between pods with isolation.
See #25638 for PR that added NetworkPolicy resource.
This PR is a replacement for #27447 .
Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)
Increase pv test ns deletion timeout to the framework's default.
**What this PR does / why we need it**:
Namespace deletion occasionally exceeds the test's timeout of 3 minutes. The test framework uses a default timeout of 5 minutes, so just making it match here.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44790
**Special notes for your reviewer**:
**Release note**:
NONE
Automatic merge from submit-queue
adds Ubuntu node e2e test
**What this PR does / why we need it**:
Add node e2e test for Ubuntu image.
**Special notes for your reviewer**:
**Release note**:
```NONE```
Automatic merge from submit-queue (batch tested with PRs 44062, 45165)
Run SSH e2e test only if SSH keys are present
**What this PR does / why we need it**:
Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
don't use build tags to mark integration tests
This is a bad pattern that leads to checked in code that isn't check for compilation. We should avoid this if it doesn't provide value, which it seems like it doesn't.
Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.
Automatic merge from submit-queue
Add google.com to e2e test only under gce/gke
We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.
Automatic merge from submit-queue (batch tested with PRs 41583, 45117, 45123)
Allow `make test-integration` to pass on OSX
**What this PR does / why we need it**: `make test-integration` isn't passing on my OSX setup (10.11.6, go1.8.1, 17.05.0-ce-rc1). Tests that startup an api server fail because the default `cert-dir` of `/var/run/kubernetes` isn't world-writable. Use a tempdir instead.
**Release note**:
```release-note
NONE
```
ref: #41595
/cc @kubernetes/sig-testing-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 44044, 44766, 44930, 45109, 43301)
e2e test: test azure disk volume
**What this PR does / why we need it**:
E2E test Azure disk volume
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
@kubernetes/sig-testing-pr-reviews
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
e2e: Remove special serviceaccount permission requirement
**What this PR does / why we need it**:
This removes the requirement for having a `default` serviceaccount with permission `create` `pod` in order to run the prestop e2e conformance test.
The `create` `pod` permissions are not needed for this particular test, and having it effectively means that as cluster must have a `default` serviceaccount with `create` `pod` permissions in order to be conformant, which I don't think is desired.
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
`/var/run` is not world-writable on my OSX 10.11.x setup, so tests that
standup a secure apiserver fail with the default cert dir. Use a
tempdir instead.
Automatic merge from submit-queue (batch tested with PRs 45033, 44961, 45021, 45097, 44938)
Disable the kubelet part of metrics collection in kubemark
Fixes https://github.com/kubernetes/kubernetes/issues/45038
This should fix it, as we are just interested in getting the apiserver metrics from kubemark master.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45033, 44961, 45021, 45097, 44938)
Add request count to APICall metric
Ref https://github.com/kubernetes/kubernetes/issues/44701
This should add beside the API call latencies, the count of the requests.
cc @wojtek-t @gmarek
Automatic merge from submit-queue
kubectl binary plugins
**What this PR does / why we need it**:
Introduces the ability to extend `kubectl` by adding third-party plugins that will be exposed through `kubectl`.
Plugins are executable commands written in any language. To be included as a plugin, a binary or script file has to
1. be located under one of the supported plugin path locations:
1.1 `~/.kubectl/plugins` dir
1.2. one or more directory set in the `KUBECTL_PLUGINS_PATH` env var
1.3. the `kubectl/plugins` dir under one or more directory set in the `XDG_DATA_DIRS` env var, which defaults to `/usr/local/share:/usr/share`
2. in any of the plugin path above, have a subfolder with the plugin file(s)
3. in the subfolder, contain at least a `plugin.yaml` file that describes the plugin
Example:
```
$ cat ~/.kube/plugins/myplugin/plugin.yaml
name: "myplugin"
shortDesc: "My plugin's short description"
command: "echo Hello plugins!"
$ kubectl myplugin
Hello plugins!
```
~~In case the plugin declares `tunnel: true`, the plugin engine will pass the `KUBECTL_PLUGIN_API_HOST` env var when calling the plugin binary. Plugins can then access the Kube REST API in "http://$KUBECTL_PLUGIN_API_HOST/api" using the same context currently in use by `kubectl`.~~
Test plugins are provided in `pkg/kubectl/plugins/examples`. Just copy (or symlink) the files to `~/.kube/plugins` to test.
**Which issue this PR fixes**:
Related to the discussions in the proposal document: https://github.com/kubernetes/kubernetes/pull/30086 and https://github.com/kubernetes/community/pull/122.
**Release note**:
```release-note
Introduces the ability to extend kubectl by adding third-party plugins. Developer preview, please refer to the documentation for instructions about how to use it.
```
* don't expect availability on Deployment creations
* switch to framework.Logf everywhere for consistency
* better use of Expect
Signed-off-by: Michail Kargakis <mkargaki@redhat.com>
Just so that we know that we need a cloud provider that
supports volumes to run this test. This is similar to
the change in 63bc42c872.
Ran into this when i was trying to run e2e tests with
local-up-cluster locally and figured out this test will
not work since we don't support local storage persistent
volumes.
Automatic merge from submit-queue (batch tested with PRs 45052, 44983, 41254)
[Federation][e2e] Add 2 new testcases to federation service e2e
**What this PR does / why we need it**:
Add 2 new test cases for federation services
- Federation service updation should update clustered service
- Federation service controller should recreate service shard in cluster, if it gets deleted.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#27623, #35827
Handles one of the tasks discussed in #41253
**Special notes for your reviewer**:
**Release note**:
`NONE`
cc @kubernetes/sig-federation-bugs, @nikhiljindal @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 45052, 44983, 41254)
Non-controversial part of #44523
For easier review of #44523, i extracted the non-controversial part out to this PR.
Automatic merge from submit-queue
Fix cockroachdb statefulset test read/write commands
Explicitly specifying `--insecure` is required on insecure clusters,
which started being enforced in a very recent release. In 2 weeks
we'll have a stable image version that we can reliably pin the
relevant statefulset yaml file to in order to avoid stupid failures
like this. I'm really sorry for the flakes!
**What this PR does / why we need it**:
It fixes the currently broken statefulset test suite - https://storage.googleapis.com/k8s-gubernator/triage/index.html?job=gci-gce-statefulset&test=CockroachDB
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
N/A
**Special notes for your reviewer**:
N/A
**Release note**:
```release-note
NONE
```
@kow3ns
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)
[Federation] Convert Daemonset to use the generic sync controller
To be rebased on master when @perotinus's configmaps PR merges.
Tested integration and e2e.
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)
Cleanup some of the tarball producing code for e2e node tests
This is some e2e node cleanup work I found sitting in a local branch while deleting old local git branches. It looks like it's still useful.
Automatic merge from submit-queue (batch tested with PRs 41106, 44346, 44929, 44979, 45027)
Log error before failing in autoscaling e2e
The gcloud alpha command in e2e fails, but no useful information (error message) is logged.
Automatic merge from submit-queue (batch tested with PRs 41106, 44346, 44929, 44979, 45027)
Make metrics filenames for e2e tests indicate the test better
Currently the names of the json files with metrics for e2e tests are named by appending the `SummaryKind` with a timestamp of the test. It took me some time to figure out which file corresponds to which e2e test due to this. Changing it to have the testname instead of the timestamp.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 44591, 44549)
[Federation][e2e] Fix a failing federation e2e testcase in gce-serial
This is to fix the failing test case in federation [gce-serial](https://k8s-testgrid.appspot.com/cluster-federation#gce-serial) tests. The test case has been failing consistently since we registered the clusters in suite-init instead of doing it in every test case.
Instead of registering and then unregistering, we will be now unregistering and then registering the cluster to federation. this test will be run in serial and will not affect other test cases too.
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Increase timeout for Stackdriver Logging e2e tests
They're failing in CI, because Stackdriver Logging's List method is too slow for this purpose. Quick fix, should be gone completely when reading is implemented properly
/cc @piosz
Automatic merge from submit-queue
e2e tests for VSAN policy support in Kubernetes for vSphere
Following e2e test cases have been implemented for VSAN policy support in Kubernetes for vSphere. These e2e tests are for PR #42974 which out for review.
A total of 8 test cases for below mentioned 5 different scenarios are implemented.
Test cases:
1. Validation of VSAN capabilities.
- hostFailuresToTolerate : Minimum 1 and Max 3. Should be integer.
- stripeWidth: Minimum is 1 and Maximum is 12. Should be integer.
- proportionalCapacity: Expressed in percentage. Should be between 0 and 100. Should be integer.
- iopsLimit: Should be greater than 0. Should be integer.
2. Use a VSAN testbed setup. Try valid VSAN capabilities which are supported by VSAN testbed. Make sure the disk is created with policy configured with it.
- Ex: Using hostFailuresToTolerate=0 and cacheReservation=12
Ex: diskStripes=1 and objectSpaceReservation=30
3. Use a VSAN testbed setup. Try valid VSAN capabilities which are not supported by VSAN testbed. Make sure that the disk creation fails and PVC status is pending.
4. Try using VSAN capabilities on a non-VSAN datastore. PVC status will be pending and it errors to the user saying to VSAN capabilities are not supported on a non-VSAN testbed.
5. Try all 1 to 5 with custom datastore specified by the user.
@jeffvance @divyenpatel
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 43395, 44960)
e2e-node: refactor lifecycle test to avoid selinux issues
Fixes#42905
Previously, the exec hook tests mounted a HostPath volume from /tmp and touched a file as a indicator that the hook had run. This is prohibited by selinux policy on Fedora/RHEL/Centos.
This PR refactors the test to avoid filesystem indication and use the same indication that the HTTP hooks use; a GET to a http endpoint. The exec hooks run `curl` to hit this endpoint and trigger the indication. This simplifies this test quite a bit as well, removing over 85 lines of code.
REVIEWER NOTE: The diff is a mess on this one. Probably better to just review the new version of the file.
@derekwaynecarr
Explicitly specifying `--insecure` is required on insecure clusters,
which started being enforced in a very recent release. In 2 weeks
we'll have a stable image version that we can reliably pin the
relevant statefulset yaml file to in order to avoid stupid failures
like this. I'm really sorry for the flakes!
Automatic merge from submit-queue
Cleanup: replace some hardcoded vars and remove unused functions
Replace some hardcoded vars and remove unused functions
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42477, 44462)
Use storage.v1 instead of v1beta1
storage.v1beta1 was used to work around GKE which did not expose v1. Now that GKE is updated, we can switch everything to v1.
This is simple sed v1beta1 -> v1 + enabled a new test + changed preference of exposed interfaces in `storage/install/install.go`.
@msau42, PTAL and let me know when GKE is updated with storage v1 API and this PR can be actually merged.
@kubernetes/sig-storage-pr-reviews
```release-note
NONE
```
Automatic merge from submit-queue
Move remain ESIPP tests to the slow suite
Continue PR of #38149.
It moves the remain ESIPP tests to the slow suite to help capture breaking changes.
/assign @thockin @freehan
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Stop treating in-cluster-config namespace as an override
Fixes#44835
The namespace of an in-cluster config should behave like the namespace specified in a kubeconfig file... it should be used as the default namespace, but be able to be overridden by namespaces specified in yaml files passed to `kubectl create -f`.
```release-note
Restored the ability of kubectl running inside a pod to consume resource files specifying a different namespace than the one the pod is running in.
```
The docker image is nowhere to be found, so lets remove it.
There have been a request for the Dockerfile here [1], but nobody
seems to care.
redis-proxy is replaced with redis-master in test-cmd-util.sh, to
ensure that the tests still works.
The redis-proxy pod in test/fixtures/doc-yaml/user-guide/multi-pod.yaml
is replaced with valid-pod from test/fixtures/doc-yaml/admin/limitrange/valid-pod.yaml,
so redis-proxy is removed every where.
[1] https://github.com/kubernetes/kubernetes/issues/4914#issuecomment-77209779
Automatic merge from submit-queue
Scheduler configurator looks for a specific key in ConfigMap.Data
**What this PR does / why we need it**: Changes scheduler configurator to look for a specific key in ConfigMap.Data instead of the old logic which expected only one entry to exist in the map. The key is a constant whose value is "policy.cfg".
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Explicit namespace from kubeconfig should override in-cluster config
Fixes#43662
If an explicitly specified namespace is read from a kubeconfig file, we should not fall back to in-cluster config
```release-note
kubectl commands run inside a pod using a kubeconfig file now use the namespace specified in the kubeconfig file, instead of using the pod namespace. If no kubeconfig file is used, or the kubeconfig does not specify a namespace, the pod namespace is still used as a fallback.
```
Automatic merge from submit-queue
Add TestEtcdStoragePath integration test
**What this PR does / why we need it**:
`TestEtcdStoragePath` prevents the accidental movement of objects stored in etcd. It creates a stub of each object and then checks the expected location in etcd. Inadvertent `GroupVersionKind` changes are prevented.
cc @liggitt @soltysh for review.
cc @ncdc because I know you love this 😄
Signed-off-by: Monis Khan <mkhan@redhat.com>
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Mark Stackdriver Logging e2e tests with a feature
Makes Stackdriver Logging e2e tests, except for the most basic one, run in the separate tests suites, prepared by https://github.com/kubernetes/test-infra/pull/2542
Automatic merge from submit-queue
De-Flake Volume E2E: force GCEPD detach to prevent timeout
**What this PR does / why we need it**:
Fix flake`[k8s.io] Volumes [Volume] [k8s.io] PD should be mountable [Flaky] 5m38s.
Flake occurs as a result of an automated detach taking longer than 5 minutes, which exceeds the timeout limit of the delete function.
This PR adds explicit detach and wait func calls before the deletion. By forcing the detach and giving GCE an appropriate timeout limit, this should squash the timeout flake. This also significantly shortens cleanup time.
This PR does not remove the [Flaky] tag. Once this PR is merged, I'll keep an eye on the test grid for ~1 week. If no flakes surface, I'll submit a PR to pull the tag off.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43977
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42177, 42176, 44721)
Job: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings Job into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that Job does not fight with other controllers over control of Pods.
Ref: #24433
**Special notes for your reviewer**:
**Release note**:
```release-note
Job controller now respects ControllerRef to avoid fighting over Pods.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42177, 42176, 44721)
CronJob: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings CronJob into compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that other controllers do not fight over control of objects that a CronJob owns.
**Special notes for your reviewer**:
**Release note**:
```release-note
CronJob controller now respects ControllerRef to avoid fighting with other controllers.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue
adding test for volume fstype validation
**What this PR does / why we need it**:
This PR is adding a test for volume fstype validation. Test verifies fstype specified in storage-class is being honored after volume creation.
Steps:
1. Create StorageClass with fstype set to valid type (default case included).
2. Create PVC which uses the StorageClass created in step 1.
3. Wait for PV to be provisioned.
4. Wait for PVC's status to become Bound.
5. Create pod using PVC on specific node.
6. Wait for Disk to be attached to the node.
7. Execute command in the pod to get fstype.
8. Delete pod and Wait for Volume Disk to be detached from the Node.
9. Delete PVC, PV and Storage Class.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
cc: @jeffvance @tusharnt
Automatic merge from submit-queue (batch tested with PRs 42272, 44696)
e2e test fix: Wait longer when first creating ELB
On any cloud (GCE or AWS), a lag between creating the LoadBalancer and
having it actually start serving traffic is expected. On AWS the lag is
larger, and we weren't correctly using the longer wait on our first
request.
Use a longer wait period on our first request.
Fix#44695
```release-note
NONE
```
Automatic merge from submit-queue
e2e: Prefer kubeconfig host to default
Previously it was necessary to pass ``-host`` to ``e2e.test`` even if ``-kubeconfig`` was specified since otherwise a localhost default would be used. This change ensures that the default is only used when kubeconfig is not set.
cc: @jayunit100
On any cloud (GCE or AWS), a lag between creating the LoadBalancer and
having it actually start serving traffic is expected. On AWS the lag is
larger, and we weren't correctly using the longer wait on our first
request.
Use a longer wait period on our first request.
Fix#44695
Automatic merge from submit-queue (batch tested with PRs 43500, 44073)
[Federation] Add option to retrieve e2e cluster config from secrets
Previously the federation e2e setup was reading member cluster configuration from the test run's kubeconfig. This change removes that dependency in favor of reading member cluster configuration from secrets in the hosting cluster, and caches the configuration to avoid having to read it separately for each test.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue
[Federation] Add simple upgrade test
This PR adds a simple upgrade test that targets all registered federated types.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
The details of the bug is described in PR #44626. We believe this bug
fixes the flakiness in this test and hence we are re-enabling this test
to get some mileage on it. If it turns out to be a problem again we are
going to revert this back.
Automatic merge from submit-queue (batch tested with PRs 44222, 44614, 44292, 44638)
Update dummy certificate used in porter image
**What this PR does / why we need it**: the dummy certificate used for the porter image is incompatible with go1.8+ since it is missing NULL parameters for the RSA public key.
I haven't pushed the new porter image yet.
I verified that this appears to fix the issue. Based on https://github.com/kubernetes/kubernetes/issues/38228#issuecomment-280213590, for the old cert:
```console
$ openssl x509 -in localhost.crt -outform der | openssl asn1parse -inform der -i
...
105:d=4 hl=2 l= 9 prim: OBJECT :rsaEncryption
116:d=3 hl=2 l= 75 prim: BIT STRING
...
```
and the new cert:
```console
$ openssl x509 -in localhost.crt -outform der | openssl asn1parse -inform der -i
...
127:d=4 hl=2 l= 9 prim: OBJECT :rsaEncryption
138:d=4 hl=2 l= 0 prim: NULL
140:d=3 hl=4 l= 271 prim: BIT STRING
...
```
**Release note**:
```release-note
NONE
```
cc @liggitt @lavalamp @luxas
Automatic merge from submit-queue
namespace deletion and kubelet restart tests for vSphere cloud provider
**What this PR does / why we need it**:
Following tests for vSphere cloud provider are added as part of this PR
**Test to verifiy that a volume mounted to a pod that is deleted while the kubelet is down, unmounts volume when the kubelet returns**
**Steps:**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node. and verify volume is mounted on the node.
6. Stop kubelet.
7. Delete pod.
8. Start kubelet.
9. Verify that volume mount not to be found on the node.
**Test to verify that deleting the Namespace of a PVC and Pod causes the successful detach of Persistent Disk.**
**Steps:**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node.
6. Delete Namespace.
7. Wait for namespace to get deleted. (Namespace deletion should trigger deletion of belonging pods)
8. Verify volume should be detached from the node.
Moved test from https://github.com/kubernetes/kubernetes/pull/41976
**Test to verify that a volume mounted to a pod remains mounted after a kubelet restarts.**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node.
6. Write to the volume
7. Restart kubelet on the node on which pod is provisioned.
8. Verify that written file is accessible after kubelet restart.
9. Delete pod.
10. Delete PV, PVC
11. Wait for VMDK to detach from node.
12. Delete VMDK
**Which issue this PR fixes**
fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc: @abrarshivani @BaluDontu @tusharnt @pdhamdhere
Automatic merge from submit-queue (batch tested with PRs 44500, 44457, 44553, 44267, 44617)
nits in service e2e test logging
Saw some weird logs from CI [gce-alpha-features](https://k8s-testgrid.appspot.com/google-gce#gce-alpha-features&width=5).
```
...
Health checking bootstrap-e2e-minion-group-3tm9, http://35.188.44.206:32733//healthz, expectedSuccess true
...
Expected %!!(MISSING)s(int=2) failures on bootstrap-e2e-minion-group-8h0q//healthz, got 0, err
...
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43000, 44500, 44457, 44553, 44267)
Updates e2e_node test to allow both kubenet and cni to be specified f…
…or the network plugin.
This adds a simple CNI configuration which is added to the node during test setup.
This also modifies the default flags in services/kubelet.go to specify the "cni-bin-dir"
and the "cni-conf-dir" and removes the "network-plugin-dir" flag. This leaves the default
network plugin to kubenet.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
TestEtcdStoragePath prevents the accidental movement of objects stored
in etcd. It creates a stub of each object and then checks the expected
location in etcd. Inadvertent GroupVersionKind changes are prevented.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Add the option to configure e2e access to member clusters from the
same secrets in the host cluster used by the federation control plane.
The default behavior will continue to be sourcing this configuration
from the e2e kubeconfig. The optional behavior can be enabled by
passing --federation-config-from-cluster=true as an argument to
ginkgo.
Automatic merge from submit-queue (batch tested with PRs 44469, 44566, 44467, 44526)
WaitForCacheSync before running attachdetach controller
@gnufied you wrote the test and @ncdc the TODO comment. Let's just run the pv and pvc informers, we do not care about them in this test. But we want to be able to stop the pod Informer at will, hence not just using informers.Start, is my understanding.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)
improve error handling in e2e helpers
**What this PR does / why we need it**:
Changes most of the volume related helper funcs to return error rather than calling `Expect`. This is a better programming practice, is consistent with Go and Kubernetes, and allows helper funcs that create multiple resources to perform cleanup.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)
Change the default CLUSTER_IP_RANGE used by e2e
The existing choice intersects with the range reserved for auto
subnets and cannot be used with some GCP features.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44569, 44398)
Move v1/refs and v1/resource
This PR moves pkg/api/v1/ref.go and pkg/api/v1/resource_helper.go to their own sub packages, it's very similar to 44299 and 44302.
The PR is mostly mechanical, except that
* i moved some utility function from resource.go to pkg/api/v1/pod and pkg/api/v1/node, as they are more appropriate
* i updated the staging/copy.sh to copy the new subpackages, so that helper functions are copied. We can get rid of this copy after client-go stops copying API types.
Automatic merge from submit-queue (batch tested with PRs 44519, 43194, 44513)
[Federation] Add type-agnostic e2e crud test
This PR proposes an e2e test that reuses the type-agnostic crudtester already used for integration testing to validate crud against a deployed cluster. It is intended to to eventually replace the existing e2e tests for simple types like secrets, but for now will run in addition to the existing testing to gain confidence in the coverage it provides.
The deletion corner cases - when orphanDependents is nil or true - do not involve operations in the member clusters and are already well-tested in integration testing and so are not reimplemented here.
Where it can be applied, this approach of abstracting a test from its execution environment - making the test 'retargetable' - can reduce the cost of test development since the bulk of the work can be iterated on as an integration test. It can also serve as a check on assumptions made in the integration test about how a deployed environment will behave.
cc: @kubernetes/sig-federation-pr-reviews @kubernetes/sig-testing-misc @smarterclayton @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 44364, 44361, 42498)
Move v1 helpers
The first 3 commits are other PRs.
This PR move pkg/api/v1/helpers.go to a subpackage, which is almost symmetric to #44296, where pkg/api/helpers.go was moved.
This PR is mostly mechanic, except that
1. moved the 3 methods of Taint and Toleration to pkg/api/methods.go
2. moved constants and types defined in v1/helpers.go to pkg/api/v1/annotataion_key_constants.go and nonstandard_types.go
3. updated staging/copy.sh to copy pkg/api/helpers to client-go, it's otherwise removed from client-go because no other code in client-go depends on the package. Some test code in pkg/controller imports client-go/pkg/api/helpers. After moving api types to its own repo, we can remove these copies of utility function from client-go and ask users to use the ones in the main repo.
(This PR breaks a cyclic import problem i met when I tried to move global variables pkg/api/Scheme and Registry to a subpackage)
