Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
Add explicit image tag to cockroachdb example and test
@gyliu513
```release-note
NONE
```
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.
Module remotecommand relies on util/exec module which will be copied to client-go/pkg/util
Automatic merge from submit-queue
Reorganize kubelet tree so apis can be independently versioned
@yujuhong @lavalamp @thockin @bgrant0607
This is an example of how we might reorganize `pkg/kubelet` so the apis it exposes can be independently versioned. This would also provide a logical place to put the `KubeletConfiguration` type, which currently lives in `pkg/apis/componentconfig`; it could live in e.g. `pkg/kubelet/apis/config` instead.
Take a look when you have a chance and let me know what you think. The most significant change in this PR is reorganizing `pkg/kubelet/api` to `pkg/kubelet/apis`, the rest is pretty much updating import paths and `BUILD` files.
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Promotes Source IP preservation for Virtual IPs from Beta to GA
Fixes#33625. Feature issue: kubernetes/features#27.
Bullet points:
- Declare 2 fields (ExternalTraffic and HealthCheckNodePort) that mirror the ESIPP annotations.
- ESIPP alpha annotations will be ignored.
- Existing ESIPP beta annotations will still be fully supported.
- Allow promoting beta annotations to first class fields or reversely.
- Disallow setting invalid ExternalTraffic and HealthCheckNodePort on services. Default ExternalTraffic field for nodePort or loadBalancer type service to "Global" if not set.
**Release note**:
```release-note
Promotes Source IP preservation for Virtual IPs to GA.
Two api fields are defined correspondingly:
- Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
- Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
```
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Revert #45101 Mark PersistentVolumes as [Feature:Volumes]
**What this PR does / why we need it**:
Reverts #45101
`Feature` tag should only be used when a test/suite has dependencies not met by core CI. That is not the case for NFS backed PV tests.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45685, 45572, 45624, 45723, 45733)
Remove a test utility function that is redundant and kinda broken
Framework.WaitForAnEndpoint() has no timeout, so if something goes wrong and the endpoint doesn't get created, the test will hang forever. (This is happening for some reason sometimes in OpenShift right now, and when the CI system eventually times out and kills the VM, it loses the logs that would explain what failed.)
There's already another nearly-identical WaitForEndpoint() method that *does* take a timeout, so people can just use that instead.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45653, 45719, 45729, 45730, 44250)
Print pod startup latency metric as perfdata
Follows #45657
This should print pod startup latency in same format as api calls latencies.
cc @wojtek-t @gmarek
Automatic merge from submit-queue
move from daemon_restart.go to framework/util.go
**What this PR does / why we need it**:
Moves the func `nodeExec` from daemon_restart.go to framework/util.go. This is the correct file for this func and is a more intuitive pkg for other callers to use. This is a small step of the larger effort of restructuring e2e tests to be more logically structured and easier for newcomers to understand.
```release-note
NONE
```
cc @timothysc @copejon
Automatic merge from submit-queue
small change to view more test info
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
small change to view more test info, think you very much
**Release note**:
```release-note
```
Automatic merge from submit-queue
util.go: format for
**What this PR does / why we need it**:
format for.
delete redundant para.
make code clean.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43067, 45586, 45590, 38636, 45599)
Make SchedulerPredicates test more resiliant to recent Node restarts
cc @kubernetes/sig-scheduling-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45200, 45203)
Add .svc.cluster.local for service host
**What this PR does / why we need it**:
Running into DNS resolve problems, testing to see if this works better
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)
Separate healthz server from metrics server in kube-proxy
From #14661, proposal is on kubernetes/community#552.
Couple bullet points as in commit:
- /healthz will be served on 0.0.0.0:10256 by default.
- /metrics and /proxyMode will be served on port 10249 as before.
- Healthz handler will verify timestamp in iptables mode.
/assign @nicksardo @bowei @thockin
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
Use dedicated UnixUserID and UnixGroupID types
**What this PR does / why we need it**:
DRYs up type definitions by using the dedicated types in apimachinery
**Which issue this PR fixes**
#38120
**Release note**:
```release-note
UIDs and GIDs now use apimachinery types
```
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
remove mvPodUidDir test
**What this PR does / why we need it**:
Removes a [HostCleanup] test which deleted the pod's uid directory and checked to see that the host was cleaned up. This test consistently failed on gci (COS) due to OS differences related to handling mounting (containerized on COS) and rm'ing files being accessed. See #44592 for more details.
```release-note
NONE
```
cc. @msau42 @copejon
Automatic merge from submit-queue (batch tested with PRs 45364, 45369, 45323)
Fix namespace deletion log in e2e test framework.
**What this PR does / why we need it**:
Current log message for namespace deletion will show under wrong condition.
**Which issue this PR fixes**
**Release note**:
NONE
Automatic merge from submit-queue
Deployment e2e fixes
* don't expect availability on Deployment creations
* switch to framework.Logf everywhere for consistency
* better use of Expect
Automatic merge from submit-queue
Add e2e network policy tests.
**What this PR does / why we need it**:
Add set of e2e tests for Network Policy. This has succesfully run againast a deployment using Calico as the network policy provider.
Specifically, adds a new e2e test file (/test/e2e/network_policy.go) which tests TCP connectivity between pods with isolation.
See #25638 for PR that added NetworkPolicy resource.
This PR is a replacement for #27447 .
Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)
Increase pv test ns deletion timeout to the framework's default.
**What this PR does / why we need it**:
Namespace deletion occasionally exceeds the test's timeout of 3 minutes. The test framework uses a default timeout of 5 minutes, so just making it match here.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44790
**Special notes for your reviewer**:
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 44062, 45165)
Run SSH e2e test only if SSH keys are present
**What this PR does / why we need it**:
Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.
Automatic merge from submit-queue
Add google.com to e2e test only under gce/gke
We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.
Automatic merge from submit-queue (batch tested with PRs 44044, 44766, 44930, 45109, 43301)
e2e test: test azure disk volume
**What this PR does / why we need it**:
E2E test Azure disk volume
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
@kubernetes/sig-testing-pr-reviews
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
e2e: Remove special serviceaccount permission requirement
**What this PR does / why we need it**:
This removes the requirement for having a `default` serviceaccount with permission `create` `pod` in order to run the prestop e2e conformance test.
The `create` `pod` permissions are not needed for this particular test, and having it effectively means that as cluster must have a `default` serviceaccount with `create` `pod` permissions in order to be conformant, which I don't think is desired.
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 45033, 44961, 45021, 45097, 44938)
Disable the kubelet part of metrics collection in kubemark
Fixes https://github.com/kubernetes/kubernetes/issues/45038
This should fix it, as we are just interested in getting the apiserver metrics from kubemark master.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45033, 44961, 45021, 45097, 44938)
Add request count to APICall metric
Ref https://github.com/kubernetes/kubernetes/issues/44701
This should add beside the API call latencies, the count of the requests.
cc @wojtek-t @gmarek
* don't expect availability on Deployment creations
* switch to framework.Logf everywhere for consistency
* better use of Expect
Signed-off-by: Michail Kargakis <mkargaki@redhat.com>
Just so that we know that we need a cloud provider that
supports volumes to run this test. This is similar to
the change in 63bc42c872.
Ran into this when i was trying to run e2e tests with
local-up-cluster locally and figured out this test will
not work since we don't support local storage persistent
volumes.
Automatic merge from submit-queue
Fix cockroachdb statefulset test read/write commands
Explicitly specifying `--insecure` is required on insecure clusters,
which started being enforced in a very recent release. In 2 weeks
we'll have a stable image version that we can reliably pin the
relevant statefulset yaml file to in order to avoid stupid failures
like this. I'm really sorry for the flakes!
**What this PR does / why we need it**:
It fixes the currently broken statefulset test suite - https://storage.googleapis.com/k8s-gubernator/triage/index.html?job=gci-gce-statefulset&test=CockroachDB
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
N/A
**Special notes for your reviewer**:
N/A
**Release note**:
```release-note
NONE
```
@kow3ns
Automatic merge from submit-queue (batch tested with PRs 41106, 44346, 44929, 44979, 45027)
Log error before failing in autoscaling e2e
The gcloud alpha command in e2e fails, but no useful information (error message) is logged.
Automatic merge from submit-queue (batch tested with PRs 41106, 44346, 44929, 44979, 45027)
Make metrics filenames for e2e tests indicate the test better
Currently the names of the json files with metrics for e2e tests are named by appending the `SummaryKind` with a timestamp of the test. It took me some time to figure out which file corresponds to which e2e test due to this. Changing it to have the testname instead of the timestamp.
cc @wojtek-t @gmarek
Automatic merge from submit-queue
Increase timeout for Stackdriver Logging e2e tests
They're failing in CI, because Stackdriver Logging's List method is too slow for this purpose. Quick fix, should be gone completely when reading is implemented properly
/cc @piosz
Explicitly specifying `--insecure` is required on insecure clusters,
which started being enforced in a very recent release. In 2 weeks
we'll have a stable image version that we can reliably pin the
relevant statefulset yaml file to in order to avoid stupid failures
like this. I'm really sorry for the flakes!
Automatic merge from submit-queue
Cleanup: replace some hardcoded vars and remove unused functions
Replace some hardcoded vars and remove unused functions
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42477, 44462)
Use storage.v1 instead of v1beta1
storage.v1beta1 was used to work around GKE which did not expose v1. Now that GKE is updated, we can switch everything to v1.
This is simple sed v1beta1 -> v1 + enabled a new test + changed preference of exposed interfaces in `storage/install/install.go`.
@msau42, PTAL and let me know when GKE is updated with storage v1 API and this PR can be actually merged.
@kubernetes/sig-storage-pr-reviews
```release-note
NONE
```
Automatic merge from submit-queue
Move remain ESIPP tests to the slow suite
Continue PR of #38149.
It moves the remain ESIPP tests to the slow suite to help capture breaking changes.
/assign @thockin @freehan
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Stop treating in-cluster-config namespace as an override
Fixes#44835
The namespace of an in-cluster config should behave like the namespace specified in a kubeconfig file... it should be used as the default namespace, but be able to be overridden by namespaces specified in yaml files passed to `kubectl create -f`.
```release-note
Restored the ability of kubectl running inside a pod to consume resource files specifying a different namespace than the one the pod is running in.
```
Automatic merge from submit-queue
Explicit namespace from kubeconfig should override in-cluster config
Fixes#43662
If an explicitly specified namespace is read from a kubeconfig file, we should not fall back to in-cluster config
```release-note
kubectl commands run inside a pod using a kubeconfig file now use the namespace specified in the kubeconfig file, instead of using the pod namespace. If no kubeconfig file is used, or the kubeconfig does not specify a namespace, the pod namespace is still used as a fallback.
```
Automatic merge from submit-queue
Mark Stackdriver Logging e2e tests with a feature
Makes Stackdriver Logging e2e tests, except for the most basic one, run in the separate tests suites, prepared by https://github.com/kubernetes/test-infra/pull/2542
Automatic merge from submit-queue
De-Flake Volume E2E: force GCEPD detach to prevent timeout
**What this PR does / why we need it**:
Fix flake`[k8s.io] Volumes [Volume] [k8s.io] PD should be mountable [Flaky] 5m38s.
Flake occurs as a result of an automated detach taking longer than 5 minutes, which exceeds the timeout limit of the delete function.
This PR adds explicit detach and wait func calls before the deletion. By forcing the detach and giving GCE an appropriate timeout limit, this should squash the timeout flake. This also significantly shortens cleanup time.
This PR does not remove the [Flaky] tag. Once this PR is merged, I'll keep an eye on the test grid for ~1 week. If no flakes surface, I'll submit a PR to pull the tag off.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43977
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42177, 42176, 44721)
Job: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings Job into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that Job does not fight with other controllers over control of Pods.
Ref: #24433
**Special notes for your reviewer**:
**Release note**:
```release-note
Job controller now respects ControllerRef to avoid fighting over Pods.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42177, 42176, 44721)
CronJob: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings CronJob into compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that other controllers do not fight over control of objects that a CronJob owns.
**Special notes for your reviewer**:
**Release note**:
```release-note
CronJob controller now respects ControllerRef to avoid fighting with other controllers.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue
adding test for volume fstype validation
**What this PR does / why we need it**:
This PR is adding a test for volume fstype validation. Test verifies fstype specified in storage-class is being honored after volume creation.
Steps:
1. Create StorageClass with fstype set to valid type (default case included).
2. Create PVC which uses the StorageClass created in step 1.
3. Wait for PV to be provisioned.
4. Wait for PVC's status to become Bound.
5. Create pod using PVC on specific node.
6. Wait for Disk to be attached to the node.
7. Execute command in the pod to get fstype.
8. Delete pod and Wait for Volume Disk to be detached from the Node.
9. Delete PVC, PV and Storage Class.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
cc: @jeffvance @tusharnt
Automatic merge from submit-queue (batch tested with PRs 42272, 44696)
e2e test fix: Wait longer when first creating ELB
On any cloud (GCE or AWS), a lag between creating the LoadBalancer and
having it actually start serving traffic is expected. On AWS the lag is
larger, and we weren't correctly using the longer wait on our first
request.
Use a longer wait period on our first request.
Fix#44695
```release-note
NONE
```
Automatic merge from submit-queue
e2e: Prefer kubeconfig host to default
Previously it was necessary to pass ``-host`` to ``e2e.test`` even if ``-kubeconfig`` was specified since otherwise a localhost default would be used. This change ensures that the default is only used when kubeconfig is not set.
cc: @jayunit100
On any cloud (GCE or AWS), a lag between creating the LoadBalancer and
having it actually start serving traffic is expected. On AWS the lag is
larger, and we weren't correctly using the longer wait on our first
request.
Use a longer wait period on our first request.
Fix#44695
Automatic merge from submit-queue (batch tested with PRs 43500, 44073)
[Federation] Add option to retrieve e2e cluster config from secrets
Previously the federation e2e setup was reading member cluster configuration from the test run's kubeconfig. This change removes that dependency in favor of reading member cluster configuration from secrets in the hosting cluster, and caches the configuration to avoid having to read it separately for each test.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue
namespace deletion and kubelet restart tests for vSphere cloud provider
**What this PR does / why we need it**:
Following tests for vSphere cloud provider are added as part of this PR
**Test to verifiy that a volume mounted to a pod that is deleted while the kubelet is down, unmounts volume when the kubelet returns**
**Steps:**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node. and verify volume is mounted on the node.
6. Stop kubelet.
7. Delete pod.
8. Start kubelet.
9. Verify that volume mount not to be found on the node.
**Test to verify that deleting the Namespace of a PVC and Pod causes the successful detach of Persistent Disk.**
**Steps:**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node.
6. Delete Namespace.
7. Wait for namespace to get deleted. (Namespace deletion should trigger deletion of belonging pods)
8. Verify volume should be detached from the node.
Moved test from https://github.com/kubernetes/kubernetes/pull/41976
**Test to verify that a volume mounted to a pod remains mounted after a kubelet restarts.**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node.
6. Write to the volume
7. Restart kubelet on the node on which pod is provisioned.
8. Verify that written file is accessible after kubelet restart.
9. Delete pod.
10. Delete PV, PVC
11. Wait for VMDK to detach from node.
12. Delete VMDK
**Which issue this PR fixes**
fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc: @abrarshivani @BaluDontu @tusharnt @pdhamdhere
Automatic merge from submit-queue (batch tested with PRs 44500, 44457, 44553, 44267, 44617)
nits in service e2e test logging
Saw some weird logs from CI [gce-alpha-features](https://k8s-testgrid.appspot.com/google-gce#gce-alpha-features&width=5).
```
...
Health checking bootstrap-e2e-minion-group-3tm9, http://35.188.44.206:32733//healthz, expectedSuccess true
...
Expected %!!(MISSING)s(int=2) failures on bootstrap-e2e-minion-group-8h0q//healthz, got 0, err
...
```
**Release note**:
```release-note
NONE
```
Add the option to configure e2e access to member clusters from the
same secrets in the host cluster used by the federation control plane.
The default behavior will continue to be sourcing this configuration
from the e2e kubeconfig. The optional behavior can be enabled by
passing --federation-config-from-cluster=true as an argument to
ginkgo.
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)
improve error handling in e2e helpers
**What this PR does / why we need it**:
Changes most of the volume related helper funcs to return error rather than calling `Expect`. This is a better programming practice, is consistent with Go and Kubernetes, and allows helper funcs that create multiple resources to perform cleanup.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)
Change the default CLUSTER_IP_RANGE used by e2e
The existing choice intersects with the range reserved for auto
subnets and cannot be used with some GCP features.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44364, 44361, 42498)
Move v1 helpers
The first 3 commits are other PRs.
This PR move pkg/api/v1/helpers.go to a subpackage, which is almost symmetric to #44296, where pkg/api/helpers.go was moved.
This PR is mostly mechanic, except that
1. moved the 3 methods of Taint and Toleration to pkg/api/methods.go
2. moved constants and types defined in v1/helpers.go to pkg/api/v1/annotataion_key_constants.go and nonstandard_types.go
3. updated staging/copy.sh to copy pkg/api/helpers to client-go, it's otherwise removed from client-go because no other code in client-go depends on the package. Some test code in pkg/controller imports client-go/pkg/api/helpers. After moving api types to its own repo, we can remove these copies of utility function from client-go and ask users to use the ones in the main repo.
(This PR breaks a cyclic import problem i met when I tried to move global variables pkg/api/Scheme and Registry to a subpackage)
Automatic merge from submit-queue (batch tested with PRs 44362, 44421, 44468, 43878, 44480)
use sudo in mv ssh cmd
**What this PR does / why we need it**:
Fixes _HostCleanup_ kubelet e2e test where `sudo` was missing from a ssh `mv` command.
The exact test is:
`host cleanup with volume mounts [Volume][HostCleanup][Flaky] Host cleanup after disrupting NFS volume [NFS] move NFS client pod's UID directory then delete pod` This test has been consistently red in the gce-flaky suite.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)
Make resource gatherer get data about etcd resource usage in kubemark…
… setup
Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)
Try in cluster config when input KubeConfig is empty
**What this PR does / why we need it**:
Allows for downstream providers to run e2es "incluster" sans kubeconfig
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
/cc @kubernetes/sig-testing-pr-reviews @marun
Automatic merge from submit-queue
Segregate storage e2es for easier assignment and dep isolation.
**What this PR does / why we need it**:
Follow on from slack conversations to isolate storage deps and have easier assignment of e2es.
As of today there is general consensus that the e2es need to be broken apart, this is 1st iteration on the sig-storage test splitting. Follow on work is needed here.
**Release note**:
```
NONE
```
/cc @saad-ali @kubernetes/sig-storage-pr-reviews @kubernetes/sig-testing-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)
Decouple remotecommand
Refactored unversioned/remotecommand to decouple it from undesirable dependencies:
- term package now is not required, and functionality required to resize terminal size can be plugged in directly in kubectl
- in order to remove dependency on kubelet package - constants from kubelet/server/remotecommand were moved to separate util package (pkg/util/remotecommand)
- remotecommand_test.go moved to pkg/client/tests module
Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)
Clean up pre-ControllerRef compatibility logic
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#43323
**Special notes for your reviewer**:
No
**Release note**:
```
NONE
```
Automatic merge from submit-queue
add kubelet tests to verify host clean up
**What this PR does / why we need it**:
Increasingly we're seeing more failures in persistent volume e2e tests where pv tests are run in parallel with disruptive tests. The quick solution is to tag the pv tests as Flaky. This pr addresses one cause of the flakiness and adds a disruptive kubelet test.
Once this pr is shown to not produce flakes the [Flaky] tag for the "HostCleanup" tests will be removed in a separate pr.
+ Adds volume tests to _kubelet.go_ motivated by issues [31272](https://github.com/kubernetes/kubernetes/issues/31272) and [37657](https://github.com/kubernetes/kubernetes/issues/37657)
+ Addresses reverted pr [41178](https://github.com/kubernetes/kubernetes/pull/41178) and negates the need for pr [41229](https://github.com/kubernetes/kubernetes/pull/41229)
**Which issue this PR fixes**
Adds regression tests to cover issues: #31272 and #37657
**Special notes for your reviewer**:
It's possible that one of the new tests, which relies on the existence of _/usr/sbin/rpc.nfsd_ in the nfs-server pod, will not work in the GCI container env. If this turns out to be true then I will add a `SkipIfProviderIs("gke")` to the `It` block.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
[e2e] Bump up pod deletion time for source pod IP test
From #44225.
Source pod IP e2e test is pretty flaky lately, and most of the failures seem to be timeout waiting for pod "kube-proxy-mode-detector" to disappear. Didn't found any other thing suspicious.
This PR bumps pod deletion timeout to DefaultPodDeletionTimeout, which is 3 minutes. Hopefully it will mitigate the flakes.
/assign @freehan
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
local up dns defaults/Privileged defaults so that [Conformance] sets mostly pass on local clusters.
Fixes#43651 So that only 4 tests fail out of the box.
Automatic merge from submit-queue
Adding load balancer src cidrs to GCE cloudprovider
**What this PR does / why we need it**:
As of January 31st, 2018, GCP will be sending health checks and l7 traffic from two CIDRs and legacy health checks from three CIDS. This PR moves them into the cloudprovider package and provides a flag for override.
Another PR will need to be address firewall rule creation for external L4 network loadbalancing #40778
**Which issue this PR fixes**
Step one of #40778
Step one of https://github.com/kubernetes/ingress/issues/197
**Release note**:
```release-note
Add flags to GCE cloud provider to override known L4/L7 proxy & health check source cidrs
```
Automatic merge from submit-queue
In 'kubectl describe', find controllers with ControllerRef, instead of showing the original creator
@enisoc @kargakis @kubernetes/sig-apps-pr-reviews @kubernetes/sig-cli-pr-reviews
```release-note
In 'kubectl describe', find controllers with ControllerRef, instead of showing the original creator.
```
Automatic merge from submit-queue
Remove alphaProvisioner in PVController and AlphaStorageClassAnnotation
remove alpha annotation and alphaProvisioner
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Remove [Flaky] from presistent volume NFS tests.
**What this PR does / why we need it**:
PV e2e test flaked because of a lack of isolation of test objects (PVs, claims). The common symptoms being volume-mounting pods timing out or an unexpected number of unbound claims being detected.
PR #43645 Introduced the selector labels to test objects that restricted binds to within each "testspace". To accomplish this the test namespace was set as a label for all PVs and selector for all Claims. This allows each test to act as if it's isolated while still creating and binding PVs in parallel w/ other tests.
This has been tested in parallel on both 3-node gci and debian clusters as
`--ginkgo.focus="\[Volume\]" --ginkgo.skip="\[Disruptive\]|\[Feature.*\]|\[Serial\]"`
with out signs of flakiness.
cc @jeffvance
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43304, 41427, 43490, 44352)
Node failure tests for cluster autoscaler
E2e tests checking whether CA is still working with a single broken node.
cc: @MaciekPytel @jszczepkowski @fgrzadkowski
Automatic merge from submit-queue (batch tested with PRs 43844, 44284)
Add a retry to cluster-autoscaler e2e
This should fix https://github.com/kubernetes/kubernetes/issues/44268.
The flake was caused by following sequence of events:
1. Cluster was at minimum size (3), some node was unneeded for a while.
2. Setup for some test (scale-down, failure) would increase node group size (to 5) and wait for new nodes to come up.
3. As soon as new node come up (cluster size 4) CA would scale-down the old unneeded node (setting node group size to 4).
4. Node group would not reach size 5 (as the target was now 4) and the test would timeout and fail.
This PR makes the setup monitor re-set the target node group size if the above scenario happens.
Automatic merge from submit-queue (batch tested with PRs 41758, 44137)
Removed hostname/subdomain annotation.
fixes#44135
```release-note
Remove `pod.beta.kubernetes.io/hostname` and `pod.beta.kubernetes.io/subdomain` annotations.
Users should use `pod.spec.hostname` and `pod.spec.subdomain` instead.
```
Automatic merge from submit-queue
test/e2e*: add/update README.md files
**What this PR does / why we need it**:
This PR is adding `README.md` files with a link to the documentation to all E2E tests.
Automatic merge from submit-queue
Add test for provisioning with storage class
This PR re-introduces e2e test for dynamic provisioning with storage classes.
It adds the same test as it was merged in PR #32485 with an extra patch adding region to AWS calls. It works well on my AWS setup, however I'm using shared company account and I can't run kube-up.sh and run the tests in the "official" way.
@zmerlynn, can you please try to run tests that led to #34961?
@justinsb, you're my AWS guru, would there be a way how to introduce fully initialized AWS cloud provider into e2e test framework? It would simplify everything. GCE has it there, but it's easier to initialize, I guess. See https://github.com/kubernetes/kubernetes/blob/master/test/e2e/pd.go#L486 for example - IMO tests should not talk to AWS directly.
Automatic merge from submit-queue (batch tested with PRs 43373, 41780, 44141, 43914, 44180)
move PD delete to AfterEach
**What this PR does / why we need it**:
Fixes a bug in PersistentVolume E2E that causes GCEPDs to be left un-deleted after a test ends.
```release-note
None
```
Automatic merge from submit-queue
fix wrong error return
Signed-off-by: Crazykev <crazykev@zju.edu.cn>
**What this PR does / why we need it**: The err return here is wrong, correct it.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 44191, 44117, 44072)
Fix pv upgrade test failure
**What this PR does / why we need it**:
2 changes were made that broke the pv upgrade test:
1. MakePod() was changed so that the volume mount point is at /mnt/volume<n> instead of /mnt. This change updates the path that the pod writes to.
2. The MakePod() call was changed to CreatePod(), which creates the pod and expects a long running pod. But the upgrade test uses a short running pod, and the call to TestContainerOutput() will also create the pod. This changes it back to MakePod()
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44168
**Special notes for your reviewer**:
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 41189, 43818)
Reduce deployment replicas in e2e tests
Fixes https://github.com/kubernetes/kubernetes/issues/41063
There are still two tests that run multiple replicas (testScaledRolloutDeployment, testIterativeDeployments), we may want to revisit them at some point.
Automatic merge from submit-queue (batch tested with PRs 44143, 44133)
Mark PD test as flaky.
**What this PR does / why we need it**: Marks "PD should be mountable" as flaky. See #43977, which shows that it has flaked at least 7 times in the last 3 days. @kubernetes/sig-storage-test-failures
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Extract e2e utility code into framework
**What this PR does / why we need it**:
There's a growing dependency on Volume e2e utilities related to creating / test against NFS volumes. For this reason, it's useful to relocate the relevant functions to the `framework` pkg. Doing so makes these utility functions available to e2e tests outside the `e2e` package.
This PR only moves code from the `e2e` package to `framework` and handle the relevant changes in calls. It does not change any logic.
```release-note
NONE
```
@jingxu97 I think there's value here in reducing duplicate code in the `common` package, given that these functions have been copied down to it. However, there's been some divergence. Can you PTAL and let me know if there's any reason we can't remove the duplicate `common` code?
cc @jeffvance
Automatic merge from submit-queue
Fix Cluster-Autoscaler e2e on testgrid
Fix an e2e test failing in CI. The failures were caused by maximum nodes in test env being lower than the number of nodes required by the test.
Automatic merge from submit-queue (batch tested with PRs 43963, 43965)
Wait for clean old RSs statuses in the middle of Recreate rollouts
After https://github.com/kubernetes/kubernetes/pull/43508 got merged, we started returning ReplicaSets with no pods but with stale statuses back to the rollout functions. As a consequence, one of our e2e tests that checks if a Recreate Deployment runs pods from different versions, started flakying because the Deployment status may be incorrect. This change simply waits for the statuses to get cleaned up before proceeding with scaling up the new RS.
Fixes https://github.com/kubernetes/kubernetes/issues/43864
@kubernetes/sig-apps-bugs
Automatic merge from submit-queue (batch tested with PRs 44104, 43903, 44109)
Make sure Teardown is called.
This will ensure that tests get a chance to clean up resources even if
setup failed part way through.
Automatic merge from submit-queue (batch tested with PRs 44097, 42772, 43880, 44031, 44066)
[Federation] Improve e2e test setup
This PR improves federation e2e test setup:
- reuses e2e framework setup (``NewDefaultFramework``) instead of duplicating it
- ensures ``FederationAfterEach`` is called if an error occurs in ``FederationBeforeEach`` (as per the [example](https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/framework.go#L161) of the e2e framework)
- skips creation of a test namespace in the hosting cluster (not used for a federation e2e test)
cc: @kubernetes/sig-federation-pr-reviews @kubernetes/sig-testing-pr-reviews
Moved remaining util functions
moved cinder specific function back to volumes.go, will have to be extracted later when a cinder e2e package is created.
remove dupe code from common/volume.go
Moved [Volume] tags to KubeDescribe
Automatic merge from submit-queue
Move autoscaling e2e tests to a separate directory
For fine-grain access control. Autoscaling team is expanding the e2e test coverage and the need for getting an approval for every PR is annoying.
cc: @MaciekPytel @jszczepkowski @fgrzadkowski @wojtek-t
Automatic merge from submit-queue
Support status.hostIP in downward API
**What this PR does / why we need it**:
Exposes pod's hostIP (node IP) via downward API.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes https://github.com/kubernetes/kubernetes/issues/24657
**Special notes for your reviewer**:
Not sure if there's more documentation that's needed, please point me in the right direction and I will add some :)
Automatic merge from submit-queue
test/e2e_node: prepull images with CRI
Part of https://github.com/kubernetes/kubernetes/issues/40739
- This PR builds on top of #40525 (and contains one commit from #40525)
- The second commit contains a tiny change in the `Makefile`.
- Third commit is a patch to be able to prepull images using the CRI (as opposed to run `docker` to pull images which doesn't make sense if you're using CRI most of the times)
Marked WIP till #40525 makes its way into master
@Random-Liu @lucab @yujuhong @mrunalp @rhatdan
Automatic merge from submit-queue (batch tested with PRs 42667, 43923)
Adding test to perform volume operations storm
**What this PR does / why we need it**:
Adding new test to perform volume operations storm
Test Steps
1. Create storage class for thin Provisioning.
2. Create 30 PVCs using above storage class in annotation, requesting 2 GB files.
3. Wait until all disks are ready and all PVs and PVCs get bind. (**CreateVolume** storm)
4. Create pod to mount volumes using PVCs created in step 2. (**AttachDisk** storm)
5. Wait for pod status to be running.
6. Verify all volumes accessible and available in the pod.
7. Delete pod.
8. wait until volumes gets detached. (**DetachDisk** storm)
9. Delete all PVCs. This should delete all Disks. (**DeleteVolume** storm)
10. Delete storage class.
This test will help validate issue reported at https://github.com/vmware/kubernetes/issues/71
**Which issue this PR fixes**
fixes #
**Special notes for your reviewer**:
executed test on 1.5.3 release with `VOLUME_OPS_SCALE` set to `5`
Will execute test with the changes made on PR - https://github.com/kubernetes/kubernetes/pull/42422, with the `VOLUME_OPS_SCALE` set to `30`
**Release note**:
```release-note
None
```
cc: @abrarshivani @BaluDontu @tusharnt @pdhamdhere @luomiao @kerneltime
