Automatic merge from submit-queue
Add node e2e tests for hostPid
For #44118. Add node e2e tests for hostPid.
cc @yujuhong @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue
Add test for provisioning with storage class
This PR re-introduces e2e test for dynamic provisioning with storage classes.
It adds the same test as it was merged in PR #32485 with an extra patch adding region to AWS calls. It works well on my AWS setup, however I'm using shared company account and I can't run kube-up.sh and run the tests in the "official" way.
@zmerlynn, can you please try to run tests that led to #34961?
@justinsb, you're my AWS guru, would there be a way how to introduce fully initialized AWS cloud provider into e2e test framework? It would simplify everything. GCE has it there, but it's easier to initialize, I guess. See https://github.com/kubernetes/kubernetes/blob/master/test/e2e/pd.go#L486 for example - IMO tests should not talk to AWS directly.
Automatic merge from submit-queue (batch tested with PRs 43373, 41780, 44141, 43914, 44180)
move PD delete to AfterEach
**What this PR does / why we need it**:
Fixes a bug in PersistentVolume E2E that causes GCEPDs to be left un-deleted after a test ends.
```release-note
None
```
Automatic merge from submit-queue
fix wrong error return
Signed-off-by: Crazykev <crazykev@zju.edu.cn>
**What this PR does / why we need it**: The err return here is wrong, correct it.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 44191, 44117, 44072)
[Federation] Cleanup e2e framework
This PR is intended to simplify maintenance of the federation e2e tests:
- refactor cluster helpers into framework so they can be reused by upgrade tests
- remove unused functions
- move backend pod management to the tests that require it rather than relying on the common cluster object
- use a slice instead of a map to manage cluster objects to make it simpler for tests that need it to find the primary cluster themselves
Commit ``fed: Refactor e2e cluster functions into framework for reuse`` was originally included in #43500, but is included here because it formed the basis for this cleanup.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue (batch tested with PRs 44191, 44117, 44072)
Fix pv upgrade test failure
**What this PR does / why we need it**:
2 changes were made that broke the pv upgrade test:
1. MakePod() was changed so that the volume mount point is at /mnt/volume<n> instead of /mnt. This change updates the path that the pod writes to.
2. The MakePod() call was changed to CreatePod(), which creates the pod and expects a long running pod. But the upgrade test uses a short running pod, and the call to TestContainerOutput() will also create the pod. This changes it back to MakePod()
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44168
**Special notes for your reviewer**:
**Release note**:
NONE
Automatic merge from submit-queue
[Federation] Add integration test for secrets
This PR adds an integration test for secrets that:
- performs create/read/update/delete on federation resources and validates that the changes are propagated to member clusters.
- uses an abstraction layer (fixture and adapter) to minimize the code required to support each federated type
- It should be possible to replace a test-specific adapter with a runtime adapter in the future (as per #41050)
- reuses fixture (federation api and clusters) across different resource types to minimize setup overhead
- on a fast machine, setup takes ~4s, and validating each type takes ~2s
- uses the [Subtest feature added in Go 1.7](https://blog.golang.org/subtests) to allow the test for a specific controller to be run in isolation
- ``make test-integration WHAT="federation -test.run=TestFederationCRUD/secret"``
Once this PR merges the test can be extended to target other federated types.
This PR targets #40705
cc: @kubernetes/sig-federation-pr-reviews @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 41189, 43818)
Reduce deployment replicas in e2e tests
Fixes https://github.com/kubernetes/kubernetes/issues/41063
There are still two tests that run multiple replicas (testScaledRolloutDeployment, testIterativeDeployments), we may want to revisit them at some point.
Automatic merge from submit-queue (batch tested with PRs 43871, 44053)
Fix original object mutation on patch retry
When applying a patch, the original state of the object and patch are captured so that retries can determine if an overlapping update was made to the object, and the patch API call should abort with a conflict.
However, the `strategicPatchObject -> applyPatchToObject -> StrategicMergeMapPatch -> mergeMap` call mutates both the original object map and the patch map, making them unsuitable for reusing for subsequent calls.
We saw this in a [downstream test](https://github.com/openshift/origin/blob/master/test/integration/patch_test.go) that exercises patch conflict retries, where the data being submitted in a patch was showing up in the original object data map.
Since mergeMap *also* mutates the patch map passed in (deletes patch directives as it encounters them), we *also* cannot reuse the patch map in `applyPatchToObject` once it has been used.
This PR:
* Builds `originalObjMap` separate from the initial patch application for later use in conflict detection
* Changes the `originalPatchMap` to a helper that returns a fresh map from original sources
* Adds an integration test that exercises retry of non-overlapping patches with patches containing `$patch` directives
Automatic merge from submit-queue (batch tested with PRs 44143, 44133)
Mark PD test as flaky.
**What this PR does / why we need it**: Marks "PD should be mountable" as flaky. See #43977, which shows that it has flaked at least 7 times in the last 3 days. @kubernetes/sig-storage-test-failures
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Extract e2e utility code into framework
**What this PR does / why we need it**:
There's a growing dependency on Volume e2e utilities related to creating / test against NFS volumes. For this reason, it's useful to relocate the relevant functions to the `framework` pkg. Doing so makes these utility functions available to e2e tests outside the `e2e` package.
This PR only moves code from the `e2e` package to `framework` and handle the relevant changes in calls. It does not change any logic.
```release-note
NONE
```
@jingxu97 I think there's value here in reducing duplicate code in the `common` package, given that these functions have been copied down to it. However, there's been some divergence. Can you PTAL and let me know if there's any reason we can't remove the duplicate `common` code?
cc @jeffvance
Automatic merge from submit-queue
Fix Cluster-Autoscaler e2e on testgrid
Fix an e2e test failing in CI. The failures were caused by maximum nodes in test env being lower than the number of nodes required by the test.
Automatic merge from submit-queue (batch tested with PRs 43963, 43965)
Wait for clean old RSs statuses in the middle of Recreate rollouts
After https://github.com/kubernetes/kubernetes/pull/43508 got merged, we started returning ReplicaSets with no pods but with stale statuses back to the rollout functions. As a consequence, one of our e2e tests that checks if a Recreate Deployment runs pods from different versions, started flakying because the Deployment status may be incorrect. This change simply waits for the statuses to get cleaned up before proceeding with scaling up the new RS.
Fixes https://github.com/kubernetes/kubernetes/issues/43864
@kubernetes/sig-apps-bugs
Automatic merge from submit-queue (batch tested with PRs 44104, 43903, 44109)
Make sure Teardown is called.
This will ensure that tests get a chance to clean up resources even if
setup failed part way through.
Automatic merge from submit-queue (batch tested with PRs 44097, 42772, 43880, 44031, 44066)
[Federation] Improve e2e test setup
This PR improves federation e2e test setup:
- reuses e2e framework setup (``NewDefaultFramework``) instead of duplicating it
- ensures ``FederationAfterEach`` is called if an error occurs in ``FederationBeforeEach`` (as per the [example](https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/framework.go#L161) of the e2e framework)
- skips creation of a test namespace in the hosting cluster (not used for a federation e2e test)
cc: @kubernetes/sig-federation-pr-reviews @kubernetes/sig-testing-pr-reviews
```release-note
kube-apiserver: --service-account-lookup now defaults to true. This enables service account tokens to be revoked by deleting the Secret object containing the token.
```
- reuse default framework setup rather than duplicating
- skip namespace creation for each test in hosting cluster
- ensure FederationAfterEach is called if BeforeEach fails
Moved remaining util functions
moved cinder specific function back to volumes.go, will have to be extracted later when a cinder e2e package is created.
remove dupe code from common/volume.go
Moved [Volume] tags to KubeDescribe
Automatic merge from submit-queue
node e2e: improve the validate OOM score test for infra containers
The test blindly checked all "pause" processes on the node, assuming
they were all infra containers. This change takes a snapshot of all
existing "pause" processes on the node, and exclude them in the
validation. The test still relies on the fact that it runs exclusively
on the node. If that assumption changes, we will need other methods to
locate the PIDs of the infra containers.
This fixes#37580
Automatic merge from submit-queue
Move autoscaling e2e tests to a separate directory
For fine-grain access control. Autoscaling team is expanding the e2e test coverage and the need for getting an approval for every PR is annoying.
cc: @MaciekPytel @jszczepkowski @fgrzadkowski @wojtek-t
Automatic merge from submit-queue
Support status.hostIP in downward API
**What this PR does / why we need it**:
Exposes pod's hostIP (node IP) via downward API.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes https://github.com/kubernetes/kubernetes/issues/24657
**Special notes for your reviewer**:
Not sure if there's more documentation that's needed, please point me in the right direction and I will add some :)
Automatic merge from submit-queue
Add/remove kargakis from a couple of places
I've replaced myself in some tests in test_owners with the actual owners of those tests and I've also picked up a bunch of deployment tests. Also due to lack of review bandwidth I am removing myself from sig_cli reviewers.
@janetkuo
Automatic merge from submit-queue
test/e2e_node: prepull images with CRI
Part of https://github.com/kubernetes/kubernetes/issues/40739
- This PR builds on top of #40525 (and contains one commit from #40525)
- The second commit contains a tiny change in the `Makefile`.
- Third commit is a patch to be able to prepull images using the CRI (as opposed to run `docker` to pull images which doesn't make sense if you're using CRI most of the times)
Marked WIP till #40525 makes its way into master
@Random-Liu @lucab @yujuhong @mrunalp @rhatdan
Automatic merge from submit-queue
tests: e2e-node: refactor node-problem-detector test to avoid selinux…
Fixes https://github.com/kubernetes/kubernetes/issues/43401
This test creates a file in /tmp on the host and creates a HostPath volume in the container to so that the host can inject messages into the logfile being read by the node problem detector in the container. However, selinux prohibits the container from reading files out of /tmp on the host.
This PR modifies the test to create the log file in an EmptyDir volume instead, which will be properly labeled for container access.
@derekwaynecarr
Automatic merge from submit-queue
Add derekwaynecarr to approvers list for test/e2e_node
I am a maintainer and have approve rights in `pkg/kubelet`.
/cc @vishh
Automatic merge from submit-queue (batch tested with PRs 42667, 43923)
Adding test to perform volume operations storm
**What this PR does / why we need it**:
Adding new test to perform volume operations storm
Test Steps
1. Create storage class for thin Provisioning.
2. Create 30 PVCs using above storage class in annotation, requesting 2 GB files.
3. Wait until all disks are ready and all PVs and PVCs get bind. (**CreateVolume** storm)
4. Create pod to mount volumes using PVCs created in step 2. (**AttachDisk** storm)
5. Wait for pod status to be running.
6. Verify all volumes accessible and available in the pod.
7. Delete pod.
8. wait until volumes gets detached. (**DetachDisk** storm)
9. Delete all PVCs. This should delete all Disks. (**DeleteVolume** storm)
10. Delete storage class.
This test will help validate issue reported at https://github.com/vmware/kubernetes/issues/71
**Which issue this PR fixes**
fixes #
**Special notes for your reviewer**:
executed test on 1.5.3 release with `VOLUME_OPS_SCALE` set to `5`
Will execute test with the changes made on PR - https://github.com/kubernetes/kubernetes/pull/42422, with the `VOLUME_OPS_SCALE` set to `30`
**Release note**:
```release-note
None
```
cc: @abrarshivani @BaluDontu @tusharnt @pdhamdhere @luomiao @kerneltime
Per Clayton's suggestion, move stuff from cluster/lib/util.sh to
hack/lib/util.sh. Also consolidate ensure-temp-dir and use the
hack/lib/util.sh implementation rather than cluster/common.sh.
Automatic merge from submit-queue (batch tested with PRs 42662, 43035, 42578, 43682)
Apply [Feature:Volumes] to subset of tests in volumes.go
**What this PR does / why we need it**:
According to [this test doc](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md#kinds-of-tests), the `[Feature:xyz]` tag indicates that the tests have a non-default requirements and should not be run in the core suite. Previously all test in _e2e/volumes.go_ were tagged `[Feature:Volumes]` but I believe only six of the tests warrant this tag: iSCSI, GlusterFS, Ceph-RBD, Ceph-FS, vSPhere, and Cinder. The remaining tests (NFS, PD, ConfigMap) do not need the `Feature:` tag.
**Special notes for your reviewer**:
I moved the `[Volume]` tags from the `It`'s to the outer `KubeDescribe` to simplify the `It` text and remove redundancy.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42662, 43035, 42578, 43682)
[Federation] Fix a typo in the ReplicaSet E2E test: s/extrace/extract/
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43726, 43643)
Make a smaller redis image for testing, based on Alpine.
**What this PR does / why we need it**:
This shrinks gcr.io/google_containers/redis from 400MB to 5MB, which should reduce flakes.
**Which issue this PR fixes**:
fixes#43631
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42617, 43247, 43509, 43644, 43820)
Adding a SSH tunnel check to GKE upgrades
**What this PR does / why we need it**: After an upgrade, it takes some time fore the master to re-establish SSH tunnels to the nodes. This adds a wait for GKE since it runs in this configuration.
**Which issue this PR fixes**: fixes#43611, #43612
Automatic merge from submit-queue (batch tested with PRs 38741, 41301, 43645, 43779, 42337)
De-Flake PersistentVolume E2E: Isolate resources with selectors
PersistentVolume tests continue to flake because tests Claims often bind to other, parallel tests' volumes. This is addressed by isolating test resource with selector labels specific to each test namespace. Doing so enables deterministic binding within each test space and allows tests to run in parallel.
1. Assign selector label to volumes and claims per test.
2. Remove `[Serial]` tag
cc @jeffvance
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 38741, 41301, 43645, 43779, 42337)
replicaset adopt and release e2e tests
**What this PR does / why we need it**: adds e2e tests for replicaset pod adoption and release
**Which issue this PR fixes** fixes#39962
**Special notes for your reviewer**: for adoption pod is created, a replicaset with a selector matching the pod's label is created and then we check for the pod being in the list of pods of the replicaset. For release, a replicaset with two replicas is created, the label of one of them is patched and then we check that that pod is no longer in the list of pods of the replicaset.
For checking the list of pods the `framework.PodsCreated` is used, maybe the name is not very adecuate given that it is used to return the pods belonging to one replicaset, no matter how they were created. Also, the patching is done through `framework.KubectlOrDie`, not sure if this is better that calling the API directly.
cc @liggitt
Comments, log lines, exported MakePersistentVolume
Moved pv/pvcConfig assignment out of diskName check, nil ptrs and configs afterEach
change label/selector code to use k8s defined types from generic string:string maps
adjust for gce refactor
Automatic merge from submit-queue
Extract GCEPD pv tests
**What this PR does / why we need it**:
This is strictly a refactor moving the GCEPD suite in Persistent Volumes E2E to it's own file. This will make future provider specific additions to pv testing much more organized and readable. It will also enable a smoother transition to providers moving out of tree by consolidating related tests.
```release-note
NONE
```
Automatic merge from submit-queue
Volume Provisioning E2E: test PVC delete causes PV delete
**What this PR does / why we need it**:
Test for a regression addressed in #21268. There was a case where the PVC being created and deleted quickly may result in a provisioned PV left behind as `Available.`
```release-note
NONE
```
cc @jeffvance
