Automatic merge from submit-queue (batch tested with PRs 68161, 68023, 67909, 67955, 67731). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
CSI: skip attach for non-attachable drivers
**What this PR does / why we need it**:
This is implementation of https://github.com/kubernetes/community/pull/2523. CSI volumes that don't need attach/detach now don't need external attacher running.
WIP:
* contains #67803 to get CSIDriver API. Ignore the first commit.
* ~~missing e2e test~~
/sig storage
cc: @saad-ali @vladimirvivien @verult @msau42 @gnufied @davidz627
**Release note**:
```release-note
CSI volume plugin does not need external attacher for non-attachable CSI volumes.
```
Automatic merge from submit-queue (batch tested with PRs 68161, 68023, 67909, 67955, 67731). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Replace git volume with configmap in emptydir wrapper conflict test
**What this PR does / why we need it**: GitRepoVolumeSource is deprecated, use a ConfigMap instead. (This test is part of the conformance suite, so it would be good to allow downstreams to disable/not support gitRepo)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67736, 68123, 68138). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Update external provisioner test to use latest nfs-provisioner
**What this PR does / why we need it**: latest nfs-provisioner will work with cri-containerd, so let's update it
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**: I want to move this test to use nfs-client-provisioner soon anyway since a lot of our e2e tests already use a containerized nfs server and it would be good to be consistent. So this can be treated as something of a stopgap but it would be nice to have ASAP to unblock https://github.com/kubernetes-incubator/external-storage/issues/432#issuecomment-417511065
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix hostpath subpath reconstruction tests are failing
**What this PR does / why we need it**:
Fix hostpath subpath reconstruction tests are failing
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes: #68093
**Special notes for your reviewer**:
/sig storage
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix multizone gce pd subpath test
**What this PR does / why we need it**:
The format pod for readonly tests also needs to fill in the NodeSelector for inline gce pd volumes.
Also rename "gce" driver to "gcepd"
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#68085
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67368, 59930, 68074). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Fix subpath tests not to fail in namespace deletion
**What this PR does / why we need it**:
This PR fixes below subpath test not to fail in namespace deletion
- subPath should support restarting containers using directory as subpath
- subPath should support restarting containers using file as subpath
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes: #68073
**Special notes for your reviewer**:
/sig storage
/sig testing
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67764, 68034, 67836). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Remove feature tag from dynamic provisioning topology tests
**What this PR does / why we need it**:
Now that the feature has been moved to beta, remove feature tag to let it run in the standard CI suite.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
This commit fixes below subpath test not to fail in namespace deletion
- subPath should support restarting containers using directory as subpath
- subPath should support restarting containers using file as subpath
Fixes: #68073
Automatic merge from submit-queue (batch tested with PRs 66577, 67948, 68001, 67982). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Refactor volume test in a similar way to csi tests
Refactoring for non-csi e2e test similar to below commit in csi e2e test.
4d11dab272 (diff-0d9ecaa3e6a0297186ad33f57aad472e)
Scopes for this refactoring are below three files:
- test/e2e/storage/volumes.go
- test/e2e/storage/volume_io.go
- test/e2e/storage/persistent_volumes-volumemode.go
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66571
**Special notes for your reviewer**:
/sig storage
@davidz627 @msau42 @jsafrane
**Release note**:
```release-note
NONE
```
Refactoring for non-csi e2e test similar to below commit in csi e2e test.
4d11dab272 (diff-0d9ecaa3e6a0297186ad33f57aad472e)
Scopes for this refactoring are below four files:
- test/e2e/storage/volumes.go
- test/e2e/storage/volume_io.go
- test/e2e/storage/persistent_volumes-volumemode.go
- test/e2e/storage/subpath.go
fixes: #66571
Automatic merge from submit-queue (batch tested with PRs 67332, 66737, 67281, 67173). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
storage e2e test: remove race when setting up loopback device
CI has shown occasional failures stemming from an -EBUSY when
test/e2e/storage/persistent_volumes-local.go aka "PersistentVolumes-local"
attempts to do losetup.
Looking at the code, it has a clear race between querying the current
free loopback device and later explicitly attempting to loopback setup a
file at the queried device node. Losetup nowadays includes the logic to
handle this for the user, if the '-f' option is used instead of naming
the desired target loopback device explicitly. It is safe to presume
a suitable losetup is present as the '-f' option is used elsewhere in
the test, and it is safe to not record the allocated device, as it is
already queried on the fly elsewhere in the test ahead of other commands
which need to know an already created loopback device's node name.
This patch should result in less flakes for this test case.
Signed-off-by: Tim Pepper <tpepper@vmware.com>
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Error in return value should be tested and fix Description Info in e2e test
**What this PR does / why we need it**:
1. Error in return value should be tested
2. fix Description Info
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Just after creating a PV, its phase is "Pending" and its finalizers
doesn't contain "kubernetes.io/pv-protection". If the e2e test performs
so faster than the target k8s cluster, the test fails because the PV is
not ready. This adds WaitForPersistentVolumePhase() for waiting the
phase "Available" to avoid such situation.
Currently, the PVC protection controller will remove its finalizer when
all Pods using a PVC reach at least a Terminating state. However,
certain volumes cannot be guaranteed to be umounted until a Pod is
deleted. Only Pods not in the current pods list can be considered
deleted, so we're removing the exception to not check Terminating Pods.
Signed-off-by: Jose A. Rivera <jarrpa@redhat.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
End2End tests for DynamicVolumeProvisioning of EBS
**What this PR does / why we need it**:
Add end2end tests to exercise `DynamicProvisioningScheduling` features for EBS. The tests make sure `WaitForFirstConsumer` and `AllowedTopologies` specified in a EBS storage class has the desired effect.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
Tests features added to 217a3d8902
**Release note**:
```
NONE
```
/sig storage
/assign @msau42 @jsafrane
CI has shown occasional failures stemming from an -EBUSY when
test/e2e/storage/persistent_volumes-local.go aka "PersistentVolumes-local"
attempts to do losetup.
Looking at the code, it has a clear race between querying the current
free loopback device and later explicitly attempting to loopback setup a
file at the queried device node. Losetup nowadays includes the logic to
handle this for the user, if the '-f' option is used instead of naming
the desired target loopback device explicitly. It is safe to presume
a suitable losetup is present as the '-f' option is used elsewhere in
the test, and it is safe to not record the allocated device, as it is
already queried on the fly elsewhere in the test ahead of other commands
which need to know an already created loopback device's node name.
This patch should result in less flakes for this test case.
Signed-off-by: Tim Pepper <tpepper@vmware.com>
Use the same pattern everywhere in the e2e test
harness, use busybox (from dockerhub) instead
of using the one from k8s.gcr.io registry.
Change-Id: I57c3b867408c1f9478a8909c26744ea0368ff003
Automatic merge from submit-queue (batch tested with PRs 66933, 66925). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Rework multi-volume test to use StatefulSet
**What this PR does / why we need it**:
The e2e test that got added as part of https://github.com/kubernetes/kubernetes/pull/66832 fails in a multi-zone environment because the volumes get provisioned in random zones. This PR reworks the test to use StatefulSet instead, which handles provisioning multiple PVCs in the same zone.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64645, 66832). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Detect if GCE PD udev link is wrong and try to correct it
**What this PR does / why we need it**:
udev can miss scsi events and not update device links properly when disks are detached and reattached to the same node in a different order. This PR workarounds the issue by comparing the udev link name, which is the PD disk name, with the scsi disk serial number returned from scsi_id, and prevents mounting of the disk if the link is wrong. It will also run `udevadm trigger` on the disk to try to correct the bad link.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
This fix prevents a GCE PD volume from being mounted if the udev device link is stale and tries to correct the link.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Part 1: Allowing volumes to choose when to use securityContext
This PR is the result of [splitting this PR]( https://github.com/kubernetes/kubernetes/pull/65793) into 2 sections:
1) This part, addressing the refactor so eligible-test-for-conformance can use get rid of privileged security context.
2) a second part that will address the promotion of the testcases to be in conformance suite.
Changes:
a) demoted privileged mode for these tests (not needed)
b) regular tests (the other ones existing in the file) will still be using privileged security context.
b) adding privilegedSecurityContext field to VolInfo, so each volume-flavor can let the test know if the security context has to be privileged or not.
This allows granular changes and updates per volume).
this PR is the result of splitting https://github.com/kubernetes/kubernetes/pull/65793 into 2 sections
1) This part, addressing the refactor so eligible-test-for-conformance can use get rid of privileged security context.
2) a second part that will address the promotion of the testcases to be in conformance suite
Changes:
a) demoted privileged mode for these tests (not needed)
b) regular tests (the other ones existing in the file) will still be using privileged security context.
c) adding privilegedSecurityContext field to VolInfo, so each volume-flavor can let the test know if the security context has to be privileged or not.
This allows granular changes and updates per volume).
d) fixing formatting issue.
Automatic merge from submit-queue (batch tested with PRs 66341, 66405, 66403, 66264, 66447). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bump subpath test pod timeout to 5 minutes
Fixes: #66376
It seems that sometimes `Attach` on GCP side can take multiple minutes (when recently detached from another node) and therefore the old timeout of 1 minute is not long enough. 5 minutes is standard elsewhere and seems to be long enough for the long tail of GCE Attach time.
Also the ext4 test was really hard to debug because all the old PD tests call all the pods "pd-injector" and all the volume mounts "pd-volume." Made a change to make that more readable by adding a length 4 random string suffix.
/kind flake
/sig storage
/assign @msau42
```release-note
NONE
```
