Commit Graph

1256 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
ac6447c76f Merge pull request #94318 from gnufied/fix-namespace-deletion
Prevent deletion  of namespace again
2020-09-10 10:46:03 -07:00
Kubernetes Prow Robot
c3b888f647 Merge pull request #93930 from pohly/multi-volume-generic-ephemeral-test-fix
e2e storage: skip multi-volume generic ephemeral volume test
2020-09-10 02:33:45 -07:00
Kubernetes Prow Robot
88e3c95d7c Merge pull request #94647 from pohly/ephemeral-fixes
storage E2E: explicitly wait for PV deletion after ephemeral test
2020-09-09 20:43:44 -07:00
Hemant Kumar
c4ce420667 Prevent deletion of namespace again 2020-09-09 11:22:08 -04:00
Jan Safranek
7df879ea95 Tolerate NotFound errors when deleting snapshots 2020-09-09 14:50:11 +02:00
Patrick Ohly
16635f5902 storage E2E: explicitly wait for PV deletion after ephemeral test
Even with foreground deletion, removal of the PVs that may have been
created for a pod with generic ephemeral volumes happens
asynchronously, in the worst case after the test has completed and the
driver for the volume got removed.

Perhaps this can be fixed in Kubernetes itself, but for now we need to
deal with it as part of the test.
2020-09-09 13:14:46 +02:00
Daniel Smith
75f835aa08 move port definitions to a common location 2020-09-02 10:48:25 -07:00
Kubernetes Prow Robot
19820b7bf1 Merge pull request #93876 from jsafrane/fix-wait-errormsg
Fix waiting for PVC to get bound
2020-09-01 18:43:42 -07:00
Kubernetes Prow Robot
e23d83eead Merge pull request #93710 from Jiawei0227/attachable2non
Detect volume attach-ability in the middle of attaching
2020-08-31 17:39:50 -07:00
Andi Li
673af97bcd Fix testpattern check 2020-08-28 13:44:42 -04:00
Andi Li
d915463451 Combine switch case into one case 2020-08-28 12:36:15 -04:00
Andi Li
5b4dffb6b7 Ran hack/update-bazel.sh but discarded changes to root build file 2020-08-28 12:36:15 -04:00
Andi Li
e748ec3de2 Move common portion of dynamic snapshot out of switch statement 2020-08-28 12:36:15 -04:00
Andi Li
7918f69055 Remove custom hash function, generate a random UUID for snapshot and snapcontent name instead 2020-08-28 12:36:15 -04:00
Andi Li
86af5a5ec3 Stack snapshot tests into one to reuse snapshot resource and reduce time taken on prow. 2020-08-28 12:36:15 -04:00
Andi Li
a4498846b1 Quick commit 2020-08-28 12:36:15 -04:00
Andi Li
854645d7c3 Remove snapshot class from prepv test because it is not needed.
Prepv snapshots only need to specify driver.
2020-08-28 12:36:15 -04:00
Andi Li
42b2d1c52c Combine two tests with overlap to reduce overall snapshot test suite time 2020-08-28 12:36:15 -04:00
Andi Li
e78aee9965 Add pre provisioned snapshot tests
Make pre provisioned snapshots using CSI driver by
1. Take a dynamic snapshot with retain policy
2. Delete the dynamic snapshot and content
3. Create a preprovisioned snapshot with snapshotHandle

This commit adds a preprovisiond test pattern, all snapshots made using
create snapshot resource become prepv snapshots. All exisitng test cases
now run again with prepv snapshots.
2020-08-28 12:36:15 -04:00
Andi Li
ed85129831 Minor cleanup in snapshot test suite. 2020-08-28 12:29:07 -04:00
Kubernetes Prow Robot
3a79bb68bd Merge pull request #94238 from jingxu97/Aug/subpathtest
Set snapshotType for tests with NTFS.
2020-08-28 08:04:55 -07:00
Kubernetes Prow Robot
f6d169c7ca Merge pull request #93120 from msau42/e2e-export
Export WaitForCSIDriverRegistrationOnAllNodes
2020-08-28 06:34:53 -07:00
Kubernetes Prow Robot
32f034de38 Merge pull request #92845 from oomichi/remove-DeprecatedMightBeMasterNode
Remove DeprecatedMightBeMasterNode()
2020-08-27 19:07:25 -07:00
Kubernetes Prow Robot
b5de8b9979 Merge pull request #94242 from jingxu97/Aug/filemode
Remove file_mode args for Windows test
2020-08-27 10:38:33 -07:00
Kubernetes Prow Robot
2770fc6128 Merge pull request #94061 from mattcary/fix-expand-offline-comment
fix misleading comment in expand_volume.go
2020-08-27 04:49:45 -07:00
Kubernetes Prow Robot
5507cb772b Merge pull request #93950 from Jiawei0227/parse-mock-logs
Make CSI mock driver log parse more flexible
2020-08-27 04:48:38 -07:00
Kubernetes Prow Robot
e3bd0de6df Merge pull request #93923 from jsafrane/reduce-cinder-sizes
Reduce test volume sizes
2020-08-27 04:48:11 -07:00
Jing Xu
d35eb56322 Fix issue on skipTest in storage suits
After PR https://github.com/kubernetes/kubernetes/pull/92555, there are a number of gce pd default fs tests skipped. Here the testpatten has SnapshotType set because some provisioning tests use snapshots. But for drivers such as In-tree gce pd driver, the tests will be skipped because of the logic in skipUnsupportedTesthttps://github.com/kubernetes/kubernetes/blob/master/test/e2e/storage/testsuites/base.go#L154
Since multiple drivers might test with the same pattern, so I think we need keep SnapshotType here.

This PR removes the part of the logic in skipUnsupportedTest. This should be ok because all snapshot tests will check whether a driver has snapshot capability or not.
2020-08-25 23:31:47 -07:00
Jing Xu
0d9c8202d5 Remove file_mode args for Windows test
Due to issue https://github.com/kubernetes/kubernetes/issues/94237,
remove file_mode arg for subPath tests for now.
2020-08-25 20:05:40 -07:00
Jing Xu
b426689ca4 Set snapshotType for tests with NTFS.
Set the snapshotType for tests with NTFS. Return error if type is not
set.
2020-08-25 19:39:30 -07:00
Jiawei Wang
a6d8e6c5c2 Detect change of volume attachability in the middle of attaching
- Add Unit tests for both volumemanager and attach/detach controller
- Add E2E test
2020-08-24 17:15:11 -07:00
Kubernetes Prow Robot
3352c44949 Merge pull request #93777 from pohly/capacity-test-flake-fix
e2e storage: avoid flaky test failure when watch dies
2020-08-19 04:47:12 -07:00
Matthew Cary
8ea57e55d7 fix misleading comment in expand_volume.go 2020-08-17 13:51:03 -07:00
Jiawei Wang
0ce86eb553 Make CSI mock driver log parse more flexible 2020-08-13 10:27:38 -07:00
Jiawei Wang
d030c64743 [e2e]Bind csi driver namespace to test namespace to avoid namespace collision 2020-08-12 10:20:30 -07:00
Patrick Ohly
1a510a1bec e2e storage: avoid flaky test failure when watch dies
As discussed in https://github.com/kubernetes/kubernetes/pull/93658,
relying on a watch to deliver all events is not acceptable, not even
for a test, because it can and did (at least for OpenShift testing) to
test flakes.

The solution in #93658 was to replace log flooding with a clear test
failure, but that didn't solve the flakiness.

A better solution is to use a RetryWatcher which "under normal
circumstances" (https://github.com/kubernetes/kubernetes/pull/93777#discussion_r467932080)
should always deliver all changes.
2020-08-12 17:56:24 +02:00
Patrick Ohly
441324e8c8 e2e storage: skip multi-volume generic ephemeral volume test
This test might flake when run on a multi-zone cluster (similar to
persistent volumes, see
https://github.com/kubernetes/kubernetes/issues/75776). We don't do
that at the moment, but it's better to fix this anyway.
2020-08-12 16:01:13 +02:00
Jan Safranek
fea06a204c Reduce test volume sizes
To reduce costs and to increase speed. Formating 5GiB volume took too long
in a badly overloaded CI cluster.

All in-tree volume plugins support at least 1GiB volumes.
2020-08-12 14:00:30 +02:00
Jan Safranek
96eeb98215 Fix error messages
Separate different errors by ": ".
2020-08-11 11:14:15 +02:00
Kubernetes Prow Robot
da5ec168e7 Merge pull request #93618 from Jiawei0227/e2e-csi-mock-flaky
Fix flaky test: wait for mock CSI Driver bringup to perform e2e test
2020-08-10 15:41:59 -07:00
Jiawei Wang
76b4973b42 Wait for mock CSI Driver bringup to perform e2e test
In our current mock CSI driver e2e test, we are not waiting
for the CSI driver register successfully to perform test
including provision PVC. This can lead to timeout when the
csi driver takes longer to register the socket.

This change adds the waiting part so that the system will
wait for up to 10 minutes for the driver to be ready. This
normally won't take this long. However, under a resource
constraint environment it can take longer than expected time.

https://github.com/kubernetes/kubernetes/issues/93358
2020-08-10 11:03:35 -07:00
Kubernetes Prow Robot
a640545167 Merge pull request #93567 from gnufied/fix-stale-attachments
Make AttachDisk for EBS idempotent again
2020-08-07 07:16:19 -07:00
Hemant Kumar
f91d448e1b Add tests for out of band attachments 2020-08-07 08:23:25 -04:00
Kubernetes Prow Robot
56c76023f7 Merge pull request #93658 from dcbw/csi-fail-test-on-close
test/e2e: fail test rather than flooding logs if PVC watch is closed prematurely
2020-08-04 18:08:42 -07:00
Kubernetes Prow Robot
ae9f76153f Merge pull request #93535 from gnufied/create-service-for-endpoint
Create service for gluster endpoint
2020-08-03 16:10:33 -07:00
Dan Williams
e0f9fe2cd3 test/e2e: fail test rather than flooding logs if PVC watch is closed prematurely
If the watch is closed prematurely the for/select loop will
spew "PVC event : <nil>" as fast as possible until the test times
out, flooding logs.

STEP: Checking PVC events
Aug  3 16:35:53.930: INFO: PVC event ADDED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"42154", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001639060), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001639080)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc000876e90), VolumeMode:(*v1.PersistentVolumeMode)(0xc000876ec0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"42159", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-e2e-csi-mock-volumes-5292"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6060), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6080)}, v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a60a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a60c0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc0012aa020), VolumeMode:(*v1.PersistentVolumeMode)(0xc0012aa030), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"43201", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-e2e-csi-mock-volumes-5292"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6100), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6120)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6140), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6160)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-221369cd-50c3-48be-8c08-dc4d8f6f36e0", StorageClassName:(*string)(0xc0012aa090), VolumeMode:(*v1.PersistentVolumeMode)(0xc0012aa0a0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"43203", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-e2e-csi-mock-volumes-5292"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6600), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6620)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6660), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6680)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-221369cd-50c3-48be-8c08-dc4d8f6f36e0", StorageClassName:(*string)(0xc0012aa450), VolumeMode:(*v1.PersistentVolumeMode)(0xc0012aa460), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
<a million lines later...>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
<test times out>
2020-08-03 13:01:00 -05:00
Kubernetes Prow Robot
d3a49f7c63 Merge pull request #93589 from jingxu97/July/csiwindowstest
Update e2e storage tests for Windows cluster
2020-08-01 20:21:54 -07:00
Jing Xu
015f2998e5 Update e2e storage tests for Windows cluster
This PR fixes a few things for e2e storage suite to run on Windows
cluster.
1. increaes timeout due to longer pod startup time for windows
2. Only set SELinuxOptions or fsGroup if os is not windows
3. Add VolumeSnapshot delete policy for windows3. Add VolumeSnapshot
delete policy for windows3. Add VolumeSnapshot delete policy for windows
2020-07-31 10:09:28 -07:00
Hemant Kumar
20934d7e50 Create service for gluster endpoint
Creating an endpoint without a service can cause endpoint to disappear

https://github.com/kubernetes/kubernetes/issues/12964
2020-07-30 16:34:11 -04:00
Jefftree
ace97738e2 Update formatting of conformance comment 2020-07-29 20:50:44 -07:00