Commit Graph

1241 Commits

Author SHA1 Message Date
Andi Li
e748ec3de2 Move common portion of dynamic snapshot out of switch statement 2020-08-28 12:36:15 -04:00
Andi Li
7918f69055 Remove custom hash function, generate a random UUID for snapshot and snapcontent name instead 2020-08-28 12:36:15 -04:00
Andi Li
86af5a5ec3 Stack snapshot tests into one to reuse snapshot resource and reduce time taken on prow. 2020-08-28 12:36:15 -04:00
Andi Li
a4498846b1 Quick commit 2020-08-28 12:36:15 -04:00
Andi Li
854645d7c3 Remove snapshot class from prepv test because it is not needed.
Prepv snapshots only need to specify driver.
2020-08-28 12:36:15 -04:00
Andi Li
42b2d1c52c Combine two tests with overlap to reduce overall snapshot test suite time 2020-08-28 12:36:15 -04:00
Andi Li
e78aee9965 Add pre provisioned snapshot tests
Make pre provisioned snapshots using CSI driver by
1. Take a dynamic snapshot with retain policy
2. Delete the dynamic snapshot and content
3. Create a preprovisioned snapshot with snapshotHandle

This commit adds a preprovisiond test pattern, all snapshots made using
create snapshot resource become prepv snapshots. All exisitng test cases
now run again with prepv snapshots.
2020-08-28 12:36:15 -04:00
Andi Li
ed85129831 Minor cleanup in snapshot test suite. 2020-08-28 12:29:07 -04:00
Kubernetes Prow Robot
3a79bb68bd Merge pull request #94238 from jingxu97/Aug/subpathtest
Set snapshotType for tests with NTFS.
2020-08-28 08:04:55 -07:00
Kubernetes Prow Robot
f6d169c7ca Merge pull request #93120 from msau42/e2e-export
Export WaitForCSIDriverRegistrationOnAllNodes
2020-08-28 06:34:53 -07:00
Kubernetes Prow Robot
32f034de38 Merge pull request #92845 from oomichi/remove-DeprecatedMightBeMasterNode
Remove DeprecatedMightBeMasterNode()
2020-08-27 19:07:25 -07:00
Kubernetes Prow Robot
b5de8b9979 Merge pull request #94242 from jingxu97/Aug/filemode
Remove file_mode args for Windows test
2020-08-27 10:38:33 -07:00
Kubernetes Prow Robot
2770fc6128 Merge pull request #94061 from mattcary/fix-expand-offline-comment
fix misleading comment in expand_volume.go
2020-08-27 04:49:45 -07:00
Kubernetes Prow Robot
5507cb772b Merge pull request #93950 from Jiawei0227/parse-mock-logs
Make CSI mock driver log parse more flexible
2020-08-27 04:48:38 -07:00
Kubernetes Prow Robot
e3bd0de6df Merge pull request #93923 from jsafrane/reduce-cinder-sizes
Reduce test volume sizes
2020-08-27 04:48:11 -07:00
Jing Xu
d35eb56322 Fix issue on skipTest in storage suits
After PR https://github.com/kubernetes/kubernetes/pull/92555, there are a number of gce pd default fs tests skipped. Here the testpatten has SnapshotType set because some provisioning tests use snapshots. But for drivers such as In-tree gce pd driver, the tests will be skipped because of the logic in skipUnsupportedTesthttps://github.com/kubernetes/kubernetes/blob/master/test/e2e/storage/testsuites/base.go#L154
Since multiple drivers might test with the same pattern, so I think we need keep SnapshotType here.

This PR removes the part of the logic in skipUnsupportedTest. This should be ok because all snapshot tests will check whether a driver has snapshot capability or not.
2020-08-25 23:31:47 -07:00
Jing Xu
0d9c8202d5 Remove file_mode args for Windows test
Due to issue https://github.com/kubernetes/kubernetes/issues/94237,
remove file_mode arg for subPath tests for now.
2020-08-25 20:05:40 -07:00
Jing Xu
b426689ca4 Set snapshotType for tests with NTFS.
Set the snapshotType for tests with NTFS. Return error if type is not
set.
2020-08-25 19:39:30 -07:00
Kubernetes Prow Robot
3352c44949 Merge pull request #93777 from pohly/capacity-test-flake-fix
e2e storage: avoid flaky test failure when watch dies
2020-08-19 04:47:12 -07:00
Matthew Cary
8ea57e55d7 fix misleading comment in expand_volume.go 2020-08-17 13:51:03 -07:00
Jiawei Wang
0ce86eb553 Make CSI mock driver log parse more flexible 2020-08-13 10:27:38 -07:00
Jiawei Wang
d030c64743 [e2e]Bind csi driver namespace to test namespace to avoid namespace collision 2020-08-12 10:20:30 -07:00
Patrick Ohly
1a510a1bec e2e storage: avoid flaky test failure when watch dies
As discussed in https://github.com/kubernetes/kubernetes/pull/93658,
relying on a watch to deliver all events is not acceptable, not even
for a test, because it can and did (at least for OpenShift testing) to
test flakes.

The solution in #93658 was to replace log flooding with a clear test
failure, but that didn't solve the flakiness.

A better solution is to use a RetryWatcher which "under normal
circumstances" (https://github.com/kubernetes/kubernetes/pull/93777#discussion_r467932080)
should always deliver all changes.
2020-08-12 17:56:24 +02:00
Jan Safranek
fea06a204c Reduce test volume sizes
To reduce costs and to increase speed. Formating 5GiB volume took too long
in a badly overloaded CI cluster.

All in-tree volume plugins support at least 1GiB volumes.
2020-08-12 14:00:30 +02:00
Kubernetes Prow Robot
da5ec168e7 Merge pull request #93618 from Jiawei0227/e2e-csi-mock-flaky
Fix flaky test: wait for mock CSI Driver bringup to perform e2e test
2020-08-10 15:41:59 -07:00
Jiawei Wang
76b4973b42 Wait for mock CSI Driver bringup to perform e2e test
In our current mock CSI driver e2e test, we are not waiting
for the CSI driver register successfully to perform test
including provision PVC. This can lead to timeout when the
csi driver takes longer to register the socket.

This change adds the waiting part so that the system will
wait for up to 10 minutes for the driver to be ready. This
normally won't take this long. However, under a resource
constraint environment it can take longer than expected time.

https://github.com/kubernetes/kubernetes/issues/93358
2020-08-10 11:03:35 -07:00
Kubernetes Prow Robot
a640545167 Merge pull request #93567 from gnufied/fix-stale-attachments
Make AttachDisk for EBS idempotent again
2020-08-07 07:16:19 -07:00
Hemant Kumar
f91d448e1b Add tests for out of band attachments 2020-08-07 08:23:25 -04:00
Kubernetes Prow Robot
56c76023f7 Merge pull request #93658 from dcbw/csi-fail-test-on-close
test/e2e: fail test rather than flooding logs if PVC watch is closed prematurely
2020-08-04 18:08:42 -07:00
Kubernetes Prow Robot
ae9f76153f Merge pull request #93535 from gnufied/create-service-for-endpoint
Create service for gluster endpoint
2020-08-03 16:10:33 -07:00
Dan Williams
e0f9fe2cd3 test/e2e: fail test rather than flooding logs if PVC watch is closed prematurely
If the watch is closed prematurely the for/select loop will
spew "PVC event : <nil>" as fast as possible until the test times
out, flooding logs.

STEP: Checking PVC events
Aug  3 16:35:53.930: INFO: PVC event ADDED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"42154", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc001639060), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc001639080)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc000876e90), VolumeMode:(*v1.PersistentVolumeMode)(0xc000876ec0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"42159", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"volume.beta.kubernetes.io/storage-provisioner":"csi-mock-e2e-csi-mock-volumes-5292"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6060), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6080)}, v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a60a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a60c0)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc0012aa020), VolumeMode:(*v1.PersistentVolumeMode)(0xc0012aa030), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"43201", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-e2e-csi-mock-volumes-5292"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6100), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6120)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6140), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6160)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-221369cd-50c3-48be-8c08-dc4d8f6f36e0", StorageClassName:(*string)(0xc0012aa090), VolumeMode:(*v1.PersistentVolumeMode)(0xc0012aa0a0), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Pending", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event MODIFIED: &v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"pvc-zrchk", GenerateName:"pvc-", Namespace:"e2e-csi-mock-volumes-5292", SelfLink:"/api/v1/namespaces/e2e-csi-mock-volumes-5292/persistentvolumeclaims/pvc-zrchk", UID:"221369cd-50c3-48be-8c08-dc4d8f6f36e0", ResourceVersion:"43203", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63732069329, loc:(*time.Location)(0x9e74040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"pv.kubernetes.io/bind-completed":"yes", "pv.kubernetes.io/bound-by-controller":"yes", "volume.beta.kubernetes.io/storage-provisioner":"csi-mock-e2e-csi-mock-volumes-5292"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string{"kubernetes.io/pvc-protection"}, ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"openshift-tests", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6600), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6620)}, v1.ManagedFieldsEntry{Manager:"kube-controller-manager", Operation:"Update", APIVersion:"v1", Time:(*v1.Time)(0xc0016a6660), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0016a6680)}}}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}}, VolumeName:"pvc-221369cd-50c3-48be-8c08-dc4d8f6f36e0", StorageClassName:(*string)(0xc0012aa450), VolumeMode:(*v1.PersistentVolumeMode)(0xc0012aa460), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"Bound", AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Capacity:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1073741824, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"1Gi", Format:"BinarySI"}}, Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
Aug  3 16:35:53.930: INFO: PVC event : <nil>
<a million lines later...>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
Aug  3 16:36:26.571: INFO: PVC event : <nil>
<test times out>
2020-08-03 13:01:00 -05:00
Kubernetes Prow Robot
d3a49f7c63 Merge pull request #93589 from jingxu97/July/csiwindowstest
Update e2e storage tests for Windows cluster
2020-08-01 20:21:54 -07:00
Jing Xu
015f2998e5 Update e2e storage tests for Windows cluster
This PR fixes a few things for e2e storage suite to run on Windows
cluster.
1. increaes timeout due to longer pod startup time for windows
2. Only set SELinuxOptions or fsGroup if os is not windows
3. Add VolumeSnapshot delete policy for windows3. Add VolumeSnapshot
delete policy for windows3. Add VolumeSnapshot delete policy for windows
2020-07-31 10:09:28 -07:00
Hemant Kumar
20934d7e50 Create service for gluster endpoint
Creating an endpoint without a service can cause endpoint to disappear

https://github.com/kubernetes/kubernetes/issues/12964
2020-07-30 16:34:11 -04:00
Jefftree
ace97738e2 Update formatting of conformance comment 2020-07-29 20:50:44 -07:00
hasheddan
efe3747c22 Wait for resources owned by pod to be cleaned up in ephemeral volume tests
sig-storage tests that delete pods need to wait for owned resources to
also be cleaned up before returning in the case that resources such as
ephemeral inline volumes are being used. This was previously implemented
by modifying the pod delete call of the e2e framework, which negatively
impacted other tests. This was reverted and now the logic has been moved
to StopPodAndDependents, which is local to the sig-storage tests.

Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
2020-07-22 17:52:19 -05:00
Kubernetes Prow Robot
a1a2f8c5f8 Merge pull request #93225 from hasheddan/storage-panic
Return error instead of panic if container index outside bounds
2020-07-21 09:15:15 -07:00
hasheddan
4e4d629af7 Return error instead of panic if container index outside bounds
Adds check for index out of bounds error instead of panic when passing
container to kubectl exec.

Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
2020-07-19 10:04:53 -05:00
Jordan Liggitt
9d83ca4b02 Deflake GCEPD namespace deletion test 2020-07-18 15:32:02 -04:00
Michelle Au
10a8c195a1 Export WaitForCSIDriverRegistrationOnAllNodes to be used by external csi driver repos
Change-Id: Ie61430b1050a778d8ba98177e0c995ff2553f9cd
2020-07-15 16:53:37 -07:00
Kubernetes Prow Robot
c46c1c043e Merge pull request #91742 from jingxu97/June/wintests
Fix subPath tests for Windows
2020-07-13 23:29:21 -07:00
Kubernetes Prow Robot
d2f4cc032b Merge pull request #92841 from chethanv28/fix-typo-e2e-tests
Fix typo in vsphere e2e test names
2020-07-11 20:56:33 -07:00
Kubernetes Prow Robot
780f7f0ce9 Merge pull request #92555 from AndiLi99/AndiLi99/snapshot_tests
Refactor tests in snapshottable and fix #73625
2020-07-11 20:55:35 -07:00
Kenichi Omichi
029b2d2197 Remove DeprecatedMightBeMasterNode()
This removes DeprecatedGetMasterAndWorkerNodes() usage from vsphere e2e
test as deprecated function cleanup.
Then all callers of DeprecatedMightBeMasterNode() have been removed.
So this removes DeprecatedMightBeMasterNode() itself also.
2020-07-11 03:18:44 +00:00
Kubernetes Prow Robot
42ae2003dd Merge pull request #92805 from chethanv28/fix-e2e-test-vsphere_utils
Fix vsphere util method - disksAreAttached
2020-07-10 15:42:51 -07:00
Kubernetes Prow Robot
2613cbc2a7 Merge pull request #92802 from divyenpatel/fix-e2e-test-vsphere_volume_datastore
fixed e2e test vsphere_volume_datastore.go
2020-07-10 15:42:41 -07:00
Kubernetes Prow Robot
0cb7e320a5 Merge pull request #92784 from pohly/generic-ephemeral-inline-volumes
generic ephemeral inline volumes
2020-07-10 15:41:46 -07:00
Patrick Ohly
2468a24b7a GenericEphemeralVolume: E2E test
This extends the existing "ephemeral volume" tests to also cover
generic ephemeral inline volumes. They get instantiated for all
drivers (CSI and others) which support persistent volume provisioning,
for several different filesystems.

Configuring the number of inline volumes via a flag with a computed
name had been identified as problematic before and now gets removed
because re-using the tests as a stress test with a higher number of
volumes should be added and configured separately.
2020-07-09 23:29:24 +02:00
Jing Xu
31db94c848 Fix subPath tests for Windows
Windows test for subPath is failing due to an issue related to
removeUnusedContainers calls. After image is changed to Agnhost, it
automatically has a args by default. However, there are places to use
container commands instead of args and causing issues.

This is the first step to fix this issue. Next plan to replace
busybox used in Linux with Agnhost which can work for both linux and
windows.

I also mark two subPath tests as LinuxOnly. I think they are not ready
for windows yet. Before they were passing due to wrong reason. The tests
checks failed container status but the contain fails due to other
reasons than what we expected.
2020-07-09 11:14:17 -07:00
Andi Li
b64f05b70a Change expected content to string constant. Cleanup DeleteAndWaitSnapshot
Use the correct format specifier
2020-07-09 16:30:16 +00:00