Commit Graph

1741 Commits

Author SHA1 Message Date
Patrick Ohly
5636418f10 e2e storage: define local PersistentVolume tests in deterministic order
Ginkgo relies on all workers defining all tests in exactly the same order. This
wasn't guaranteed for these tests, with the result that some tests might have
been executed more than once and others not at all when running in parallel.

This was noticed when some of these tests started to flake and then were
reported both as failure and success, as if they had been retried.
2023-01-02 16:22:11 +01:00
Patrick Ohly
4d7e2894b2 e2e storage: fix "Stress with local volumes [Serial] should be able to process many pods and reuse local volumes"
The background goroutine was started with the context from ginkgo.BeforeEach,
which then led to "context canceled" errors. While at it, the entire goroutine
start/stop gets moved into the BeforeEach and simplified.
2022-12-19 11:23:53 +01:00
Patrick Ohly
2f6c4f5eab e2e: use Ginkgo context
All code must use the context from Ginkgo when doing API calls or polling for a
change, otherwise the code would not return immediately when the test gets
aborted.
2022-12-16 20:14:04 +01:00
Patrick Ohly
bf1d1dfd0f e2e storage: pass around topologyTest pointer
Copying the instance makes no sense because the fields are meant to be shared
between test and cleanup code.
2022-12-16 19:42:26 +01:00
Kubernetes Prow Robot
093c5964f7 Merge pull request #114289 from gnufied/add-recovery-expansion-tests
Add recovery expansion tests
2022-12-16 06:16:28 -08:00
Hemant Kumar
3d33178dab Start adding e2e for recovery from expansion failure 2022-12-14 15:28:05 -05:00
jinxu
e46172014c Fix issues in volumesnapshot test for ephemeral storage
Make sure snapshot and restored volumes are cleaned up concurrently
without strict ordiering

Change-Id: Id7e567861fa3618e142915b7cd6991526e4eb6a4
2022-12-14 10:21:15 -08:00
Patrick Ohly
f30d1867e6 e2e storage: remove obsolete testCleanups 2022-12-14 07:39:48 +01:00
Kubernetes Prow Robot
546d4054ac Merge pull request #113930 from gnufied/refactor-mock-tests
Refactor mock tests in different files
2022-12-13 17:27:34 -08:00
Hemant Kumar
9e5d0828e0 use defercleanup in all the places 2022-12-13 15:54:55 -05:00
Kubernetes Prow Robot
233f9c210a Merge pull request #114092 from anggao/update_method_name
e2e: Update GetExternalIP and GetInternalIP method name to make them more explicit.
2022-12-13 12:39:46 -08:00
Kubernetes Prow Robot
cffc7a1391 Merge pull request #110473 from lzhecheng/disruptive-skip-ssh-not-working
Skip tests if SSH not working
2022-12-13 12:39:35 -08:00
Hemant Kumar
6dd94f5663 Refactor mock tests in different files
Change code to use pointer for mock driver setup
2022-12-13 14:26:07 -05:00
Zhecheng Li
21c41e79e5 Skip tests if SSH not working
Functions involves TestVolumeUnmountsFromDeletedPodWithForceOption
need to check if SSH works.

Signed-off-by: Zhecheng Li <zhechengli@microsoft.com>
2022-12-13 14:38:21 +00:00
Patrick Ohly
d4729008ef e2e: simplify test cleanup
ginkgo.DeferCleanup has multiple advantages:
- The cleanup operation can get registered if and only if needed.
- No need to return a cleanup function that the caller must invoke.
- Automatically determines whether a context is needed, which will
  simplify the introduction of context parameters.
- Ginkgo's timeline shows when it executes the cleanup operation.
2022-12-13 08:09:01 +01:00
Ang Gao
3540f948e9 Update method name to make it more explicit 2022-12-12 21:50:07 +00:00
Patrick Ohly
5c09ca57ff e2e storage: remove context.WithCancel
The context provided by Ginkgo will get cancelled automatically.
2022-12-12 11:30:07 +01:00
Patrick Ohly
8c27313cfd e2e storage: don't panic when server stops
Now that everything is connected to a per-test context, the gRPC server might
encounter an error before it gets shut down normally. We must not panic in that
case because it would kill the entire Ginkgo worker process. This is not even
an error, so just log it as info message.
2022-12-12 11:30:07 +01:00
Patrick Ohly
0d73c0d0e5 e2e: fix linter errors
Adding "ctx" as parameter in the previous commit led to some linter errors
about code that overwrites "ctx" without using it.

This gets fixed by replacing context.Background or context.TODO in those code
lines with the new ctx parameter.

Two context.WithCancel calls can get removed completely because the context
automatically gets cancelled by Ginkgo when the test returns.
2022-12-10 20:35:46 +01:00
Patrick Ohly
df5d84ae81 e2e: accept context from Ginkgo
Every ginkgo callback should return immediately when a timeout occurs or the
test run manually gets aborted with CTRL-C. To do that, they must take a ctx
parameter and pass it through to all code which might block.

This is a first automated step towards that: the additional parameter got added
with

    sed -i 's/\(framework.ConformanceIt\|ginkgo.It\)\(.*\)func() {$/\1\2func(ctx context.Context) {/' \
        $(git grep -l -e framework.ConformanceIt -e ginkgo.It )
    $GOPATH/bin/goimports -w $(git status | grep modified: | sed -e 's/.* //')

log_test.go was left unchanged.
2022-12-10 19:50:18 +01:00
Kubernetes Prow Robot
d34bdeea41 Merge pull request #113853 from jsafrane/add-volumepath
Fix subpath disruptive tests
2022-11-15 07:56:52 -08:00
Kubernetes Prow Robot
e4d46148de Merge pull request #113826 from jsafrane/add-openstack
Add CSI migration of OpenStack Cinder volumes
2022-11-11 11:00:07 -08:00
Kubernetes Prow Robot
e451cf5ffe Merge pull request #113169 from soltysh/use_waitgroup
Use goroutine to speed up volume clenaups
2022-11-11 10:59:55 -08:00
Jan Safranek
3fdd4ca7a4 Fix subpath disruptive tests
Add volumePath parameter to all disruptive checks, so subpath tests can use
"/test-volume" and disruptive tests can use "/mnt/volume1" for their
respective Pods.
2022-11-11 14:03:33 +01:00
Jan Safranek
6005c3b512 Remove OpenStack cloud provider
And keep OpenStack Cinder CSI translation
2022-11-11 12:04:55 +01:00
Jan Safranek
5b284a50b7 Revert "Remove references to openstack and cinder"
This reverts commit 9bbf01bae9.
2022-11-10 18:25:30 +01:00
Kubernetes Prow Robot
d94261e904 Merge pull request #113186 from ttakahashi21/KEP-3294
Introduce APIs to support CrossNamespaceSourceProvisioning
2022-11-10 08:06:54 -08:00
Maciej Szulik
0253397fbd Use goroutine to speed up volume clenaups
This removes WaitTimeoutForPodNoLongerRunningOrNotFoundInNamespace
introduced in f2b9479f8e and changes
the test to use goroutines to speed up the cleanups.
2022-11-10 13:15:06 +01:00
Jan Safranek
ee12e3ff42 Add [Feature:SELinux] to all tests that require SELinux
Most CI jobs run an OS that does not support SELinux, therefore tests that
need it should be skipped by default.

* [Feature:SELinux] marks tests that need SELinux (for any feature)
* [Feature:SELinuxMountReadWriteOncePod] marks tests that need
  SELinuxMountReadWriteOncePod alpha gate enabled.

Currently, all SELinux tests have both, but it will change in the future.
2022-11-10 10:18:11 +01:00
Takafumi Takahashi
87c1ca88d4 Add API and validation for CrossNamespaceVolumeDataSource 2022-11-09 20:58:25 +00:00
Hemant Kumar
8cc30f5e0b Fix broken readwriteOncePod serial tests
These tests can't yet run in non-alpha clusters
2022-11-08 15:58:53 -05:00
Jan Safranek
d6c36736d5 Add mock CSI driver test for SELinux mount 2022-11-08 13:37:09 +01:00
Jan Safranek
802979c295 Add SELinux disruptive test 2022-11-08 12:42:20 +01:00
Kubernetes Prow Robot
fbde6ab05c Merge pull request #111724 from dobsonj/csi-inline-conformance-tests
CSI Inline Volumes: promote API tests to conformance
2022-11-07 16:00:16 -08:00
Roman Bednar
8209066c4c graduate RetroactiveDefaultStorageClass to beta
Change feature to beta and remove e2e test feature flags since they're
not needed anymore.
2022-11-02 09:25:10 +01:00
Kubernetes Prow Robot
f163fae7d5 Merge pull request #113409 from gnufied/disable-generic-ephemeral-expansion
Disable expansion in SC, if driver does not support it
2022-10-27 13:20:42 -07:00
Hemant Kumar
fa242ff102 Disable expansion in SC, if driver does not support it 2022-10-27 13:35:36 -04:00
Jonathan Dobson
5d1725f17b CSI Inline Volumes: promote API tests to conformance 2022-10-26 07:41:01 -06:00
Kubernetes Prow Robot
bb12faf15b Merge pull request #113141 from dobsonj/csi-inline-pod-test-flake
test: remove flaky pod update test in CSIInlineVolumes e2e
2022-10-25 16:26:45 -07:00
Jonathan Dobson
e377d071bb test: remove flaky pod update test in CSIInlineVolumes e2e 2022-10-25 11:55:59 -06:00
Kubernetes Prow Robot
fc866d87dc Merge pull request #113208 from soltysh/azure_timeouts
Set default test timeouts first, only then modify the required ones
2022-10-20 12:07:20 -07:00
Kubernetes Prow Robot
0a08757181 Merge pull request #113128 from pohly/e2e-storage-cleanup
e2e storage: cleanup with context
2022-10-20 10:29:00 -07:00
Maciej Szulik
24d30bf8b2 Set default test timeouts first, only then modify the required ones 2022-10-20 15:35:17 +02:00
Patrick Ohly
f15d7f6cca e2e storage: eliminate context.TODO and cleanup callbacks
Tests should accept a context from Ginkgo and pass it through to all functions
which may block for a longer period of time. In particular all Kubernetes API
calls through client-go should use that context. Then if a timeout occurs,
the test returns immediately because everything that it could block on will
return.

Cleanup code then needs to run in a separate Ginkgo node, typically
DeferCleanup, which ensures that it gets a separate context which has not timed
out yet.
2022-10-20 13:56:59 +02:00
Patrick Ohly
d8d3dc9476 e2e storage: eliminate explicit test driver cleanup
Cleanup can be handled inside the setup code with ginkgo.DeferCleanup. This
simplifies the code.
2022-10-20 13:56:59 +02:00
Kubernetes Prow Robot
6be327c053 Merge pull request #113046 from jsafrane/add-multiplepv-cap
Add capability for tests with multiple PVs with the same VolumeHandle
2022-10-18 16:27:12 -07:00
Jan Safranek
bda42b6b53 Add capability for tests with multiple PVs with the same VolumeHandle
Some CSI drivers do not like multiple PVs with the same VolumeHandle,
therefore make that tests optional.
2022-10-18 13:44:54 +02:00
Maciej Szulik
f2b9479f8e Wait for pod not running or gone in storage tests 2022-10-18 13:18:54 +02:00
Kubernetes Prow Robot
216ba39632 Merge pull request #112987 from chrishenzie/e2e-test-rwop
e2e test for ReadWriteOncePod PV access mode
2022-10-17 10:01:08 -07:00
Chris Henzie
f6d9c27722 e2e test for ReadWriteOncePod PV access mode
Adds two tests for the enforcement of the ReadWriteOncePod
PersistentVolume access mode.

1. Tests that when two Pods are scheduled that reference the same
   ReadWriteOncePod PVC, the latter-scheduled Pod will be marked
   unschedulable because the PVC is in-use.
2. Tests that when two Pods are scheduled on the same node (setting
   Pod.Spec.NodeName to bypass scheduling for the second Pod), the
   latter Pod will fail to start because the PVC is already mounted on
   the Node.

Included are changes to update the hostpath CSI driver to accept new CSI
access modes. Its sidecar containers are already at supported versions
for ReadWriteOncePod and don't need updating. The GCP PD CSI driver does
not yet support the new CSI access modes, but its sidecar containers are
at supported versions and so the feature will work.

To support ReadWriteOncePod, the following CSI sidecars must be updated
to these versions or greater:
 - csi-provisioner:v3.0.0+
 - csi-attacher:v3.3.0+
 - csi-resizer:v1.3.0+

For more details, see:
https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode/README.md
2022-10-14 09:15:36 -07:00