Commit Graph

18973 Commits

Author SHA1 Message Date
Patrick Ohly
d43308e64c e2e storage: simpler port forwarding
Instead of trying to use the client-go portforward package as-is it is
simpler to copy some code from it and then use the http stream
directly. That way we don't need to go through a local listening
socket and error handling and logging becomes simpler.
2021-03-01 19:22:37 +01:00
Patrick Ohly
3adcf11b45 e2e storage: use embedded mock CSI driver
This replaces embedding of JavaScript code into the mock driver that
runs inside the cluster with Go callbacks which run inside the
e2e.test suite itself. In contrast to the JavaScript hooks, they have
direct access to all parameters and can fabricate arbitrary responses,
not just error codes.

Because the callbacks run in the same process as the test itself, it
is possible to set up two-way communication via shared variables or
channels. This opens the door for writing better tests. Some of the
existing tests that poll mock driver output could be simplified, but
that can be addressed later.

For now, only tests using hooks use embedding. How gRPC calls are
retrieved is abstracted behind the CSIMockTestDriver interface, so
tests don't need to be modified when switching between embedding
and remote mock driver.
2021-03-01 19:22:37 +01:00
Patrick Ohly
92bac8afc1 mock driver: fix no-op setDefaultCreds
The function must modify the content of the "creds" pointer, not the
pointer.

Found via hack/verify-staticcheck.sh after importing the code into
Kubernetes. It is uncertain whether this bug had any consequences.
2021-03-01 19:22:37 +01:00
Patrick Ohly
a2a34bb744 mock driver fixes 2021-03-01 19:22:37 +01:00
Patrick Ohly
ab365c091c mock driver: replace logrus with klog
klog now has structured logging.
2021-03-01 19:22:37 +01:00
Patrick Ohly
df6d3bc7dd CSI mock driver: fix faulty error message
Caught by verify-typecheck.sh after importing the code into
Kubernetes:

ERROR(linux/arm): /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/drivers/csi-test/mock/service/controller.go:404:20: math.MaxUint32 (untyped int constant 4294967295) overflows int
ERROR(linux/arm): /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/drivers/csi-test/mock/service/controller.go:795:20: math.MaxUint32 (untyped int constant 4294967295) overflows int
ERROR(linux/386): /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/drivers/csi-test/mock/service/controller.go:404:20: math.MaxUint32 (untyped int constant 4294967295) overflows int
ERROR(linux/386): /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/drivers/csi-test/mock/service/controller.go:795:20: math.MaxUint32 (untyped int constant 4294967295) overflows int
ERROR(windows/386): /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/drivers/csi-test/mock/service/controller.go:404:20: math.MaxUint32 (untyped int constant 4294967295) overflows int
ERROR(windows/386):
/home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/drivers/csi-test/mock/service/controller.go:795:20:
math.MaxUint32 (untyped int constant 4294967295) overflows int

Instead of producing our own error message, we can show the original
value and the error from strconv.
2021-03-01 19:22:37 +01:00
Patrick Ohly
7008196093 csi-test: import copyright clarification
The code originates in csi-test and this copyright change was made by
the original author in https://github.com/kubernetes-csi/csi-test/pull/324
2021-03-01 19:22:37 +01:00
Patrick Ohly
7f2b438020 e2e: import csi-test mock driver
This is a verbatim copy of the corresponding files in csi-test v4.0.2.

They'll be modified in future commits to make the code usable when
embedded in e2e.test. Some of those changes may be worthwhile
backporting to csi-test, but this is uncertain at this time.
2021-03-01 19:22:37 +01:00
Patrick Ohly
21ffdd1a28 test: update CSI mock driver
The new release properly checks for existence of the staging
directory. The names of the volumes are now the same as in
https://github.com/kubernetes-csi/csi-test/blob/master/mock/example/deploy/csi-mock-driver-deployment.yaml
2021-03-01 19:22:37 +01:00
Patrick Ohly
99248b1406 e2e: reduce worker threads during mock testing
We don't need much concurrency and having too many worker threads has
one disadvantage (besides resource usage): when the sidecar looses the
connection to the CSI driver, it calls klog.Fatal, which prints all
gouroutines. This can lead to much output.
2021-03-01 19:22:37 +01:00
Clayton Coleman
18f43e4120 daemonset: Implement MaxSurge on daemonset update
If MaxSurge is set, the controller will attempt to double up nodes
up to the allowed limit with a new pod, and then when the most recent
(by hash) pod is ready, trigger deletion on the old pod. If the old
pod goes unready before the new pod is ready, the old pod is immediately
deleted. If an old pod goes unready before a new pod is placed on that
node, a new pod is immediately added for that node even past the MaxSurge
limit.

The backoff clock is used consistently throughout the daemonset controller
as an injectable clock for the purposes of testing.
2021-03-01 13:21:12 -05:00
Kubernetes Prow Robot
5498ee641b Merge pull request #99561 from BenTheElder/remove-bazel
Remove Bazel
2021-03-01 09:55:27 -08:00
Antonio Ojea
1b61f49995 fix e2e test kube-proxy tcp_wait 2021-03-01 00:54:57 +01:00
Benjamin Elder
56e092e382 hack/update-bazel.sh 2021-02-28 15:17:29 -08:00
Benjamin Elder
5eedf2a0f1 nit: newlines 2021-02-28 13:04:21 -08:00
Benjamin Elder
5ed5a5fe25 add test/conformance/gen-conformance-docs.sh 2021-02-28 13:01:07 -08:00
Kubernetes Prow Robot
c410910c1f Merge pull request #99512 from knabben/add-netpol-ports
Adding specific protocol network policy test
2021-02-28 07:36:39 -08:00
Kubernetes Prow Robot
a185bafa0c Merge pull request #99549 from BenTheElder/conformance-make
conformance.yaml: generate with make
2021-02-28 05:30:39 -08:00
Benjamin Elder
025e6de3eb don't run generated_files when generating specsummaries.json 2021-02-28 00:54:54 -08:00
Benjamin Elder
ed09168e8a fix generated conformance filepaths when test binary is not stripped 2021-02-28 00:54:54 -08:00
Benjamin Elder
c1c3fdefe3 port conformance yaml generation to hack 2021-02-28 00:54:54 -08:00
wojtekt
749922fa46 Cleanup SIGDescribe in test/e2e/network 2021-02-27 20:29:27 +01:00
lala123912
5faa51e5d6 Replace top-level ginkgo.Describe with SIGDescribe 2021-02-27 14:26:36 +08:00
Kubernetes Prow Robot
f2d5ee2d2d Merge pull request #99508 from wojtek-t/cleanup_describe_10
Move node_recreate test to cloud/gcp directory
2021-02-26 18:06:38 -08:00
Amim Knabben
20263a07a6 Added protocol specific netpol 2021-02-26 18:11:45 -05:00
Kubernetes Prow Robot
157c4bbf85 Merge pull request #99390 from pacoxu/use-random-node/e2e
[e2e] use random node to avoid node0 overloaded
2021-02-26 14:50:54 -08:00
Kubernetes Prow Robot
1428969a73 Merge pull request #96610 from andrewsykim/service-lb-node-port-e2e
test/integration: add integration tests for service.spec.allocateLoadBalancerNodePorts
2021-02-26 14:50:38 -08:00
wojtekt
4b98a9f1e6 Move node_recreate test to cloud/gcp directory 2021-02-26 21:47:27 +01:00
Kubernetes Prow Robot
e5e3c8d4f5 Merge pull request #99467 from wojtek-t/cleanup_describe_5
Remove KubeDescribe from SIGDescribe calls
2021-02-26 11:32:44 -08:00
Kubernetes Prow Robot
dbb6cfa7fb Merge pull request #99445 from pohly/generic-ephemeral-volume-test
test: fix skipping of GenericEphemeralVolume volume type
2021-02-26 11:32:36 -08:00
drfish
38ad860c2f Unroll PreemptHandle to Handle 2021-02-27 00:44:05 +08:00
Kubernetes Prow Robot
43e8405d9c Merge pull request #99434 from pacoxu/e2e/npe-fix
fix nil error e2e/network/kube_proxy.go
2021-02-26 05:34:15 -08:00
wojtekt
cad9957f95 Remove KubeDescribe from SIGDescribe calls 2021-02-26 08:49:25 +01:00
James Sturtevant
232a9ac892 Updating owners with sig-windows tech leads 2021-02-25 17:17:06 -08:00
Kubernetes Prow Robot
58052f4dff Merge pull request #99469 from wojtek-t/cleanup_describe_6
Move common/volume.go test to the storage dir
2021-02-25 14:46:17 -08:00
Kubernetes Prow Robot
ca916183db Merge pull request #99460 from wojtek-t/transfer_secrets_ownership
Align ownership of secret and configmap tests to sig-node
2021-02-25 13:52:56 -08:00
wojtekt
67de069dd1 Move common/volume.go test to the storage dir 2021-02-25 20:40:08 +01:00
Kubernetes Prow Robot
7791be16eb Merge pull request #99450 from jkaniuk/revert-5m-dump
Revert "tests: Set a 5 minute timeout for kubectl cluster-info dump"
2021-02-25 10:01:53 -08:00
Kubernetes Prow Robot
8d42920e94 Merge pull request #99380 from ii/Riaankl_ReplicaSet
Write ReplicaSet Replace and Patch Test +2 Endpoints
2021-02-25 10:01:37 -08:00
Kubernetes Prow Robot
4fefc856db Merge pull request #99213 from AliceZhang2016/affinity
parse pod's node affinity once in preFilter
2021-02-25 10:01:24 -08:00
Kubernetes Prow Robot
5feb39fdb3 Merge pull request #99458 from wojtek-t/cleanup_describe_1
Replace some nested KubeDescribe calls with ginkgo.Describe
2021-02-25 08:01:25 -08:00
wojtekt
8df355b80a Replace some nested KubeDescribe calls with ginkgo.Describe 2021-02-25 14:44:39 +01:00
wojtekt
22429ab642 Align ownership of secret and configmap tests to sig-node 2021-02-25 14:38:02 +01:00
Kubernetes Prow Robot
2b930d99bd Merge pull request #99374 from wojtek-t/cleanup_describe_3
Cleanup tests/e2e/common directory - part 1
2021-02-25 05:13:25 -08:00
Jacek Kaniuk
f3f4d535d5 Revert "tests: Set a 5 minute timeout for kubectl cluster-info dump"
This reverts commit bc16100549.
2021-02-25 11:27:29 +01:00
lala123912
652f8fd3e9 typo fixed 2021-02-25 17:52:17 +08:00
Patrick Ohly
a255b74d11 test: fix skipping of GenericEphemeralVolume volume type
The test should run for test drivers which support dynamic
provisioning, but was skipped because of the volume type check:

External Storage [Driver: hostpath.csi.k8s.io]
[90m/home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Generic Ephemeral-volume (default fs) [Feature:GenericEphemeralVolume] (late-binding)] ephemeral
  [90m/home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    [36mshould support multiple inline ephemeral volumes [BeforeEach]
    [90m/home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:211

    [36mDriver "hostpath.csi.k8s.io" does not support volume type "GenericEphemeralVolume" - skipping

    /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
2021-02-25 10:09:40 +01:00
Alexander Minbaev
5b73122105 Fix typo in util.go 2021-02-25 00:54:37 -06:00
pacoxu
6b94cad4e9 fix nil error e2e/network/kube_proxy.go 2021-02-25 12:00:42 +08:00
pacoxu
9488decd5e use random node to avoid node0 overloaded 2021-02-25 11:10:02 +08:00