Commit Graph

465 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
9dc13f4f57
Merge pull request #111011 from ii/promote-patch-pod-status-test
Promote patchCoreV1NamespacedPodStatus test - +1 endpoint
2022-07-08 09:07:52 -07:00
Dave Chen
2f3028c00d Define the timeout to 24h for Ginkgo V2
Default timeout setting has been reduced from `24h` down to `1h` in
Ginkgo V2, but for some long running test this is too short.

How long to abort the test was controlled by the the linux command `timeout`
in V1. e.g. `'timeout -k 30s 150m ...`, and is configured in the file
like `sig-network-misc.yaml`.

Set the timeout manually for Ginkgo V2 to avoid the early aborting.

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:46:11 +08:00
Dave Chen
20498fd65d Generate conformance test spec with offset decorator
`FullStackTrace` is not available in v2 if no exception found
with test execution.

The change is needed for conformance test's spec validation.

pls see: https://github.com/onsi/ginkgo/issues/960 for details.

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:46:11 +08:00
Dave Chen
b57bade50c Switch to use dry-run option to generate test spec
`dryRun` option is deprecated and will not be able to
generate test spec.

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:46:11 +08:00
Dave Chen
375b2a5fb2 Build Ginkgo binary
Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:46:11 +08:00
Dave Chen
857458cfa5 update ginkgo from v1 to v2 and gomega to 1.19.0
- update all the import statements
- run hack/pin-dependency.sh to change pinned dependency versions
- run hack/update-vendor.sh to update go.mod files and the vendor directory
- update the method signatures for custom reporters

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:46 +08:00
Stephen Heywood
c288ebe53c Promote PodStatus e2e test to Conformance 2022-07-08 11:51:58 +12:00
Kubernetes Prow Robot
92c5f3f01c
Merge pull request #110583 from Riaankl/node_status_ineligible
Update ineligible_endpoints.yaml
2022-06-16 12:33:04 -07:00
Riaan Kleinhans
39d3d9439f
Update ineligible_endpoints.yaml 2022-06-17 06:24:35 +12:00
Riaan Kleinhans
6d19ddc7bf
Update ineligible_endpoints.yaml 2022-06-15 10:27:33 +12:00
Wes Panther
6ac853fe60 Update Debian base images to latest
- debian-base:bullseye-v1.3.0
- debian-iptables:bullseye-v1.4.0
- setcap:bullseye-v1.3.0
2022-06-13 21:36:46 -07:00
Kubernetes Prow Robot
b4fbd12c3a
Merge pull request #106441 from yjjy0921/master
create resultsDir if the folder not exists
2022-06-01 13:39:46 -07:00
Davanum Srinivas
50bea1dad8
Move from k8s.gcr.io to registry.k8s.io
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-05-31 10:16:53 -04:00
Kubernetes Prow Robot
5e9a6a256e
Merge pull request #109464 from liggitt/gomodule-install
Use module mode when building/installing
2022-05-05 16:28:52 -07:00
Kubernetes Prow Robot
f853a05e24
Merge pull request #109534 from ii/promote-job-lifecycle-test
Promote Batchv1JobLifecycleTest +4 Endpoints
2022-05-05 10:50:21 -07:00
Jordan Liggitt
2a10ca650d drop vendor from test targets 2022-05-05 08:47:33 -04:00
Claudiu Belu
0570f611a2 tests: Enables a few Conformance tests for Windows
Some of these tests could not be run previously, especially on Windows
Docker containers. But now, by using Windows Containerd, we can finally
run them:

- HostNetwork=true tests: This can now be enabled on Windows Privileged Containers.
- /etc/hosts related tests: These were not supported because it required single
  file mappings, which is possible in Containerd.
- termination message as non-root user: Requires RunAsUsername, and single file
  mappings.
2022-05-04 13:07:24 -07:00
Kubernetes Prow Robot
70ca1dbb81
Merge pull request #107444 from sayaoailun/master
fix comment of e2e test case garbage_collector
2022-05-03 17:18:23 -07:00
Stephen Heywood
18d33db623 Promote e2e job lifecycle test to Conformance 2022-04-19 09:35:27 +12:00
Stephen Augustus
cddd83c997
Update Debian base images
- debian-base:bullseye-v1.2.0
- debian-iptables:bullseye-v1.3.0
- setcap:bullseye-v1.2.0

Signed-off-by: Stephen Augustus <foo@auggie.dev>
2022-04-13 16:23:54 -04:00
James Sturtevant
3d09201dbd Disable node->pod tests for Windows 2022-04-05 16:17:13 -07:00
Kubernetes Prow Robot
66369f47ec
Merge pull request #109294 from Riaankl/NetworkPolicyStatus_endpoints
Add NetworkPolicyStatus to ineligible_endpoints.yaml
2022-04-05 13:33:49 -07:00
Kubernetes Prow Robot
a1a9dc066c
Merge pull request #109256 from chendave/nullpointer
Fix the nil pointer dereference for comformance spec validation
2022-04-05 13:33:37 -07:00
Kubernetes Prow Robot
5163909f64
Merge pull request #108867 from pohly/storage-capacity-conformance
storage capacity conformance
2022-04-05 13:33:24 -07:00
Kubernetes Prow Robot
c13d3369d5
Merge pull request #109267 from ii/promote-job-status-test
Promote Read, Replace, Patch BatchV1NamespacedJobStatus test - +3 endpoints
2022-04-05 08:23:49 -07:00
Riaan Kleinhans
a4a6d70168
Add NetworkPolicyStatus to ineligible list 2022-04-05 07:36:38 +12:00
Kubernetes Prow Robot
a315ec3388
Merge pull request #108590 from claudiubelu/tests/enable-windows-tests
tests: Enables a few Conformance tests for Windows (part 1)
2022-04-04 08:56:11 -07:00
Patrick Ohly
556794d469 storage capacity: promote CRUD test to conformance 2022-04-04 07:53:05 +02:00
Stephen Heywood
c709baa9a0 Promote e2e job status test to Conformance 2022-04-04 13:22:57 +12:00
Dave Chen
1b5d8b0b9b Fix the nil pointer dereference for comformance spec validation
`verify-conformance-yaml.sh` is used to verify the test spec of
conformance e2e test are valid.

Bascically, it calls `e2e.test` binary to dump the test spec and
then walk the source to generate the `conformance.yaml`.

If the `e2e.test` binary is outdated, it's possible that `testInfo`
is nil.

Access the field from nil will result to `nil pointer dereference`

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-04-02 16:53:39 +08:00
Kevin Delgado
f234cb142a Adjust testing for server-side validation as default 2022-03-28 04:11:27 +00:00
Kubernetes Prow Robot
99e36a93b2
Merge pull request #108781 from SergeyKanzhelev/conformanceruntimeClass
RuntimeClass and PodOverhead tests promotion to Conformance
2022-03-23 22:31:41 -07:00
Sergey Kanzhelev
29e8bef315 RuntimeClass and PodOverhead promotion to GA 2022-03-17 21:49:32 +00:00
Aldo Culquicondor
2c5d0a273c Graduate IndexedJob to stable
- Lock feature gate to true and schedule for deletion in 1.26
- Remove checks on feature gate
- Graduate E2E test to Conformance

Change-Id: I6814819d318edaed5c86dae4055f4b050a4d39fd
2022-03-15 13:41:06 -04:00
Kubernetes Prow Robot
f9be590b25
Merge pull request #108431 from ii/promote-replace-podtemplate
Promote replaceCoreV1NamespacedPodTemplate test - +1 endpoint coverage
2022-03-14 16:24:58 -07:00
Riaan Kleinhans
454571b1bc
add log endpoint to ineligible_endpoints.yaml 2022-03-14 09:59:21 +13:00
Claudiu Belu
87123c49a0 tests: Enables a few Conformance tests for Windows
Some of these tests could not be run previously, especially on Windows
Docker containers. But now, by using Windows Containerd, we can finally
run them:

- HostNetwork=true tests: This can now be enabled on Windows Privileged Containers.
- /etc/hosts related tests: These were not supported because it required single
  file mappings, which is possible in Containerd.
- termination message as non-root user: Requires RunAsUsername, and single file
  mappings.
2022-03-06 09:36:39 -08:00
Stephen Heywood
5597b0c4fb Promote replace podtemplate e2e test to Conformance 2022-03-02 10:33:02 +13:00
ahrtr
fe95aa614c io/ioutil has already been deprecated in golang 1.16, so replace all ioutil with io and os 2022-02-03 05:32:12 +08:00
sayaoailun
36697d2e5d fix comment of e2e test case garbage_collector
Signed-off-by: sayaoailun <guojianwei007@126.com>
2022-01-12 10:59:13 +08:00
Davanum Srinivas
9682b7248f
OWNERS cleanup - Jan 2021 Week 1
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-01-10 08:14:29 -05:00
Kubernetes Prow Robot
1c2f24f106
Merge pull request #107349 from Riaankl/add-HPA-1
Update ineligible_endpoints.yaml to include HPA
2022-01-05 19:08:48 -08:00
Kubernetes Prow Robot
faf1590950
Merge pull request #107179 from hakman/remove_dockershim_tests
Clean up dockershim in tests
2022-01-05 13:54:21 -08:00
Riaan Kleinhans
d42d547f0f
Update ineligible_endpoints.yaml 2022-01-06 09:52:27 +13:00
Kubernetes Prow Robot
2036bf0ba3
Merge pull request #106856 from pmorie/cronjob-e2e-typo
conformance: correct typo for cronjob forbidconcurrent
2022-01-04 11:27:14 -08:00
Ciprian Hacman
a0abe5aa33 Clean up dockershim in tests
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
2021-12-22 13:05:34 +02:00
Kubernetes Prow Robot
87a54cbd3f
Merge pull request #106313 from aojea/conformance
conformance tests should have at least 2 untainted nodes
2021-12-15 01:19:21 -08:00
Wei Cao
6f45cbd7a5 create resultsDir if the folder not exists 2021-12-14 18:02:58 +08:00
Kubernetes Prow Robot
aed75bf2ed
Merge pull request #106880 from ii/promote-proxy-301-redirect
Promote Pod- & ServiceProxy Test to Conformance - +12 endpoint coverage
2021-12-10 21:31:57 -08:00
Davanum Srinivas
9405e9b55e
Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-12-09 21:31:26 -05:00
Stephen Heywood
27a247df85 Promote pod and service Proxy e2e test to Conformance 2021-12-08 15:21:40 +13:00
Paul Morie
48a84fe9c7 conformance: correct typo for cronjob forbidconcurrent 2021-12-07 16:55:03 -05:00
Kubernetes Prow Robot
203d145b6a
Merge pull request #106281 from ii/promote-delete-service-collection
Promote DeleteCollection service e2e test to conformance - +1 endpoint
2021-11-18 07:47:03 -08:00
Aaron Crickenberger
357ad91d78 demote kubelet events test from Conformance
This test has been part of the Conformance suite since at least
Kubernetes 1.2 (2015-10-xx). Some years later, around 2018-10-xx, we
drafted a rigorous set of rules for tests to follow in order to be
eligible for promotion to Conformance. We explicitly disallowed any
tests that check for specific Events, since they are not an API, and we
make no guarantees about their contents nor their delivery.

Unfortunately, we neglected to go through the existing corpus of
Conformance tests with a fine-toothed comb after drafting these rules.
The very nature of what this test is attempting to exercise and verify
is specific Events, and their delivery, thus making it ineligible for
Conformance. We should have caught and demoted this test back then.

Better late than never?
2021-11-17 15:36:30 -08:00
Antonio Ojea
9723fbf3f1 conformance tests should have at least 2 untainted nodes 2021-11-10 16:32:30 +01:00
Stephen Heywood
675be94024 Promote DeleteCollection service e2e test to conformance 2021-11-10 10:19:46 +13:00
Kubernetes Prow Robot
c20ff9839e
Merge pull request #97045 from claudiubelu/tests-windows-containerd-enabled
tests: Enable some [LinuxOnly] tests for Windows Containerd
2021-10-06 19:53:45 -07:00
Stephen Augustus
1a15eeef38
test/conformance/image: Update runner image to base-debian11
Signed-off-by: Stephen Augustus <foo@auggie.dev>
2021-09-20 21:05:06 -04:00
Stephen Augustus
531eb712c2
releng: Update build images to Debian 11 (Bullseye)
- debian-base:bullseye-v1.0.0
- debian-iptables:bullseye-v1.0.0
- go-runner:v2.3.1-go1.17.1-bullseye.0
- kube-cross:v1.23.0-go1.17.1-bullseye.0
- setcap:bullseye-v1.0.0

Signed-off-by: Stephen Augustus <foo@auggie.dev>
2021-09-20 20:51:32 -04:00
Kubernetes Prow Robot
af551a3b22
Merge pull request #105006 from Riaankl/Riaankl-change-upper_case-to-lower_case
update ineligible_endpoints removing upper case `Endpoint` to `endpoint`
2021-09-14 11:30:05 -07:00
Kubernetes Prow Robot
c7074017f7
Merge pull request #104948 from ishangupta-ds/test-conf-update
update github.com/pkg/errors in test/conformance/image/go-runner/ with native go pkg
2021-09-14 10:21:08 -07:00
Riaan Kleinhans
e675e74ce5
update ineligible_endpoints removing upper case 2021-09-15 02:05:08 +12:00
ishangupta-ds
c6b82bddd6 removing usage of github.com/pkg/errors from test/conformance/image/go-runner/ directory
Signed-off-by: ishangupta-ds <ishangupta.ds@gmail.com>

formatted using gofmt script

Signed-off-by: ishangupta-ds <ishangupta.ds@gmail.com>

adding nil check for error

Signed-off-by: ishangupta-ds <ishangupta.ds@gmail.com>

adding nil check for errors

Signed-off-by: ishangupta-ds <ishangupta.ds@gmail.com>

formatted

Signed-off-by: ishangupta-ds <ishangupta.ds@gmail.com>
2021-09-14 08:39:00 +05:30
Kubernetes Prow Robot
5cbe2656c5
Merge pull request #104978 from Riaankl/Riaankl-update-ineligible_enpoint-RBAC
Update ineligible_endpoint.yaml to include RBAC
2021-09-13 15:44:11 -07:00
Kubernetes Prow Robot
caf853b596
Merge pull request #104696 from PushkarJ/bump-conformance-images
Update conformance image to use debian-base:buster-v1.9.0
2021-09-13 13:28:02 -07:00
Riaan Kleinhans
6fa2859bf5
Update ineligible_endpoint.yaml to include RBAC 2021-09-14 08:02:49 +12:00
Pushkar Joglekar
8ed31517ff Bump conformance images to use debian:buster-v1.9.0
- Debian base used was older (v2.1.3)  missing multiple fixed CVEs
- Minor update to distroless debian image name to explicitly point
  to debian 10
- Debian base image now points to buster-1.9.0
2021-09-13 09:22:36 -07:00
Kubernetes Prow Robot
4e30c058c4
Merge pull request #104290 from nak3/fix-obsolete-link
Update obsolete link in the Conformance Test doc
2021-09-05 04:02:39 -07:00
Antonio Ojea
2a5ad65a9a e2e test apiserver endpoint and endpointslices
The e2e test "should have Endpoints and EndpointSlices pointing to
the API Server Service" was veryfing the current endpoints
reconciler implementation on the apiservers, however, users may
disable the endpoint reconciler and create their own.

This e2e test is also a conformance test, so we should test the
behaviour and not the implementation details. The test verifies
that a kubernetes.default service exist, an endpoint and endpoint
slices object referencing that service exist and are equivalent.
2021-08-31 11:29:06 +02:00
Claudiu Belu
91c095a11f tests: Enable some [LinuxOnly] tests for Windows Containerd
Windows Containerd has more features than Windows Docker. One of them is single file
mappings, allowing us to also map individual files into containers, not just folders.

This will set the tag [Excluded:WindowsDocker] for those tests instead of [LinuxOnly].

Co-authored-by: Mark Rossetti <marosset@microsoft.com>
2021-08-16 07:43:24 -07:00
Kubernetes Prow Robot
e6c7837afe
Merge pull request #101063 from claudiubelu/tests/lifecycle-hooks
tests: Spawn poststart / prestop pods on the same node as the http pod
2021-08-12 10:19:49 -07:00
Kenjiro Nakayama
4f7cb4fb69 Update obsolete link in the Conformance Test doc
This patch updates the link for kubelet.go.

Recent conformance test docs always contain the invalid
link - e.g. https://github.com/cncf/k8s-conformance/blob/master/docs/KubeConformance-1.21.md#output
since 3b438eb4e5 moved the code.
2021-08-11 15:11:41 +09:00
Kubernetes Prow Robot
58cde6c442
Merge pull request #103874 from liggitt/move-conformance
Move conformance image
2021-08-05 08:17:34 -07:00
Jordan Liggitt
4115bef826 Update references to test/conformance/image 2021-07-23 11:26:54 -04:00
Jordan Liggitt
024930df11 Move cluster/images/conformance to test/conformance/image 2021-07-23 11:23:55 -04:00
Mike Miranda
0cce9a4a6c Remove conformance status from a sysctl test and relabel 2021-07-22 20:12:26 +00:00
Kubernetes Prow Robot
9ca75c1f49
Merge pull request #103243 from ii/promote-statefulset-status-test
Promote to Conformance StatefulSet Patch, Read and Replace Status test +3
2021-07-01 14:28:02 -07:00
Kubernetes Prow Robot
3334703eb2
Merge pull request #103242 from ii/promote-deployment-status-test
Promote to Conformance Patch, Read and Replace DeploymentStatus test +1
2021-07-01 13:18:04 -07:00
Kubernetes Prow Robot
cd94e840cb
Merge pull request #103241 from ii/promote-statefulset-list-deletecollection
Promote to Conformance StatefulSet List, Patch & DeleteCollection Test +3
2021-07-01 13:17:52 -07:00
Nabarun Pal
68b334d02b
test: setup embedded file sources for manifests
Signed-off-by: Nabarun Pal <pal.nabarun95@gmail.com>
2021-06-29 19:16:46 +05:30
Stephen Heywood
1b73a0040a Promote Stateful Status e2e test to Conformance 2021-06-28 11:24:59 +12:00
Stephen Heywood
405e9a5172 Promote Deployment Status e2e test to Conformance 2021-06-28 11:04:30 +12:00
Stephen Heywood
d3ad1d77f3 Promote Statefulset list and deleteCollection e2e test to Conformance 2021-06-28 10:06:27 +12:00
Claudiu Belu
f9e4a015e2 tests: Spawn poststart / prestop pods on the same node as the http pod
In the case of multinode clusters, the http server pod and the test cluster can
spawn on different nodes, which can be problematic for  poststart / prestop hooks,
as they are executed by the kubelet itself, and the cross-node lifecycle hook might
fail (according to the Kubernetes network model, it is not mandatory for kubelet to
be able to access pods on a different node).

This commit ensures that the test pod spawns on the same node as the http server pod.
2021-06-19 23:13:39 -07:00
Kubernetes Prow Robot
8e48d992aa
Merge pull request #102349 from ii/promote-replicaset-status-test
Promote Read, Patch & Replace ReplicaSet Status +3 endpoints
2021-06-16 12:48:00 -07:00
Kubernetes Prow Robot
727b4307f0
Merge pull request #102348 from ii/promote-daemonset-status-test
Promote Read, Patch & Replace DaemonSet Status e2e test to Conformance +3 endpoints
2021-06-16 09:43:59 -07:00
Kubernetes Prow Robot
d5e37cf9b6
Merge pull request #102346 from ii/promote-daemonset-list-deletecollection
Promote Daemonset list and deleteCollection e2e test to Conformance +2 Endpoints
2021-06-15 19:49:59 -07:00
Stephen Heywood
5ca47deb0a Promote DaemonSet Status e2e test to Conformance 2021-06-16 13:08:22 +12:00
Stephen Heywood
f7d02f4d25 Promote ReplicaSet Status e2e test to Conformance 2021-06-16 12:52:56 +12:00
Stephen Heywood
e89d2a7779 Promote Daemonset list and deleteCollection e2e test to Conformance 2021-05-26 14:30:43 +12:00
Stephen Heywood
5e4c59ba99 Promote Replicaset list and deleteCollection e2e test to Conformance 2021-05-26 14:24:39 +12:00
Riaan Kleinhans
94c550a050 Add NetworkPolicy Endpoint to Ineligible endpoints 2021-04-23 09:18:33 +12:00
Kubernetes Prow Robot
27a625cf89
Merge pull request #100724 from liggitt/eviction-v1beta1
Add policy/v1 Eviction support
2021-04-16 10:02:22 -07:00
Jordan Liggitt
a07d4460bf Promote e2e eviction test to conformance 2021-04-14 09:12:39 -04:00
Monis Khan
91241eac9b
Prune stale entries from OWNERS files
Signed-off-by: Monis Khan <mok@vmware.com>
2021-04-13 20:54:50 -04:00
Kubernetes Prow Robot
770d3f181c
Merge pull request #99734 from wgahnagl/sysctls-conformance
Promote sysctls e2e test to Conformance
2021-03-26 18:26:43 -07:00
Skyler Clark
c6b99025a6
adds sysctls conformance tests 2021-03-25 09:28:25 -04:00
Swetha Repakula
9c1f9c3962 Promote EndpointSlice API test to Conformance 2021-03-19 23:28:35 +00:00
Kubernetes Prow Robot
da341b5cfc
Merge pull request #100290 from ii/promote-service-status-test
Promote ServiceStatus lifecycle e2e test to Conformance  +4 endpoint coverage
2021-03-16 15:57:16 -07:00
Stephen Heywood
af1f64bee8 Promote ServiceStatus lifecycle e2e test to Conformance 2021-03-16 10:07:05 +13:00