Commit Graph

258 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
111741071f
Merge pull request #91743 from spiffxp/fix-kubeconform-link
Merge behavior loading code
2020-06-10 17:13:10 -07:00
Jordan Liggitt
6fb7d40a72 Move kube-apiserver client cert back to simple e2e 2020-06-10 15:28:26 -04:00
Stephen Heywood
04627cef49 Address issues with test failures 2020-06-08 20:41:32 +12:00
Jordan Liggitt
ea8f4cb536 Update CSR e2e to use v1 2020-06-05 00:50:01 -04:00
Aaron Crickenberger
d960bf1903 behaviors: enforce behavior id must have suite id as prefix 2020-06-03 16:59:34 -07:00
Aaron Crickenberger
85724cf2c3 behaviors: fix kubeconform link
kubeconform was choking on a typo in the description field, so I fixed
the typo while adding friendlier logging to tell me which file was
invalid

I got curious why tests didn't catch this, and it turns out kubeconform
and the behavior tests use different codepaths to load and validate. So
I merged them together
2020-06-03 16:59:27 -07:00
Kubernetes Prow Robot
bb11561ace
Merge pull request #80545 from spiffxp/promote-preemption-tests
Promote SchedulerPreemption e2e tests to Conformance
2020-06-02 15:28:16 -07:00
Kubernetes Prow Robot
65c0a1e64b
Merge pull request #89716 from Jefftree/bdd-initial-link
Add behavior <-> conformance test linkage example for pods
2020-05-26 23:24:41 -07:00
Aaron Crickenberger
4857893a0c Promote preemption e2e tests to Conformance 2020-05-20 23:05:15 -07:00
Kubernetes Prow Robot
57b0c310b7
Merge pull request #90390 from ii/promote-serviceaccount-resource-lifecycle-test
Promote ServiceAccount resource lifecycle test - +3 conformance coverage
2020-05-20 16:44:33 -07:00
Kubernetes Prow Robot
62a4cb930b
Merge pull request #90367 from julianvmodesto/conformance-diff-dry-run
Promote diff and dry-run to conformance
2020-05-20 16:44:19 -07:00
Jefftree
c7216b9a1b Fix unmarshal for tests without behaviors 2020-05-18 23:35:25 -07:00
Jefftree
ceb67482c4 link 2 pod conformance tests with behaviors 2020-05-18 23:35:25 -07:00
Julian V. Modesto
7af0011884 Promote diff and dry-run to conformance 2020-05-13 13:04:41 -04:00
Davanum Srinivas
b64e904ca8
Drop conformance for a HostPath test
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2020-05-07 22:40:00 -04:00
Davanum Srinivas
3292bb6145
Drop conformance tag for tests that rely directly on kubelet /logs API
In caf0d1d61874a2c8687b7deb773eca30ddaee5b6 we documented a policy to
ensure that conformance tests should not rely in existence or use of
kubelet apis directly. So based on that we should drop conformance for
the two tests here that use the "/logs" endpoint directly.

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2020-04-29 20:16:51 -04:00
Kubernetes Prow Robot
c12aeb2b77
Merge pull request #89753 from ii/promote-event-resource-lifecycle-test
Promote Event resource lifecycle test  +5 test endpoint coverage
2020-04-23 03:57:45 -07:00
Caleb Woodbine
e09e781a0c Promote ServiceAccount resource lifecycle test 2020-04-23 16:32:54 +12:00
Kubernetes Prow Robot
b7a6d53f5e
Merge pull request #88924 from dims/add-support-for-listing-conformance-tests-in-e2e.test
Add support for listing conformance tests in e2e.test
2020-04-15 13:14:45 -07:00
Kubernetes Prow Robot
53ffaa7175
Merge pull request #76443 from mgdevstack/promote-service-sessionaffinity
Promote e2e "verifying service's sessionAffinity for ClusterIP and NodePort services" to Conformance
2020-04-07 18:47:44 -07:00
Kubernetes Prow Robot
3072d1e22e
Merge pull request #88036 from ii/promote-podtemplate-lifecycle
Promote: PodTemplate Lifecycle test - +3 conformance endpoint coverage
2020-04-07 15:27:43 -07:00
Kubernetes Prow Robot
f21bd9efb9
Merge pull request #89745 from johnbelamaric/kubeconform-cleanup
Various cleanup for the kubeconform command
2020-04-07 12:49:44 -07:00
Kubernetes Prow Robot
667e028f38
Merge pull request #83378 from ii/promote-pod-replicaset-verification
Promote pod PreemptionExecutionPath verification - +3 conformance endpoint coverage
2020-04-06 19:09:43 -07:00
Mayank Gaikwad
7aa94a6549 Promote e2e verifying service's session affinity behaviours 2020-04-06 21:42:22 +05:30
Caleb Woodbine
31fea24c39 Promote Event resource lifecycle test 2020-04-02 15:36:49 +13:00
John Belamaric
de5d8d612b Various cleanup for the kubeconform command 2020-04-01 14:39:52 -07:00
Davanum Srinivas
d1c948acdc
Add support for listing conformance tests
we print yaml, so you can use yaml tools like `yq`:
```
e2e.test --list-conformance-tests | yq r - --collect *.testname
```

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2020-04-01 10:48:53 -04:00
Caleb Woodbine
ce97b49965 Update conformance metadata 2020-04-01 12:54:09 +13:00
Aaron Crickenberger
ec0d6cb693 Fix bazel location of testdata 2020-03-31 16:11:18 -07:00
Caleb Woodbine
63632118b7 Fix conformance testdata 2020-04-01 09:43:56 +13:00
Caleb Woodbine
02f983d8c8 Update metadata 2020-03-27 08:59:33 +13:00
Jefftree
074a676dbc kubetestgen -> kubeconform 2020-03-25 14:37:09 -07:00
Jefftree
77edd42619 Add CLI script for listing untested conformance behaviors 2020-03-25 14:37:09 -07:00
Kubernetes Prow Robot
1da4f75e24
Merge pull request #85128 from kevtaylor/kep/VolumeSubpathExpansion-Conformance
Promote VolumeSubpathEnvExpansion e2e tests to Conformance
2020-03-23 15:32:47 -07:00
Jefftree
40d80195f2 Only check testname for missing 2020-03-18 14:54:06 -07:00
Jefftree
90cd496e72 Include testname 2020-03-18 14:54:06 -07:00
Kevin Taylor
468b3b40d8 Promote VolumeSubpathExpansion e2e tests to Conformance 2020-03-12 20:05:28 +00:00
Jefftree
db714336a9 Update conformance.yaml to remove comments not part of Description 2020-03-06 11:37:35 -08:00
Jefftree
6da6380d1b Capture behaviors in walk.go 2020-03-06 11:09:25 -08:00
fengzixu
076132e38a cleanup: move the test of TaintBasedEvictions features to sig-node
1. move the integration test of TaintBasedEvictions to test/integration/node
2. move the e2e test of TaintBasedEvictions e2e test/e2e/node
3. modify the conformance file to adapt the TaintBasedEviction test
2020-03-04 10:28:00 +09:00
Caleb Woodbine
51d3f64b81 Fix conformance testdata 2020-03-02 14:23:29 +13:00
Caleb Woodbine
c33e3bdc28 Fix conformance testdata 2020-03-02 14:17:24 +13:00
Caleb Woodbine
67fb7c5989 Fix release number 2020-03-02 14:09:20 +13:00
Caleb Woodbine
40dadaaa9c Promote Pod Preemption Verification 2020-03-02 14:08:30 +13:00
Caleb Woodbine
633a72ce28 Promote: PodTemplate Lifecycle test 2020-03-02 13:57:17 +13:00
John Schnake
2683b1065c Update the conformance list and doc generation logic
The existing walk.go and conformance.txt have a few shortcomings
which we'd like to resolve:
 - difficult to get the full test name due to test context nesting
 - complicated AST logic and understanding necessary due to the
different ways a test can be invoked and written

This changes the AST parsing logic to be much more simple and simply
looks for the comments at/around a specific line. This file/line
information (and the full test name) is gathered by a custom ginkgo
reporter which dumps the SpecSummary data to a file.

Also, the SpecSummary dump can, itself, be potentially useful for
other post-processing and debugging tasks.

Signed-off-by: John Schnake <jschnake@vmware.com>
2020-02-24 14:00:44 -08:00
Kubernetes Prow Robot
23045f9247
Merge pull request #88057 from julianvmodesto/remove-deprecated-rolling-update
Remove deprecated rolling-update command
2020-02-21 18:31:30 -08:00
Kubernetes Prow Robot
5ab1f3464c
Merge pull request #88126 from johnbelamaric/behaviors-readme
Add a README describing behaviors
2020-02-18 14:04:41 -08:00
John Belamaric
ca399e6315 Add a README describing behaviors 2020-02-18 09:23:46 -08:00
Julian V. Modesto
932500c015 Remove deprecated rolling-update command 2020-02-13 23:52:53 -05:00
Jefftree
da4b0b0fa1 Validation for behaviors 2020-02-04 10:45:53 -08:00
Kubernetes Prow Robot
2a17cfbd45
Merge pull request #85960 from johnbelamaric/podspec-behaviors
Initial behavior examples for conformance
2020-02-02 14:47:21 -08:00
John Belamaric
becdd79067 Initial example behaviors
Organizes the behaviors directory based on SIG, and provides a few
example behavior descriptions for Pods and Services to start. Note
that these are very incomplete lists of behaviors at this point.
2020-01-31 11:25:55 -08:00
Kubernetes Prow Robot
dba8d60f8c
Merge pull request #87077 from soltysh/remove_run_generators
Remove kubectl run generators
2020-01-29 12:16:14 -08:00
Kubernetes Prow Robot
7883d93877
Merge pull request #87262 from ii/promote-secret-patching-test
Promote: Secret patching test
2020-01-29 07:32:28 -08:00
Kubernetes Prow Robot
1590c7b31c
Merge pull request #87371 from ii/promote-limit-range-defaults-test
Add: promotion for LimitRange defaults test to Conformance
2020-01-28 20:46:14 -08:00
Kubernetes Prow Robot
426b353890
Merge pull request #87260 from ii/promote-test-for-kubernetes-service-in-default-namespace
Promote: find Kubernetes Service in default Namespace
2020-01-25 03:05:01 -08:00
Kubernetes Prow Robot
4a3cbfea89
Merge pull request #87256 from ii/promote-namespace-patch-test
Promote: Namespace patch test
2020-01-23 17:40:45 -08:00
Maciej Szulik
82f97775b7
Remove kubectl run generators 2020-01-22 16:44:00 +01:00
Kubernetes Prow Robot
3091ae7aac
Merge pull request #85486 from spiffxp/conformance-liveness-tcp-probe
Promote a TCP socket probe test to Conformance
2020-01-21 16:26:35 -08:00
Caleb Woodbine
bc2aabaaeb Add: promotion for LimitRange defaults test to Conformance 2020-01-19 20:44:42 +00:00
Caleb Woodbine
8676fb3cdb Promote: Secret patching test 2020-01-15 20:37:23 +00:00
Caleb Woodbine
4efcb21af8 Promote: find Kuberntes Service in default Namespace 2020-01-15 20:16:02 +00:00
Caleb Woodbine
0f528a27ad Promote: Namespace patch test 2020-01-15 19:42:16 +00:00
Kubernetes Prow Robot
91e55c7927
Merge pull request #85154 from johnbelamaric/johnbelamaric-conformance-owners
Add johnbelamaric as conformance code reviewer/approver
2020-01-09 11:32:47 -08:00
Aaron Crickenberger
4d731d92bf Demote Delete Grace Period test to [Flaky]
This is currently the top flake against PRs, so I'm tagging it
as [Flaky]. Flaky tests can't be conformance tests, so I'm
removing it from [Conformance] as well until this is resolved.
2020-01-08 10:12:47 -08:00
Slava Semushin
02885845ff
kubetestgen: improve errors handling
- don't proceed when a file creation has been failed
- handle a possible error from WriteString()
2020-01-06 15:21:54 +00:00
Aaron Crickenberger
877381550d Revert "promote e2e tests for taint-tolerations in predicates"
This reverts commit 4d3d364d2b.

This commit promoted a test to conformance and also changed the test at
the same time. It should have been split up into two PRs to verify
whether the change had the intended effect. The test is now failing
consistently, so reverting.
2019-12-18 10:01:00 -05:00
Kubernetes Prow Robot
7fdefe5002
Merge pull request #84735 from liggitt/wardle-example-e2e
wardle example e2e
2019-12-17 12:00:28 -08:00
Jordan Liggitt
8111bb3d77 Update wardle e2e image 2019-12-16 11:06:11 -05:00
Jefftree
4d3d364d2b promote e2e tests for taint-tolerations in predicates 2019-12-05 16:54:02 -08:00
Aaron Crickenberger
3cbc07bb64 Promote a TCP probe test to Conformance 2019-11-21 12:12:58 -08:00
Dr. Stefan Schimanski
69282dc4d8 e2e: promote CRD defaulting test to conformance test 2019-11-15 15:19:45 +01:00
John Belamaric
3d745c4ad4 Add johnbelamaric as conformance code reviewer/approver 2019-11-12 13:51:16 -08:00
Jefftree
fd7eaffd99 kubetestgen implementation 2019-11-08 15:40:22 -08:00
John Belamaric
c0e6a8704c Add conformance-behavior-approvers to OWNERS_ALIASES 2019-10-29 11:24:06 -07:00
Timothy St. Clair
97055841b1 Audit of test/* OWNERS files 2019-10-10 15:52:51 -05:00
Kubernetes Prow Robot
c449359119
Merge pull request #74977 from mgdevstack/master-conformance-pqdn
Promote DNS e2e verifying PQDN resolution for cluster
2019-09-26 14:39:36 -07:00
Kubernetes Prow Robot
c0ad143f64
Merge pull request #82632 from wojtek-t/node_lease_conformance
Promote lease API test to Conformance
2019-09-26 13:02:49 -07:00
John Schnake
d0ab1f06fb Allow disruptive conformance tests and marks 2
- Adjusts tooling to allow disrutpive tests to be marked conformance
 - Updates the corresponding tests
 - Marks 2 tests as disruptive
2019-09-24 14:17:37 -05:00
wojtekt
4ec66c842e Promote lease API test to Conformance 2019-09-12 19:58:04 +02:00
John Belamaric
84070403da Add johnbelamaric as conformance approver 2019-09-11 12:36:13 -07:00
Mayank Gaikwad
eb07002d41 Promote PQDN for services DNS e2e to Conformance 2019-09-06 02:14:46 +05:30
Haowei Cai
fd68c41e42 promote crd discovery e2e to conformance 2019-08-30 14:07:19 -07:00
Kubernetes Prow Robot
1e6686e6af
Merge pull request #81864 from jpbetz/crd-conformance
Promote CRD e2e tests to conformance
2019-08-29 19:55:59 -07:00
Joe Betz
68bafe4404 Promote admission webhook e2e tests to conformance 2019-08-29 13:50:14 -07:00
Joe Betz
c7919e64ff Promote CRD e2e tests to conformance 2019-08-29 11:50:24 -07:00
Kubernetes Prow Robot
56aded4db0
Merge pull request #81721 from mattjmcnaughton/mattjmcnaughton/close-open-files-in-conformance
Close leaking files in `conformance/walk.go`
2019-08-23 00:51:45 -07:00
mattjmcnaughton
f7128535bd
Close leaking files in conformance/walk.go
We want to ensure we close this file handler if it was opened
successfully.
2019-08-21 08:59:35 -04:00
Mayank Gaikwad
7c64fc9d98 Promote e2e verifying service type updates 2019-08-14 16:07:59 +05:30
Kubernetes Prow Robot
9d3d20114a
Merge pull request #78331 from mgdevstack/master-promote-resourcequota
Promote resourceQuota e2e verifying 'object count quota' and 'quota scope' to Conformance
2019-08-08 17:57:25 -07:00
Kubernetes Prow Robot
cc4301025d
Merge pull request #80655 from johnbelamaric/promote-nodeport-creation
Promote NodePort service creation e2e test to conformance
2019-08-06 23:56:50 -07:00
John Belamaric
60a51ce020 Promote NodePort service creation e2e test to conformance 2019-08-06 16:23:40 -07:00
Aaron Crickenberger
e0750bd8db Promote taint-based eviction e2e tests to Conformance 2019-07-26 11:50:25 -07:00
Kubernetes Prow Robot
9e4b126e18
Merge pull request #80550 from spiffxp/promote-jobs-complete-after-restart
Promote job completion after failure e2e test to Conformance
2019-07-25 22:35:58 -07:00
Kubernetes Prow Robot
8497639221
Merge pull request #80544 from spiffxp/promote-jobs-adopt-release-orphan-pods
Promote job pod orphaning/adotion e2e test to Conformance
2019-07-25 20:48:36 -07:00
Kubernetes Prow Robot
2321d1e9e8
Merge pull request #80150 from ii/promote-statefulset-replica-scaling
Promote StatefulSet Replica scaling
2019-07-25 16:02:07 -07:00
Mayank Gaikwad
a921970745 Promote resourceQuota e2e verifying 'object count quota' and 'quota scope' to Conformance 2019-07-25 20:01:35 +05:30
Aaron Crickenberger
e0b9d77609 Promote job completion after failure e2e test to Conformance 2019-07-24 17:17:07 -07:00
Aaron Crickenberger
275b7b2fa0 Promote job pod orphaning/adotion e2e test to Conformance 2019-07-24 16:28:10 -07:00