Commit Graph

229 Commits

Author SHA1 Message Date
Caleb Woodbine
63632118b7 Fix conformance testdata 2020-04-01 09:43:56 +13:00
Caleb Woodbine
02f983d8c8 Update metadata 2020-03-27 08:59:33 +13:00
Jefftree
074a676dbc kubetestgen -> kubeconform 2020-03-25 14:37:09 -07:00
Jefftree
77edd42619 Add CLI script for listing untested conformance behaviors 2020-03-25 14:37:09 -07:00
Kubernetes Prow Robot
1da4f75e24
Merge pull request #85128 from kevtaylor/kep/VolumeSubpathExpansion-Conformance
Promote VolumeSubpathEnvExpansion e2e tests to Conformance
2020-03-23 15:32:47 -07:00
Jefftree
40d80195f2 Only check testname for missing 2020-03-18 14:54:06 -07:00
Jefftree
90cd496e72 Include testname 2020-03-18 14:54:06 -07:00
Kevin Taylor
468b3b40d8 Promote VolumeSubpathExpansion e2e tests to Conformance 2020-03-12 20:05:28 +00:00
Jefftree
db714336a9 Update conformance.yaml to remove comments not part of Description 2020-03-06 11:37:35 -08:00
Jefftree
6da6380d1b Capture behaviors in walk.go 2020-03-06 11:09:25 -08:00
fengzixu
076132e38a cleanup: move the test of TaintBasedEvictions features to sig-node
1. move the integration test of TaintBasedEvictions to test/integration/node
2. move the e2e test of TaintBasedEvictions e2e test/e2e/node
3. modify the conformance file to adapt the TaintBasedEviction test
2020-03-04 10:28:00 +09:00
Caleb Woodbine
51d3f64b81 Fix conformance testdata 2020-03-02 14:23:29 +13:00
Caleb Woodbine
c33e3bdc28 Fix conformance testdata 2020-03-02 14:17:24 +13:00
Caleb Woodbine
67fb7c5989 Fix release number 2020-03-02 14:09:20 +13:00
Caleb Woodbine
40dadaaa9c Promote Pod Preemption Verification 2020-03-02 14:08:30 +13:00
Caleb Woodbine
633a72ce28 Promote: PodTemplate Lifecycle test 2020-03-02 13:57:17 +13:00
John Schnake
2683b1065c Update the conformance list and doc generation logic
The existing walk.go and conformance.txt have a few shortcomings
which we'd like to resolve:
 - difficult to get the full test name due to test context nesting
 - complicated AST logic and understanding necessary due to the
different ways a test can be invoked and written

This changes the AST parsing logic to be much more simple and simply
looks for the comments at/around a specific line. This file/line
information (and the full test name) is gathered by a custom ginkgo
reporter which dumps the SpecSummary data to a file.

Also, the SpecSummary dump can, itself, be potentially useful for
other post-processing and debugging tasks.

Signed-off-by: John Schnake <jschnake@vmware.com>
2020-02-24 14:00:44 -08:00
Kubernetes Prow Robot
23045f9247
Merge pull request #88057 from julianvmodesto/remove-deprecated-rolling-update
Remove deprecated rolling-update command
2020-02-21 18:31:30 -08:00
Kubernetes Prow Robot
5ab1f3464c
Merge pull request #88126 from johnbelamaric/behaviors-readme
Add a README describing behaviors
2020-02-18 14:04:41 -08:00
John Belamaric
ca399e6315 Add a README describing behaviors 2020-02-18 09:23:46 -08:00
Julian V. Modesto
932500c015 Remove deprecated rolling-update command 2020-02-13 23:52:53 -05:00
Jefftree
da4b0b0fa1 Validation for behaviors 2020-02-04 10:45:53 -08:00
Kubernetes Prow Robot
2a17cfbd45
Merge pull request #85960 from johnbelamaric/podspec-behaviors
Initial behavior examples for conformance
2020-02-02 14:47:21 -08:00
John Belamaric
becdd79067 Initial example behaviors
Organizes the behaviors directory based on SIG, and provides a few
example behavior descriptions for Pods and Services to start. Note
that these are very incomplete lists of behaviors at this point.
2020-01-31 11:25:55 -08:00
Kubernetes Prow Robot
dba8d60f8c
Merge pull request #87077 from soltysh/remove_run_generators
Remove kubectl run generators
2020-01-29 12:16:14 -08:00
Kubernetes Prow Robot
7883d93877
Merge pull request #87262 from ii/promote-secret-patching-test
Promote: Secret patching test
2020-01-29 07:32:28 -08:00
Kubernetes Prow Robot
1590c7b31c
Merge pull request #87371 from ii/promote-limit-range-defaults-test
Add: promotion for LimitRange defaults test to Conformance
2020-01-28 20:46:14 -08:00
Kubernetes Prow Robot
426b353890
Merge pull request #87260 from ii/promote-test-for-kubernetes-service-in-default-namespace
Promote: find Kubernetes Service in default Namespace
2020-01-25 03:05:01 -08:00
Kubernetes Prow Robot
4a3cbfea89
Merge pull request #87256 from ii/promote-namespace-patch-test
Promote: Namespace patch test
2020-01-23 17:40:45 -08:00
Maciej Szulik
82f97775b7
Remove kubectl run generators 2020-01-22 16:44:00 +01:00
Kubernetes Prow Robot
3091ae7aac
Merge pull request #85486 from spiffxp/conformance-liveness-tcp-probe
Promote a TCP socket probe test to Conformance
2020-01-21 16:26:35 -08:00
Caleb Woodbine
bc2aabaaeb Add: promotion for LimitRange defaults test to Conformance 2020-01-19 20:44:42 +00:00
Caleb Woodbine
8676fb3cdb Promote: Secret patching test 2020-01-15 20:37:23 +00:00
Caleb Woodbine
4efcb21af8 Promote: find Kuberntes Service in default Namespace 2020-01-15 20:16:02 +00:00
Caleb Woodbine
0f528a27ad Promote: Namespace patch test 2020-01-15 19:42:16 +00:00
Kubernetes Prow Robot
91e55c7927
Merge pull request #85154 from johnbelamaric/johnbelamaric-conformance-owners
Add johnbelamaric as conformance code reviewer/approver
2020-01-09 11:32:47 -08:00
Aaron Crickenberger
4d731d92bf Demote Delete Grace Period test to [Flaky]
This is currently the top flake against PRs, so I'm tagging it
as [Flaky]. Flaky tests can't be conformance tests, so I'm
removing it from [Conformance] as well until this is resolved.
2020-01-08 10:12:47 -08:00
Slava Semushin
02885845ff
kubetestgen: improve errors handling
- don't proceed when a file creation has been failed
- handle a possible error from WriteString()
2020-01-06 15:21:54 +00:00
Aaron Crickenberger
877381550d Revert "promote e2e tests for taint-tolerations in predicates"
This reverts commit 4d3d364d2b.

This commit promoted a test to conformance and also changed the test at
the same time. It should have been split up into two PRs to verify
whether the change had the intended effect. The test is now failing
consistently, so reverting.
2019-12-18 10:01:00 -05:00
Kubernetes Prow Robot
7fdefe5002
Merge pull request #84735 from liggitt/wardle-example-e2e
wardle example e2e
2019-12-17 12:00:28 -08:00
Jordan Liggitt
8111bb3d77 Update wardle e2e image 2019-12-16 11:06:11 -05:00
Jefftree
4d3d364d2b promote e2e tests for taint-tolerations in predicates 2019-12-05 16:54:02 -08:00
Aaron Crickenberger
3cbc07bb64 Promote a TCP probe test to Conformance 2019-11-21 12:12:58 -08:00
Dr. Stefan Schimanski
69282dc4d8 e2e: promote CRD defaulting test to conformance test 2019-11-15 15:19:45 +01:00
John Belamaric
3d745c4ad4 Add johnbelamaric as conformance code reviewer/approver 2019-11-12 13:51:16 -08:00
Jefftree
fd7eaffd99 kubetestgen implementation 2019-11-08 15:40:22 -08:00
John Belamaric
c0e6a8704c Add conformance-behavior-approvers to OWNERS_ALIASES 2019-10-29 11:24:06 -07:00
Timothy St. Clair
97055841b1 Audit of test/* OWNERS files 2019-10-10 15:52:51 -05:00
Kubernetes Prow Robot
c449359119
Merge pull request #74977 from mgdevstack/master-conformance-pqdn
Promote DNS e2e verifying PQDN resolution for cluster
2019-09-26 14:39:36 -07:00
Kubernetes Prow Robot
c0ad143f64
Merge pull request #82632 from wojtek-t/node_lease_conformance
Promote lease API test to Conformance
2019-09-26 13:02:49 -07:00
John Schnake
d0ab1f06fb Allow disruptive conformance tests and marks 2
- Adjusts tooling to allow disrutpive tests to be marked conformance
 - Updates the corresponding tests
 - Marks 2 tests as disruptive
2019-09-24 14:17:37 -05:00
wojtekt
4ec66c842e Promote lease API test to Conformance 2019-09-12 19:58:04 +02:00
John Belamaric
84070403da Add johnbelamaric as conformance approver 2019-09-11 12:36:13 -07:00
Mayank Gaikwad
eb07002d41 Promote PQDN for services DNS e2e to Conformance 2019-09-06 02:14:46 +05:30
Haowei Cai
fd68c41e42 promote crd discovery e2e to conformance 2019-08-30 14:07:19 -07:00
Kubernetes Prow Robot
1e6686e6af
Merge pull request #81864 from jpbetz/crd-conformance
Promote CRD e2e tests to conformance
2019-08-29 19:55:59 -07:00
Joe Betz
68bafe4404 Promote admission webhook e2e tests to conformance 2019-08-29 13:50:14 -07:00
Joe Betz
c7919e64ff Promote CRD e2e tests to conformance 2019-08-29 11:50:24 -07:00
Kubernetes Prow Robot
56aded4db0
Merge pull request #81721 from mattjmcnaughton/mattjmcnaughton/close-open-files-in-conformance
Close leaking files in `conformance/walk.go`
2019-08-23 00:51:45 -07:00
mattjmcnaughton
f7128535bd
Close leaking files in conformance/walk.go
We want to ensure we close this file handler if it was opened
successfully.
2019-08-21 08:59:35 -04:00
Mayank Gaikwad
7c64fc9d98 Promote e2e verifying service type updates 2019-08-14 16:07:59 +05:30
Kubernetes Prow Robot
9d3d20114a
Merge pull request #78331 from mgdevstack/master-promote-resourcequota
Promote resourceQuota e2e verifying 'object count quota' and 'quota scope' to Conformance
2019-08-08 17:57:25 -07:00
Kubernetes Prow Robot
cc4301025d
Merge pull request #80655 from johnbelamaric/promote-nodeport-creation
Promote NodePort service creation e2e test to conformance
2019-08-06 23:56:50 -07:00
John Belamaric
60a51ce020 Promote NodePort service creation e2e test to conformance 2019-08-06 16:23:40 -07:00
Aaron Crickenberger
e0750bd8db Promote taint-based eviction e2e tests to Conformance 2019-07-26 11:50:25 -07:00
Kubernetes Prow Robot
9e4b126e18
Merge pull request #80550 from spiffxp/promote-jobs-complete-after-restart
Promote job completion after failure e2e test to Conformance
2019-07-25 22:35:58 -07:00
Kubernetes Prow Robot
8497639221
Merge pull request #80544 from spiffxp/promote-jobs-adopt-release-orphan-pods
Promote job pod orphaning/adotion e2e test to Conformance
2019-07-25 20:48:36 -07:00
Kubernetes Prow Robot
2321d1e9e8
Merge pull request #80150 from ii/promote-statefulset-replica-scaling
Promote StatefulSet Replica scaling
2019-07-25 16:02:07 -07:00
Mayank Gaikwad
a921970745 Promote resourceQuota e2e verifying 'object count quota' and 'quota scope' to Conformance 2019-07-25 20:01:35 +05:30
Aaron Crickenberger
e0b9d77609 Promote job completion after failure e2e test to Conformance 2019-07-24 17:17:07 -07:00
Aaron Crickenberger
275b7b2fa0 Promote job pod orphaning/adotion e2e test to Conformance 2019-07-24 16:28:10 -07:00
Kubernetes Prow Robot
65b094676f
Merge pull request #78350 from globervinodhn/hostip_hostport_promote
Promote existing E2Es for hostport/HostIP resolution to conformance
2019-07-19 14:25:24 -07:00
Caleb Woodbine
5db35e67ee Fix placement of test in conformance.txt 2019-07-16 21:36:55 +00:00
Caleb Woodbine
91da6fd565 Promote API metadata return value 2019-07-15 21:40:14 +00:00
Caleb Woodbine
a062367251 Promote StatefulSet Replica scaling 2019-07-15 02:02:39 +00:00
Kubernetes Prow Robot
f7f1b2d5f7
Merge pull request #72168 from brahmaroutu/conformance_hdr
Allow version field in the title to be changeable
2019-07-01 19:09:08 -07:00
Aaron Crickenberger
ffe70f1a6e Conformance walker should handle nested/adjacent Describes
This also revealed that the regex for Context was too generous and would
catch things like SecurityContext or ContextWithFoo(...) calls

This ensures that test suites with a ineligible tag in their top-level
Describe will be rejected from promotion to conformance
2019-06-25 21:41:38 -07:00
pontiyaraja
40146c85f4 re-named conformance test name for pod creating with CPU and memory resources 2019-06-17 14:54:08 +05:30
Kubernetes Prow Robot
16bc6c1d02
Merge pull request #74982 from mgdevstack/promote-dns-hostname
Promote DNS e2e verifying DNS resolution for hostname and subdomain
2019-06-13 19:13:11 -07:00
Kubernetes Prow Robot
62eae8d058
Merge pull request #70639 from mgdevstack/promote-security-context
Promote security context NodeConformance tests to Conformance suite
2019-06-13 17:48:20 -07:00
globervinodhn
faf5cda046 Promote existing E2Es for hostport/HostIP resolution to conformance 2019-06-06 11:39:27 +05:30
Aaron Crickenberger
67be4f5d06 Error on tests ineligible for promotion to conformance
If there are tags in the test name that describe qualities of the
test that make it ineligible for conformance, raise an error. This
is basically the "skip list" that heptio's e2e image used to use.

Thankfully all of our existing Conformance tests lack these tags. I
considered added [Slow] to the list, but let's save that for another
day.
2019-06-05 10:36:59 -07:00
Mayank Gaikwad
fc2af0e5a4 Promote e2e verifying DNS resolution for o pod's fqdn, hostname and subdomain 2019-05-14 23:53:00 +05:30
Kubernetes Prow Robot
f712c3ee0e
Merge pull request #77620 from johnbelamaric/add-to-conformance-reviewers
Add johnbelamaric as conformance test reviewer
2019-05-10 17:56:24 -07:00
Kubernetes Prow Robot
7f86cb140f
Merge pull request #72167 from brahmaroutu/conformance_context
Comment association to ConformanceIt block should be validated properly.
2019-05-08 20:12:48 -07:00
John Belamaric
4f3653f026 Add johnbelamaric as conformance test reviewer 2019-05-08 16:37:25 -07:00
Kubernetes Prow Robot
7faeee22b1
Merge pull request #77556 from spiffxp/add-spiffxp-to-conformance-code-approvers
Promote spiffxp to approver, add oomichi as reviewer
2019-05-07 22:15:10 -07:00
Srini Brahmaroutu
e9bab1c896 Allow version field in the title to be changeable 2019-05-07 15:05:04 -07:00
Aaron Crickenberger
8ee8b77eed Promote spiffxp to approver, add oomichi as reviewer 2019-05-07 12:11:36 -07:00
Srini Brahmaroutu
1cb5872cd3 Comment association to ConformanceIt block should be validated properly. 2019-05-07 09:16:44 -07:00
Srini Brahmaroutu
37e700050b Add Release information to each of the conformance tests. 2019-05-07 09:14:08 -07:00
Davanum Srinivas
7b8c9acc09
remove unused code
Change-Id: If821920ec8872e326b7d85437ad8d2620807799d
2019-04-19 08:36:31 -04:00
Kubernetes Prow Robot
03c5079ac1
Merge pull request #75134 from mgdevstack/promote-runtime-3
Promote container runtime e2e verifying termination message
2019-04-16 02:11:44 -07:00
Kubernetes Prow Robot
9217cb5b6e
Merge pull request #76350 from jpbetz/watch-consistency-conformance
Promote 'watch-consistency' e2e test to be a conformance test
2019-04-15 18:05:20 -07:00
Kubernetes Prow Robot
94a1458638
Merge pull request #76368 from SataQiu/cleanup-test-20190410
Cleanup command prompt for readme.md
2019-04-15 10:26:06 -07:00
Mayank Gaikwad
82822f10e1 Promote container runtime e2e verifying termination message 2019-04-15 16:37:18 +05:30
SataQiu
9f8ee74a52 cleanup command prompt for readme.md 2019-04-13 19:10:36 +08:00
Kubernetes Prow Robot
9aac5ced1f
Merge pull request #74702 from globervinodhn/secret_empty_promote
Promote empty secret key e2e test to conformance test
2019-04-12 21:38:02 -07:00
Kubernetes Prow Robot
2a6efd3fe7
Merge pull request #75473 from pontiyaraja/podCNF
promote pod should be submitted and removed e2e test case to conformance
2019-04-12 15:56:18 -07:00
Kubernetes Prow Robot
2d5f5a9da5
Merge pull request #75471 from pontiyaraja/sharedVolCNF
promote shared volumes between the container in a pod e2e to conformance
2019-04-12 15:56:04 -07:00