`FullStackTrace` is not available in v2 if no exception found
with test execution.
The change is needed for conformance test's spec validation.
pls see: https://github.com/onsi/ginkgo/issues/960 for details.
Signed-off-by: Dave Chen <dave.chen@arm.com>
- update all the import statements
- run hack/pin-dependency.sh to change pinned dependency versions
- run hack/update-vendor.sh to update go.mod files and the vendor directory
- update the method signatures for custom reporters
Signed-off-by: Dave Chen <dave.chen@arm.com>
`verify-conformance-yaml.sh` is used to verify the test spec of
conformance e2e test are valid.
Bascically, it calls `e2e.test` binary to dump the test spec and
then walk the source to generate the `conformance.yaml`.
If the `e2e.test` binary is outdated, it's possible that `testInfo`
is nil.
Access the field from nil will result to `nil pointer dereference`
Signed-off-by: Dave Chen <dave.chen@arm.com>
The existing walk.go and conformance.txt have a few shortcomings
which we'd like to resolve:
- difficult to get the full test name due to test context nesting
- complicated AST logic and understanding necessary due to the
different ways a test can be invoked and written
This changes the AST parsing logic to be much more simple and simply
looks for the comments at/around a specific line. This file/line
information (and the full test name) is gathered by a custom ginkgo
reporter which dumps the SpecSummary data to a file.
Also, the SpecSummary dump can, itself, be potentially useful for
other post-processing and debugging tasks.
Signed-off-by: John Schnake <jschnake@vmware.com>
This also revealed that the regex for Context was too generous and would
catch things like SecurityContext or ContextWithFoo(...) calls
This ensures that test suites with a ineligible tag in their top-level
Describe will be rejected from promotion to conformance
If there are tags in the test name that describe qualities of the
test that make it ineligible for conformance, raise an error. This
is basically the "skip list" that heptio's e2e image used to use.
Thankfully all of our existing Conformance tests lack these tags. I
considered added [Slow] to the list, but let's save that for another
day.
To detect whether conformance tests have been changed, the test
argument with the `ConformanceIt` is recorded to conformance.txt.
This change remove additional tags (e.g., [Feature:]) and trim spaces,
so that the detection is less-prone to noise.
This change also updates conformance.txt with normalized names.
This test creates a golden list of existing conformance tests. Efforts
to add or remove conformance tests will require you to rebuild the
golden list, and changes to the golden list will be reviewed by
sig-architecture.
