Some tests have a short timeout for starting the pods (1 minute), but if
those tests happen to be the first ones to run, and the images have to be
pulled, then the test could timeout, especially with larger images. This
commit will allow us to prepull commonly used E2E test images, so this issue
can be avoided.
The Topology Manager e2e tests wants to run on real multi-NUMA system
and want to consume real devices supported by device plugins; SRIOV
devices happen to be the most commonly available of such devices.
CI machines aren't multi NUMA nor expose SRIOV devices, so the biggest portion
of the tests will just skip, and we need to keep it like this until we
figure out how to enable these features.
However, some organizations can and want to run the testsuite on bare metal;
in this case, the current test will skip (not fail) with misconfigured
boxes, and this reports a misleading result. It will be much better to
fail if the test preconditions aren't met.
To satisfy both needs, we add an option, controlled by an environment
variable, to fail (not skip) if the machine on which the test run
doesn't meet the expectations (multi-NUMA, 4+ cores per NUMA cell,
expose SRIOV VFs).
We keep the old behaviour as default to keep being CI friendly.
Signed-off-by: Francesco Romani <fromani@redhat.com>
The profile gatherer has been removed in
https://github.com/kubernetes/kubernetes/pull/85304, so those options
are unused since then and can therefore be removed.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Instead of allowing the cloud provider to guess at the zones that
should be applied for a cluster under test, allow the explicit list
of zones to consider to be passed as a new test context flag -gce-zones.
Only the GCE test cloud provider recognizes this value because only
the GCE test cloud provider makes assumptions about zones for verifying
values, and the default assumptions for GKE do not always match non-GKE
providers.
A number of e2e tests are useful to run after the system has been
disrupted or is in the progress of being disrupted, but the current
suite and test logic blocks progress waiting for all nodes to be
healthy.
By passing -1 to --minStartupPods or --allowed-not-ready-nodes flags
the caller can bypass wait logic before and after test suites that
would prevent running e2e during disruption. This allows use of parts
of the e2e suite during cluster duress to verify that controllers or
components still function.
Both of these are explicit arguments and are more elegantly logged
in a test framework by logging the arguments to the test.
The namespaces to be deleted are already logged inside
WaitForNamespacesDeleted
- re-enable e2e_node services
- call GenerateSecureToken for e2e_node Conformance test-suite
- add log messages indicating location in process
- move log messages to some more accurate locations
we print yaml, so you can use yaml tools like `yq`:
```
e2e.test --list-conformance-tests | yq r - --collect *.testname
```
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
The image "gcr.io/authenticated-image-pulling/windows-nanoserver:v1" is not a
manifest list, and it is only useful for Windows Server 1809, which means that the
test "should be able to pull from private registry with secret" will fail for
environments with Windows Server 1903, 1909, or any other future version we might
want to test.
This commit adds the the ability to have an alternative private image to pull by
using a configurable docker config file which contains the necessary credentials
needed to pull the image.
The existing walk.go and conformance.txt have a few shortcomings
which we'd like to resolve:
- difficult to get the full test name due to test context nesting
- complicated AST logic and understanding necessary due to the
different ways a test can be invoked and written
This changes the AST parsing logic to be much more simple and simply
looks for the comments at/around a specific line. This file/line
information (and the full test name) is gathered by a custom ginkgo
reporter which dumps the SpecSummary data to a file.
Also, the SpecSummary dump can, itself, be potentially useful for
other post-processing and debugging tasks.
Signed-off-by: John Schnake <jschnake@vmware.com>
We cannot anticipate all the possible configurations
needed by the SRIOV device plugin: there is too much variety.
Hence, we need to allow the test environment to supply
a host-specific ConfigMap to properly configure the device
plugin and avoid false negatives.
We still provide a the default config map as fallback and reference.
Signed-off-by: Francesco Romani <fromani@redhat.com>
One common frustration of end users running the e2e suite is that
they take a significant amount of time and it is difficult to
gauge progress.
Even if tailing the logs it can be difficult to see where one
test starts and another ends or understand the if there have been
failures in the past 1h of logs.
This change adds a new custom reporter which prints summary information
as tests complete. This includes the number of tests to run and how
many have been passed/failed/skipped along with which tests have failed.
A new flag can be set which pushes these values to an endpoint. This is
intended for integration with Sonobuoy but any endpoint could consume and
surface this data to the user so they can better understand the state
of the current test run.
e2e_node.test does not set default kubectlPath, which lead to test
errors as following:
[Fail] [sig-storage] EmptyDir volumes [It] pod should support
shared volumes between containers [Conformance]
When the test trying to read file in shared volume, it uses
"kubeclt exec namespace -c container_name -- cat file_name".
However, as variable framework.TestContext.KubectlPath not set,
kubectl binary can not be found in the test and the tast fails.
This patch move kubectlPath flag from RegisterClusterFlags to
RegisterCommonFlags, thus default value for
framework.TestContext.KubectlPath will be set,and
user can also use --kubectl-path flag to set kubectl path.
Signed-off-by: Howard Zhang <howard.zhang@arm.com>
Adds a new flag which allows users to specify a regexp
which will effectively whitelist certain taints and
allow the test framework to startup tests despite having
tainted nodes.
Fixes an issue where e2e tests were unable to be run
in tainted environments due to the framework waiting for
all the nodes to be schedulable without any tolerations.
HandleFlags() was used at e2e package and it depends on sub e2e
framework "config" in core e2e framework. That was invalid dependency.
So this moves HandleFlags() to e2e package for simple dependency.
Tests should never directly add to the global command line, because
some users of the tests might not want them there. For example,
options might only get set directly from a config file.
To achieve that, e2e/framework/config, e2e/framework/viperconfig, and
e2e/framework/test_context.go avoid using the global flag set and
instead expect to be told by the caller which flag set to use. Tests
that called flag directly either get updated or obsolete flags get
removed.
The exception is framework.HandleFlags, which as before directly
implements global command line handling.
This is a breaking change for test suites which do not use that
function (and only those): they now need to ensure that they copy
individual flags from tests. Because the RegisterCommonFlags prototype
has changed, test suite authors will notice due to the resulting
compilation errors.
