Tests should never directly add to the global command line, because
some users of the tests might not want them there. For example,
options might only get set directly from a config file.
To achieve that, e2e/framework/config, e2e/framework/viperconfig, and
e2e/framework/test_context.go avoid using the global flag set and
instead expect to be told by the caller which flag set to use. Tests
that called flag directly either get updated or obsolete flags get
removed.
The exception is framework.HandleFlags, which as before directly
implements global command line handling.
This is a breaking change for test suites which do not use that
function (and only those): they now need to ensure that they copy
individual flags from tests. Because the RegisterCommonFlags prototype
has changed, test suite authors will notice due to the resulting
compilation errors.
Need to add bracket in the tag for sig-windows. Also fix an issue: for
current testing structure, it first init driver and then set up the
framework. So when initialize the driver, it does not know what OS is
and we can not set up the capabilities correctly. Instead we have to add
all the capabilities and supported fs types including both linux and
windows. Later in the code, we will check the Node OS and decide how to
run the test.
CreateDriver (now called SetupTest) is a potentially expensive
operation, depending on the driver. Creating and tearing down a
framework instance also takes time (measured at 6 seconds on a fast
machine) and produces quite a bit of log output.
Both can be avoided for tests that skip based on static
information (like for instance the current OS, vendor, driver and test
pattern) by making the test suite responsible for creating framework
and driver.
The lifecycle of the TestConfig instance was confusing because it was
stored inside the DriverInfo, a struct which conceptually is static,
while the TestConfig is dynamic. It is cleaner to separate the two,
even if that means that an additional pointer must be passed into some
functions. Now CreateDriver is responsible for initializing the
PerTestConfig that is to be used by the test.
To make this approach simpler to implement (= less functions which
need the pointer) and the tests easier to read, the entire setup and
test definition is now contained in a single function. This is how it
is normally done in Ginkgo. This is easier to read because one can see
at a glance where variables are set, instead of having to trace values
though two additional structs (TestResource and TestInput).
Because we are changing the API already, also other changes are made:
- some function prototypes get simplified
- the naming of functions is changed to match their purpose
(tests aren't executed by the test suite, they only get defined
for later execution)
- unused methods get removed (TestSuite.skipUnsupportedTest is redundant)
Exposing framework.VolumeTestConfig as part of the testsuite package
API was confusing because it was unclear which of the values in it
really have an effect. How it was set also was a bit awkward: a test
driver had a copy that had to be overwritten at test runtime and then
might have been updated and/or overwritten again by the driver.
Now testsuites has its own test config structure. It contains the
values that might have to be set dynamically at runtime. Instead of
overwriting a copy of that struct inside the test driver, the test
driver takes some common defaults (specifically, the framework pointer
and the prefix) when it gets initialized and then manages its own
copy. For example, the hostpath driver has to lock the pods to a
single node.
framework.VolumeTestConfig is still used internally and test drivers
can decide to run tests with a fully populated instance if needed (for
example, after setting up an NFS server).
This makes it possible to use the testsuites package out-of-tree
without pulling in unnecessary dependencies and code (in
test/e2e/storage/vsphere) that defines tests that are not wanted in a
custom test suite.
Different drivers support different volume sizes. Some have certain
minimum sizes, some maximum sizes. Instead of hard-coding some kind of
default into the testsuites, now each driver that supports dynamic
provisioning has to provide the size.
The setup of the V0 hostpath driver was done with copy-and-paste and
then changing just the driver name and the manifests. The same can be
achieved by making the base struct a bit more configurable, which
simplifies future changes (less code).
Renaming the provisioner container was unnecessary and was reverted to
make it possible to use the same patch configuration.
While at it, also fix the InitHostV0PathCSIDriver typo.
