Add script to run tests locally

The script uses the test_description.json file to get the test
names and the commands to be run.

Signed-off-by: Catalin Dumitru <catdum@amazon.com>
This commit is contained in:
Catalin Dumitru
2021-08-20 12:25:16 +03:00
committed by Andreea Florescu
parent cd3d97f923
commit 555474aaba
2 changed files with 68 additions and 0 deletions

View File

@@ -294,3 +294,32 @@ pytest rust-vmm-ci/integration_tests/test_benchmark.py -s
Note that performance is highly dependent on the underlying platform that the
tests are running on. The raw numbers obtained are likely to differ from their
counterparts on a CI instance.
### Running the tests locally
To run the integration tests locally, you can run the following from the crate you need to test.
You can find the latest container version in the
[script](.buildkite/autogenerate_pipeline.py)
that autogenerates the pipeline. For example:
```bash
cd ~/vm-superio
CRATE="vm-superio"
LATEST=12
docker run -it \
--security-opt seccomp=unconfined \
--volume $(pwd):/${CRATE} \
--volume ~/.ssh:/root/.ssh \
rustvmm/dev:v${LATEST}
cd vm-superio
./rust-vmm-ci/test_run.py
```
Known issues:
- When running the `cargo-audit` test, the following error may occur:
```
test_cargo-audit (__main__.TestsContainer) ... error: couldnt fetch advisory database: git operation failed: reference refs/heads/master not found; class=Reference (4); code=NotFound (-3)
```
A fix for this is to remove `~/.cargo/advisory-db` in the container, and then rerun `test_run.py`:
```
rm -rf ~/.cargo/advisory-db
./rust-vmm-ci/test_run.py
```

39
test_run.py Executable file
View File

@@ -0,0 +1,39 @@
#!/usr/bin/env python3
import json
import subprocess
import platform
import pathlib
import unittest
PARENT_DIR = pathlib.Path(__file__).parent.resolve()
class TestsContainer(unittest.TestCase):
pass
def make_test_function(command):
def test(self):
subprocess.run(command, shell=True, check=True)
return test
def retrieve_test_list(
config_file=f"{PARENT_DIR}/.buildkite/test_description.json"
):
with open(config_file) as jsonFile:
test_list = json.load(jsonFile)
jsonFile.close()
return test_list
if __name__ == '__main__':
test_config = retrieve_test_list()
for test in test_config['tests']:
command = test['command']
command = command.replace("{target_platform}", platform.machine())
test_func = make_test_function(command)
setattr(TestsContainer, 'test_{}'.format(test['test_name']), test_func)
unittest.main(verbosity=2)