readme update: cosmetic changes

Signed-off-by: Alexandra Iordache <aghecen@amazon.com>
This commit is contained in:
Alexandra Iordache
2020-03-04 19:22:32 +02:00
committed by Alexandru Agache
parent 265df53b5c
commit 9dd386c2e6

View File

@@ -23,29 +23,33 @@ git submodule add https://github.com/rust-vmm/rust-vmm-ci.git
git commit -s -m "Added rust-vmm-ci as submodule"
```
2. Create the coverage test configuration file named coverage_config_ARCH.json in
the root of the repository, where ARCH is the architecture of the machine.
1. Create the coverage test configuration file named
`coverage_config_ARCH.json` in the root of the repository, where `ARCH` is the
architecture of the machine.
There are two coverage test configuration files, one per each platform.
The example of the configuration file for x86_64 architecture can be found
in [coverage_config_x86_64.json.sample](coverage_config_x86_64.json.sample),
and the example of the configuration file for aarch64 architecture can be found
in [coverage_config_aarch64.json.sample](coverage_config_aarch64.json.sample).
The example of the configuration file for the `x86_64` architecture can be
found in
[coverage_config_x86_64.json.sample](coverage_config_x86_64.json.sample),
and the example of the configuration file for the `aarch64` architecture can be
found in
[coverage_config_aarch64.json.sample](coverage_config_aarch64.json.sample).
The json must have the following fields:
- *coverage_score*: The coverage of the repository.
- *exclude_path*: This field is used for excluding files from the report. It
should be used to exclude autogenerated files. Files in *exclude_path* are
separated by one coma. If the repository does not have any autogenerated
files, *exclude_path* should be an empty string.
- *crate_features*: `cargo kcov` does not build crate features by default. To
get the coverage report including optional features these need to be
specified in *crate_features* separated by coma. If the crate does not have
- `coverage_score`: The coverage of the repository.
- `exclude_path`: This field is used for excluding files from the report. It
should be used to exclude autogenerated files. Files in `exclude_path` are
separated by one comma. If the repository does not have any autogenerated
files, `exclude_path` should be an empty string.
- `crate_features`: `cargo kcov` does not build crate features by default. To
get the coverage report including optional features, these need to be
specified in `crate_features` separated by comma. If the crate does not have
any features, this field should be empty.
This file is required for the coverage integration so it needs to be added
to the repository as well.
3. Create a new pipeline definition in Buildkite. For this step ask one of the
1. Create a new pipeline definition in Buildkite. For this step ask one of the
rust-vmm Buildkite [admins](CODEOWNERS) to create one for you. Add a pipeline
step that is uploading the rust-vmm-ci pipeline:
@@ -53,8 +57,11 @@ step that is uploading the rust-vmm-ci pipeline:
buildkite-agent pipeline upload rust-vmm-ci/.buildkite/pipeline.yml
```
4. The code owner of the repository will have to setup a WebHook for triggering
the CI on Pull Requests and pushes.
1. The code owners of the repository will have to setup a WebHook for
triggering the CI on
[pull request](https://developer.github.com/v3/activity/events/types/#pullrequestevent)
and [push](https://developer.github.com/v3/activity/events/types/#pushevent)
events.
## Buildkite Pipeline
@@ -62,7 +69,7 @@ The [Buildkite](https://buildkite.com) pipeline is the definition of tests to
be run as part of the CI. It includes steps for running unit tests and linters
(including coding style checks), and computing the coverage.
Currently the tests can run on Linux x86_64 and aarch64 hosts.
Currently the tests can run on Linux `x86_64` and `aarch64` hosts.
Example of step that checks the build:
@@ -117,9 +124,9 @@ always-pull: true
The integration tests of some repositories have dependencies on external
resources. One example is
[linux-loader](https://github.com/rust-vmm/linux-loader/) which needs to
[`linux-loader`](https://github.com/rust-vmm/linux-loader/) which needs to
download a bzImage before running the unit tests. Because this is specific
to the linux-loader crate, the logic for downloading the required resources
to the `linux-loader` crate, the logic for downloading the required resources
cannot be part of the common pipeline. The mechanism used here is
[Repository Hooks](https://buildkite.com/docs/agent/v3/hooks#repository-hooks).
The hooks are defined per repository and live in the crate repository under
@@ -148,9 +155,9 @@ rm -r ${EXTRACT_PATH}
rm -f ${DEB_PATH}
```
In this example the post-checkout hook downloads a deb image, extracts the
contents of it and places it in `linux-loader/src/bzimage`. The unit tests
will use the relative path `src/bzimage` which does not depend on the image
In this example the post-checkout hook downloads a deb image, extracts its
contents and places it in `linux-loader/src/bzimage`. The unit tests will use
the relative path `src/bzimage` which does not depend on the image
being downloaded.
## Integration Tests
@@ -162,17 +169,20 @@ complex tests defined in [integration_tests](integration_tests).
### Test Profiles
The integration tests support two test profiles:
- **devel**: this is the recommended profile for running the integration tests
on a local development machine.
- **ci** (default option): this is the profile used when running the
integration tests as part of the the Continuous Integration (CI).
The test profiles are applicable to tests that run using pytest. Currently only
the [coverage test](tests/test_coverage.py) follows this model as all the other
integration tests are run using the
[Buildkite pipeline](https://buildkite.com/rust-vmm/kvm-ioctls-ci).
The test profiles are applicable to
[`pytest`](https://docs.pytest.org/en/latest/), the integration test framework
used with rust-vmm-ci. Currently only the
[coverage test](tests/test_coverage.py) follows this model as all the other
integration tests are run using the Buildkite pipeline.
The difference between is declaring tests as passed or failed:
- with the **devel** profile the coverage test passes if the current coverage
is equal or higher than the upstream coverage value. In case the current
coverage is higher, the coverage file is updated to the new coverage value.
@@ -188,12 +198,14 @@ The line coverage is saved in [tests/coverage](tests/coverage). To update the
coverage before submitting a PR, run the coverage test:
```bash
crate="kvm-ioctls"
container_version=5
docker run --device=/dev/kvm \
-it \
--security-opt seccomp=unconfined \
--volume $(pwd)/${CRATE}:/${CRATE} \
rustvmm/dev:v${LATEST}
cd ${CRATE}
--volume $(pwd)/${crate}:/${crate} \
rustvmm/dev:v{$container_version}
cd ${crate}
pytest --profile=devel rust-vmm-ci/integration_tests/test_coverage.py
```
@@ -241,6 +253,7 @@ The test requires [`criterion`](https://github.com/bheisler/criterion.rs)
benchmarks to be exported by the crate. The test expects the entry point
into the performance benchmarks to be named `main`. In other words, the
following configuration is expected in `Cargo.toml`:
```toml
[[bench]]
name = "main"
@@ -248,11 +261,10 @@ name = "main"
All benchmarks need to be collected in a main.rs file placed in `benches/`.
`criterion` collects performance
results by running a function for a user-configured number of iterations,
timing the runs, and applying statistics. The individual benchmark tests must
be added in the crate. They can be run outside the CI with:
`criterion` collects performance results by running a function for a
user-configured number of iterations, timing the runs, and applying statistics.
The individual benchmark tests must be added in the crate. They can be run
outside the CI with:
```bash
cargo bench [--all-features] OR [--features <features>]