mirror of
https://github.com/rust-vmm/rust-vmm-ci.git
synced 2026-08-05 03:08:31 +00:00
readme update: cosmetic changes
Signed-off-by: Alexandra Iordache <aghecen@amazon.com>
This commit is contained in:
committed by
Alexandru Agache
parent
265df53b5c
commit
9dd386c2e6
82
README.md
82
README.md
@@ -23,29 +23,33 @@ git submodule add https://github.com/rust-vmm/rust-vmm-ci.git
|
||||
git commit -s -m "Added rust-vmm-ci as submodule"
|
||||
```
|
||||
|
||||
2. Create the coverage test configuration file named coverage_config_ARCH.json in
|
||||
the root of the repository, where ARCH is the architecture of the machine.
|
||||
1. Create the coverage test configuration file named
|
||||
`coverage_config_ARCH.json` in the root of the repository, where `ARCH` is the
|
||||
architecture of the machine.
|
||||
There are two coverage test configuration files, one per each platform.
|
||||
The example of the configuration file for x86_64 architecture can be found
|
||||
in [coverage_config_x86_64.json.sample](coverage_config_x86_64.json.sample),
|
||||
and the example of the configuration file for aarch64 architecture can be found
|
||||
in [coverage_config_aarch64.json.sample](coverage_config_aarch64.json.sample).
|
||||
The example of the configuration file for the `x86_64` architecture can be
|
||||
found in
|
||||
[coverage_config_x86_64.json.sample](coverage_config_x86_64.json.sample),
|
||||
and the example of the configuration file for the `aarch64` architecture can be
|
||||
found in
|
||||
[coverage_config_aarch64.json.sample](coverage_config_aarch64.json.sample).
|
||||
|
||||
The json must have the following fields:
|
||||
- *coverage_score*: The coverage of the repository.
|
||||
- *exclude_path*: This field is used for excluding files from the report. It
|
||||
should be used to exclude autogenerated files. Files in *exclude_path* are
|
||||
separated by one coma. If the repository does not have any autogenerated
|
||||
files, *exclude_path* should be an empty string.
|
||||
- *crate_features*: `cargo kcov` does not build crate features by default. To
|
||||
get the coverage report including optional features these need to be
|
||||
specified in *crate_features* separated by coma. If the crate does not have
|
||||
|
||||
- `coverage_score`: The coverage of the repository.
|
||||
- `exclude_path`: This field is used for excluding files from the report. It
|
||||
should be used to exclude autogenerated files. Files in `exclude_path` are
|
||||
separated by one comma. If the repository does not have any autogenerated
|
||||
files, `exclude_path` should be an empty string.
|
||||
- `crate_features`: `cargo kcov` does not build crate features by default. To
|
||||
get the coverage report including optional features, these need to be
|
||||
specified in `crate_features` separated by comma. If the crate does not have
|
||||
any features, this field should be empty.
|
||||
|
||||
This file is required for the coverage integration so it needs to be added
|
||||
to the repository as well.
|
||||
|
||||
3. Create a new pipeline definition in Buildkite. For this step ask one of the
|
||||
1. Create a new pipeline definition in Buildkite. For this step ask one of the
|
||||
rust-vmm Buildkite [admins](CODEOWNERS) to create one for you. Add a pipeline
|
||||
step that is uploading the rust-vmm-ci pipeline:
|
||||
|
||||
@@ -53,8 +57,11 @@ step that is uploading the rust-vmm-ci pipeline:
|
||||
buildkite-agent pipeline upload rust-vmm-ci/.buildkite/pipeline.yml
|
||||
```
|
||||
|
||||
4. The code owner of the repository will have to setup a WebHook for triggering
|
||||
the CI on Pull Requests and pushes.
|
||||
1. The code owners of the repository will have to setup a WebHook for
|
||||
triggering the CI on
|
||||
[pull request](https://developer.github.com/v3/activity/events/types/#pullrequestevent)
|
||||
and [push](https://developer.github.com/v3/activity/events/types/#pushevent)
|
||||
events.
|
||||
|
||||
## Buildkite Pipeline
|
||||
|
||||
@@ -62,7 +69,7 @@ The [Buildkite](https://buildkite.com) pipeline is the definition of tests to
|
||||
be run as part of the CI. It includes steps for running unit tests and linters
|
||||
(including coding style checks), and computing the coverage.
|
||||
|
||||
Currently the tests can run on Linux x86_64 and aarch64 hosts.
|
||||
Currently the tests can run on Linux `x86_64` and `aarch64` hosts.
|
||||
|
||||
Example of step that checks the build:
|
||||
|
||||
@@ -117,9 +124,9 @@ always-pull: true
|
||||
|
||||
The integration tests of some repositories have dependencies on external
|
||||
resources. One example is
|
||||
[linux-loader](https://github.com/rust-vmm/linux-loader/) which needs to
|
||||
[`linux-loader`](https://github.com/rust-vmm/linux-loader/) which needs to
|
||||
download a bzImage before running the unit tests. Because this is specific
|
||||
to the linux-loader crate, the logic for downloading the required resources
|
||||
to the `linux-loader` crate, the logic for downloading the required resources
|
||||
cannot be part of the common pipeline. The mechanism used here is
|
||||
[Repository Hooks](https://buildkite.com/docs/agent/v3/hooks#repository-hooks).
|
||||
The hooks are defined per repository and live in the crate repository under
|
||||
@@ -148,9 +155,9 @@ rm -r ${EXTRACT_PATH}
|
||||
rm -f ${DEB_PATH}
|
||||
```
|
||||
|
||||
In this example the post-checkout hook downloads a deb image, extracts the
|
||||
contents of it and places it in `linux-loader/src/bzimage`. The unit tests
|
||||
will use the relative path `src/bzimage` which does not depend on the image
|
||||
In this example the post-checkout hook downloads a deb image, extracts its
|
||||
contents and places it in `linux-loader/src/bzimage`. The unit tests will use
|
||||
the relative path `src/bzimage` which does not depend on the image
|
||||
being downloaded.
|
||||
|
||||
## Integration Tests
|
||||
@@ -162,17 +169,20 @@ complex tests defined in [integration_tests](integration_tests).
|
||||
### Test Profiles
|
||||
|
||||
The integration tests support two test profiles:
|
||||
|
||||
- **devel**: this is the recommended profile for running the integration tests
|
||||
on a local development machine.
|
||||
- **ci** (default option): this is the profile used when running the
|
||||
integration tests as part of the the Continuous Integration (CI).
|
||||
|
||||
The test profiles are applicable to tests that run using pytest. Currently only
|
||||
the [coverage test](tests/test_coverage.py) follows this model as all the other
|
||||
integration tests are run using the
|
||||
[Buildkite pipeline](https://buildkite.com/rust-vmm/kvm-ioctls-ci).
|
||||
The test profiles are applicable to
|
||||
[`pytest`](https://docs.pytest.org/en/latest/), the integration test framework
|
||||
used with rust-vmm-ci. Currently only the
|
||||
[coverage test](tests/test_coverage.py) follows this model as all the other
|
||||
integration tests are run using the Buildkite pipeline.
|
||||
|
||||
The difference between is declaring tests as passed or failed:
|
||||
|
||||
- with the **devel** profile the coverage test passes if the current coverage
|
||||
is equal or higher than the upstream coverage value. In case the current
|
||||
coverage is higher, the coverage file is updated to the new coverage value.
|
||||
@@ -188,12 +198,14 @@ The line coverage is saved in [tests/coverage](tests/coverage). To update the
|
||||
coverage before submitting a PR, run the coverage test:
|
||||
|
||||
```bash
|
||||
crate="kvm-ioctls"
|
||||
container_version=5
|
||||
docker run --device=/dev/kvm \
|
||||
-it \
|
||||
--security-opt seccomp=unconfined \
|
||||
--volume $(pwd)/${CRATE}:/${CRATE} \
|
||||
rustvmm/dev:v${LATEST}
|
||||
cd ${CRATE}
|
||||
--volume $(pwd)/${crate}:/${crate} \
|
||||
rustvmm/dev:v{$container_version}
|
||||
cd ${crate}
|
||||
pytest --profile=devel rust-vmm-ci/integration_tests/test_coverage.py
|
||||
```
|
||||
|
||||
@@ -241,6 +253,7 @@ The test requires [`criterion`](https://github.com/bheisler/criterion.rs)
|
||||
benchmarks to be exported by the crate. The test expects the entry point
|
||||
into the performance benchmarks to be named `main`. In other words, the
|
||||
following configuration is expected in `Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[[bench]]
|
||||
name = "main"
|
||||
@@ -248,11 +261,10 @@ name = "main"
|
||||
|
||||
All benchmarks need to be collected in a main.rs file placed in `benches/`.
|
||||
|
||||
|
||||
`criterion` collects performance
|
||||
results by running a function for a user-configured number of iterations,
|
||||
timing the runs, and applying statistics. The individual benchmark tests must
|
||||
be added in the crate. They can be run outside the CI with:
|
||||
`criterion` collects performance results by running a function for a
|
||||
user-configured number of iterations, timing the runs, and applying statistics.
|
||||
The individual benchmark tests must be added in the crate. They can be run
|
||||
outside the CI with:
|
||||
|
||||
```bash
|
||||
cargo bench [--all-features] OR [--features <features>]
|
||||
|
||||
Reference in New Issue
Block a user