Instead of having a static pipeline.yml, autogenerate it from a JSON
configuration file. The json approach is taken because we also want to
be able to run tests locally, and thus the json can be parsed once for
generating the pipeline, and once for generating a local test run.
Signed-off-by: Catalin Dumitru <catdum@amazon.com>
In the other crates, the tests reside in rust-vmm-ci/intergation_tests.
When we test the CI itself, the tests reside in integration_tests.
Therefore we needed to make the command find the tests independently of
the crate it is run from.
Signed-off-by: Catalin Dumitru <catdum@amazon.com>
In order to be able to run the CI on rust-vmm-ci we needed to
make it a rust crate. Also the path to the coverage config
file needed to be updated in test_coverage.py so that the path
does not depend on the crate; a breadth-first search was used
to guarantee that the config file belongs to the crate that is
being tested.
Signed-off-by: Catalin Dumitru <catdum@amazon.com>
Set default values for the remote name and
base branch when the test is run locally and
the Buildkite variables are not available and
the variables BASE_BRANCH and REMOTE are not set.
Signed-off-by: Catalin Dumitru <catdum@amazon.com>
This version is needed because there are features (for example passing
devices to a container) that are not available in the old version.
Signed-off-by: Andreea Florescu <fandree@amazon.com>
Although we check commit authors in test_commit_format.py to skip tests
for dependabot's commits, the current code only checks the author of the
head commit.
This PR make it check each commit's author so that we can test branches
where both a dependabot's commit and other commits exist.
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
When dependabot updates the rust-vmm-ci submodule, the git commit
sumamry is over 50 chars. To be able to merge PR, exclude the dependabot
user from this checks.
Fixes: https://github.com/rust-vmm/rust-vmm-ci/issues/61
Signed-off-by: Andreea Florescu <fandree@amazon.com>
This is needed for running the integration tests in
vmm-reference. They need access to resources from tmp.
Signed-off-by: Laura Loghin <lauralg@amazon.com>
Until we find a safe way to run the commit format test for
repositories specified with git, we should skip that test
for those repos.
Added a conditional for the corresponding buildkite step
that will run the commit test only if BUILDKITE_REPO
doesn't start with `git@`.
Signed-off-by: Laura Loghin <lauralg@amazon.com>
This commit adds a pytest command line option `--no-cleanup` for
the coverage test. Running the coverage test with this option will
let users save the `kcov_output` directory so that it is easier to
inspect the coverage report.
Signed-off-by: Henry Wang <henry.wang@arm.com>
Commit b3acb307f add the cargo-audit label but indentation is wrong,
making it an invalid YAML, at least from buildkite's point of view.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Without this flag, it looks like not all workspace files
are checked. You can test this by adding for example an
unused import (`use std::clone;`) in
https://github.com/rust-vmm/vmm-reference/pull/11,
file src/vmm/tests/integration_tests.rs. cargo check will
fail only if we pass `--workspace` flag too.
For clippy, this flag doesn't seem to have an obvious
effect, this should be further investigated in #21.
Signed-off-by: Laura Loghin <lauralg@amazon.com>
Added `--workspace` flag to `cargo test` commands
because it is needed for workspace crates (an example
would be the vmm-reference).
The coverage test is already run on all tests for
workspaces too.
Fixes: #38.
Signed-off-by: Laura Loghin <lauralg@amazon.com>
Pytest closes stdin by default, so any processes
spawned from it won't have it. As some tests in
various rust-vmm crates might need stdin to be
open, we mock it with a (currently empty) byte
stream, which we can extend in the future.
Signed-off-by: Alexandra Iordache <aghecen@amazon.com>
- Added defines for PR branch, result files
- Updated variables & function names to be more explicit about what
they do
Signed-off-by: Andreea Florescu <fandree@amazon.com>
When there are no benchmarks on the master branch, instead of
crashing first check if the error. Pass the test in case
cargo bench failed due to main bench not being present.
Fixes: https://github.com/rust-vmm/rust-vmm-ci/issues/30
Signed-off-by: Andreea Florescu <fandree@amazon.com>
The test that was checking the commit message format
was not comparing the PR's HEAD to the right master,
even though `origin` was set to the pipeline repository.
Fixed this by fetching that repo. This way, `FETCH_HEAD`
points to the latest upstream master.
Signed-off-by: Laura Loghin <lauralg@amazon.com>
Added test that checks if commits follow the 50/72 git
commit rule and if they are signed (git commit -s).
Fixes: #9, #14.
Signed-off-by: Laura Loghin <lauralg@amazon.com>
This version adds git, critcmp, cargo-audit and
cargo-license.
Also, the Rust version is updated to 1.44.1.
Signed-off-by: Laura Loghin <lauralg@amazon.com>
Warnings should be checked by custom pipelines. The common
pipeline should check warnings on all features instead.
Signed-off-by: Andreea Florescu <fandree@amazon.com>
1. Enabled the aarch64 coverage test in `test_coverage` script.
2. Seperated the coverage json sample file to two sample files for
different machine achitectures.
3. Updated Buildkite pipeline container version.
4. Updated README for the description of coverage files and container
version.
Signed-off-by: Henry Wang <henry.wang@arm.com>
Because aarch64 is a tier-2 Rust architecture, clippy is not always
available with the latest release. Keeping the clippy tests on arrch64
prevent us from freely moving to newer Rust releases.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
If we clone rust-vmm-ci with SSH Buildkite requires an ssh key
when cloning the repository for some reason.
Signed-off-by: Andreea Florescu <fandree@amazon.com>
The container now runs Rust 1.35. This is needed as versions 1.34.0
up to 1.34.2 (which is the first one to include a fix) have a security
vulnerability.
For details check the CVE: https://www.cvedetails.com/cve/CVE-2019-12083/
Signed-off-by: Andreea Florescu <fandree@amazon.com>
This pipeline contains the tests that are currently run for
kvm-ioctls. The pipeline also has a step that runs the coverage
test defined in integration_tests/test_coverage.py.
The purpose of this pipeline is to be used by all rust-vmm
repositories to ensure we keep the same quality bar for all crates.
Signed-off-by: Andreea Florescu <fandree@amazon.com>
