We now check whether the test defines if_changed and crate_path before
using the ones from the workspace. If the tests does define these two,
then we don't overwrite them with the default workspace configuration.
This is needed so that custom pipelines for tests can specify as if
changed the crate that they're intended to. Without this change we
end up running the custom tests for all crates that are changed instead
of running it just for the crate that they're intended.
Signed-off-by: Andreea Florescu <andreea.florescu15@gmail.com>
Add a README section spelling out the three goals: per-crate
supported platforms, custom tests defined per crate, and a minimal
set of workspace-scoped tests for cross-crate compatibility. Also
describe how `--workspace-selective` uses Buildkite's native
`if_changed` property to realize them.
Signed-off-by: Ruoqing He <ruoqing.he@lingcage.com>
Running the full pipeline for every change in a Cargo workspace
retests crates a pull request never touched. Add a
`--workspace-selective` parameter that, when set to `True`, emits one
step per workspace member tagged with Buildkite's native `if_changed`
property, so each crate's shared crate-scoped tests only run when
files in that crate (or in a workspace-global path) change. Workspace
members are discovered by parsing the workspace's `Cargo.toml` directly
so the script does not need `cargo` on the host. The shared test
description is supplied via `-t`; a crate may extend it with its own
`.buildkite/test_description.json`, whose tests run only when files in
that crate change.
Signed-off-by: Ruoqing He <ruoqing.he@lingcage.com>
Dependabot is reporting several vulnerability alerts related to
`codeowners/requirements.txt`, let's update them.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
New `black` is complaining about reformatting some of our files,
let's make it happy:
$ black . --check
would reformat rust-vmm-ci/test_run.py
would reformat rust-vmm-ci/integration_tests/conftest.py
would reformat rust-vmm-ci/.buildkite/autogenerate_pipeline.py
Oh no! 💥💔💥
3 files would be reformatted, 5 files would be left unchanged.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
cargo-audit 0.22 is the first version that can parse cvss 4.0 scores
and avoid the following error in our pipelines:
$ cargo audit -q --deny warnings
error: error loading advisory database: parse error: TOML parse error at line 8, column 8
|
8 | cvss = "CVSS:4.0/AV:N/AC:L/AT:P/PR:L/UI:N/VC:L/VI:L/VA:L/SC:N/SI:N/SA:N"
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
unsupported CVSS version: 4.0
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Make repository_setup.sh automatically discover whether we have a multi
crate cargo workspace, or just a single crate repository, and then
generate a publish github action for each of them that triggers when a
tag named '$crate_name-v*' (or just 'v*' for single-crate repos) is
published.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Add a github action sample workflow file for auto-publishing crates on
creation of 'v*' tags. The sample workflow assumes a non-workspace
setup, so that cargo publish can be run from the repository root (=
crate root).
For multi-crate repositories, repository_setup.sh will generate
derivatives of this file.
There is a useless `cd .` in the `run` step which only exists so that
sed can easily replace it with the path for crates in a workspace when
auto-generating workflows.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Replace the manual instructions for setting up dependabot and .platform
file with instructions to run the new script. Move the explanation of
the .platform file to the testing section (where we were incorrectly
claiming that we only support x86_64 and aarch64, so fix that up too).
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Add a repository_setup.sh script that automates a couple of setup steps
for setup of new rust-vmm repositories. It also supports updating the
configuration of already created repositories by comparing the
repository's configuration to the defaults in rust-vmm-ci and giving the
option to update.
Currently, the script supports dependabot setup and .platform file
generation. It does not deal with the coverage files, since I am still
hoping to switch us over to codecov.io in the future, at which point
that will all changes completely anyway.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
The environment variable that we pass to indicate the a docs build is
happening was misspelling "RUST" as "RUTS".
Signed-off-by: Patrick Roy <patrick.roy@linux.dev>
Update container version to g6300e70 which has SBI_V01 enabled during
kernel compilation for according RISC-V test cases.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Some crates, such as vhost, use compile_error!() to indicate that
mutually exclusive feature have been specified. This leads to problems
when running cargo doc --all-features, which will inevitably trigger
these conditions and then lead to documentation not building (for this
reason, the vhost documentation currently fails to build on docs.rs for
example).
Add a custom cfg that gets passed via RUSTFLAGS in the cargo doc CI step
on whose absense these compile_error!() macros can be gated, so that
they dont cause issues when building documentation in CI
A custom cfg is needed because cargo doc does not pass `--cfg doc` down
to dependencies.
See also rust-vmm/vhost#326
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Due to us running riscv tests in an emulated environment, compiling
multiple permutations of crates is simply too slow and causes buildkite
timeouts. Revert to using --all-features here.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Turns out any cargo subcommand can be ran like this, instead of just the
ones documented on the cargo-all-features crates.io page.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Ensure that every permutation of feature flags is both build-tested, and
has their unit-tests ran.
The unit-testing part can be reverted to `--all-features` at some point,
once we have made sure that all our cargo features are actually
additive.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
We already run `cargo build` with `RUSTFlAGS=-Dwarnings`, so also
running `cargo check` with the same arguments, on the same
architectures, and with the same rustflags gains us nothing.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
This ensures tests, binaries, benchmarks, examples and libraries are all
build-tested, instead of just the default targets.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
There seems to be a bug in dependabot where sometimes it refuses to open
PRs due to hitting the pull request limit despite no PRs being open. The
rust-vmm/kvm repository ran into this. To avoid this in the future, drop
the open-pull-requests-limit key, which fixed the issue for
rust-vmm/kvm. In practice, due to us only having a single submodule and
due to cargo updates being grouped, we can only get at most one PR per
ecosystem at a time anyway.
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Clean up and standardize our examples by always mounting the crate in
/crate and using --workdir to avoid `cd ..` in the container.
While we're here, let's remove a strange mount of ~/.ssh in the last
example.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
We recently changed the container tags from vNN to gSHA-1, but we didn't
update README.md. Also, we now push `latest`, which is always updated,
so use this tag in the examples and mention to look at CONTAINER_VERSION
in .buildkite/autogenerate_pipeline.py for a specific version.
Reported-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Mention that `crate_features` is now optional, and also update a very
outdated reference to `cargo kcov`
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Currently, if a crate wishes to run the coverage test with some cargo
features enabled, it must explicitly list them out in the
`crate_features` key of the coverage json files.
Change this so that if this key is omitted from the json file, then
test_coverage.py passes `--all-features` to cargo.
No functional change, as currently `crate_features` is a mandatory key,
so to opt-in to this new behavior, crates must remove this key from
their coverage json file(s).
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
We are missing `rsync` for `seccompiler`. And after bumping to
ubuntu:24.04, some dependencies required by `vhost-devices` could be
installed from ubuntu sources, so @stefano-garzarella reworked the
preparation of `vhost-devices` dependencies in
rust-vmm/rust-vmm-container#128.
Update to g82bd1a3 dev image to apply the changes.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`libc-bin` is broken in previous images which causes problem in
rust-vmm/vhost-device#820.
We ended up upgrading to ubuntu 24:04 to address this.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Rust does optimizations which can cause havoc
in `--release` mode, so add another set of unit tests
with all optimizations enabled.
Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
It seems that the raw output of llvm-cov changed at some point, and we
ended upcomparing functoin coverage instead of line coverage. Fix this
by instead using the new json output, which will hopefully prevent such
goofs in the future.
Fixes#170
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Before checking out different commits, clean the working directory to
avoid checkout failures to to temporary files created during benchmark
execution (as is done by linux-loader)
Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
Version v47 [1] incorporates:
- chore: Move RUST_TOOLCHAIN into build_container.sh
- chore: Refactor dialing to reduce useless hints
- riscv64: Set MIN_CORES to 4
[1] rust-vmm/rust-vmm-container#118
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
After `.platform` mechanism was introduced in #159, it silently filters
out tests with no `platform` specified. Enable them to execute by adding
`platform is not None` predicate preceeds allowlist check.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
