Centralize test configuration (filters, retries, fail-fast,
sequential/parallel scheduling) into nextest profiles, replacing
scattered flags across shell scripts. This simplifies the scripts
and provides a single source of truth for test behavior.
Enable JUnit XML output per profile, giving CI systems structured
test results for better reporting.
Not all test invocations are converted to profiles yet. Just the
ones that are repeated across scripts.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Focal has served us well for many years but is now beyond EOL. Remove
all remaining use of focal images from the CI.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Pass `-p cloud-hypervisor` to all `cargo nextest run` invocations in
the integration test scripts so test discovery and execution are
scoped to the cloud-hypervisor package only, avoiding running tests
from other workspace crates.
This avoids flooding the output messages like below
`test result: ok. 0 passed; 0 failed; 0 ignored;
0 measured; 0 filtered out; finished in 0.00s
Running unittests src/lib.rs`
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Skip sha1sum verification for firmware files that were
provided via CH_CUSTOM_FIRMWARE or CH_CUSTOM_OVMF environment
variables. Custom files will not match the expected checksums,
so we filter them out of the sha1sums list before running
sha1sum --check.
Updated scripts:
- run_integration_tests_x86_64.sh
- run_integration_tests_aarch64.sh
Assisted-by: GitHub Copilot:Claude-Opus-4.6
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Reduce `nr_hugepages` from 12 GB to 6 GB on both architectures. The
number if huge pages needed (if all the tests run at once) is 4GiB so
this gives 50% headroom.
This should reduce the number of tests that fail/flake out due to lack
of memory.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Move the live migration tests themselves into the common scopes allowing
the tests to now run interleaved together hopefully reducing CI time.
On MSHV the live migration tests are now not compiled in rather than
compiled in and skipped (as the helpers are not compiled in for MSHV.)
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Move the live migration test running from their own script into the
x86-64 script (on aarch64 they were already in the same script.) They
were historically separate as they were new. Now they are established it
makes sense for them to be combined.
The timeout in the GitHub workflow has been extended to accommodate the
extra work in the same step.
The Rust test scopes are unchanged - the running of the tests has been
moved.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Increase the number of parallel integration tests in CI to save ~3-5
minutes per x86_64 run. The thread limit is driven by RAM and disk
space constraints, not CPU availability.
A new `PARALLEL_INTEGRATION_TESTS_NUM` environment variable controls
the thread count. In CI it is set explicitly (12 for x86_64, 25 for
ARM64); locally it falls back to `nproc / 4`, preserving the previous
behavior.
Only the first test group (`common_parallel`, `live_migration_parallel`)
uses the overridden value - subsequent groups (dbus_api, fw_cfg,
ivshmem, aarch64_acpi) continue to use the `nproc / 4` default.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Rename download_ovmf to download_amd64_ovmf and add a separate
download_aarch64_ovmf for CLOUDHV_EFI.fd
Replace build_edk2 with download_aarch64_ovmf in aarch64 scripts
Update OVMF firmware tag to ch-1e1b96f126
Update CLOUDHV.fd sha1sum in sha1sums-x86_64
Add CLOUDHV_EFI.fd sha1sum in sha1sums-aarch64-common
Fixes: #7622
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Switch to a git hash of virtiofsd as unfortunately the last release is
some time ago and does not incorporate a dependency bump that enables it
to work with snapshot-restore.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This patch adds the skeleton of the CVM test
support and modify existing scripts and test framework
to enable such scenario. Split the sha1sum to support both
regular and CVM guest. Add one test case for CVM. Will further
add more test cases.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
X64_64 image download steps is being used for both
regular and CVM guest. Keeping the steps withing a function
in the test-util.sh
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Testing generic vhost-user devices will require virtiofsd to support the
--tag option, which v1.8.0 does not support.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
It is a common use case to run a subset of tests locally to verify
certain functionalities.
The default behaviour for nextest is to error out if no tests are run.
That causes the test scripts to return a non-zero value (failure). Pass
`--no-tests=pass` to nextest to match what `cargo test` does if no tests
are run.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Add tests:
- zlib: test_virtio_block_qcow2_zlib()
- zstd: test_virtio_block_qcow2_zstd()
Both these tests use zlib- and zstd-compressed images as OS image.
Modify test_virtio_block_qcow2_backing_file() test: it is practical
to test qcow2 file-backing with compression, so use zlib-compressed
image as a backing file.
Signed-off-by: Eugene Korenevsky <ekorenevsky@aliyun.com>
TL;DR: Massive quality of life improvement for devs
Cloud Hypervisor uses the Cargo test framework for multiple tests:
- normal unit tests
- unit tests requiring special environment (the Tap device tests)
- integration tests requiring a special environment
This prevented the execution of `cargo test --workspace`, which results
in a very poor developer experience. Although
`./scripts/run_unit_tests.sh` exists, there are valid reasons why devs
cannot or even don't want to use it.
By adding a new `chv_testenv` rustc config, we can conditionally only
activate tests when the `./scripts/` magic runs them. This improves
the general developer experience by a lot.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Due to the fw_cfg test being disabled on MSHV this results in no tests
being runnable which results in an error in nextest. Reduce that error
to a warning use --no-tests=warn
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The ivshmem tests are all failing in the CI for MSHV because Cloud
Hypervisor is built without the mshv feature.
Error: Cloud Hypervisor exited with the following chain of errors:
0: Failed to open hypervisor interface (is hypervisor interface
available?)
1: Failed to create the hypervisor
2: no supported hypervisor
Modify the build command to include the mshv feature.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
test_fw_cfg is frequently failing in the CI for MSHV. Exclude it for
now. It needs further investigation. See issue #7434 for details.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Only use 75% of the available threads - this will reduce dislk and
memory pressure. Reducing the chance of flaky tests.
See: #7405
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Running one or two tests in a tight loop can cause the download
functions to quickly hit GitHub's API rate limit. That causes the test
script to fail for no apparent reason.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
`prepare_linux` is capable of determining whether we need to invoke
`build_custom_linux` for building linux from source or `download_linux`
for downloading pre-built.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add release and target params to `cargo test` since we collect
the code coverage reports from `xx/$BUILD_TARGET/release/`.
Signed-off-by: Songqian Li <sionli@tencent.com>
The Linux kernel fork repository for Cloud Hypervisor now produces
prebuilt x86-64 and aarch64 binaries. Speed up the CI by using those
binaries.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Move the duplicated logic to download OVMF into a helper function.
Explicitly specify the OVMF_FW_TAG instead of downloading the latest
so that new OVMF versions can be easily tested.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
The following tests have been temporarily disabled:
1. Live upgrade/migration test with ovs-dpdk (#5532);
2. Disk hotplug tests on windows guests (#6037);
This patch has been tested with PR #6048.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Tested-by: Bo Chen <chen.bo@intel.com>
There is no need to reconstruct it from within the scripts since the
main script already constructed it once.
Drop the previously useless setting of BUILD_TARGET from various
scripts. The value was always overwritten at a later point.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Now feature "mshv" can be built together with "kvm". There is no need to
use "--no-default-features" any more.
Fixes: #5647
Signed-off-by: Bo Chen <chen.bo@intel.com>
The VFIO tests based on the NVMe emulation framework require cause a
high number of file descriptors to be opened on the system. On systems
with a low limit on opened files, tests like test_vfio_user can possibly
fail with "too many open files" exeption. This issue is fixed by raising
the corresponding limit.
A tricky detail here is, that the limit has to be changed in the test
container and not on the host. As the the container is executed in the
privileged mode, the setting in it will override the host value.
Related: #5426
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This test case creates a new qcow2 file using the image of ubuntu as
its backing file, and boot a virtual machine with this image file.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
In this way, our integration tests exercise the same set of build
features (e.g. "kvm,mshv") being used for releases.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Implemented a `TargetApi` enum to make the process of implementing
tests for the D-Bus and HTTP API more convenient.
Refactored `test_api_{create_boot, shutdown, pause_resume, delete}` tests
with the `TargetApi` enum to also implement tests for the D-Bus API.
Added a new test named `test_api_dbus_and_http_interleaved` that uses
both the HTTP and D-Bus API at the same time.
Modified integration test scripts to enable the `dbus_api` feature when
compiling and start a dbus-session when integration tests are run.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
The tests require a fixed amount of huge page memory to run, rather than
encode that as fixed number of huge pages it should be calculated from
the size of memory required and the huge page size
Signed-off-by: Dom <peng6662001@163.com>
