Reduce `nr_hugepages` from 12 GB to 6 GB on both architectures. The
number if huge pages needed (if all the tests run at once) is 4GiB so
this gives 50% headroom.
This should reduce the number of tests that fail/flake out due to lack
of memory.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Move the live migration tests themselves into the common scopes allowing
the tests to now run interleaved together hopefully reducing CI time.
On MSHV the live migration tests are now not compiled in rather than
compiled in and skipped (as the helpers are not compiled in for MSHV.)
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Move the live migration test running from their own script into the
x86-64 script (on aarch64 they were already in the same script.) They
were historically separate as they were new. Now they are established it
makes sense for them to be combined.
The timeout in the GitHub workflow has been extended to accommodate the
extra work in the same step.
The Rust test scopes are unchanged - the running of the tests has been
moved.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
The aarch64 integration script hardcoded `LAST_RELEASE_VERSION="v39.0"`
for the live-upgrade binary download, while the live-migration runner
already accepts a `MIGRATABLE_VERSION` env override with a `vxx.0`
regex check. Standardise the aarch64 script on the same env-override
block so both arches honour the same knob with the same validation.
Default is unchanged (v39.0).
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Increase the number of parallel integration tests in CI to save ~3-5
minutes per x86_64 run. The thread limit is driven by RAM and disk
space constraints, not CPU availability.
A new `PARALLEL_INTEGRATION_TESTS_NUM` environment variable controls
the thread count. In CI it is set explicitly (12 for x86_64, 25 for
ARM64); locally it falls back to `nproc / 4`, preserving the previous
behavior.
Only the first test group (`common_parallel`, `live_migration_parallel`)
uses the overridden value - subsequent groups (dbus_api, fw_cfg,
ivshmem, aarch64_acpi) continue to use the `nproc / 4` default.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
During development, it is common to cancel a running test and
rerun it after making changes. However, pressing Ctrl+C while
dev_cli.sh runs long-running container commands (wget, qemu-img,
cargo build, etc.) does not reliably terminate the process.
Bash defers signal handling while a foreground process is
running, so the trap only fires after the docker run command
returns. This makes it difficult to cancel and restart quickly.
Fix this by introducing a run_container() wrapper that runs
docker in the background and uses 'wait', which is immediately
interruptible by signals. A cleanup() trap handler is set for
SIGINT and SIGTERM that kills the named container, the tracked
background PID, and any remaining child processes.
Each docker run invocation is assigned a unique --name based
on the script PID (clh-dev-$$) to allow targeted cleanup.
The interactive shell (cmd_shell) is left unwrapped since it
needs foreground terminal I/O.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The unit test container runs with Docker default seccomp
profile which blocks io_uring_setup, io_uring_enter and
io_uring_register. This causes all qcow_async unit tests to
fail with EPERM when creating an io_uring instance.
Add --security-opt seccomp=unconfined to the unit test docker
run invocation. The container already has --device access and
cap_net_admin, so this does not materially change the security
posture.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The updated image is configured in a same way as the
previously used 2022.
SAC, SSH, and RDP are configured.
All Windows updates to the curent date are installed.
Includes latest stable virtio-win 0.1.285 drivers.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Rename download_ovmf to download_amd64_ovmf and add a separate
download_aarch64_ovmf for CLOUDHV_EFI.fd
Replace build_edk2 with download_aarch64_ovmf in aarch64 scripts
Update OVMF firmware tag to ch-1e1b96f126
Update CLOUDHV.fd sha1sum in sha1sums-x86_64
Add CLOUDHV_EFI.fd sha1sum in sha1sums-aarch64-common
Fixes: #7622
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Add an `iommufd` flag to existing VFIO integration tests. When false,
tests use the legacy vfio container/group backend (existing behavior).
When true, tests use vfio cdev with iommufd and vfio_p2p_dma=off.
vfio_p2p_dma=off is required because the VFIO test runner uses a stock
Ubuntu 24.04 kernel (v6.8) which does not support mapping device MMIO
pages (VM_PFNMAP) through iommufd, causing IOMMU_IOAS_MAP to fail with
-EFAULT on MMIO BAR regions.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The aarch64 dbus, fw_cfg & ivshmem tests don't build the mshv feature
causing them to fail when run on MSHV. Fix by building the mshv feature
too just like the x86 version of the script does.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Use IFS-based splitting instead of parameter expansion to
correctly separate '#'-delimited volume paths in
process_volumes_args(). The previous approach placed all
volumes into a single array element, causing Docker to
receive malformed --volume arguments.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Switch to a git hash of virtiofsd as unfortunately the last release is
some time ago and does not incorporate a dependency bump that enables it
to work with snapshot-restore.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add --test-exclude to process_common_args in test-util.sh and forward
it to the performance-metrics binary from run_metrics.sh.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Rename the aarch64 sha1sums file to sha1sums-aarch64-common to follow
the same naming convention as sha1sums-x86_64-common. This allows
run_metrics.sh to use the generic sha1sums-${TEST_ARCH}-common
pattern for all architectures, removing the need for aarch64-specific
conditionals.
Update run_integration_tests_aarch64.sh to reference the renamed file.
Signed-off-by: Souradeep <schakrabarti@microsoft.com>
Split the vDPA preparation flow into helper functions for
building modules, validating availability, loading modules,
and creating devices.
Build the vdpa_sim modules only on Ubuntu, where the script
installs dependencies and compiles them from the matching
kernel source. On other distributions, reuse the installed
kernel modules and verify that they are available before
continuing.
This makes the script easier to follow and supports systems
such as Azure Linux, where the modules are provided by the
kernel package.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
- Modified the integration test script to support CVM test threads
- Add more parameters to cargo nextest to match other files
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Accept arguments after -- in 'dev_cli.sh shell' and forward them
to 'bash -c' inside the container. When no arguments are given,
an interactive shell is started as before. This enables running
one-off commands in the CI container without an interactive session,
for example:
./scripts/dev_cli.sh shell -- rustup toolchain install nightly \&\& cargo +nightly fmt --all -- --check
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The focal image checksums have been moved in the -common
sha1sums file. Use the correct file for metrics.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When running manual tests locally, it is sometimes necessary to
generate a cloud-init file at a custom path instead of defaulting
to /tmp. This is useful for developers and higher-level management
layers where files in /tmp may be cleaned up automatically.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch adds the skeleton of the CVM test
support and modify existing scripts and test framework
to enable such scenario. Split the sha1sum to support both
regular and CVM guest. Add one test case for CVM. Will further
add more test cases.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
X64_64 image download steps is being used for both
regular and CVM guest. Keeping the steps withing a function
in the test-util.sh
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Testing generic vhost-user devices will require virtiofsd to support the
--tag option, which v1.8.0 does not support.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
build_edk2 was leaving behind .built markers
even when compilation failed.
Gate creation of.built marker to occur only on
successful build
Modify build_edk2() to exit with error code
when arm64 firmware artifact : CLOUDHV_EFI.fd
is not produced
Fixes#7608
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
build_edk2() module in scripts/common-aarch64.sh
does not produce the UEFI firmware for aarch64 as
the commits used to assemble sources for acpica,
edk2-platforms and edk2 do not compile after GCC
version upgraded from 11.4.0 to 13.3.0 in the
developer container (ubuntu 22.04 to 24.04)
Apply minimum upgrade to EDK2_REPO and ACPICA_REPO
required to compile with GCC 13.3.0
while still assuring guest VM boot for all
integration tests
BaseTools: Brotli compression submodule that was
previously failing has been fixed following commit
bump
Developers can now produce UEFI firmware for
aarch64 using the following commands
```
./scripts/dev_cli.sh shell
source scripts/test-util.sh
source scripts/common-aarch64.sh
build_edk2
```
Update docs/uefi.md
Fixes#7608
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
CI reports:
In scripts/test-util.sh line 216:
cleanup() {
^-- SC2329 (info): This function is never invoked. Check usage (or ignored if invoked indirectly).
The shellcheck can't trace calling in trap, so we need add hint
to make it happy.
Signed-off-by: Yi Wang <foxywang@tencent.com>
It is a common use case to run a subset of tests locally to verify
certain functionalities.
The default behaviour for nextest is to error out if no tests are run.
That causes the test scripts to return a non-zero value (failure). Pass
`--no-tests=pass` to nextest to match what `cargo test` does if no tests
are run.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Add tests:
- zlib: test_virtio_block_qcow2_zlib()
- zstd: test_virtio_block_qcow2_zstd()
Both these tests use zlib- and zstd-compressed images as OS image.
Modify test_virtio_block_qcow2_backing_file() test: it is practical
to test qcow2 file-backing with compression, so use zlib-compressed
image as a backing file.
Signed-off-by: Eugene Korenevsky <ekorenevsky@aliyun.com>
TL;DR: Massive quality of life improvement for devs
Cloud Hypervisor uses the Cargo test framework for multiple tests:
- normal unit tests
- unit tests requiring special environment (the Tap device tests)
- integration tests requiring a special environment
This prevented the execution of `cargo test --workspace`, which results
in a very poor developer experience. Although
`./scripts/run_unit_tests.sh` exists, there are valid reasons why devs
cannot or even don't want to use it.
By adding a new `chv_testenv` rustc config, we can conditionally only
activate tests when the `./scripts/` magic runs them. This improves
the general developer experience by a lot.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Due to the fw_cfg test being disabled on MSHV this results in no tests
being runnable which results in an error in nextest. Reduce that error
to a warning use --no-tests=warn
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is required to support exclusive locking on files which is needed
for safe test ID generation when using nextest (since it runs each test
as a separate process.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
To get that list, I've used
```
git log | grep --fixed-strings -- "-by:" | head -n 100000 | sort | less
```
on the Linux kernel's git repository.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Replace the use of `guestmount and `guestunmount` with standard tools
(losetup, mount) to modify cloud disk image. This eliminates
dependency on guestmount which requires /dev/kvm and is not available
in MSHV root partition.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
Refactor MSHV hypervisor selection logic to enable aarch64 support
instead of exiting. Use `--features mshv` to conditionally build &
test when hypervisor is mshv replacing hard exit for aarch64 on mshv.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
With the current syntax docker gives 'docker: invalid reference format'
error. Also during parsing /xxx:/yyy in process_volumes_args
with \" inside variable i.e arr_vols=("${arg_vols//#/ }")
gives wrong output.
Example:
scripts/dev_cli.sh tests --integration --volumes /mshv:/mshv
Error: The volume /mshv /mshv does not exist.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The ivshmem tests are all failing in the CI for MSHV because Cloud
Hypervisor is built without the mshv feature.
Error: Cloud Hypervisor exited with the following chain of errors:
0: Failed to open hypervisor interface (is hypervisor interface
available?)
1: Failed to create the hypervisor
2: no supported hypervisor
Modify the build command to include the mshv feature.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
