After updating the Linux kernel to 6.19.6 the second segment (segment=1)
is now under the 2nd IOMMU group (which it a more logical setup) and as
such the added device which is on that segment is in that second IOMMU
group.
The same check is made in test_vdpa_block so also test there.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Resignal every 10ms the thread if it has not acknowledged the signal via
setting the atomic when the vCPU thread was acknowledged. Further, avoid
an infinite loop by generating an error if it takes more than 1000ms to
interrupt the thread.
The retry helps mitigate a race condition where the signal is received
between checking the pause atomic and entering KVM_RUN ioctl when
pausing. Hitting this race condition would leave the
wait_untial_signal_acknowledged() method spinning indefinitely.
The timeout error avoids the VMM process being blocked indefinitely.
See: #7427
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Dump kernel logs after running the tests in the MSHV workflow to help
debug failures.
In addition to getting the kernel logs using `dmesg` also use AzCli to
retrieve the serial console logs. If the VM is hung or panicked, the
workflow would be unable to SSH into it and execute `dmesg`. In this
case the serial console logs would be helpful.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Replace the use of `guestmount and `guestunmount` with standard tools
(losetup, mount) to modify cloud disk image. This eliminates
dependency on guestmount which requires /dev/kvm and is not available
in MSHV root partition.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
Refactor MSHV hypervisor selection logic to enable aarch64 support
instead of exiting. Use `--features mshv` to conditionally build &
test when hypervisor is mshv replacing hard exit for aarch64 on mshv.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
There are several copy-pasted code fragments in impl QcowFile. All of
them add L2 entry to the cache and one of them (in file_offset_write())
does also allocating new L2 entry if necessary.
Fold all these code fragments (except of one in l2_table() which does
error handling in special way) into cache_l2_cluster() method without
changing the logic.
This will make the code more compact and clean.
Signed-off-by: Eugene Korenevsky <ekorenevsky@aliyun.com>
With the current syntax docker gives 'docker: invalid reference format'
error. Also during parsing /xxx:/yyy in process_volumes_args
with \" inside variable i.e arr_vols=("${arg_vols//#/ }")
gives wrong output.
Example:
scripts/dev_cli.sh tests --integration --volumes /mshv:/mshv
Error: The volume /mshv /mshv does not exist.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The API documentation tells users to expect a message when Cloud
Hypervisor is launched, this message was removed in commit 13724db
This change updates the documentation to reflect how the program
actually functions, which is to say no message.
Signed-off-by: Ariel Chenet <achenet@fastmail.com>
Consuming `&Arc<T>` as argument is almost always an antipattern as it
hides whether the callee is going to take over (shared) ownership
(by .clone()) or not. Instead, it is better to consume `&dyn T` or
`Arc<dyn T>` to be more explicit. This commit cleans up the code.
The change is very mechanic and was very easy to implement across the
code base.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Disable test_snapshot_restore_hotplug_virtiomem for mshv. It is failing
frequently in the CI. It needs to be stabilized before enabling again.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The ivshmem tests involving snapshot/restore & live migration are
failing in the MSHV CI with this error:
Could not get vCPU state GetMsrEntries(
Hypercall 80 failed with 0x5 : InvalidParameter ...<snip>)
This needs more investigation. It is worth noting that the general live
migration tests are also not run in the CI for MSHV.
Disable these tests for MSHV for the time being.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The ivshmem tests are all failing in the CI for MSHV because Cloud
Hypervisor is built without the mshv feature.
Error: Cloud Hypervisor exited with the following chain of errors:
0: Failed to open hypervisor interface (is hypervisor interface
available?)
1: Failed to create the hypervisor
2: no supported hypervisor
Modify the build command to include the mshv feature.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The existing concurrency configuration causes workflow runs for one Pull
Request to cancel in-progress runs for another Pull Request. When the
workflow is triggered by the `pull_request_target event`, `github.ref`
is always set to the base branch, regardless of which branch raised the
PR. This causes every new pull request to share the same concurrency key
and thus cancel the running CI of any other open pull request.
This commit updates the `concurrency.group` to use
`github.event.pull_request.number` for unique isolation when running on
PR, falling back to `github.ref` for non-PR push (e.g., merge group).
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Disable the following common parallel and common sequential tests for
mshv since these are failing consistently in the CI.
Common parallel:
- test_tpm
- test_cpu_topology_421
- test_cpu_topology_142
- test_cpu_topology_262
- test_cpu_hotplug
Common sequential:
- test_snapshot_restore_basic
- test_snapshot_restore_with_fd
- test_snapshot_restore_pvpanic
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
test_fw_cfg is frequently failing in the CI for MSHV. Exclude it for
now. It needs further investigation. See issue #7434 for details.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Move the GICv2M MSI frame SPI base from 32 to 128 to avoid conflicts
with platform device interrupts.
While at it, rename the constants to make it explicit that they are
associated with the GICv2m MSI Frame.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
This was added in 7be69edf51 to deal with
changes to the KVM bindings that made run() and set_immediate_exit()
take &mut self. Instead adopt a Box<> value in Vcpu allowing the removal
of this internal Mutex.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The virtio spec defines the feature bit range and 4 x 32-bit pages.
However no features are currently defined with bits > 63. However Linux
has now started acking features in those higher pages. Since this is
harmless (we drop the acks to those feature pages) and quietly return 0
for available features in those higher pages this warning can be
removed.
Fixes: #7414
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Fix a race condition that happens in resume()-pause() cycles.
It is odd that for pause(), the CpuManager waited via `state.paused`
for the vCPU thread to ACK the state change but not for `resume()`.
In the `resume()` case, oddly CpuManager "owned" the state change
in `state.paused`. This commit changes this so that the vCPU ACKs
its state change itself in `state.paused` when it transitions from
pause->run. Further, `CpuManager::resume()` now gracefully waits for
the vCPU to be resumed.
More technical:
This change ensures proper synchronization and prevents situations
in that park() follows right after unpark(), causing deadlocks and
other weird behavior due to race conditions.
Calling resume() now takes slightly longer, very similar to pause().
This is, however, even for 254 vCPUs in the range of less than 10ms,
and ultimately we now have correct behaviour.
## Reproducer
Since [0] is merged, the underlying problem can be tested without this
commit by modifying the pause() API call to run `CpuManager::pause()`
and `CpuManager::resume()` in a loop a thousand times.
`ch-remote --api-socket ... pause`
```patch
diff --git a/vmm/src/vm.rs b/vmm/src/vm.rs
index d7bba25cc..35557d58f 100644
--- a/vmm/src/vm.rs
+++ b/vmm/src/vm.rs
@@ -2687,6 +2687,10 @@ impl Pausable for Vm {
MigratableError::Pause(anyhow!("Error activating pending virtio devices: {:?}", e))
})?;
+ for _ in 0..1000 {
+ self.cpu_manager.lock().unwrap().pause()?;
+ self.cpu_manager.lock().unwrap().resume()?;
+ }
self.cpu_manager.lock().unwrap().pause()?;
self.device_manager.lock().unwrap().pause()?;
```
## Outlook
Decades of experience in VMM development showed us that using
many AtomicBools is a footgun. They are not synchronized with each
other at all. On the long term, we might want to refactor things to
have a single shared AtomicU64 with different bits having different
meanings.
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7290
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
These bindings actually refer to atomic bool shared across all
vCPUs to instruct single vCPUs with their next action. As there
are already enough Arc<AtomicBool>, this helps while debugging
things to see that different bindings refer to the same atomic
bool.
In other words: This naming really confused us while debugging.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This gives a reliable way of identifying if the VM has booted as well as
a reliable way to validate the addition of the network interface.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Add a second L1 network interface definition to the guest Cloud Init
configuration, including an additional host IP. Do this by splitting the
network range into two /25s. For clarity the network struct members have
also been renamed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Previous RISC-V ACPI related change introduced MSIS which will be
rejected by typo check, let's extend the typos to accept it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add necessary definitions and RISC-V ACPI tables to enable ACPI feature.
More specifically, this commit add MADT definitions for RISC-V AIA
interrupt chips.
Signed-off-by: Can Zhang <icloud9957@gmail.com>
Rename the MSHV CI workflow to better align with the existing naming
scheme and also make the arch explicit.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Add step-level timeout for running integration tests so that the
workflow doesn't get stuck on any test.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
We are actually using v0.3.0 of vm-fdt since main branch of that crate
already includes everything. Let's pin vm-fdt to v0.3.0 instead.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Pull requests from forked repos cannot access GitHub secrets which
results in failure of MSHV CI. Switching to pull_request_target
resolves this. It allows the workflow to run with access to repo secrets
and ensures that code from the base branch is used instead of forked
code, preventing potential security risks.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Add workflow to run integration tests on mshv. It calls the azure
infra setup workflow and executes integration tests in the
provisioned environment.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Add workflow to setup Azure infra to validate MSHV.
This is used to provision the environment before executing tests
on it.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
