Replace the use of `guestmount and `guestunmount` with standard tools
(losetup, mount) to modify cloud disk image. This eliminates
dependency on guestmount which requires /dev/kvm and is not available
in MSHV root partition.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
Refactor MSHV hypervisor selection logic to enable aarch64 support
instead of exiting. Use `--features mshv` to conditionally build &
test when hypervisor is mshv replacing hard exit for aarch64 on mshv.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
There are several copy-pasted code fragments in impl QcowFile. All of
them add L2 entry to the cache and one of them (in file_offset_write())
does also allocating new L2 entry if necessary.
Fold all these code fragments (except of one in l2_table() which does
error handling in special way) into cache_l2_cluster() method without
changing the logic.
This will make the code more compact and clean.
Signed-off-by: Eugene Korenevsky <ekorenevsky@aliyun.com>
With the current syntax docker gives 'docker: invalid reference format'
error. Also during parsing /xxx:/yyy in process_volumes_args
with \" inside variable i.e arr_vols=("${arg_vols//#/ }")
gives wrong output.
Example:
scripts/dev_cli.sh tests --integration --volumes /mshv:/mshv
Error: The volume /mshv /mshv does not exist.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The API documentation tells users to expect a message when Cloud
Hypervisor is launched, this message was removed in commit 13724db
This change updates the documentation to reflect how the program
actually functions, which is to say no message.
Signed-off-by: Ariel Chenet <achenet@fastmail.com>
Consuming `&Arc<T>` as argument is almost always an antipattern as it
hides whether the callee is going to take over (shared) ownership
(by .clone()) or not. Instead, it is better to consume `&dyn T` or
`Arc<dyn T>` to be more explicit. This commit cleans up the code.
The change is very mechanic and was very easy to implement across the
code base.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Disable test_snapshot_restore_hotplug_virtiomem for mshv. It is failing
frequently in the CI. It needs to be stabilized before enabling again.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The ivshmem tests involving snapshot/restore & live migration are
failing in the MSHV CI with this error:
Could not get vCPU state GetMsrEntries(
Hypercall 80 failed with 0x5 : InvalidParameter ...<snip>)
This needs more investigation. It is worth noting that the general live
migration tests are also not run in the CI for MSHV.
Disable these tests for MSHV for the time being.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The ivshmem tests are all failing in the CI for MSHV because Cloud
Hypervisor is built without the mshv feature.
Error: Cloud Hypervisor exited with the following chain of errors:
0: Failed to open hypervisor interface (is hypervisor interface
available?)
1: Failed to create the hypervisor
2: no supported hypervisor
Modify the build command to include the mshv feature.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The existing concurrency configuration causes workflow runs for one Pull
Request to cancel in-progress runs for another Pull Request. When the
workflow is triggered by the `pull_request_target event`, `github.ref`
is always set to the base branch, regardless of which branch raised the
PR. This causes every new pull request to share the same concurrency key
and thus cancel the running CI of any other open pull request.
This commit updates the `concurrency.group` to use
`github.event.pull_request.number` for unique isolation when running on
PR, falling back to `github.ref` for non-PR push (e.g., merge group).
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Disable the following common parallel and common sequential tests for
mshv since these are failing consistently in the CI.
Common parallel:
- test_tpm
- test_cpu_topology_421
- test_cpu_topology_142
- test_cpu_topology_262
- test_cpu_hotplug
Common sequential:
- test_snapshot_restore_basic
- test_snapshot_restore_with_fd
- test_snapshot_restore_pvpanic
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
test_fw_cfg is frequently failing in the CI for MSHV. Exclude it for
now. It needs further investigation. See issue #7434 for details.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Move the GICv2M MSI frame SPI base from 32 to 128 to avoid conflicts
with platform device interrupts.
While at it, rename the constants to make it explicit that they are
associated with the GICv2m MSI Frame.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
This was added in 7be69edf51 to deal with
changes to the KVM bindings that made run() and set_immediate_exit()
take &mut self. Instead adopt a Box<> value in Vcpu allowing the removal
of this internal Mutex.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The virtio spec defines the feature bit range and 4 x 32-bit pages.
However no features are currently defined with bits > 63. However Linux
has now started acking features in those higher pages. Since this is
harmless (we drop the acks to those feature pages) and quietly return 0
for available features in those higher pages this warning can be
removed.
Fixes: #7414
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Fix a race condition that happens in resume()-pause() cycles.
It is odd that for pause(), the CpuManager waited via `state.paused`
for the vCPU thread to ACK the state change but not for `resume()`.
In the `resume()` case, oddly CpuManager "owned" the state change
in `state.paused`. This commit changes this so that the vCPU ACKs
its state change itself in `state.paused` when it transitions from
pause->run. Further, `CpuManager::resume()` now gracefully waits for
the vCPU to be resumed.
More technical:
This change ensures proper synchronization and prevents situations
in that park() follows right after unpark(), causing deadlocks and
other weird behavior due to race conditions.
Calling resume() now takes slightly longer, very similar to pause().
This is, however, even for 254 vCPUs in the range of less than 10ms,
and ultimately we now have correct behaviour.
## Reproducer
Since [0] is merged, the underlying problem can be tested without this
commit by modifying the pause() API call to run `CpuManager::pause()`
and `CpuManager::resume()` in a loop a thousand times.
`ch-remote --api-socket ... pause`
```patch
diff --git a/vmm/src/vm.rs b/vmm/src/vm.rs
index d7bba25cc..35557d58f 100644
--- a/vmm/src/vm.rs
+++ b/vmm/src/vm.rs
@@ -2687,6 +2687,10 @@ impl Pausable for Vm {
MigratableError::Pause(anyhow!("Error activating pending virtio devices: {:?}", e))
})?;
+ for _ in 0..1000 {
+ self.cpu_manager.lock().unwrap().pause()?;
+ self.cpu_manager.lock().unwrap().resume()?;
+ }
self.cpu_manager.lock().unwrap().pause()?;
self.device_manager.lock().unwrap().pause()?;
```
## Outlook
Decades of experience in VMM development showed us that using
many AtomicBools is a footgun. They are not synchronized with each
other at all. On the long term, we might want to refactor things to
have a single shared AtomicU64 with different bits having different
meanings.
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7290
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
These bindings actually refer to atomic bool shared across all
vCPUs to instruct single vCPUs with their next action. As there
are already enough Arc<AtomicBool>, this helps while debugging
things to see that different bindings refer to the same atomic
bool.
In other words: This naming really confused us while debugging.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This gives a reliable way of identifying if the VM has booted as well as
a reliable way to validate the addition of the network interface.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Add a second L1 network interface definition to the guest Cloud Init
configuration, including an additional host IP. Do this by splitting the
network range into two /25s. For clarity the network struct members have
also been renamed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Previous RISC-V ACPI related change introduced MSIS which will be
rejected by typo check, let's extend the typos to accept it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add necessary definitions and RISC-V ACPI tables to enable ACPI feature.
More specifically, this commit add MADT definitions for RISC-V AIA
interrupt chips.
Signed-off-by: Can Zhang <icloud9957@gmail.com>
Rename the MSHV CI workflow to better align with the existing naming
scheme and also make the arch explicit.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Add step-level timeout for running integration tests so that the
workflow doesn't get stuck on any test.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
We are actually using v0.3.0 of vm-fdt since main branch of that crate
already includes everything. Let's pin vm-fdt to v0.3.0 instead.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Pull requests from forked repos cannot access GitHub secrets which
results in failure of MSHV CI. Switching to pull_request_target
resolves this. It allows the workflow to run with access to repo secrets
and ensures that code from the base branch is used instead of forked
code, preventing potential security risks.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Add workflow to run integration tests on mshv. It calls the azure
infra setup workflow and executes integration tests in the
provisioned environment.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Add workflow to setup Azure infra to validate MSHV.
This is used to provision the environment before executing tests
on it.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
This provides clearer life-cycle management of resources around vfio,
and aligns better with the kernel behavior as reported below with vfio
legacy mode (with vfio container/group).
Fixes: #7328
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Yi Wang <foxywang@tencent.com>
We recently moved many of our tests to use focal to jammy as the guest
images, including rate-limiter tests (#7367). We forgot to update the
rate-limiter scripts to reflect such change.
Our CI pipeline failed to report such error because our self-host runner
happened to be not working at the time we land the changes (see #7405).
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Only use 75% of the available threads - this will reduce dislk and
memory pressure. Reducing the chance of flaky tests.
See: #7405
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
While an IGVM is provided validation fails as there is
no kernel or firmware. This patch fixes the sev_snp boot
failure.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
MSHV (Microsoft Hypervisor) doesn't expose an ITS to guests. Instead it
exposes a GICv2m MSI Frame for mapping MSIs to SPIs.
Update the MADT generation code to create the GIC MSI Frame structure
instead of the ITS structure whenever the guest vGIC is using a GICv2m
frame. This allows booting guests via firmware on MSHV.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
In preparation for removing focal support - start by using a more
generic "disk_config" variable name for the DiskConfig struct vs
embedding the image type.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Rust's aliasing rules do not permit mutating a `usize` through a shared
reference. Unsafe blocks and FFI are no exception.
This PR fixes such a violation detected in the AMX support checks by
passing a raw mutable pointer instead of a shared reference in the
syscall.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP <oliver.anderson@sap.com>
For a graceful resource management of externally provided FDs in Cloud
Hypervisor, corresponding FDs need to be closed on a device removal.
This is the case for virtio-net devices using external FDs, for example.
With the fix introduced in this commit, we allow management software to
properly clean up resources, e.g., libvirt can clean up tap devices.
PS: CHV uses "added" and "removed", which has the same meaning as
hot device attach/hotplug and hot device detach/unplug.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Fix clippy warning `uninlined_format_args` reported by rustc rustc
1.89.0 (29483883e 2025-08-04).
```console
warning: variables can be used directly in the `format!` string
--> block/src/lib.rs:649:17
|
649 | info!("{} failed to create io_uring instance: {}", error_msg, e);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
= note: `#[warn(clippy::uninlined_format_args)]` on by default
help: change this to
|
649 - info!("{} failed to create io_uring instance: {}", error_msg, e);
649 + info!("{error_msg} failed to create io_uring instance: {e}");
|
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
"Update configs must have a unique combination of 'package-ecosystem',
'directory', and 'target-branch'. Ecosystem 'cargo' has overlapping
directories for target branch 'main'."
Drop `target-branch` in rust-vmm group to make those two unique.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
## TL;DR
This unifies error handling, implementation, and logging of config
objects that are populated with additional FDs received as part of
the HTTP API request from the management software. This prevents current
and future code in repetition.
In the future, if we want to support let's say virtio-blk with external
FDs, the AddBlk API handler simply needs a call to `attach_fds_to_cfg()`
and gets all the magic for free.
## Motivation
The motivation of the new abstraction is to provide a verbose, solid,
and bulletproof solution for a complex domain. The interaction between
- the management layer,
- the passing of file descriptors over UNIX domain sockets via
SCM_RIGHTS,
- the attachment of configurations to those FDs,
- and the ability to give new developers clear insights into what
happens under the hood
is not trivial. These factors justify encapsulating the complexity
behind a convenient and well-documented abstraction, making the system
both robust in production and approachable for new developers.
In addition, it allows us to perform unit testing. Further, We get rid
of existing partial code duplication and inconsistencies.
Finally, while this approach may initially result in more code, every
new handler that accepts FDs benefits from reduced duplication and a
correct implementation by relying on the shared abstraction. This will
also enable future functionality, such as virtio-blk devices backed by
FDs which can then be integrated with ease.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Deserializing values as `-1` makes sense to prevent errors, so let's
keep it. However, serializing them differently adds confusion. For
example, a `ch-remote info` call should not report `-1` but the actual
FDs.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
(1) The old messages are missing the "why" part. With this change, users
of Cloud Hypervisor have somehow more context and people looking at
the code perfectly know what's going on.
(2) Using warn! implies that the user should take action, but in this
case, there’s nothing the user can do. If the API is used correctly
and file descriptors are passed via an SCM_RIGHTS message over a
UNIX domain socket, everything works as intended. In that case,
there's no need to issue a warning — debug! is sufficient.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Live migration, state save/resume, and hotplug are not trivial when it
comes to virtio-net devices backed by externally provided FDs. As the
mechanism behind it can be considered as quite "multi-step magic" even
for experienced programmers, it makes sense to thoroughly document this
to ease debugging and to improve the mental model of developers working
on this in the future.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This change enables easier integration with third-party
tools by removing the requirement for a dummy IP address
when configuring tap devices. The modification applies to
both CLI and API interactions.
Previously, cloud-hypervisor would automatically set a
default static IP address (192.168.249.1) if none was provided.
This could lead to:
* multiple devices without explicit IP configurations
would end up with the same default IP
* unnecessary inclusion of this IP in firewall rules
* the IP address could clash with host networking and
routing
This introduces a new constraint:
When providing an IP, the mask must also be provided.
Removes warnings introduced in #7179.
Closes issue #7083.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
The function `read_local_stream_port` had no proper handling for
unexpected or incomplete input.
When the control socket of the VSOCK device was closed without sending
the expected `CONNECT <PORT>\n` statement completely, the thread
got stuck in an infinite loop as it attempted to read from a closed
socket over and over again which never returned any data.
This resulted in the thread responsible for `epoll` being completely
blocked. New VSOCK connections could not be established and existing
ones became defunct, effectively leading to a Denial of Service of
the entire VSOCK device.
The issue can be reproduced by opening a socket and immediately
closing it.
```
socat - UNIX-CONNECT:/socket.vsock
<Ctrl-C>
```
Instead of applying a quick fix by handling the `EPOLLHUP` event before
reading, the function is refactored to remove the error-prone `while`
loop and multiple `read`s.
Notably, we now check if the number of bytes read is zero, which occurs
when `event_set == EPOLLHUP | EPOLLIN`, indicating that the socket has
been closed by the client.
Additionally, the actual parsing code is now extracted into a dedicated
function that is tested.
Fixes: #6798
Signed-off-by: Maximilian Güntner <code@mguentner.de>
For MSHV customers don't want to make everything
default during partition creation. For example
nested support, some synthetic features could be
controlled from CLI through platform argument.
Create_vm API getting messy after adding more flags.
This patch introduces common data struct to be passed
from vmm crate to hypervisor crate during partition creation.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Handle MSHV reset intercepts on arm64 such that guests can gracefully
shutdown/reboot instead of panicking due to unhandled intercept.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Reference link to set_device_attribut for aarch64
is invalid. Looks like the code browsing does not
have the reference anymore.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This message makes it easier to confirm a successful shutdown when
scanning logs during development and in production. If the message is
missing and the last log entry is not an error, the most likely cause
is a livelock (e.g. contention on a lock) introduced while developing
new Cloud Hypervisor features.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This commit includes all simple clippy fixes excluding the
collapsing of nested ifs using the let-chains feature. This
follows in the next commit.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Bump mshv-ioctls and mshv-bindings to 0.6.0. Most notably, this version
contains fixes and new bindings for arm64 guests.
Bump the vfio crates too so that they point to the latest mshv crates.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Virtio console is activated much later in boot. The output it spits out
lacks the initial CPU configuration of the guest.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
These can differ between platforms, so it's better to use centralized
definitions of them. We can't currently do this for the KVM and VFIO
ioctls, because the corresponding crates don't publicly expose the
ioctl numbers.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The previously linked file was converted to ReStructuredText, then
later removed from the kernel, with part of the justification being
"Most of what booting-without-of.rst contains is now in the DT
specification", so point to that instead.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Since the user has to explicitly ask for Landlock to be enabled in
Cloud Hypervisor, it's surprising that, even when they've done that,
Landlock will silently not be enabled if the kernel doesn't support
it.
To prevent accidental absence of a desired security protection, exit
with an error if Landlock, or the one feature of it we use in Cloud
Hypervisor (file access) is not supported.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
arch::aarch64::fdt::create_cpu_nodes will always look at this if it
exists. (If it doesn't exist, this is a no-op —
add_rule_with_access() won't add rules for paths that don't exist.)
Fixes: b3e5738b4 ("vmm: Introduce ApplyLandlock trait")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
MSHV now supports movable pages i.e VA backed guest.
Also with more features and stability in the MSHV
Kernel drives MSHV now supports more test scenario.
This patch enables more integration test cases.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Without write access to the directory the socket will go in, it's not
possible to create the socket.
I've tested outgoing connections from the VM, and they don't seem to
need read permissions on that directory to connect to a socket on the
host.
Fixes: b3e5738b4 ("vmm: Introduce ApplyLandlock trait")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
path_beneath_rules() just needs the paths given to it to be
AsRef<Path>, so there's no need to create new PathBufs for it.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Raise the max number of supported (v)CPUs on kvm x86_64 hosts
to 8192 (the max allowed value of CONFIG_NR_CPUS in the Linux kernel).
Other platfroms keep their existing CPU limits pending further
development and testing.
The change has been tested on Intel and AMD hosts.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
These are now used by Cloud Hypervisor when linked with Glibc 2.42.
These values should be correct for all currently supported Cloud
Hypervisor platforms, although they are not for all Linux platforms.
Closes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7276
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This commit removes the SGX support from cloud hypervisor. SGX support
was deprecated in May as part of #7090.
Signed-off-by: Shubham Chakrawar <schakrawar@crusoe.ai>
This patch gives user an option to override the
default migratable version to any later release.
This option makes MSHV specific tests suitable for
tests since MSHV is stable after some breaking changes.
This patch is also necessary for MSHV CI.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This step was done manually by searching for "unsafe fn" in
the code base and adding corresponding Safety sections.
`clippy::missing_safety_doc` only works for public functions
but none of the corresponding functions is public.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Implement the batch submission function for raw disk, default it is
enabled. After parsing the requests this method is
called for better IO latency and bandwidth.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Cache and batch IO requests after parsing all
items in the queue, improving performance—especially
for small block sizes—by reducing per-request overhead.
Introduced two methods in the AsyncIo trait for batch
submission, with implementation in the raw disk backend.
This method should be called during/after parsing all block IO requests
in the available queue. If the batch submission is not enabled, by
default it does the old way of submitting requests.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Instead of returning boolean return an struct of completion status
so that it can be cached for batch submission.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch changes the read-only check using acked features bit, which
will help to check more features.
Signed-off-by: Songqian Li <sionli@tencent.com>
Also tweak extended interrupt handlings, as needed.
Most credit should go to Neel Natu, who figured out the magic
bits needed to make things work and provided detailed comments.
This patch is still NOOP, as VM config allows only up to
254 vCPUs on x86_64.
Note: changes in this and related previous patches/PRs have
only been tested on Linux hosts running on Intel x86_64 hardware.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
test_direct_kernel_boot_bzimage runs only on x86, so the cfg!() branch
for selecting grep_cmd is unnecessary. Remove it for clarity.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
One can call `to_vec()` anyway if one needs an owned copy. This change
further helps to prevent needless copies in upcoming changes.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
On aarch64 and RISC-V, calling load_firmware() through load_kernel()
provides no benefit and only duplicates checks already performed in
load_payload(). load_payload() now directly invokes load_firmware() or
load_kernel(), removing unnecessary indirection and redundancy.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Both functions are defined separately for the two architecture with
minor differences.
* `load_firmware()`: call `arch::uefi::load_uefi` which are available on
both architecture;
* `load_kernel()`: manually align to `arch::layout::KERNEL_START` 2MB
for both architecture (e.g. no-op for `aarch64`);
Signed-off-by: Bo Chen <bchen@crusoe.ai>
With 254 vCPUs, pausing now takes ~4ms instead of >254ms. This
improvement is visible when running `ch-remote pause` and is
particularly important for live migration, where every millisecond
of downtime matters.
For the wait logic, it is fine to stick to the approach of
sleeping 1ms on the first missed ACK as:
1) we have to wait anyway
2) we give time to the OS, enabling it to schedule a vCPU thread next
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Enable kvm build test and clippy test on RISC-V 64-bit platform to
ensure whole projects builds properly.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
MP table is a legacy device that is incompatible
with x2apic CPU IDs exceeding 254. The Linux kernel
is perfectly happy without MP table in these cases.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
Currently, the following scenarios are supported by Cloud Hypervisor to
bootstrap a VM:
1. provide firmware
2. provide kernel
3. provide kernel + cmdline
4. provide kernel + initrd
5. provide kernel + cmdline + initrd
As the difference between `--firmware` and `--kernel` is not very clear
currently, especially as both use/support a Xen PVH entry, adding this
helps to identify the cause of misconfiguration.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This is necessary to use the let-chains feature in a
follow-up. After upgrading to Rust edition 2024, clippy
wants to collapse various if's with let-chains.
Update image to 20250815-0 since MSRV in Dockerfile is updated.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This patch introduces the inter-vm shared memory(ivshmem) device
to share a memory region between multiple processes running
different guests and the host.
This patch supports the basic ivshmem functions like ivshmem-plain
in QEMU[1].
[1] https://www.qemu.org/docs/master/specs/ivshmem-spec.html
Signed-off-by: Yi Wang <foxywang@tencent.com>
Signed-off-by: Songqian Li <sionli@tencent.com>
Move UserspaceMapping to vm-device to avoid redefinition since
UserspaceMapping is used by both `virtio-devices` and `device`
crate.
Signed-off-by: Songqian Li <sionli@tencent.com>
It is always called with topology provided, so there is no
need to pass topology as an Option. Simplifying the signature
makes further topology-related changes to arc/src/x86_64 module
simpler.
Signed-off-by: Peter Oskolkov <posk@google.com>
This is the second patch in a series intended to let Cloud Hypervisor
support more than 255 vCPUs in guest VMs; the first patch/commit is
https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7231
At the moment, CPU topology in Cloud Hypervisor is using
u8 for components, and somewhat inconsistently:
- struct CpuTopology in vmm/src/vm_config.rs uses four components
(threads_per_core, cores_per_die, dies_per_package, packages);
- when passed around as a tuple, it is a 3-tuple of u8, with
some inconsistency:
- in get_x2apic_id in arch/src/x86_64/mod.rs the three u8
are assumed to be (correctly)
threads_per_core, cores_per_die, and dies_per_package, but
- in get_vcpu_topology() in vmm/src/cpu.rs the three-tuple is
threads_per_core, cores_per_die, and packages (dies_per_package
is assumed to always be one? not clear).
So for consistency, a 4-tuple is always passed around.
In addition, the types of the tuple components is changed from u8 to
u16, as on x86_64 subcomponents can consume up to 16 bits.
Again, config constraints have not been changed, so this patch
is mostly NOOP.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
This is the first change to Cloud Hypervisor in a series of changes
intended to increase the max number of supported vCPUs in guest VMs,
which is currently limited to 255 (254 on x86_64).
No user-visible/behavior changes are expected as a result of
applying this patch, as the type of boot_cpus and related
fields in config structs remains u8 for now, and all configuration
validations remain the same.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
This allows us to enable/disable the fw_cfg device via the cli
We can also now upload files into the guest vm using fw_cfg_items
via the cli
Signed-off-by: Alex Orozco <alexorozco@google.com>
This allows the fw_cfg device to be recognized by the guest linux
kernel. This becomes more relavnt in the following cl where I add
the option to load files into the guest via fw_cfg. The Linux kernel
already has a fw_cfg driver that will automatically load these files
under /sys when CONFIG_FW_CFG_SYSFS is enabled in the kernel config
For arm we must add fw_cfg to the devices tree
Signed-off-by: Alex Orozco <alexorozco@google.com>
We pass a reference to the guest memory when we create the device
in DeviceManager. This allows us to access the guest memory for DMA.
Signed-off-by: Alex Orozco <alexorozco@google.com>
The acpi tables are created in the same place the acpi tables would be
created for the regular bootflow, except here we add them to the
fw_cfg device to be measured by the fw and then the fw will put the
acpi tables into memory.
Signed-off-by: Alex Orozco <alexorozco@google.com>
The kernel and initramfs are passed to the fw_cfg device as
file references. The cmdline is passed directly.
Signed-off-by: Alex Orozco <alexorozco@google.com>
Here we add the fw_cfg device as a legacy device to the device manager.
It is guarded behind a fw_cfg flag in vmm at creation of the
DeviceManager. In this cl we implement the fw_cfg device with one
function (signature).
Signed-off-by: Alex Orozco <alexorozco@google.com>
This partially reverts
ed8f347fe6 from #7183 and
6277d7d5f2 from #7201.
# Output how it was merged for v47 (#7066)
```
Error: Cloud Hypervisor exited with the following chain of errors:
0: Error booting VM
1: The VM could not boot
2: Error manipulating firmware file
3: No such file or directory (os error 2)
Debug Info: VmBoot(VmBoot(FirmwareFile(Os { code: 2, kind: NotFound, message: "No such file or directory" })))
```
# Output after #7183 and #7201
```
cloud-hypervisor: 31.385730ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:27 -- Error: Cloud Hypervisor exited with the following chain of errors:
cloud-hypervisor: 31.417961ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 0: Error booting VM
cloud-hypervisor: 31.448078ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 1: The VM could not boot
cloud-hypervisor: 31.486711ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 2: Error manipulating firmware file
cloud-hypervisor: 31.513331ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 3: No such file or directory (os error 2)
cloud-hypervisor: 31.548037ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:44 --
cloud-hypervisor: 31.568045ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:45 -- Debug Info: VmBoot(VmBoot(FirmwareFile(Os { code: 2, kind: NotFound, message: "No such file or directory" })))
```
The "proper logger" has indeed the advantage that messages can
be gracefully redirected to log files etc. However, this makes the
error message hardly readable.
Therefore, I propose to use error!() only for runtime errors messages
but not a pretty-printed version of those.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Error::UefiLoad is required for load_firmware to propagate errors
encountered, define it for riscv64.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
uefi_flash field in memory_manager is required for uefi loading and
booting, enable it for riscv64 architecture.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The option parser did not allow quoting any special characters, which
means that some strings could not be passed via command-line options.
Fix this by allowing double quotes to quote all the special characters,
such as '@', ',', '[', and ']'. '"' can be escaped by doubling it. The
implementation uses some tricks to stay modular.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
rustc 1.90.0-beta.1 (788da80fc 2025-08-04) suggests using library
feature `unsigned_is_multiple_of`. It is stabled in Rust 1.87.0.
Update image to 20250807-0 since MSRV in Dockerfile is updated.
Signed-off-by: Songqian Li <sionli@tencent.com>
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Now 'create_acpi_tables_internal()' can generate ACPI tables with
different dsdt offset, so that it can be reused for generating ACPI
tables for `fw_cfg`.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Alex Orozco <alexorozco@google.com>
Optimize lychee workflow to check only changed
files in pull requests, avoiding excessive API
calls to prevent rate limits.
Fixes#7056
Signed-off-by: Shubham Chakrawar <schakrawar@crusoe.ai>
In case of CVM guest rsdp is set to none. Unwrapping it
make the vmm crashed. Don't call configure system if the
rsdb address is none.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Many of the workspace members in the Cloud-hypervisor workspace share
common dependencies. Making these workspace dependencies reduces
duplication and improves maintainability.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The current documentation only mentions Windows
server support. Since only minimal adjustments
are needed for Windows 11 support, these were
added where applicable.
Also contains whitespace fixes, adds missing `\`
in example code and adds more cross references.
Signed-off-by: Lucas Grosche <lucas.grosche@cyberus-technology.de>
The other workspace members in the Cloud-hypervisor workspace
currently declare log version 0.4.22, but the tpm crate
has an older version. This inconsistency is addressed by this PR
which opens the door for declaring log as a workspace dependency.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Running one or two tests in a tight loop can cause the download
functions to quickly hit GitHub's API rate limit. That causes the test
script to fail for no apparent reason.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Generate the data disk under a temporary directory so that multiple
instances of the test suites can run at the same time.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The other workspace members in the Cloud-hypervisor workspace
currently declare libc crate version 0.2.167, but the tpm crate
has an older version. This inconsistency is addressed by this PR
which opens the door for declaring libc as a workspace dependency.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The other workspace members in the Cloud-hypervisor workspace
currently declare anyhow version 1.0.94, but the tpm crate
has an older version. This inconsistency is addressed by this PR
which opens the door for declaring anyhow as a workspace dependency.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
It shouldn't be necessary to lock the file for the adaptor. This removes
two layers of indirection for QcowDiskSync and VhdxDiskSync.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The PL031 RTC provides two features: a real-time counter and an alarm
interrupt. To use the alarm, the driver normally writes a time value
into the match register RTCMR, and when the counter reaches that value
the device triggers the interrupt.
At the moment the implementation ignores programming of the alarm, as
the feature seems rarely used in VMs. However the interrupt is still
triggered arbitrarily when the guest writes to registers, and the line
is never cleared. This really confuses the Linux driver, which loops in
the interrupt handler until Linux realizes that no one is dealing with
the interrupt (200000 unanswered calls) and disables the handler.
One way to fix this would be implementing the alarm function properly,
which isn't too difficult but requires adding some async timer logic
which probably won't ever get used. In addition the device's interrupt
is level-triggered and we don't support level interrupts at the moment,
though we could probably get away with changing this interrupt to edge.
The simplest fix, though, is to just disable the interrupt logic
entirely, so that the alarm function still doesn't work but the guest
doesn't see spurious interrupts.
Add a default() implementation to satisfy clippy's new_without_default
check, since Rtc::new() doesn't take a parameter after this change.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
The read_exact() call was introduced in 82ac114b8 ("virtio-devices:
vsock: handle short read in muxer") to solve a crash when a connection
disconnected without sending any data, but it introduced a problem of
its own: because the socket is non-blocking, read_exact() may read
some data, then return ErrorKind::WouldBlock. In that case, the data
it read will be discarded. So for example if it read "CONNECT ",
and then nothing else was available to read yet, "CONNECT " would be
discarded, and so the next time this function was called, when epoll
triggered again for the socket, only the following data would end up
in command.buf, causing an error due to just a port number being an
invalid command.
Contrary to that commit message, this code was actually designed to
handle short reads just fine — in the case of a short read, it stores
the data it has read in command, and returns
Error::UnixRead(ErrorKind::WouldBlock), which is ignored by the
caller, and the function gets called again when there is more data to
read, building up command potentially over the course of several
reads. The only thing it didn't handle correctly, as far as I can
tell, was a 0-byte read, which happens when a client disconnects from
the socket without writing anything. All that's needed to fix this is
to avoid an invalid subtraction in that case, so this change reverts
82ac114b8, fixing the issue with partial commands being discarded, and
instead handles the 0-byte read by using slice::get, and treating an
empty command as an incomplete command, which of course it is.
Fixes: 82ac114b8 ("virtio-devices: vsock: handle short read in muxer")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This makes it possible to run cargo test just for the virtio-devices
crate (as long as either KVM or MSHV is specified).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
`std::mem::offset_of` could be used for calculating nested fields, use
this feature to shorten aarch64 reg offset calculation.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`std::mem::offset_of` is stabilized since Rust 1.77, let's use
implementation provided by std instead of manual implementation.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually implemented `_offset_of` and `offset_of` in
`arch/riscv64/mod.rs` are not used now, remove them.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`std::mem::offset_of` supports calculating offset of nested structures,
let's use implementation provided by std instead of manual
implementation.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Other lines are already logged using `log::error!` and
`env_logger` is initialized before calling
`start_net_backend` in `main()`.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
Unify log formatting and printing as `eprintln!` and `log::error!`
would be used alongside each other.
When using e.g. `env_logger` lines printed with `eprintln!` would
lack formatting / colors.
Currently only relevant in `ch-remote` + `cli_print_error_chain`.
Note that the replaced messages now also end up in the logfile of
`cloud-hypervisor` when configured and not any longer in stderr.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
Until now all messages generated using `log::level!`
(e.g., `warn!`) have not been printed as `ch-remote` did not
register a logger.
Furthermore, replace all `eprintln!` with `error!`
to align formatting for consistency.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
Recently vfio crates have moved to crates.io, thus we should start
consuming the crate from crates.io instead git url.
This results in better versioning instead of tracking some git commit
sha.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since action-rs/cross is deprecrated, thus move to
houseabsolute/actions-rust-cross.
We should pin the cross-version to the latest version to fix the build
issues with virtio-bindings crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
When invoking the script chown shows a warning.
chown: warning: '.' should be ':': ‘1000.1000’
From `info coreutils 'chown invocation'`.
Some older scripts may still use ‘.’ in place of the ‘:’ separator.
POSIX 1003.1-2001 (*note Standards conformance::) does not require
support for that, but for backward compatibility GNU ‘chown’ supports
‘.’ so long as no ambiguity results, although it issues a warning and
support may be removed in future versions. New scripts should avoid the
use of ‘.’ because it is not portable, and because it has undesirable
results if the entire OWNER‘.’GROUP happens to identify a user whose
name contains ‘.’.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
That feature was dropped when consolidating the UUID dependency because
somehow building the whole project worked. The CI system was happy.
However, building the block crate alone is broken. The vhdx code uses
Uuid::new_v4, which requires `v4` to be enabled.
Add the feature back.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When a proper Rust backtrace is printed, the Rust std wants to use the
SYS_getcwd(79) system call to prettify some paths while printing. In
Cloud Hypervisor, this is at least relevant for printing panics or if
a `anyhow::Error` value is printed using `{e:?}` (but not `{e:#?}`).
The syscall cause can be found in `impl fmt::Display for Backtrace {}`
in `library/std/src/backtrace.rs`.
Without this addition, the seccomp violation of the SYS_getcwd (79)
hinders the proper error message including a full backtrace from showing
up. This annoying behaviour already delayed many debugging efforts. With
this fix, things just work. The new syscall itself should be pretty
harmless for normal operation.
```
thread 'vmm' panicked at virtio-devices/src/rng.rs:224:9:
Yikes, things went horribly wrong!
==== Possible seccomp violation ====
Try running with `strace -ff` to identify the cause and open an issue: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/new
[1] 287683 invalid system call (core dumped) RUST_BACKTRACE=full cargo run --bin cloud-hypervisor -- --api-socket --kerne
```
```
thread 'vmm' panicked at virtio-devices/src/rng.rs:224:9:
Yikes, things went horribly wrong!
stack backtrace:
0: 0x557d91286b62 - std::backtrace_rs::backtrace::libunwind::trace::hc20b48b31ee52608
at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
1: 0x557d91286b62 - std::backtrace_rs::backtrace::trace_unsynchronized::h5d207cd20f193d88
at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
...
67: 0x0 - <unknown>
Error: Cloud Hypervisor exited with the following error:
Failed to join on VMM thread: Any { .. }
Debug Info: ThreadJoin(Any { .. })
```
- add any panic, for example into the create or drop function of a
device
- add --seccomp=true|log to analyze the situation
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The virtio_features_to_tap_offload() defined in ctrl_queue.rs
is duplicated. Remove it and use the one defined in lib.rs
instead.
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Checksumming a type that has padding would use the padding in the
checksum, which is definitely wrong and is often unsound.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Cargo fuzz build report an error:
error: lifetime flowing from input to output with different syntax can be confusing
--> /home/runner/work/cloud-hypervisor/cloud-hypervisor/block/src/lib.rs:747:13
|
747 | fn file(&mut self) -> MutexGuard<F>;
| ^^^^^^^^^ ------------- the lifetime gets resolved as `'_`
| |
| this lifetime flows to the output
error: lifetime flowing from input to output with different syntax can be confusing
--> /home/runner/work/cloud-hypervisor/cloud-hypervisor/block/src/async_io.rs:68:11
|
68 | fn fd(&mut self) -> BorrowedDiskFd;
| ^^^^^^^^^ -------------- the lifetime gets resolved as `'_`
| |
| this lifetime flows to the output
Signed-off-by: Yi Wang <foxywang@tencent.com>
Remote server errors are transferred as raw HTTP body. This way,
we lose the nested structured error information.
This is an attempt to retrieve the errors from the HTTP response
and to align the output with the normal error output.
For example, this produces the following chain of errors. Note
that everything after level 0 was retrieved from the HTTP server
response:
```
Error: ch-remote exited with the following chain of errors:
0: http client error
1: Server responded with InternalServerError
2: Error from API
3: The disk could not be added to the VM
4: Failed to validate config
5: Identifier disk1 is not unique
Debug Info: HttpApiClient(ServerResponse(InternalServerError, Some("Error from API<br>The disk could not be added to the VM<br>Failed to validate config<br>Identifier disk1 is not unique")))
```
In case the JSON can't be parsed properly, ch-remote will print:
```
Error: ch-remote exited with the following chain of errors:
0: http client error
X: Can't get remote's error messages from JSON response: EOF while parsing a value at line 1 column 0: body=''
Debug Info: HttpApiClient(ServerResponse(InternalServerError, Some("")))
```
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
With the foundations of each error type implementing std::error::Error,
we can now nicely walk the `.source()` chain and print an error trace.
This commit introduces improved user-facing error printing when:
- Cloud Hypervisor fails with an error
- ch-remote fails (client error)
- ch-remote fails (remote error)
The additional context is a clear improvement in UX for both users and
developers. In the following example, the new behaviour is shown for
a direct invocation of Cloud Hypervisor leading to a failure. This
looks similar for ch-remote.
```
Old Style
`target/release/cloud-hypervisor --api-socket /tmp/chv2.sock --kernel /etc/bootitems/linux/kernel_minimal/stable.bzImage --cmdline console=ttyS0 --serial tty --console off --disk path=img.raw --initramfs /etc/bootitems/linux/initrd_minimal/default`
Error booting VM: VmBoot(LockingError(BlockError(LockDiskImage(AlreadyLocked)))
```
```
`target/release/cloud-hypervisor --api-socket /tmp/chv2.sock --kernel /etc/bootitems/linux/kernel_minimal/stable.bzImage --cmdline console=ttyS0 --serial tty --console off --disk path=img.raw --initramfs
/etc/bootitems/linux/initrd_minimal/default`
Error: Cloud Hypervisor exited with the following chain of errors:
0: Error booting VM
1: The VM could not boot
2: Error locking disk images: Another instance likely holds a lock
3: Cannot lock images of all block devices
4: Failed to get Write lock for disk image: ./img.raw
5: The file is already locked
Debug Info: VmBoot(VmBoot(LockingError(DiskLockError(LockDiskImage { error: AlreadyLocked, lock_type: Write, path: "./raw_disk.bin" })))
```
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The changes were mostly automatically applied using the following
Python script:
```python
import os, re
for root, _, files in os.walk("."):
for f in files:
if not f.endswith(".rs"):
continue
p = os.path.join(root, f)
with open(p, "r", encoding="utf-8") as file:
lines = file.readlines()
changed = False
for i in range(len(lines) - 1):
if re.search(r'#\[error\(".*: \{0[^}]*\}"\)\]', lines[i]) and "#[source]" in lines[i + 1].strip():
lines[i] = re.sub(r': \{0[^}]*\}"\)\]', '")]', lines[i])
changed = True
if changed:
with open(p, "w", encoding="utf-8") as file:
file.writelines(lines)
print("Fixed:", p)
```
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
# Conflicts:
# vmm/src/api/http/mod.rs
Having them sorted alphabetically makes more sense since there are
already many, and the list is growing. This improves the UX for users
and developers.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Instead of exiting on IO errors, report the errors to the guest with
VIRTIO_BLK_S_IOERR. For example, the guest kernel will log something
similar to this if the nbd behind /dev/vdc is unexpectedly disconnected:
[ 166.033957] I/O error, dev vdc, sector 264 op 0x1:(WRITE) flags 0x9800 phys_seg 1 prio class 2
[ 166.035083] Aborting journal on device vdc-8.
[ 166.037307] Buffer I/O error on dev vdc, logical block 9, lost sync page write
[ 166.038471] JBD2: I/O error when updating journal superblock for vdc-8.
[...]
[ 174.234470] EXT4-fs (vdc): I/O error while writing superblock
In case the rootfs is not located on the affected block device, this
will not crash the guest.
Fixes: #6995
Signed-off-by: Gauthier Jolly <contact@gjolly.fr>
This was caught by the nightly compiler during cargo fuzz build.
error: lifetime flowing from input to output with different syntax can be confusing
--> /home/runner/work/cloud-hypervisor/cloud-hypervisor/hypervisor/src/arch/x86/emulator/mod.rs:493:26
|
493 | pub fn new(platform: &mut dyn PlatformEmulator<CpuState = T>) -> Emulator<T> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ----------- the lifetime gets resolved as `'_`
| |
| this lifetime flows to the output
|
= note: `-D mismatched-lifetime-syntaxes` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(mismatched_lifetime_syntaxes)]`
help: one option is to remove the lifetime for references and use the anonymous lifetime for paths
|
493 | pub fn new(platform: &mut dyn PlatformEmulator<CpuState = T>) -> Emulator<'_, T> {
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
cargo fuzz build complaints about some un-used function in the
instruction emultator. Silence the warning by allowing dead code
generation.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since it is used by multiple components at this point, it is better to
move it to workspace level dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are a little many cpuid logs now. When starting a vm with
64 vcpu, we can get more than four thousand INFO messages:
cat vm1.log |grep 'arch/src/x86_64/mod.rs:891' |wc -l
4352
Signed-off-by: Yi Wang <foxywang@tencent.com>
This commit adds the warning to deprecate the SGX support with the
intention to remove the support from code base in two release cycles.
See: #6960
Signed-off-by: Bo Chen <bchen@crusoe.ai>
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
When booting a Linux guest in SMP configuration,
the following kernel warning can be observed:
[Firmware Bug]: CPUID leaf 0xb subleaf 1 APIC ID mismatch 1 != 0
The reason is that we announce the presence of the extended topology
leaf, but fail to announce the x2apic ID in EDX.
Signed-off-by: Thomas Prescher <thomas.prescher@cyberus-technology.de>
On-behalf-of: SAP thomas.prescher@sap.com
No sane guest OS will allow hotplugging all cpus.
However, the REST API currently allows specifying
`ch-remote resize --cpus 0`.
On Linux, we can then observe the following error in the kernel log:
processor cpu0: Offline failed.
Subsequent resize commands via ch-remote will then fail with
VcpuPendingRemovedVcpu because the removal of cpu0 was never
successful.
Fix this by disallowing resizing to zero vCPUs.
Signed-off-by: Thomas Prescher <thomas.prescher@cyberus-technology.de>
On-behalf-of: SAP thomas.prescher@sap.com
For use in QEMU, I would like GuestMemoryRegion to return a BitmapSlice
instead of a &Bitmap. This adds some flexibility that QEMU needs in
order to support a single global dirty bitmap that is sliced by the
various GuestMemoryRegions.
However, this removes access to the methods of AtomicBitmap, and in
particular reset() and get_and_reset(). Fortunately, cloud-hypervisor
always uses GuestMemoryMmap, and therefore `region` is known to be a
&GuestRegionMmap. Dereferencing it returns the MmapRegion to which the
bitmap is attached, thus calling MmapRegion::bitmap(); this has the
same effect as `<GuestRegionMmap as GuestRegion>::bitmap()`, and works
both with or without https://github.com/rust-vmm/vm-memory/pull/324.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Allow tap interfaces to be configured with an IPv6 address. The change
is fairly straightforward: we need to update the API types and CLI
parsing to accept either an IPv6 or IPv4 and then match on the IP
address type when the tap device is configured.
For IPv6 addresses, the netmask (prefix) must be provided at the same
time as the address itself (in the SIOCSIFADDR ioctl). They cannot be
configured separately. So we remove the separate "set_netmask" function
and convert "set_ip_addr" to also accept a netmask. For IPv4 addresses,
the IP address and netmask were already always set together, so this
should have no functional impact for users of IPv4 addresses.
Signed-off-by: Gregory Anders <ganders@cloudflare.com>
The DeviceManagerError type is among the types missing the Error trait
so far. To streamline the code and to simplify usage on higher levels
of this error type, this type now implements Display and Error.
As not all variant values are Error yet, `#[source]` is not everywhere
where it could be. This is done in the next commits.
The high level goal is: Enable future work to improve the error output
of cloud hypervisor.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The new locking behavior uncovered that unfortunate test situation:
Many tests running in parallel access the same disk image with
rw permissions. Luckily, none of the tests actually writes to
the disk. Therefore, we can set it to readonly=true. In case this
changes, the test needs to be moved to the sequential test module.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
# What
This commit introduces file-based advisory locking for the files backing
up the block devices by using the fcntl() syscall with OFD locks. The
per-open-file-descriptor (OFD) locks are more robust than traditional
POSIX locks (F_SETLK) as they are not tied to process IDs and avoid
common issues in multithreaded or multi-fd scenarios [1]. Therefore,
we don't use `std::fs::File::try_lock()`, which is backed by F_SETLKW.
The locking mechanism is aware of the `readonly` property and allows
`n` readers or `1` writer (exclusive mode).
As the locks are advisory, multiple cloud-hypervisor processes can
prevent themselves from writing to the same file. However, this is not
a system-wide file-system level locking mechanism preventing to open()
a file.
The introduced new locking mechanism does not cover vhost-user devices.
# Why
To prevent misconfiguration and improve safety, it is good practice to
protect disk image files with a locking mechanism. Experience and common
best practices suggest that advisory locks are preferable over mandatory
locks due to better compatibility and fewer pitfalls (in fs space).
The introduced functionality is aligned with the approach taken by
QEMU [0], and is also recommended in [1].
# Implementation Details
We need to ensure that not only normal operation keeps working but also
state save/resume and live-migration. Especially for live migration,
it is crucial that the sender VMM releases the locks when the VM stops
so the receiver VMM can acquire them right after that.
Therefore, the locking and releasing happen directly on the block
device struct. The device manager knows all block devices and can
forward requests to these types.
Last but not least, this commit uses on explicit lock acquiring
but implicit lock releasing (FD close). It only explicitly releases
the locks where this integrates more smoothly into the existing
code.
# Testing
I tested
- normal operation
- state save/resume,
- device hot plugging,
- and live-migration
with read/shared and write/exclusive locks.
One can use the `fcntl-tool` to test if locks are actually acquired
or released [2].
# Links
[0] 825b96dbce/util/osdep.c (L266)
[1] https://apenwarr.ca/log/20101213
[2] https://crates.io/crates/fcntl-tool
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
To support PCIe P2P between VFIO devices, we populate IOMMU mappings for
the non-emulated MMIO regions of all VFIO devices via
`VFIO_IOMMU_MAP_DMA` (f0c1f8d), but the patch did not properly update
the IOMMU mappings with BAR reprogramming.
Fixes: #7027
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The way how we handle PCI configuration space for vfio and vfio-user
devices are different from the rest of PCI devices. Besides accesses to
BAR registers (trapped to access the shadowing PCI config space we
maintained), accesses to other registers (including the COMMAND
register) are handled directly by the underline vfio or vfio-user
device.
This patch adds the proper handling of pending BAR reprogramming for
vfio and vfio-user devices.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The Memory Space Enable (MSE) bit from the COMMAND register in the
PCI configuration space controls whether a PCI device responds to memory
space accesses, e.g. read and write cycles to the device MMIO regions
defined by its BARs. The MSE bit is used by the device drivers to ensure
the correctness of BAR reprogramming. A common workflow is, the driver
first clears the MSE bit, then writes new values to the BAR registers,
and finally set the MSE bit to finish the BAR reprogramming.
This patch changes how we handle BAR reprogramming for all PCI
devices (e.g. virtio-pci, vfio, vfio-user, etc.), so that we follow the
same convention, e.g. moving PCI BARs when its MSE bit is set.
Note that some device drivers (such as edk2) only clear and set MSE once
while reprogramming multiple BARs of a single device. To support such
behavior, this patch adds support for multiple pending BAR reprogramming.
See: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7027#issuecomment-2853642959
Signed-off-by: Bo Chen <bchen@crusoe.ai>
A BAR reprogramming of a PCI device will only happen when the (guest)
kernel write to its PCI config space, e.g. the detection of bar
reprogramming (`detect_bar_repgraomming()`) can be embedded to the PCI
config space write (`write_config_register()`). It simplifies APIs
exposed by the `struct PciConfiguration` and `trait PciDevice`. It also
prepares for easier handling of pending bar reprogramming when the MSE
bit of the COMMAND register is not enabled at the time of changing BAR
registers.
See: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7027#issuecomment-2853642959
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Fixing the following clippy issue using `cargo clippy --fix`:
error: variables can be used directly in the `format!` string
--> build.rs:25:27
|
25 | version.push_str(&format!("-{}", extra_version));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Currently a lot of functions are stubbed out with unimplemented feature
tag. Add the missing implementation to successfully boot ARM64 guests on
MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Num SPI and Base SPI nodes should be added before the end node to be
included in the device tree generation.
Fixes: eac44e6 (arch: Extend FDT for GICv2M device for ARM64 on MSHV)
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
It seems like address allocation has been spread into different files
and different location for x86 vs ARM. This makes it hard to follow the
code. Thus, unify it a single location which satisfies all the
requirement.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
It just simplifies code and improves the code read-ability without much
affecting the boot performance.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This will satisfy the requirement of MSHV i.e., setting the GICD base
address before initializing the VM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For MSHV guests, we would need interrupt controller to be initialized
before the VM gets initialized. This is because we are registering the
base address of GIC distributor with MSHV as part of interrupt
controller initialization workflow. And MSHV mandates that this property
is set before we initialize the VM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For ARM64 guests we need to set three important partition property:
1) PPI interrupt ID for timer interrupt
2) PPI interrupt ID for PMU interrupts.
3) Hiding LPI support from the guest because MSHV does emulate ITS for
the guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since this would be used in other places inside the hypervisor and
hypervisor crate cannot take a dependency on arch crate, as that creates
cyclic dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently PPI interrupt ID are hardcoded as numbers, it would be ideal
to define them as constants and could be reused in other parts of the
hypervisor crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Along with also bump the vfio-bindings crates to use the latest
mshv-bindings.
There is a breaking change in the new mshv crate which requires an
additional step to initialize vm after creating it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
On x86 MSHV guests only used to support MSI based interrupts via IOAPIC
but ARM64 guests uses legacy interrupt for its functioning. Thus, extend
the logic to create routing entry to support legacy interrupts for ARM64
guests on MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For MSHV arm64 guest, there is an in-hypervisor GICv2M emulation and for
that to work, it needs to be enlightened with the base address of GIC
redistributor exposed to guest via FDT.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
GICv2M requires two additional properties to be exposed via FDT:
1) Base SPI number and 2) Total number of SPIs. SPIs in general starts
from 32 and goes upto 1019. But currently we are limiting the range to
96 as that should be good enough for any normal Linux guest to function.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since there are not a wide availability of RISC-V boards available at
the moment, it is easiest to develop with a QEMU virtual machine. I had
a hard time setting one up, but with the assistance of Ruoqing, I was
able to get one running. These are the steps I took to do so.
Signed-off-by: abm-77 <andrewmiller77@protonmail.com>
Ubuntu 24.04 LTS (Noble Numbat) is tested against cloud-hypervisor v45.0
release and it is working, document status in README.md.
Signed-off-by: Bingxin Li <bl497@cam.ac.uk>
ARM64 system register constants are not 1:1 mapped to MSHV definition of
those registers so we need a small helper function to translate that
mapping before retrieving those system registers.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are other potential users of these registers definitions in the
hypervisor crate. And hypervisor crate cannot use definitions from arch
crate because it creates cyclic dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
In order to be on-pair with what's we're using from micro-http, let's
also add the proper status code here as well (as it will be used by
`ch-remote`).
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Although the CPU manager gives us a quite descriptive error, on the
application side (the part calling Cloud Hypervisor) we have absolutely
no way to distinguish such error from any other error that may happen
when resizing a VM.
With this in mind, let's be more specific and return a TooManyRequests
(429) error, allowing the caller to have a chance to decide whether they
want to retry the operation or not.
https://datatracker.ietf.org/doc/html/rfc6585#section-4
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
As the coming patches in this series will take advantage of a status
code that was recently added there.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
New MSHV version updates the get_msr_list output as
vector instead of fam-wrapper. It avoids unnecessary
conversions.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Latest mshv crates contains some IOCTL changes that
enhances VM creation and configures the features in correct
way. Also adds some features that improves register access.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
MSHV allows VMM to map the VP register page into root.
This feature helps VMM to faster process most of the frequent
used registers. This patch uses the VP register page for port
handling in CPU run method.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
As part of this configure the program counter, pstate and X0 registers.
Program counter will point to the start address of the kernel/firmware
in the guest memory. X0 will point to start of the FDT.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
As part of this configuration, two things are being done:
1. Setting up the base address of GIC Distributor
2. Setting up the base address of GIC Interrupt Translator
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Initial PSTATE value would be same for both KVM and MSHV. Thus, move it
to common register definition pool.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are a lot of internal functions that are not and probably should
not be called from other places.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
If a device returns 0xff as a capability pointer bad things happen.
The code before the previous commits would crash in debug builds due
to integer overflow. With the two lowest bits masked out, it sends the
code into an endless loop.
Be more robust by at least handling the case where the capability
appears to point to itself.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
The PCI standard mandates that the lower bits of the capability
pointer are masked out before using the pointer. See PCI Local Bus
Specification 3.0 Chapter 6.7 "Capabilities List".
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Currently, the code tries to follow the PCI capabilities list in
offset 0x34 in the config space regardless of whether the status
registers says this is valid. Fix by adding the appropriate check.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
OSDev has cranked up its bot protection. The following link works for
me locally after clicking the "I'm a human" button. I guess the CI
fails this check...
Without this exception the CI fails the link check stage:
* [403] [https://wiki.osdev.org/IOAPIC](https://wiki.osdev.org/IOAPIC) | Network error: Forbidden
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Use the context from Unmapped Gpa exit from the hypervisor to initialize
the MshvEmulatorContext and later call the emulator to decode the
instruction.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently it would be using the syndrome register for instruction
decoding which is what KVM has been using in-kernel to decode
instructions for ARM64 guests. In future, it could be extended with an
actual instruction emulator if required. But most Linux guests works
well with the instruction decoder using syndrome register.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This helps in implementing an instruction decoder for MSVH which does
not support in-kernel instruction decoding like KVM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Some tests are specifically designed for KVM hypervisor platform. Thus,
guard them using appropriate feature flags.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are a bunch of unused variables as of now on the MSHV side and
compiler warns about them. Thus, mark them as unused for the time being.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
MSHV does not emulate a GICv3-ITS for guests to support MSI interrupts,
instead it exposes a GICv2m device. Currently adding a skeleton code
which would be modified later on with complete implementation.
With this we can start compiling cloud-hypervisor for MSHV on ARM64.
This will make sure that we don't regress in future in terms of basic
compilation test.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
`zerocopy` is bumped to 0.8.x after 0.3.4 of igvm crates, bump to rev
d062818 to capture `zerocopy` upgrade, but we should bump to 0.3.5
later.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually bump zerocopy to 0.8.24 since our dependabot could not perform
the upgrade properly.
Manually bump acpi_tabls as well since it's depending on zerocopy.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The dependency `bitfield-struct` 0.10.x of `igvm` 0.3.5 requires MSRV
1.83.0, bump to catch up.
Update image to 20250412-0 because MSRV in Dockerfile is updated.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This aligns with our CLI syntax. The correctness of `DiskConfig` will be
ensured via `VmConfig::validate()`, e.g. `path` and `socket` are
mutually exclusive.
Fixes: #7016
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Use the builtin function instead of using `==` operator.
Warning from the beta compiler:
error: use `std::ptr::eq` when comparing raw pointers
--> pci/src/vfio.rs:1616:24
if host_addr == libc::MAP_FAILED {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: try: `std::ptr::eq(host_addr, libc::MAP_FAILED)`
= help: for further information visit
= https://rust-lang.github.io/rust-clippy/master/index.html#ptr_eq
= note: `-D clippy::ptr-eq` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::ptr_eq)]`
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Use the builtin function to improve the readability of the code.
Warning from beta compiler:
error: manual arithmetic check found
--> vm-allocator/src/address.rs:151:30
|
|let adjust = if alignment > 1 { alignment - 1 } else { 0 };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| help: replace it with: `alignment.saturating_sub(1)`
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#implicit_saturating_sub
= note: `-D clippy::implicit-saturating-sub` implied by `-D warnings`
= help: to override `-D warnings` add`#[allow(clippy::implicit_saturating_sub)]`
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Rust has a new way of constructing other error and clippy complains if
we are still using the older way to construct error message. Thus,
migrate to the new approach suggested by the clippy.
Warning from beta compiler:
error: this can be `std::io::Error::other(_)`
--> block/src/vhdx/mod.rs:142:17
|
| / std::io::Error::new(
| | std::io::ErrorKind::Other,
| | format!("Failed to update VHDx header: {e}"),
| | )
| |_________________^
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#io_other_error
help: use `std::io::Error::other`
std::io::Error::other(
format!("Failed to update VHDx header: {e}"),
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
`serde_json` crate is referenced by multiple components, centralize it
to workspace to better manage this crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
KVM supports GICv3-ITS emulation and the current GicState is modelled
around the KVM implementation. We should refactor this to accomodate
other hypervisor requirements. For example, MSHV only support GICv2M
emulation for guests for delivering MSI interrupts instead of GICv3-ITS.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Add `riscv.md` to guide developers/users on preparing guest kernel,
image, and direct boot VM on riscv64 platforms; document support status
and known limitations.
Co-authored-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The larger the VM memory, the greater the memory pressure, and the
greater the stop_dirty_log() overhead. Moving stop_dirty_log() outside
the downtime period can reduce downtime.
Signed-off-by: Jinrong Liang <cloudliang@tencent.com>
`riscv64_set_one_reg_to_vcpu` macro is used to set value of specific
RISC-V `$reg_name` stored in `state` to KVM Vcpu.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`riscv64_get_one_reg_from_vcpu` macro is used to extract RISC-V register
data from KVM Vcpu according to `$reg_name` provided to `state`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`igvm` crates are referenced by multiple components, centralize them to
workspace to better manage those crates.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Certain MSHV ioctls are only available on x86 architecture. Thus,
conditionally compile seccomp filter for x86 and in general enable
seccomp filters when compiling for MSHV on aarch64.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The CLI has grown to a big variety of options. clap prints them in the
help message (--help) in the order they were defined. We now are at a
point where grouping things logically together doesn't work well.
Further, there is no support by clap for logical grouping and the
current code base wasn't consistent. Therefore, this commit introduces
two changes:
- a new structure to define arguments (all in an array)
- an alphabetical ordering of the arguments
No other changes have been made. No options have been altered.
This significantly improves:
- code maintainability and extensibility
- readability of the --help output
A unit test ensures they stay sorted. A better approach to check if the
list of arguments (known at build time) is sorted would be a compile
time check (`const`), but this currently isn't possible in stable Rust.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Use the definitions from the rust-vmm/mshv crate for various
datastructures such as StandardRegisters, RegList, VcpuInit etc.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
We are moving the metrics and rate-limiter workers to a bare-metal
system, so that we can have more consistent results (particularly for
the block device metrics)
Signed-off-by: Bo Chen <bchen@crusoe.ai>
v0.5.0 of `seccompiler` has supported riscv64, let's bump from v0.4.0 to
v0.5.0 to capture that support.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
In addition to the BDF information on the guest, reporting the device
path of the VFIO device on the host is more useful when it comes to
debug DMA map/unmap errors particularly ones caused by failing hardware
on the host.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
`test_snapshot_restore_with_fd` uses unsafe file descriptors and with
rust 1.82.0 it errors with:
```
fatal runtime error: IO Safety violation: owned file descriptor already
closed
```
so has been skipped for now.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We are having complains from Rust 1.86.0-beta.1 (f0cb41030 2025-02-17)'
clippy, which suggests us to replace `repeat().take()` with
`repeat_n()`. While `repeat_n()` is stablized in Rust 1.82.0.
Update image to 20250307-2 because MSRV in Dockerfile is updated.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
While executing `rustup toolchain add --profile minimal
1.82.0-x86_64-unknown-linux-musl`, it errors out with:
```
error: toolchain '1.82.0-x86_64-unknown-linux-musl' may not be able to
run on this system
note: add the `--force-non-host` flag to install the toolchain anyway
```
Add `--force-non-host` to install minimal musl toolchain for x86_64.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Currently we are just storing the StandardRegisters in the Vcpu state
which would be required for saving and restoring the ARM64 guest on
MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The reason `test_vfio_user` fails is as @likebreath pointed: our ARM
host does not support SVE, while the nvme_tgt binary built from the
container image requires it. As a result, we encountered a SIGILL when
running the nvme_tgt binary. This also explains why this is not
happening when the container is built on the same host itself.
And quote from @rbradford:
When a job is run on one of the workers it looks to see if there is a
container locally matching the name as specified in the dev_cli.sh
script - if there is then it uses it. Otherwise it will try and download
it from the container registry - if that fails then it will built
locally. For the x86-64 workers started dynamically it will never have a
local version as they are a fresh VM. But on the ARM64 builder is a
local container image cache.
This can lead to an issue where if the image is build with one version
(a handcrafted datestamp) and then the Dockerfile is changed without
changing the timestamp then an old version may be fetched from the cache
or server. It is there for essential to always bump the datestamp (there
is a number after the - that can be used for this.)
However there is also the added complexity that image that is build and
uploaded to the container registry is not the same as the built locally
and thus used for the initial testing of the Dockerfile change. This
leads to the issue we have seen where different CPU compiler flags (from
-march=native) from the QEMU cross build in the hosted GHA action and
the local ARM64 build. Resulting in a binary in the remotely built
container not working locally.
We end up specifying TARGET_ARCHITECTURE="armv8.2-a" for building spdk,
and put built `python/spdk/` folder into `/usr/local/bin/spdk-nvme`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`arm64` build in ubuntu:22.04 errors out with `error processing package
libc-bin`. This issue is a known issue between the binfmt (running
different architectures via QEMU) and the libc ldconfig binary running
in container. We're "suddenly" having issues as ubuntu-latest (which is
the OS version we run the GH action container with) was recently changed
from 22.04 to 24.04 and hence why upgrading the container userspace from
22.04 to 24.04 solves the problem.
Removed deprecated package `python3-distutils`.
Update image name from `20250111-0` to `20250222-0`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Currently a bunch of KVM specific interfaces are leaked into the vmm
crate which should ideally does not contain any hypervisor specific data
structures.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Use `lychee` to check availability of links in cloud-hypervisor.
The urls explictly excluded in config file are manually checked.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We have lost track to releases before v27.0 since these projects no
longer exists. Delete links to those projects.
Update links to a detailed view specific to each group of release.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Previous repo hosts details of `Rust Style` is removed, point `Rust
Style` to `style-guide` in `rust-lang/rust` repo.
Link to provide illustration on `signed-off-by language` is also
removed, use a snapshot found in web archive [1] instead.
Format `CONTRIBUTING.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
[1] https://web.archive.org
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
TDX homepage was moved to elsewhere, and `sgx` support is upstreamed
since v5.11 kernel.
Format `intel_sgx.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
TDX homepage was moved to elsewhere, and `tdx-tools` repo was removed.
Provide a valid link of TDX homepage and change all reference to
`tdx-tools` to `tdx-linux`.
Format `intel_tdx.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
There is a link referencing `rate-limiter` module of `firecracker`, but
that module no longer exsits.
Point the link to a commit with the same date in `firecracker` when this
commit was merged to `cloud-hypervisor`.
Format `io_throttling.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`docs/arm64.md` was removed and splited into `README.md`, `building.md`
and `uefi.md` in #4991.
Let's point it to
`8ab15b9a98/docs/arm64.md`
the commit right before `docs/arm64.md` was removed in main branch.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Previous link `docs/logging` is not valid, replacing `docs/logging` with
`docs/logging.md`.
Format `logging.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Use this to add any extra versioning information to the binary. It is
useful when packaging Cloud Hypervisor.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2025-02-04 19:33:10 +00:00
282 changed files with 16649 additions and 10221 deletions
We follow the [Rust Style](https://github.com/rust-dev-tools/fmt-rfcs/blob/master/guide/guide.md)
We follow the [Rust Style](https://github.com/rust-lang/rust/tree/HEAD/src/doc/style-guide/src)
convention and enforce it through the Continuous Integration (CI) process calling into `rustfmt`
for each submitted Pull Request (PR).
@@ -36,7 +36,7 @@ commit you make.
## Certificate of Origin
In order to get a clear contribution chain of trust we use the [signed-off-by language](https://01.org/community/signed-process)
In order to get a clear contribution chain of trust we use the [signed-off-by language](https://web.archive.org/web/20230406041855/https://01.org/community/signed-process)
used by the Linux kernel project.
## Patch format
@@ -79,12 +79,12 @@ you want to merge your changes to `cloud-hypervisor`:
1. Fork the [cloud-hypervisor](https://github.com/cloud-hypervisor/cloud-hypervisor) project
into your github organization.
2. Within your fork, create a branch for your contribution.
3. [Create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/)
1. Within your fork, create a branch for your contribution.
1. [Create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/)
against the main branch of the Cloud Hypervisor repository.
4. To update your pull request amend existing commits whenever applicable and
1. To update your pull request amend existing commits whenever applicable and
then push the new changes to your pull request branch.
5. Once the pull request is approved it can be integrated.
1. Once the pull request is approved it can be integrated.
The `fw_cfg` device is a QEMU-compatible device that allows the hypervisor to pass configuration and data to the guest operating system. This is particularly useful for firmware to access information like ACPI tables, kernel images, initramfs, kernel command lines, and other arbitrary data blobs.
Cloud Hypervisor implements the `fw_cfg` device with DMA-enabled access.
## Purpose
The `fw_cfg` device serves as a generic information channel between the VMM and the guest. It can be used to:
* Load the kernel, initramfs, and kernel command line for direct kernel boot with firmware.
* Provide ACPI tables to the guest firmware or OS.
* Pass custom configuration files or data blobs (e.g., attestation data, SEV-SNP launch secrets) to the guest.
* Supply an E820 memory map to the guest.
## Enabling `fw_cfg`
The `fw_cfg` device is enabled via the `fw_cfg` feature flag when building Cloud Hypervisor:
```bash
cargo build --features fw_cfg
```
## Guest Kernel Configuration
For the guest Linux kernel to recognize and use the `fw_cfg` device via sysfs, the following kernel configuration option must be enabled:
*`CONFIG_FW_CFG_SYSFS=y`
This option allows the kernel to expose `fw_cfg` entries under `/sys/firmware/qemu_fw_cfg/by_name/`.
## Command Line Options
The `fw_cfg` device is configured using the `--fw-cfg-config` command-line option.
**Parameters:**
*`e820=on|off`: (Default: `on`) Whether to add an E820 memory map entry to `fw_cfg`.
*`kernel=on|off`: (Default: `on`) Whether to add the kernel image (specified by `--kernel`) to `fw_cfg`.
*`cmdline=on|off`: (Default: `on`) Whether to add the kernel command line (specified by `--cmdline`) to `fw_cfg`.
*`initramfs=on|off`: (Default: `on`) Whether to add the initramfs image (specified by `--initramfs`) to `fw_cfg`.
*`acpi_table=on|off`: (Default: `on`) Whether to add generated ACPI tables to `fw_cfg`.
*`items=[... : ...]`: A list of custom key-value pairs to be exposed via `fw_cfg`.
*`name=<guest_sysfs_path>`: The path under which the item will appear in the guest's sysfs (e.g., `opt/org.example/my-data`).
*`file=<host_file_path>`: The path to the file on the host whose content will be provided to the guest for this item.
**Example Usage:**
1.**Direct kernel boot with custom `fw_cfg` entries:**
The `raw` file contains the binary content of the host file provided.
Standard items like kernel, initramfs, cmdline, and ACPI tables also have predefined names (e.g., `etc/kernel`, `etc/cmdline`) if they are enabled to be passed via `fw_cfg`.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
