This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Allow tap interfaces to be configured with an IPv6 address. The change
is fairly straightforward: we need to update the API types and CLI
parsing to accept either an IPv6 or IPv4 and then match on the IP
address type when the tap device is configured.
For IPv6 addresses, the netmask (prefix) must be provided at the same
time as the address itself (in the SIOCSIFADDR ioctl). They cannot be
configured separately. So we remove the separate "set_netmask" function
and convert "set_ip_addr" to also accept a netmask. For IPv4 addresses,
the IP address and netmask were already always set together, so this
should have no functional impact for users of IPv4 addresses.
Signed-off-by: Gregory Anders <ganders@cloudflare.com>
The DeviceManagerError type is among the types missing the Error trait
so far. To streamline the code and to simplify usage on higher levels
of this error type, this type now implements Display and Error.
As not all variant values are Error yet, `#[source]` is not everywhere
where it could be. This is done in the next commits.
The high level goal is: Enable future work to improve the error output
of cloud hypervisor.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The new locking behavior uncovered that unfortunate test situation:
Many tests running in parallel access the same disk image with
rw permissions. Luckily, none of the tests actually writes to
the disk. Therefore, we can set it to readonly=true. In case this
changes, the test needs to be moved to the sequential test module.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
# What
This commit introduces file-based advisory locking for the files backing
up the block devices by using the fcntl() syscall with OFD locks. The
per-open-file-descriptor (OFD) locks are more robust than traditional
POSIX locks (F_SETLK) as they are not tied to process IDs and avoid
common issues in multithreaded or multi-fd scenarios [1]. Therefore,
we don't use `std::fs::File::try_lock()`, which is backed by F_SETLKW.
The locking mechanism is aware of the `readonly` property and allows
`n` readers or `1` writer (exclusive mode).
As the locks are advisory, multiple cloud-hypervisor processes can
prevent themselves from writing to the same file. However, this is not
a system-wide file-system level locking mechanism preventing to open()
a file.
The introduced new locking mechanism does not cover vhost-user devices.
# Why
To prevent misconfiguration and improve safety, it is good practice to
protect disk image files with a locking mechanism. Experience and common
best practices suggest that advisory locks are preferable over mandatory
locks due to better compatibility and fewer pitfalls (in fs space).
The introduced functionality is aligned with the approach taken by
QEMU [0], and is also recommended in [1].
# Implementation Details
We need to ensure that not only normal operation keeps working but also
state save/resume and live-migration. Especially for live migration,
it is crucial that the sender VMM releases the locks when the VM stops
so the receiver VMM can acquire them right after that.
Therefore, the locking and releasing happen directly on the block
device struct. The device manager knows all block devices and can
forward requests to these types.
Last but not least, this commit uses on explicit lock acquiring
but implicit lock releasing (FD close). It only explicitly releases
the locks where this integrates more smoothly into the existing
code.
# Testing
I tested
- normal operation
- state save/resume,
- device hot plugging,
- and live-migration
with read/shared and write/exclusive locks.
One can use the `fcntl-tool` to test if locks are actually acquired
or released [2].
# Links
[0] 825b96dbce/util/osdep.c (L266)
[1] https://apenwarr.ca/log/20101213
[2] https://crates.io/crates/fcntl-tool
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
To support PCIe P2P between VFIO devices, we populate IOMMU mappings for
the non-emulated MMIO regions of all VFIO devices via
`VFIO_IOMMU_MAP_DMA` (f0c1f8d), but the patch did not properly update
the IOMMU mappings with BAR reprogramming.
Fixes: #7027
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The way how we handle PCI configuration space for vfio and vfio-user
devices are different from the rest of PCI devices. Besides accesses to
BAR registers (trapped to access the shadowing PCI config space we
maintained), accesses to other registers (including the COMMAND
register) are handled directly by the underline vfio or vfio-user
device.
This patch adds the proper handling of pending BAR reprogramming for
vfio and vfio-user devices.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The Memory Space Enable (MSE) bit from the COMMAND register in the
PCI configuration space controls whether a PCI device responds to memory
space accesses, e.g. read and write cycles to the device MMIO regions
defined by its BARs. The MSE bit is used by the device drivers to ensure
the correctness of BAR reprogramming. A common workflow is, the driver
first clears the MSE bit, then writes new values to the BAR registers,
and finally set the MSE bit to finish the BAR reprogramming.
This patch changes how we handle BAR reprogramming for all PCI
devices (e.g. virtio-pci, vfio, vfio-user, etc.), so that we follow the
same convention, e.g. moving PCI BARs when its MSE bit is set.
Note that some device drivers (such as edk2) only clear and set MSE once
while reprogramming multiple BARs of a single device. To support such
behavior, this patch adds support for multiple pending BAR reprogramming.
See: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7027#issuecomment-2853642959
Signed-off-by: Bo Chen <bchen@crusoe.ai>
A BAR reprogramming of a PCI device will only happen when the (guest)
kernel write to its PCI config space, e.g. the detection of bar
reprogramming (`detect_bar_repgraomming()`) can be embedded to the PCI
config space write (`write_config_register()`). It simplifies APIs
exposed by the `struct PciConfiguration` and `trait PciDevice`. It also
prepares for easier handling of pending bar reprogramming when the MSE
bit of the COMMAND register is not enabled at the time of changing BAR
registers.
See: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7027#issuecomment-2853642959
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Fixing the following clippy issue using `cargo clippy --fix`:
error: variables can be used directly in the `format!` string
--> build.rs:25:27
|
25 | version.push_str(&format!("-{}", extra_version));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Currently a lot of functions are stubbed out with unimplemented feature
tag. Add the missing implementation to successfully boot ARM64 guests on
MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Num SPI and Base SPI nodes should be added before the end node to be
included in the device tree generation.
Fixes: eac44e6 (arch: Extend FDT for GICv2M device for ARM64 on MSHV)
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
It seems like address allocation has been spread into different files
and different location for x86 vs ARM. This makes it hard to follow the
code. Thus, unify it a single location which satisfies all the
requirement.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
It just simplifies code and improves the code read-ability without much
affecting the boot performance.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This will satisfy the requirement of MSHV i.e., setting the GICD base
address before initializing the VM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For MSHV guests, we would need interrupt controller to be initialized
before the VM gets initialized. This is because we are registering the
base address of GIC distributor with MSHV as part of interrupt
controller initialization workflow. And MSHV mandates that this property
is set before we initialize the VM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For ARM64 guests we need to set three important partition property:
1) PPI interrupt ID for timer interrupt
2) PPI interrupt ID for PMU interrupts.
3) Hiding LPI support from the guest because MSHV does emulate ITS for
the guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since this would be used in other places inside the hypervisor and
hypervisor crate cannot take a dependency on arch crate, as that creates
cyclic dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently PPI interrupt ID are hardcoded as numbers, it would be ideal
to define them as constants and could be reused in other parts of the
hypervisor crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Along with also bump the vfio-bindings crates to use the latest
mshv-bindings.
There is a breaking change in the new mshv crate which requires an
additional step to initialize vm after creating it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
On x86 MSHV guests only used to support MSI based interrupts via IOAPIC
but ARM64 guests uses legacy interrupt for its functioning. Thus, extend
the logic to create routing entry to support legacy interrupts for ARM64
guests on MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For MSHV arm64 guest, there is an in-hypervisor GICv2M emulation and for
that to work, it needs to be enlightened with the base address of GIC
redistributor exposed to guest via FDT.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
GICv2M requires two additional properties to be exposed via FDT:
1) Base SPI number and 2) Total number of SPIs. SPIs in general starts
from 32 and goes upto 1019. But currently we are limiting the range to
96 as that should be good enough for any normal Linux guest to function.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since there are not a wide availability of RISC-V boards available at
the moment, it is easiest to develop with a QEMU virtual machine. I had
a hard time setting one up, but with the assistance of Ruoqing, I was
able to get one running. These are the steps I took to do so.
Signed-off-by: abm-77 <andrewmiller77@protonmail.com>
Ubuntu 24.04 LTS (Noble Numbat) is tested against cloud-hypervisor v45.0
release and it is working, document status in README.md.
Signed-off-by: Bingxin Li <bl497@cam.ac.uk>
ARM64 system register constants are not 1:1 mapped to MSHV definition of
those registers so we need a small helper function to translate that
mapping before retrieving those system registers.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are other potential users of these registers definitions in the
hypervisor crate. And hypervisor crate cannot use definitions from arch
crate because it creates cyclic dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
In order to be on-pair with what's we're using from micro-http, let's
also add the proper status code here as well (as it will be used by
`ch-remote`).
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Although the CPU manager gives us a quite descriptive error, on the
application side (the part calling Cloud Hypervisor) we have absolutely
no way to distinguish such error from any other error that may happen
when resizing a VM.
With this in mind, let's be more specific and return a TooManyRequests
(429) error, allowing the caller to have a chance to decide whether they
want to retry the operation or not.
https://datatracker.ietf.org/doc/html/rfc6585#section-4
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
As the coming patches in this series will take advantage of a status
code that was recently added there.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
