Take the recent thread refactor further. Bundle a device's worker
JoinHandles together with the kill event that stops them into a single
WorkerThreads value, owned by VirtioCommon. Its Drop signals the workers
to exit, unparks any parked for migration, and joins them.
This makes a detached/leaked worker unrepresentable. reset(),
wait_for_epoll_threads() and VhostUserCommon::shutdown() now happen when
dropping the WorkerThreads, and the unpark-before-join teardown now
lives in one place.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Rob Bradford <rbradford@meta.com>
Replace use of spawn_virtio_thread() helper with the new method on
VirtioCommon to handle thread management as well as spawning. As a
result this cleanly handles reset if it fails to spawn the thread.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add check_range calls on request and status descriptor addresses to
reject buffers that extend past guest memory.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
IommuEpollHandler::request_queue() can fail because the guest put in a
bad request or because of a fatal error. Handle those cases differently,
letting the guest continue, but see the error if it can.
This makes debugging from the guest easier as one mistake doesn't cause
a VM reset if it's avoidable.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Worker threads spawned through spawn_virtio_thread previously wrote to
exit_evt on any clean Err return, taking the whole VMM down on a single
failed device worker. A guest induced fault in any virtio device thus
propagated into a host wide failure.
Route the Err return through the shared mark_device_needs_reset helper
instead. The helper sets the DEVICE_NEEDS_RESET bit on device_status
and triggers a config change interrupt, so the device goes idle and
the guest is informed. The thread exits cleanly without killing the
rest of the VMM.
The panic and the seccomp filter apply paths keep writing to exit_evt.
A panicked worker may have left poisoned locks or partially mutated
state, so a hard exit remains the right policy there.
spawn_virtio_thread now takes the device_status and the interrupt
callback. Every native virtio and vhost-user call site is updated to
pass them in.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The per-domain mapping cap bounds memory inside one domain, but a
guest can still grow the domains map indefinitely with ATTACH
requests for distinct domain IDs. Reject ATTACH with
VIRTIO_IOMMU_S_NOMEM at 64K domains.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The virtio spec says the device must never present a value other
than 0 or 1 for bypass. Mask off the upper bits on write so a
later read does not return whatever the driver wrote.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The device offers VIRTIO_IOMMU_F_INPUT_RANGE when the address
width is constrained, but never validates that guest MAP requests
fall inside the advertised range. The virtio spec requires such
requests to fail with VIRTIO_IOMMU_S_RANGE.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The virtio spec mandates VIRTIO_IOMMU_S_NOENT, not S_INVAL, when
the target domain does not exist.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The handler accepted unknown flag bits, unaligned ranges, and
overlapping mappings, and returned the wrong status code when the
target domain did not exist. The virtio spec requires explicit
rejections for each of these.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The virtio spec requires the device to reject ATTACH with a
non-zero reserved field, an unknown flag bit, or a bypass flag
that conflicts with an existing domain. The current handler
silently accepts all three.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The virtio spec requires the device to leave the reply buffer
untouched and report a used length of zero for an unrecognised
request type, so the driver can tell the request was not handled.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
Domain::mappings only shrinks on UNMAP. Without a bound a guest can
issue MAP for arbitrarily many distinct virt_start values and drive
the VMM heap until the host runs out.
Reject MAP with VIRTIO_IOMMU_S_NOMEM at 1M entries per domain.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
A concurrent DETACH between the read-lock check and the write-lock
get_mut().unwrap() in MAP/UNMAP would panic the worker. Replace the
unwrap with a let-else.
A failure on a later endpoint in the per-endpoint MAP loop, or a
DETACH that races the missing-domain branch, must roll back the
external mappings already installed; otherwise domain.mappings
diverges from VFIO state.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
An UNMAP that would split an existing mapping must be rejected with
VIRTIO_IOMMU_S_RANGE without removing anything. The previous start-only
retain silently left mappings that started outside the unmap range but
overlapped it.
Walk bookkeeping under a read lock and reject before touching VFIO so a
rejection cannot leave VFIO and bookkeeping out of sync.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The DmaRemapping translate_gva and translate_gpa entry points discarded
the size argument that AccessPlatform's signature already carries and
only checked the base address. A buffer beginning inside a mapping but
extending past it was treated as fully translated, allowing reads or
writes outside the IOMMU-authorized window.
Add `size` to the trait, validate the full span fits in a single
mapping, and propagate it through AccessPlatformMapping.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
The reply length was `hdr_len + size_of::<tail>()`, computed twice.
Make it explicit via checked_add and reuse the result.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:claude-opus-4-7
Catch u64 overflows on map so that later translation requests from the
guest don't have a vector for causing a host panic.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Previously the interrupt was created in VirtioPciDevice, moved via the
Option::take() to the VirtioPciDeviceActivator and then moved to the
VirtioDevice upon activation. On reset it would be moved back ready for
reactivation.
Since this already an Arc type remove the wrapping Option and instead
refcount it such that the VirtioPciDevice can continue to hold onto it
for later activations.
This significantly simplifies the reset() logic as there is no need to
hand back the interrupt.
A few devices used whether the interrupt was Some to make triggering an
interrupt a no-op. However the MSI-X interrupt routing already drops the
interrupt if the driver hasn't yet configured the vector so it is safe
to trigger the interrupt before device activation (e.g. balloon resize
request before driver loaded).
VirtioCommon still retains an Option<..> for the interrupt as the
interrupt is not known until activation time (after this has been
created). A helper VirtioCommon::trigger_interrupt() has been added to
handle this.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Fix the _reserved field in VirtioIommuConfig from [u8; 7] to
[u8; 3], correcting the struct size from 44 bytes to the
spec-mandated 40 bytes.
The virtio specification v1.2, Section 5.13.4 defines struct
virtio_iommu_config as 40 bytes total.
The kernel UAPI header linux/virtio_iommu.h matches this
layout with __u8 reserved[3] since kernel 5.17. Prior to that,
the struct was 36 bytes with no bypass field at all.
The incorrect [u8; 7] made the packed struct 44 bytes. Since
the struct is exposed to the guest, the guest saw a 44 byte
device specific configuration region instead of 40 bytes.
While well behaved guest drivers only access fields at known
offsets and would not observe data corruption from the extra
4 zero bytes at the tail, the oversized config region is a
spec violation.
The write_config path is not affected because it validates
the exact offset of the bypass field before allowing writes,
and the bypass field sits at offset 36 regardless of the
trailing reserved size.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This is a follow-up of [0].
# Advantages
- This saves dozens of unneeded clone()s across the whole code base
- Makes it much easier to reason about how parameters are used
(often we passed owned Arc/Rc versions without actually needing
ownership)
# Exceptions
For certain code paths, the alternatives would require awkward or overly
complex code, and in some cases the functions are the logical owners of
the values they take. In those cases, I've added
#[allow(clippy::needless_pass_by_value)].
This does not mean that one should not improve this in the future.
[0] 6a86c157af
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Consuming `&Arc<T>` as argument is almost always an antipattern as it
hides whether the callee is going to take over (shared) ownership
(by .clone()) or not. Instead, it is better to consume `&dyn T` or
`Arc<dyn T>` to be more explicit. This commit cleans up the code.
The change is very mechanic and was very easy to implement across the
code base.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Fix clippy warning `uninlined_format_args` reported by rustc rustc
1.89.0 (29483883e 2025-08-04).
```console
warning: variables can be used directly in the `format!` string
--> block/src/lib.rs:649:17
|
649 | info!("{} failed to create io_uring instance: {}", error_msg, e);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
= note: `#[warn(clippy::uninlined_format_args)]` on by default
help: change this to
|
649 - info!("{} failed to create io_uring instance: {}", error_msg, e);
649 + info!("{error_msg} failed to create io_uring instance: {e}");
|
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Rust has a new way of constructing other error and clippy complains if
we are still using the older way to construct error message. Thus,
migrate to the new approach suggested by the clippy.
Warning from beta compiler:
error: this can be `std::io::Error::other(_)`
--> block/src/vhdx/mod.rs:142:17
|
| / std::io::Error::new(
| | std::io::ErrorKind::Other,
| | format!("Failed to update VHDx header: {e}"),
| | )
| |_________________^
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#io_other_error
help: use `std::io::Error::other`
std::io::Error::other(
format!("Failed to update VHDx header: {e}"),
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently, Cloud Hypervisor does not set a VIRTIO_IOMMU_F_INPUT_RANGE
feature bit for the VirtIO IOMMU device, which, according to spec[1],
means that the guest may use the whole 64-bit address space is for
IOMMU purposes:
>If the feature is not offered, virtual mappings span over the whole
>64-bit address space (start = 0, end = 0xffffffff ffffffff)
As far as I am aware, there are currently no host platforms on
the market capable of addressing the whole 64-bit address space.
For example, I am currently working with a host platform that reports
39-bit address space for IOMMU purposes:
>DMAR: Host address width 39
When running a VFIO pass-through guest on such a platform, NVIDIA
driver in guest gets DMA mapping failures when working with large data,
and this results in Cloud Hypervisor exiting with the following error:
>cloud-hypervisor: 1501.220535s: <__iommu>
>ERROR:virtio-devices/src/thread_helper.rs:53 -- Error running worker:
>HandleEvent(Failed to process request queue : ExternalMapping(Custom
>{ kind: Other, error: "failed to map memory for VFIO container, iova
>0x7fff00000000, gpa 0x24ce25000, size 0x1000: IommuDmaMap(Error(22))"
>}))
Passing "--platform iommu_address_width=39" to Cloud Hypervisor built
with this change fixes this.
[1]: https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/
virtio-v1.3-csd01.html#x1-5420006
Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>
Remove an erroneous optimisation that used the page size mask to reduce
the range to iterate through on the set of mappings. This doesn't work
as the virtio-iommu ranges are larger than a single page. This may have
worked in the past when the mappings were limited to a single page.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
By introducing `imports_granularity="Module"` format strategy,
effectively groups imports from the same module into one line or block,
improving maintainability and readability.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Historically the Cloud Hypervisor coding style has been to ensure that
all imports are ordered and placed in a single group. Unfortunately
cargo fmt has no support for ensuring that all imports are in a single
group so if whitespace lines were added as part of the import statements
then they would only be odered correctly in the group.
By adopting "group_imports="StdExternalCrate" we can enforce a style
where imports are placed in at most three groups for std, external
crates and the crate itself. Choosing a style enforceable by the tooling
reduces the reviewer burden.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Properly detach a device from a domain if that device is already
attached to another domain on an attach request (following section
5.13.6.3.2 of the virtio-iommu spec). Resolves nested virtualization
reboot.
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
Ensures that any endpoints already attached to the domain are properly
mapped to a new endpoint on said endpoint's attach request. This is done
by search for all previous mappings in the domain and then issuing map
requests for the newly attached endpoint.
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
According to the virtio iommu spec (section 5.13.6.6), all mappings
within the entire range from virt_start to virt_end in an unmap
request must be removed. This change adds this functionality,
iterating through all mappings that fall within an unmap request
for that domain and removing them.
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
This change is important to do a proper resource cleanup. We decided
to do this repetitive approach as VirtioCommon can't implement Drop
without major changes to the corresponding code. Also, devices such as
Net can't easily use the epoll_threads-abstraction from VirtioCommon as
it has multiple threads with different semantics.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
The information about the identifier related to a Snapshot is only
relevant from the BTreeMap perspective, which is why we can get rid of
the duplicated identifier in every Snapshot structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
