pause() returns DeviceDisconnected without calling into the backend when
VhostUserCommon already knows the socket is gone. DeviceManager treats
only that sentinel as log-and-continue, so one dead vhost-user device
does not abort the whole pause iteration.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Add common PCI device configuration to the virtio-console device
configuration. This allows setting the device ID (the name), ID, the
PCI segment, and the PCI device ID (BDF), which were previously not
configurable for the virtio-console device.
This gives management software, such as libvirt, more control over PCI
resource assignment and aligns virtio-console with other devices that
already support this functionality [0].
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/issues/8175
On-behalf-of: Philipp Schuster <philipp.schuster@sap.com>
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
This commit introduces a new struct `CommonConsoleConfig` which is the
base for the split into `ConsoleConfig` and `SerialConfig`. This is a
pre-requisite for allowing more configurable PCI options for the
virtio-console device.
The commit doesn't change or add any functionality.
On-behalf-of: Philipp Schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
The PciDeviceHandle::VfioUser arm in eject_device propagated dma_unmap
failures with ?, which short-circuited the subsequent
remove_dma_mapping_handler loop and left stale Arc<VfioUserDmaMapping>
entries in every virtio-mem device's handler map. Log unmap errors
with warn! and continue so the handler map cleanup always runs.
This only happens if a vfio-user process crashes and the same device is
later removed. However, given we've seen similar issues on vhost-user
this is probably worth cleaning up.
Signed-off-by: Dylan Reid <dgreid@fb.com>
The BusDevice read and write arms for B0EJ_FIELD_OFFSET and
PSEG_FIELD_OFFSET opened with assert!/assert_eq! on data.len(), so a
guest 1/2/8-byte MMIO access to either register panicked the vCPU
thread. Replace each assert with a warn! and early return so unusual
access widths are logged and ignored instead of crashing the VMM.
Signed-off-by: Dylan Reid <dgreid@fb.com>
self.balloon and self.virtio_mem_devices are not updated when
eject_device removes the underlying device, leaving stale
Arc<Mutex<...>> entries that resize_balloon / balloon_size and the
virtio-mem DMA-handler iteration would dereference if reached after
eject. Clear self.balloon and retain-out the matching virtio-mem entry
in the PciDeviceHandle::Virtio eject arm, identifying the ejected
device by Arc pointer-equality against the already-resolved
Arc<Mutex<dyn VirtioDevice>>.
This change is defensive: DeviceManager::remove_device currently
rejects VirtioDeviceType::Balloon and VirtioDeviceType::Mem with
RemovalNotAllowed before pci_devices_down is set, so the guest never
sees an eject notification and eject_device is never reached for
either type today. If the allowlist is extended later, this cleanup
keeps the post-eject state consistent.
Signed-off-by: Dylan Reid <dgreid@fb.com>
The old implementation used an ever monotonically increasing u32 counter
to allocate new GSIs. The counter increased every time a new GSI was
allocated, and freeing GSIs was not possible. Thus, Cloud Hypervisor
can run out of GSIs and panics. This currently happened at the 1024th
GSI [0]. Further, this caused the `KVM_SET_GSI_ROUTING` ioctl to carry
much more payload than needed.
This new implementation uses a bitmap for proper tracking of resources
and can gracefully free GSIs - this is abstracted in type
InterruptAllocator.
Please note that this commit only replaces the old mechanism. The next
commit will introduce freeing used GSIs automatically when an
InterruptRoute is dropped.
While being on this, we also propagate the errors that the allocator may
throw where necessary.
Co-authored-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
On-behalf-of: Philipp Schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Allow VFIO devices to list BAR indices that should not be
mmapped into the guest. This lets operators skip large BARs that
are known not to be used by their workload.
When a BAR is skipped, the log also calls out that P2P DMA
mapping is skipped because the VFIO DMA map path uses the same
mmap backing.
Signed-off-by: Damian Barabonkov <dbctl@pm.me>
Assisted-by: OpenCode:gpt-5.5
Rather than calculate in the DeviceManager and pass it through do it in
the device where it already has all the required information.
Signed-off-by: Rob Bradford <rbradford@meta.com>
All disk format backends now implement AsyncFullDiskFile directly.
The DiskBackend enum that dispatched between Legacy and Next arms
is no longer needed since the factory returns trait objects and vmm
no longer constructs format types manually.
Replace DiskBackend with Box<dyn AsyncFullDiskFile> in the Block
struct and its constructor. Remove the DiskBackend::Next wrapping
in device_manager and the fuzz target.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Replace the manual match block that constructed each disk format
backend with a single call to block::factory::open_disk. The factory
handles file opening, format detection, async/sync backend selection
and logging internally.
Remove imports and errors for individual format types and helper
functions that are no longer called directly. The factory returns
BlockError with path and operation context attached, surfaced via
the existing Disk variant.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
DeviceManager and VfioPciDevice both hold Arc<MmapRegion> for
each VFIO BAR mmap window. During VM shutdown, VfioPciDevice
drops after DeviceManager::Drop::drop (via device_tree
field drop). Without clearing DeviceManager's clones first,
VfioPciDevice::unmap_mmio_regions decrements the Arc but
does not reach zero, munmap never fires, the VFIO device
file VMAs survive, and VFIO_GROUP_UNSET_CONTAINER returns
EBUSY.
Clear DeviceManager's mmio_regions in Drop::drop so
VfioPciDevice is the sole Arc owner at drop time and ensure
VFIO_GROUP_UNSET_CONTAINER ioctl success.
Remove redundant .clone() on the mmio_regions() return value
in the eject_device() hot-unplug path.
Add detail comments
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Boot-time block devices on PCI segment 0 use 32-bit BARs so early
firmware can access them without additional identity mapping in the
firmware page tables. However, hot-plugged block devices are only ever
seen by the OS kernel which handles 64-bit BARs natively.
Switch hot-plugged block devices to 64-bit BARs to avoid exhausting the
scarce 32-bit MMIO window (typically 2-3 GB between RAM and 4 GB) when
many devices are hot-plugged.
Extract the BAR sizing decision into use_64bit_bar_for_virtio_device()
and thread an is_hotplug flag through add_virtio_pci_device(). Add unit
tests covering all relevant combinations.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Introduce the SevFd abstraction that wraps /dev/sev and implements the
KVM_SEV_INIT2 and KVM_SEV_SNP_LAUNCH_START ioctls for SEV-SNP VM
initialization on KVM.
Key changes:
- Add sev.rs with KvmSevInit and KvmSevSnpLaunchStart ioctl structs
matching the kernel layout (linux/arch/x86/include/uapi/asm/kvm.h)
- Implement KVM_SEV_INIT2 and KVM_SEV_SNP_LAUNCH_START ioctls
- Set KVM_MEMORY_ATTRIBUTE_PRIVATE on newly created memory regions
when guest_memfd is supported
- Widen SevSnpPageAccessProxy cfg gates from mshv-only to all
sev_snp-enabled builds
- Make sev_snp_init a required trait method (remove default impl)
- Include KVM_SEV_SNP_LAUNCH_START in the seccomp allowlist
- Parse VMSA SEV features from IGVM and include them in the
KVM_SEV_INIT2 ioctl
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Co-authored-by: Rob Bradford <rbradford@meta.com>
Signed-off-by: Rob Bradford <rbradford@meta.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Following the pattern used by the existing virtio devices make the
balloon device work with confidential VMs (e.g. SEV-SNP). This requires
advertising the VIRTIO_F_ACCESS_PLATFORM feature. Do not expose this to
the user as a controllable option and instead only enable in on the
"force" case.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This a clearer name for it's purpose and now matches more closely what
is used for the virtio devices themselves.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Replace the stored AccessPlatform reference with one to the
VirtioDevice. By doing this not only does it allow the code to be
simplified but also now makes it virtio spec compliant by only
translating via the access platform if the feature is acknowledged.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Our bus slots are now Reserved/Allocated/Free so change the method to
free it to free_device_id() and update error.
Also update to take u8 to match the other methods.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Use two passes to first reserve PCI device IDs and then allocate them
when adding the devices to the bus. This prevents a situation where an
anonymous PCI device allocation clashes with an explicitly allocated PCI
device ID.
Signed-off-by: Rob Bradford <rbradford@meta.com>
We pass the device ID from the config to the allocation routine, where
it is then used as the preferred device ID alongside the existing PCI
segment ID.
Signed-off-by: Pascal Scholz <pascal.scholz@cyberus-technology.de>
On-behalf-of: SAP pascal.scholz@sap.com
Signed-off-by: Rob Bradford <rbradford@meta.com>
This can be used in a two pass approach where all configs that can hold
PCI devices are evaluated to reserve any specific PCI device IDs they
may need. Those device IDs will later be allocated when the devices are
added to the bus. The tri-state Free, Reserved, Allocated also catches
the problem of hotplugging a device with a specific, already used,
device ID.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This commit refactors the PCI bus struct. It has two major focuses.
First, we change the type of `device_ids` in `PciBus` to an array. A
fixed-size array better reflects real PCI bus constraints, especially
its limited number of PCI devices. Moreover, it can't be grown
accidentally.
The second focus is changing the type of the key of `devices` in
`PciBus` to `u8`, since device IDs are not allowed to exceed 31. We
furthermore replace magic numbers with constants and make them publicly
available so we can use them in a follow-up change when parsing user
input.
Signed-off-by: Pascal Scholz <pascal.scholz@cyberus-technology.de>
On-behalf-of: SAP pascal.scholz@sap.com
Calling vm.add-user-device a second time with a socket path already
in use makes the VMM thread block indefinitely inside
vfio_user::Client::new(). libvfio-user servers (SPDK, the reference
libvfio-user daemon) accept a single active client per socket, so
the second connect(2) succeeds at the OS level but the handshake
recvmsg(2) waits for a response that never arrives.
All subsequent API requests queue behind the stuck VMM event loop
and also hang (vm.info, vmm.ping, vm.remove-device). The VM itself
keeps running on vcpu threads, making the symptom confusing: the
guest looks healthy, only the API is unreachable.
This is easy to hit from management software that uses an idempotent
reconcile / ensure pattern for user devices.
Reject the call up-front when another user_device already has the
same socket path, returning an HTTP 500 with a descriptive
UserDeviceSocketInUse error in milliseconds instead of hanging.
Signed-off-by: Max Makarov <maxpain@linux.com>
Plumb ACPI S5 shutdown through guest_exit_evt instead of the shared
exit path.
This keeps guest-triggered shutdown separate from fatal VMM exit
handling. Management software, for example libvirt, expects that
distinction, and making it explicit aligns Cloud Hypervisor more
closely with QEMU.
Only the guest shutdown path is moved here. Reboot handling stays on
reset_evt and non-guest exit paths are left unchanged.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Introduce a dedicated guest_exit_evt and a matching epoll dispatch
path for guest-triggered shutdowns.
This series is needed because managment software such as libvirt may
still need the Cloud Hypervisor process to stay alive after the guest
has shut down.
Today a guest-triggered shutdown can make the VMM disappear immediately,
which means the managment software can lose track of the VM run-state.
This must only apply to guest-triggered shutdowns. Fatal error paths
and other internal exit paths must keep using the existing VMM exit
handling.
For now GuestExit still calls vmm_shutdown(), so this commit only adds
the separate plumbing and keeps the current behavior unchanged.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Extract AcpiCpuHotplugController from CpuManager and move the BusDevice
implementation to the new type. This separates VMM-internal vCPU
management from the guest-visible ACPI CPU hotplug MMIO interface.
Besides clarifying responsibilities and reducing technical debt, this
fixes a rare deadlock involving pause handling and MMIO access.
New responsibilities:
- CpuManager manages VMM-internal vCPU lifecycle and coordination
- AcpiCpuHotplugController implements the guest-visible ACPI CPU hotplug
MMIO interface
A vCPU thread may exit KVM_RUN to perform an MMIO access previously
handled by CpuManager. If the VMM thread begins processing a `pause`
event before that MMIO operation acquires access to CpuManager,
CpuManager::pause() will block waiting for the vCPU thread to ACK
the pause, while the vCPU thread is blocked waiting to complete the MMIO
operation through the same CpuManager - which it can never lock - the
VMM is deadlocked.
This can occur during early boot or CPU hotplug when pause events race
with MMIO accesses. The issue is rare and timing-dependent, but real.
For reproducing: run `ch-remote pause|resume` in a loop while booting
a Linux VM (via direct kernel boot).
With the new design, these MMIO operations no longer depend on
CpuManager, which removes the deadlock path entirely.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
When io_uring is available and not disabled, open QCOW2 images
with QcowDiskAsync for asynchronous reads. Falls back to
QcowDiskSync otherwise.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This struct has the same members and it can be reused to reduce
complexity now and if other common PCI related fields need to be
added in the future.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch VsockConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch VdpaConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch UserDeviceConfig over to using the newly extracted struct members
as used by all PCI based devices. The use of #[serde(flatten)] means
that this change has no impact on the JSON format that the data is
stored as.
As VFIO user devices do not support being placed behind an IOMMU an
error is now raised if iommu is set. This can't happen via the CLI but
could via the JSON/API.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch DeviceConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch PmemConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch GenericVhostUserConfig over to using the newly extracted struct
members as used by all PCI based devices. The use of #[serde(flatten)]
means that this change has no impact on the JSON format that the data is
stored as.
As generic vhost-user devices do not support being placed behind an
IOMMU an error is now raised if iommu is set. This can't happen via the
CLI but could via the JSON/API.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch FsConfig over to using the newly extracted struct members as used
by all PCI based devices. The use of #[serde(flatten)] means that this
change has no impact on the JSON format that the data is stored as.
As virtio-fs does not support being placed behind an IOMMU an error is
now raised if iommu is set. This option is not exposed via the CLI but
could happen with a miscontructed JSON/API call.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch NetConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch DiskConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The MMIO64 allocator size is computed with alignment truncation:
size = (range / alignment) * alignment
This loses up to one alignment unit (4 GiB) at the top of the
address space. When a guest (Windows with virtio-win 0.1.285)
programs a BAR near the top of the physical address space, the
allocation fails because the address falls in the truncated gap.
Give the last PCI segment allocator all remaining space up to
the end of the device area, so no addresses are lost.
The `end` parameter of create_mmio_allocators() is an inclusive
address (the last valid byte). Fix the 32-bit caller and tests
to pass inclusive values, consistent with the 64-bit caller
which already uses the inclusive end_of_device_area().
Signed-off-by: CMGS <ilskdw@gmail.com>
Add a user-configurable option to control whether VFIO device MMIO BAR
regions are DMA-mapped into the host IOMMU address space.
This mapping is required for peer-to-peer DMA between devices (e.g.
NVLink, RDMA NIC accessing GPU VRAM). However, iommufd on upstream
kernels does not support mapping device MMIO pages (VM_PFNMAP), causing
IOMMU_IOAS_MAP to fail with -EFAULT. Kernels with the NVIDIA PFNMAP
workaround or future kernels with DMABUF-based mapping
(IOMMU_IOAS_MAP_FILE) handle this correctly.
The option defaults to `on` to preserve existing behavior. Users on
vanilla kernels using iommufd should set `vfio_p2p_dma=off` to skip
MMIO BAR DMA mapping.
A validation check ensures that `x_nv_gpudirect_clique` (which depends
on P2P DMA) cannot be used when `vfio_p2p_dma=off`.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
When `--platform iommufd=on` is set, use the vfio cdev interface backed
by iommufd instead of the legacy vfio container/group interface for
device passthrough.
The cdev path opens '/dev/iommu' via IommuFd, allocates an IOAS, and
binds VFIO devices through VfioIommufd. The legacy container/group path
remains the default and is used when iommufd is not enabled.
Add iommufd-ioctls as a workspace dependency and enable the "vfio_cdev"
feature on vfio-ioctls for KVM builds.
Fixes: #6892
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Refactor PciDevice::allocate_bars trait and all implementations
to take &mut SystemAllocator instead of &Arc<Mutex<SystemAllocator>>,
removing double indirection.
The caller in device_manager.rs now acquires the lock before
calling allocate_bars.
Signed-off-by: Chinmoy <daschinmoyy21@gmail.com>
Remove the old async_io::DiskFile trait implementation from
RawFileDiskAio, now that the new disk_file trait hierarchy
is fully implemented.
Clean up unused imports: DiskFile and DiskFileResult from
crate::async_io.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Classify the io::Error as BlockErrorKind::Io with ErrorOp::Open.
Update vmm CreateFixedVhdDiskSync to take BlockError.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
