DeviceManager and VfioPciDevice both hold Arc<MmapRegion> for
each VFIO BAR mmap window. During VM shutdown, VfioPciDevice
drops after DeviceManager::Drop::drop (via device_tree
field drop). Without clearing DeviceManager's clones first,
VfioPciDevice::unmap_mmio_regions decrements the Arc but
does not reach zero, munmap never fires, the VFIO device
file VMAs survive, and VFIO_GROUP_UNSET_CONTAINER returns
EBUSY.
Clear DeviceManager's mmio_regions in Drop::drop so
VfioPciDevice is the sole Arc owner at drop time and ensure
VFIO_GROUP_UNSET_CONTAINER ioctl success.
Remove redundant .clone() on the mmio_regions() return value
in the eject_device() hot-unplug path.
Add detail comments
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
QEMU supports passing inline string values to the guest via fw_cfg
(-fw_cfg name=...,string=...). Cloud Hypervisor previously only
supported file-backed fw_cfg items. This adds the 'string' option
so users can pass values like OVMF's X-PciMmio64Mb without creating
a temporary file on the host.
Each fw_cfg item now accepts exactly one of 'file' or 'string'.
The FwCfgInvalidItem invariant is validated in PayloadConfig::validate()
(via FwCfgConfig::validate()), covering both CLI and JSON API paths.
The populate_fw_cfg match arm uses unreachable!() since validation
guarantees the invariant holds at that point.
CLI syntax:
--fw-cfg-config items=[name=opt/ovmf/X-PciMmio64Mb,string=262144]
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Move FwCfgMissingKernel/Cmdline/Initramfs error variants from
ValidationError into PayloadConfigError. Change FwCfgConfig::validate()
to take &PayloadConfig instead of &VmConfig and return
PayloadConfigError. Wire the call through PayloadConfig::validate()
so both CLI and JSON API paths are covered.
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Boot-time block devices on PCI segment 0 use 32-bit BARs so early
firmware can access them without additional identity mapping in the
firmware page tables. However, hot-plugged block devices are only ever
seen by the OS kernel which handles 64-bit BARs natively.
Switch hot-plugged block devices to 64-bit BARs to avoid exhausting the
scarce 32-bit MMIO window (typically 2-3 GB between RAM and 4 GB) when
many devices are hot-plugged.
Extract the BAR sizing decision into use_64bit_bar_for_virtio_device()
and thread an is_hotplug flag through add_virtio_pci_device(). Add unit
tests covering all relevant combinations.
Signed-off-by: Dylan Reid <dgreid@fb.com>
A bootloader/firmware (e.g. stage0) and the VMSA page require dedicated
memory regions at fixed GPAs.
Add reserve_region_for_stage0() to allocate these regions before IGVM
loading begins:
- Stage0 at GPA 0xffc0_0000 (4 MB)
- VMSA page at GPA 0xffff_ffff_f000 (4 KB)
These reservations are KVM-only; MSHV handles stage0/VMSA placement
through its own isolated import path.
Also add fw_cfg device creation and SYS_statx to the vCPU seccomp
allowlist (needed by stage0's file access pattern).
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Adapt the IGVM loader to work with both MSHV and KVM backends, which
differ in page type constants, CPUID page layout, and VMSA handling.
Abstract page types into a PageTypeConfig struct populated at runtime
from the detected hypervisor, replacing hardcoded mshv_bindings constants.
Apply the VMSA register state to each vCPU via setup_sev_snp_regs(),
translating SevSelector attributes to KVM segment format using a bitfield
decoder.
KVM's SNP launch path sanitizes certain CPUID bits that could lead to
an insecure guest. If the VMM sets these bits, KVM rejects the CPUID
page import on the first attempt, requiring a retry with the
firmware-corrected values.
Pre-clear the known problematic bits before import to avoid the
reject-and-retry cycle:
- Leaf 0x1, ECX bit 24: TSC_DEADLINE (filtered by KVM)
- Leaf 0x7, EBX bit 1: SGX (filtered by KVM)
- Leaf 0x7, EDX: clear entirely (contains speculative features)
- Leaf 0x80000008, EBX bit 25: filtered by KVM
- Leaf 0x80000021, ECX: clear entirely
This keeps the CPUID page stable across launch updates and avoids
noisy error logs from the retry path.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Co-authored-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Add the KVM_SEV_SNP_LAUNCH_FINISH ioctl, which finalizes the SNP
launch sequence and transitions the VM into a runnable encrypted
state.
Additionally, add KVM_SEV_SNP_LAUNCH_FINISH to the seccomp allowlist.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Implement the KVM_SEV_SNP_LAUNCH_UPDATE ioctl.
Extend Vm::import_isolated_pages() with a uaddrs parameter carrying
host virtual addresses, which KVM needs, unlike MSHV. Compute uaddrs
from guest memory mappings in the IGVM loader.
Add KVM_SEV_SNP_LAUNCH_UPDATE to the seccomp allowlist.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Introduce the SevFd abstraction that wraps /dev/sev and implements the
KVM_SEV_INIT2 and KVM_SEV_SNP_LAUNCH_START ioctls for SEV-SNP VM
initialization on KVM.
Key changes:
- Add sev.rs with KvmSevInit and KvmSevSnpLaunchStart ioctl structs
matching the kernel layout (linux/arch/x86/include/uapi/asm/kvm.h)
- Implement KVM_SEV_INIT2 and KVM_SEV_SNP_LAUNCH_START ioctls
- Set KVM_MEMORY_ATTRIBUTE_PRIVATE on newly created memory regions
when guest_memfd is supported
- Widen SevSnpPageAccessProxy cfg gates from mshv-only to all
sev_snp-enabled builds
- Make sev_snp_init a required trait method (remove default impl)
- Include KVM_SEV_SNP_LAUNCH_START in the seccomp allowlist
- Parse VMSA SEV features from IGVM and include them in the
KVM_SEV_INIT2 ioctl
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Co-authored-by: Rob Bradford <rbradford@meta.com>
Signed-off-by: Rob Bradford <rbradford@meta.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Move IGVM file parsing from load_igvm() into a dedicated parse_igvm()
helper in igvm/mod.rs, and parse the file upfront in Vm::new() so the
resulting IgvmFile struct is available throughout VM initialization.
This is a prerequisite for extracting VMSA SEV features from the parsed
IGVM before issuing KVM_SEV_INIT2, which needs sev_features.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
The load_payload and load_payload_async functions previously received a
sev_snp_enabled flag to decide whether to call load_igvm with or
without the host_data parameter. Replace this with a single code path
that always passes host_data behind a cfg(feature = "sev_snp") gate,
removing the runtime branch and the extra parameter threaded through
three call sites.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Change configure_system to take an Option<GuestAddress>
since rsdp is wrapped into an option anyways (we use configure
system to setup the mptables).
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
The SNP guest policy (AMD SEV-SNP ABI bits controlling SMT, migration,
debug, etc.) was previously hardcoded inside the MSHV implementation.
Widen Vm::sev_snp_init() to accept an SnpPolicy parameter so each
hypervisor backend receives the policy at init time.
Add get_default_sev_snp_guest_policy() in the VMM to construct the
default policy.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Previously, the payload validation rejected an IGVM file combined with
a kernel or firmware.
Relax this constraint to allow an IGVM carrying a firmware (e.g Oak
stage0) to be paired with a separate kernel image.
This enables fw_cfg-style boot where stage0 loads a kernel provided
through fw_cfg rather than embedded in the IGVM file itself.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Add support for guest_memfd (available in Linux kernel v6.8+), which
enables private memory for confidential VMs.
Key changes:
- Introduce UserMemoryRegion abstraction with guest_memfd fields
- Add From impls between kvm_userspace_memory_region2 and UserMemoryRegion
- Convert all KVM memory region operations from kvm_userspace_memory_region
to kvm_userspace_memory_region2, with automatic fallback to v1 when
guest_memfd is not supported
- Add set_user_memory_region() wrapper that dispatches to v1/v2 based on
kvm_guest_memfd_supported capability
- Create guest_memfd via KVM_CREATE_GUEST_MEMFD ioctl when supported
- Extend KvmDirtyLogSlot to preserve region2 fields across dirty log
start/stop cycles
This is prerequisite infrastructure for KVM-based confidential computing
that requires private guest memory backed by guest_memfd.
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Following the pattern used by the existing virtio devices make the
balloon device work with confidential VMs (e.g. SEV-SNP). This requires
advertising the VIRTIO_F_ACCESS_PLATFORM feature. Do not expose this to
the user as a controllable option and instead only enable in on the
"force" case.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This a clearer name for it's purpose and now matches more closely what
is used for the virtio devices themselves.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Replace the stored AccessPlatform reference with one to the
VirtioDevice. By doing this not only does it allow the code to be
simplified but also now makes it virtio spec compliant by only
translating via the access platform if the feature is acknowledged.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Our bus slots are now Reserved/Allocated/Free so change the method to
free it to free_device_id() and update error.
Also update to take u8 to match the other methods.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Use two passes to first reserve PCI device IDs and then allocate them
when adding the devices to the bus. This prevents a situation where an
anonymous PCI device allocation clashes with an explicitly allocated PCI
device ID.
Signed-off-by: Rob Bradford <rbradford@meta.com>
We pass the device ID from the config to the allocation routine, where
it is then used as the preferred device ID alongside the existing PCI
segment ID.
Signed-off-by: Pascal Scholz <pascal.scholz@cyberus-technology.de>
On-behalf-of: SAP pascal.scholz@sap.com
Signed-off-by: Rob Bradford <rbradford@meta.com>
Validate the PCI device ID are within range and not using the reserved
value. We need this option to ensure that invalid device IDs received
via an API call result in an error as soon as possible. In this case,
this would be after deserialization. On this code path, validation via
`parse` is skipped and must be invoked by calling `validate`.
Signed-off-by: Pascal Scholz <pascal.scholz@cyberus-technology.de>
On-behalf-of: SAP pascal.scholz@sap.com
Signed-off-by: Rob Bradford <rbradford@meta.com>
For those devices types that have the the ability to support specifying
the PCI device ID add it to their help syntax.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This can be used in a two pass approach where all configs that can hold
PCI devices are evaluated to reserve any specific PCI device IDs they
may need. Those device IDs will later be allocated when the devices are
added to the bus. The tri-state Free, Reserved, Allocated also catches
the problem of hotplugging a device with a specific, already used,
device ID.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Next to tests for `allocate_device_id`, we introduce a new constructor
`new_without_address_manager`, only available in the test build. As
there is no way to instantiate an `AddressManager` in the tests, we use
this constructor to work around this.
Signed-off-by: Pascal Scholz <pascal.scholz@cyberus-technology.de>
On-behalf-of: SAP pascal.scholz@sap.com
Signed-off-by: Rob Bradford <rbradford@meta.com>
This commit refactors the PCI bus struct. It has two major focuses.
First, we change the type of `device_ids` in `PciBus` to an array. A
fixed-size array better reflects real PCI bus constraints, especially
its limited number of PCI devices. Moreover, it can't be grown
accidentally.
The second focus is changing the type of the key of `devices` in
`PciBus` to `u8`, since device IDs are not allowed to exceed 31. We
furthermore replace magic numbers with constants and make them publicly
available so we can use them in a follow-up change when parsing user
input.
Signed-off-by: Pascal Scholz <pascal.scholz@cyberus-technology.de>
On-behalf-of: SAP pascal.scholz@sap.com
Calling vm.add-user-device a second time with a socket path already
in use makes the VMM thread block indefinitely inside
vfio_user::Client::new(). libvfio-user servers (SPDK, the reference
libvfio-user daemon) accept a single active client per socket, so
the second connect(2) succeeds at the OS level but the handshake
recvmsg(2) waits for a response that never arrives.
All subsequent API requests queue behind the stuck VMM event loop
and also hang (vm.info, vmm.ping, vm.remove-device). The VM itself
keeps running on vcpu threads, making the symptom confusing: the
guest looks healthy, only the API is unreachable.
This is easy to hit from management software that uses an idempotent
reconcile / ensure pattern for user devices.
Reject the call up-front when another user_device already has the
same socket path, returning an HTTP 500 with a descriptive
UserDeviceSocketInUse error in milliseconds instead of hanging.
Signed-off-by: Max Makarov <maxpain@linux.com>
Several error mappings in vm.rs dropped the underlying error with
map_err(|_| ...), making failures harder to diagnose. Preserve the
source error by adding #[source] fields to InitramfsLoad and ErrorNmi.
- InitramfsLoad: now wraps std::io::Error from seek/rewind operations
- ErrorNmi: now wraps cpu::Error from the CPU manager nmi() call
Partially addresses #7563
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Add a CLI-only --no-shutdown flag that keeps the VMM process alive
after a guest-triggered shutdown.
Management software may still need the Cloud Hypervisor process
after the guest has powered off. Exposing this separately lets
management software, for example libvirt, keep the VMM around in a
way that is closer to QEMU.
The flag only affects the GuestExit path. Fatal exits and other
existing VMM shutdown paths remain unchanged.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Plumb ACPI S5 shutdown through guest_exit_evt instead of the shared
exit path.
This keeps guest-triggered shutdown separate from fatal VMM exit
handling. Management software, for example libvirt, expects that
distinction, and making it explicit aligns Cloud Hypervisor more
closely with QEMU.
Only the guest shutdown path is moved here. Reboot handling stays on
reset_evt and non-guest exit paths are left unchanged.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Introduce a dedicated guest_exit_evt and a matching epoll dispatch
path for guest-triggered shutdowns.
This series is needed because managment software such as libvirt may
still need the Cloud Hypervisor process to stay alive after the guest
has shut down.
Today a guest-triggered shutdown can make the VMM disappear immediately,
which means the managment software can lose track of the VM run-state.
This must only apply to guest-triggered shutdowns. Fatal error paths
and other internal exit paths must keep using the existing VMM exit
handling.
For now GuestExit still calls vmm_shutdown(), so this commit only adds
the separate plumbing and keeps the current behavior unchanged.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Extract AcpiCpuHotplugController from CpuManager and move the BusDevice
implementation to the new type. This separates VMM-internal vCPU
management from the guest-visible ACPI CPU hotplug MMIO interface.
Besides clarifying responsibilities and reducing technical debt, this
fixes a rare deadlock involving pause handling and MMIO access.
New responsibilities:
- CpuManager manages VMM-internal vCPU lifecycle and coordination
- AcpiCpuHotplugController implements the guest-visible ACPI CPU hotplug
MMIO interface
A vCPU thread may exit KVM_RUN to perform an MMIO access previously
handled by CpuManager. If the VMM thread begins processing a `pause`
event before that MMIO operation acquires access to CpuManager,
CpuManager::pause() will block waiting for the vCPU thread to ACK
the pause, while the vCPU thread is blocked waiting to complete the MMIO
operation through the same CpuManager - which it can never lock - the
VMM is deadlocked.
This can occur during early boot or CPU hotplug when pause events race
with MMIO accesses. The issue is rare and timing-dependent, but real.
For reproducing: run `ch-remote pause|resume` in a loop while booting
a Linux VM (via direct kernel boot).
With the new design, these MMIO operations no longer depend on
CpuManager, which removes the deadlock path entirely.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
When io_uring is available and not disabled, open QCOW2 images
with QcowDiskAsync for asynchronous reads. Falls back to
QcowDiskSync otherwise.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This struct has the same members and it can be reused to reduce
complexity now and if other common PCI related fields need to be
added in the future.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This can then be used with the OptionParser::add_all() API to reduce the
number of locations the same options are added to the parser. The only
quirk is that some devices do not support an IOMMU (because they are
vhost-user / vfio-user based). There are two different versions of the
array to support that.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The IommuNotSupportedOnSegment variant is no longer needed as the common
PciDeviceCommonConfig::validate() handles this case with the
OnIommuSegment variant along with more use of the IommuNotSupported
error variant.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch VsockConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch VdpaConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch UserDeviceConfig over to using the newly extracted struct members
as used by all PCI based devices. The use of #[serde(flatten)] means
that this change has no impact on the JSON format that the data is
stored as.
As VFIO user devices do not support being placed behind an IOMMU an
error is now raised if iommu is set. This can't happen via the CLI but
could via the JSON/API.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch DeviceConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch PmemConfig over to using the newly extracted struct members as
used by all PCI based devices. The use of #[serde(flatten)] means that
this change has no impact on the JSON format that the data is stored as.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch GenericVhostUserConfig over to using the newly extracted struct
members as used by all PCI based devices. The use of #[serde(flatten)]
means that this change has no impact on the JSON format that the data is
stored as.
As generic vhost-user devices do not support being placed behind an
IOMMU an error is now raised if iommu is set. This can't happen via the
CLI but could via the JSON/API.
Signed-off-by: Rob Bradford <rbradford@meta.com>
