Storing the snapshot causes issues when needing to do a subsequent
hotplug instead just pass it through on all the methods that need it
making the lifecycle cleaner.
Assisted-by: Claude:Opus-4.6
Signed-off-by: Rob Bradford <rbradford@meta.com>
The serial-manager thread was the only VMM-managed thread without a
seccomp filter. Add a Thread::SerialManager variant and whitelist the
31 syscalls needed for its epoll-based I/O loop (read, write, socket
ops, signal handling, memory allocation, glibc internals).
The filter is computed in start_thread() and applied before the epoll
loop, matching the pattern used by other VMM threads.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Assisted-by: Pi-agent:Claude-Opus-4.7
Only Thread::Vmm and Thread::Vcpu need to know the hypervisor type.
Make the type optional, and then simplify the users.
Assisted-by: Pi-agent:Claude-Opus-4.7
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Userfaultfd is a great mechanism for providing fast restore to Cloud
Hypervisor VMs. But that means the price to pay for bringing pages in
happens at runtime, which might slow down the guest when it's touching
pages which haven't been brought in yet.
By prefaulting the pages in the background, we're trying to get the best
of both worlds. That means we still get a very fast restore with the
uffd handler, but within a few seconds (depending on VM's RAM size), we
also get the pages fully faulted and we can stop the uffd handler thread
at that point.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-7
The JSON parser expects the field to be named
`iommu_address_width_bits`, but the code declared it as
`iommu_address_width`. This mismatch caused the field to be
unrecognized when deserializing configuration from JSON.
Rename the field to `iommu_address_width_bits` to match the
expected schema.
Signed-off-by: tchaton <thomas.chaton.ai@gmail.com>
This reverts commit ced3762a67.
This change lead to a serious memory regression when not using hugepages
or shared=on.
`MAP_PRIVATE` creates an anonymous memory allocation for every page
written when the backing store is a file. This CoW behaviour is useful
but leads to double allocations when the backing store is an empty file
created by `memfd_create()`. When the page is written to, the CoW
semantics require a real page to be created in the memory for the memfd
(previously before the page was touched they would all point to the zero
page). This real page is filled with zeroes because in theory this page
would be accessible via read/write syscalls on the FD even though in our
implementation it is only ever `mmap()`ed.
The intention of the commit was to enable `fallocate()` to be used to
punch holes but that would only affect the inaccessible backing page and
the page in the CoW anonymous memory would be unaffected. Leading it
likely not to have the desired effect.
Fixes: #8211
Signed-off-by: Rob Bradford <rbradford@meta.com>
When reading from an unregistered PIO address, pio_read() wasn't
initialising the buffer, so guests were reading stale bytes from the
previous PIO transaction rather than all 0xff bytes like master abort
on real hardware.
Fill data with 0xff on invalid reads.
Correct 'read to unregistered address' info message to 'read from
unregistered address' while we're touching this block.
Signed-off-by: Chris Webb <chris@arachsys.com>
When reading from an unregistered MMIO address, mmio_read() wasn't
initialising the buffer, so guests were reading stale bytes from the
previous MMIO transaction rather than all 0xff bytes like master abort
on real hardware.
Fill data with 0xff on invalid reads.
Correct 'read to unregistered address' info message to 'read from
unregistered address' while we're touching this block.
Signed-off-by: Chris Webb <chris@arachsys.com>
I started by looking at all `Option<Vec<T>>` values in config.rs and
vm_config.rs, and replaced them with `Option<Box<[T]>>`. This has the
advantage that one now can see at a glance if this field will ever
resize during operation or not, reducing cognitive load and increasing
maintainability. All fields that need the properties of a Ver or where
this change was not trivial are kept intact.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
When no image_type is specified, sector 0 writes are disabled as a
safety measure for autodetected raw images. Extend this protection
to autodetected fixed VHD images, which carry metadata in the last
sector and are equally susceptible to accidental overwrites of the
first sector when the format is not explicitly acknowledged.
Update the corresponding warning in the virtio block worker to be
format agnostic.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Explanatory comments for the rollback paths in both PCI BAR
relocation branches.
Assisted-by: Claude (Anthropic)
Signed-off-by: CMGS <ilskdw@gmail.com>
After free(old_base), if allocate(new_base) fails the allocator
treats old_base as free even though the MMIO/PIO bus still maps the
device there. Subsequent allocations pick old_base, mmio_bus.insert
hits the live mapping and returns Overlap.
Restore old_base on the failure path in both the Memory*BitRegion
and IoRegion branches before bubbling the error up.
PR #7950 added restore_bar_addr() so the BAR config register stays
consistent on failed move_bar(); this completes the same picture
for the allocator side.
Signed-off-by: CMGS <ilskdw@gmail.com>
resume() mirrors pause() for backend communication: it skips the
vhost-user backend call when the device is already disconnected, and it
marks newly failed resume_vhost_user() calls disconnected only when the
classifier identifies transport loss.
Signed-off-by: Dylan Reid <dgreid@fb.com>
pause() returns DeviceDisconnected without calling into the backend when
VhostUserCommon already knows the socket is gone. DeviceManager treats
only that sentinel as log-and-continue, so one dead vhost-user device
does not abort the whole pause iteration.
Signed-off-by: Dylan Reid <dgreid@fb.com>
User-defined zones may be mapped private. Create a memfd for private
zones so that fallocate operations are available on all regions, not
just shared ones. This prepares for zone management via hole punching.
The MAP_ANONYMOUS flag is now omitted since the memory becomes
tmpfs-backed via memfd.
Signed-off-by: JP Kobryn <inwardvessel@gmail.com>
Add common PCI device configuration to the virtio-console device
configuration. This allows setting the device ID (the name), ID, the
PCI segment, and the PCI device ID (BDF), which were previously not
configurable for the virtio-console device.
This gives management software, such as libvirt, more control over PCI
resource assignment and aligns virtio-console with other devices that
already support this functionality [0].
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/issues/8175
On-behalf-of: Philipp Schuster <philipp.schuster@sap.com>
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
`--serial` still accepted `iommu=on|off` from the shared serial/console
configuration shape, even though only virtio-console has a meaningful
IOMMU connection.
Cloud Hypervisor wires virtio-iommu support through DMA-capable
virtio/VFIO PCI endpoints. The serial devices are legacy UARTs accessed
through PIO or MMIO registers, so exposing an IOMMU option there is
misleading.
On-behalf-of: Philipp Schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
This commit introduces a new struct `CommonConsoleConfig` which is the
base for the split into `ConsoleConfig` and `SerialConfig`. This is a
pre-requisite for allowing more configurable PCI options for the
virtio-console device.
The commit doesn't change or add any functionality.
On-behalf-of: Philipp Schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
The PciDeviceHandle::VfioUser arm in eject_device propagated dma_unmap
failures with ?, which short-circuited the subsequent
remove_dma_mapping_handler loop and left stale Arc<VfioUserDmaMapping>
entries in every virtio-mem device's handler map. Log unmap errors
with warn! and continue so the handler map cleanup always runs.
This only happens if a vfio-user process crashes and the same device is
later removed. However, given we've seen similar issues on vhost-user
this is probably worth cleaning up.
Signed-off-by: Dylan Reid <dgreid@fb.com>
The BusDevice read and write arms for B0EJ_FIELD_OFFSET and
PSEG_FIELD_OFFSET opened with assert!/assert_eq! on data.len(), so a
guest 1/2/8-byte MMIO access to either register panicked the vCPU
thread. Replace each assert with a warn! and early return so unusual
access widths are logged and ignored instead of crashing the VMM.
Signed-off-by: Dylan Reid <dgreid@fb.com>
self.balloon and self.virtio_mem_devices are not updated when
eject_device removes the underlying device, leaving stale
Arc<Mutex<...>> entries that resize_balloon / balloon_size and the
virtio-mem DMA-handler iteration would dereference if reached after
eject. Clear self.balloon and retain-out the matching virtio-mem entry
in the PciDeviceHandle::Virtio eject arm, identifying the ejected
device by Arc pointer-equality against the already-resolved
Arc<Mutex<dyn VirtioDevice>>.
This change is defensive: DeviceManager::remove_device currently
rejects VirtioDeviceType::Balloon and VirtioDeviceType::Mem with
RemovalNotAllowed before pci_devices_down is set, so the guest never
sees an eject notification and eject_device is never reached for
either type today. If the allowlist is extended later, this cleanup
keeps the post-eject state consistent.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Although a response is sent to the client these errors are not logged in
the log file making it hard to cross reference these with other log
entries.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The old implementation used an ever monotonically increasing u32 counter
to allocate new GSIs. The counter increased every time a new GSI was
allocated, and freeing GSIs was not possible. Thus, Cloud Hypervisor
can run out of GSIs and panics. This currently happened at the 1024th
GSI [0]. Further, this caused the `KVM_SET_GSI_ROUTING` ioctl to carry
much more payload than needed.
This new implementation uses a bitmap for proper tracking of resources
and can gracefully free GSIs - this is abstracted in type
InterruptAllocator.
Please note that this commit only replaces the old mechanism. The next
commit will introduce freeing used GSIs automatically when an
InterruptRoute is dropped.
While being on this, we also propagate the errors that the allocator may
throw where necessary.
Co-authored-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
On-behalf-of: Philipp Schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
When an SEV-SNP guest transitions pages from private to shared via
KVM_HC_MAP_GPA_RANGE, punch holes in the corresponding guest_memfd
backing it. Without this the balloon driver's `set_memory_decrypted()`
path transitions the page attributes but the physical memory stays
pinned in guest_memfd, making virtio-balloon ineffective for memory
overcommit with confidential VMs. Even without ballooning these pages
are unused by the guest so consume resources. This mirrors the hole
punching that the balloon device does for releasing pages.
The memory_slots Arc is cloned into each KvmVcpu at creation so the
punch can happen in the vcpu thread.
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Rob Bradford <rbradford@meta.com>
Prefer /dev/userfaultfd (Linux 6.1+) over the userfaultfd(2) syscall
for obtaining userfaultfd file descriptors. The device path bypasses
the capable(CAP_SYS_PTRACE) and vm.unprivileged_userfaultfd sysctl
checks that block the syscall in user-namespaced containers, using
file permissions instead.
Falls back to the syscall on older kernels or when the device node
does not exist.
Signed-off-by: Roberto Campesato <render@metalabs.org>
Assisted-by: Claude:claude-opus-4-6
Import common std types used by the userfaultfd wrapper instead of
spelling the full paths at every use site.
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Allow VFIO devices to list BAR indices that should not be
mmapped into the guest. This lets operators skip large BARs that
are known not to be used by their workload.
When a BAR is skipped, the log also calls out that P2P DMA
mapping is skipped because the VFIO DMA map path uses the same
mmap backing.
Signed-off-by: Damian Barabonkov <dbctl@pm.me>
Assisted-by: OpenCode:gpt-5.5
The unit tests added in bf3279f09 built sparse files by writing only
at one offset and assuming the surrounding pages stayed unallocated.
That breaks on shmem/tmpfs with huge=within_size: kernel 6.10+ added
large-folio support to shmem, and on first write the kernel allocates
one folio whose order is the largest power-of-two number of pages
that fits inside the file size (capped at PMD-size). For a 64 KiB
test file the very first pwrite anywhere allocates a 64 KiB folio
covering the whole file, so SEEK_HOLE never reports a hole and
written_pages_show_as_data_extents,
sparse_file_yields_extents_at_written_positions, and
single_extent_at_zero_offset all fail. memfd_create lives on shmem
too and inherits the same THP policy from
/sys/kernel/mm/transparent_hugepage/shmem_enabled, so the problem is
not /tmp-specific.
Fix the fixtures, not the production code: build each test file via
a new sparse_layout() helper that writes the requested data extents
and then fallocate(FALLOC_FL_PUNCH_HOLE)s every gap. PUNCH_HOLE is
the explicit "deallocate these pages" syscall and is honored by every
Linux filesystem we run tests on (tmpfs, ext4, xfs, btrfs); the
kernel splits any large folio overlapping the punched range. The
resulting SEEK_DATA/SEEK_HOLE map matches the spec exactly regardless
of folio/THP policy.
For single_extent_at_zero_offset the dst side still loses to the
folio allocator -- writing 8 KiB into a 64 KiB tmpfs file allocates
a 64 KiB folio whether we want it or not -- so the previous
meta.blocks()-based sparseness assertion (which tested the filesystem,
not our code) is replaced with a sentinel pre-fill: dst starts filled
with 0xFE and the post-condition is that bytes outside the
source-data extent are still 0xFE. That directly verifies
write_region_sparse only touched the data extent without depending on
dst-side hole reporting.
Side effect: extent_at_non_zero_src_offset,
two_regions_in_same_destination_file_at_dst_offset, and
round_trip_sparse_write_then_read previously passed by accident on
hosts with mTHP-on-shmem -- their src memfds reported the whole file
as data so write_region_sparse silently fell into a dense copy of
zeros + data. With sparse_layout() the sources are genuinely sparse
and those tests now exercise the sparse path on every host.
Tested on tmpfs (huge=within_size) and ext4 (TMPDIR=/var/tmp); all 9
tests pass on both with no skips.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Dylan Reid <dgreid@fb.com>
Rate limiting is implemented in the virtio device layer and does not
apply to vhost-user devices which delegate I/O handling to an external
process.
Add validation to reject configurations where vhost_user is enabled
along with rate limiting options (bw_size, ops_size, or
rate_limit_group) for both disk and network devices.
This prevents users from mistakenly configuring rate limiting that would
be silently ignored when using vhost-user backends.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This extends migration to also support paused VMs, preserving the
paused state on the destination.
Changes:
- Add CompletePaused protocol command that finalizes migration without
resuming the VM on the destination
- Skip the pause step during migration if the VM is already paused
- On migration failure, only restore the running state if
the VM was originally running (not paused)
Signed-off-by: Nguyen Dinh Phi <phind.uet@gmail.com>
KSM will not attempt to merge pages that are mapped as MAP_SHARED, so
configuring memory with both mergeable and shared options is invalid.
Add validation to reject configurations where both options are enabled
for memory or memory zones.
Signed-off-by: Rob Bradford <rbradford@meta.com>
With the `mshv` feature enabled, the immutable `data` `Vec` is mutated
via a pointer. This violates Rust aliasing rules. Fixed by cloning the
`Vec` to a mutable instance when the `mshv` feature is active.
Signed-off-by: Julian Schindel <mail@arctic-alpaca.de>
Add KvmSevSnpIdBlock and KvmSevSnpIdAuth structs matching the AMD
SEV-SNP Firmware ABI Spec (Rev 1.58), and build them from the IGVM
SNP ID block directive during launch finish. This properly populates
id_block_uaddr/id_auth_uaddr in KVM_SEV_SNP_LAUNCH_FINISH and derives
auth_key_en from the assembled author key, matching QEMU's behavior.
Thread the guest policy from sev_snp_init to launch_finish via an
atomic on KvmVm so the ID block gets the correct policy value.
Also track has_snp_id_block in IgvmLoadedInfo to enable the ID block
based on whether the IGVM file actually contains one, rather than
hardcoding it for KVM.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Extract the SNP guest policy from IGVM initialization headers when
available, falling back to the default policy. This matches QEMU's
behaviour where only a non-zero IGVM policy overrides the default.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Preserve the original IGVM import order for KVM SNP launch updates.
The launch digest is order-sensitive, so only coalesce adjacent pages
that already share the same page type and size. MSHV continues to
sort by GPA for hypercall batching.
Signed-off-by: Kevin Hui <kevinhui@meta.com>
Introduce the KVM_SNP_PAGE_TYPE_ZERO page type for ZERO
pages. AMD SEV SNP can accept ZERO pages as a page in which
the page memory is functionally just zeroes
Signed-off-by: Kevin Hui <kevinhui@meta.com>
Verify the SEV hash table layout, GUID placement, kernel/initrd/cmdline
digest values, and the setup_sects > boot_params size branch. These
guard against silent regressions in the launch digest computation.
Signed-off-by: Kevin Hui <kevinhui@meta.com>
This introduces the kernel hashes measured boot table into
cloud hypervisor if a cmdline and kernel is passed into an
SEV-SNP CVM, incorporating a kernel/cmdline/optional initrd
into a memory page that is measured into the launch digest
of a SEV-SNP CVM. If both --kernel and --cmdline are not
provided, we do not insert this data page
Signed-off-by: Kevin Hui <kevinhui@meta.com>
