As we now have more than one parameter for the receive migration call,
this commit also adds parsing and validation for those parameters. We
maintain backwards compatibility by also correctly parsing the case
where the caller only provides a URL.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
For TLS we have to parse the hostname from the given migration URL. For
that we have to make a few assumptions about the URL (e.g. it always has
a port). To catch problems early, we tighten the URL validation.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Extend ReceiveListener with a TLS-backed listener variant for migration
receivers.
Store the TCP listener together with the server TLS configuration, wrap
accepted sockets in TlsStream::new_server(), and preserver the existing
listener cloning and fd polling behavior so receive-side migration code
can treat TLS listeners like the existing TCP and UNIX cases.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Teach the migration transport to handle TLS-backed streams alongside
plain TCP and UNIX sockets.
Introduce a Tls variant in SocketStream and implement the necessary
traits.
Also updates the local-migration error path to reject any non-UNIX
transport, which now includes TLS-wrapped TCP connections.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
ReadVolatile already provides a default read_volatile_exact()
implementation, and WriteVolatile a default write_volatile_exact()
implementation. Overriding these functions adds no behavioral value, but
duplicates logic and needs to be updated whenever SocketStream gains or
changes a variant.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Split `add_pci_device()` into two phases: `allocate_pci_bars()` which
only allocates BAR address space, and `commit_pci_device()` which
makes the device visible to the guest on the PCI bus.
All callers now follow the pattern: allocate BARs → perform device-
specific setup (ioeventfd, device_tree, mmio mapping) → commit device.
This eliminates a race window where the guest could discover a
partially-initialized device via `acpiphp_check_bridge()` during rapid
sequential hotplug, causing BAR reprogramming to fail because
ioeventfds and device_tree entries were not yet in place.
Signed-off-by: wuxinyue <wuxinyue.wxy@antgroup.com>
Assisted-by: Claude:Opus-4.6
`PciBus::register_mapping()` operates on `mmio_bus` and `io_bus`
which are passed in as external parameters and have nothing to do
with PciBus internal state. Move this logic into
`DeviceManager::register_bar_mapping()` where it belongs, and move
the `PioInsert`/`MmioInsert` error variants from `PciRootError` to
`DeviceManagerError` accordingly.
Signed-off-by: wuxinyue <wuxinyue.wxy@antgroup.com>
Assisted-by: Claude:Opus-4.6
Remove stale #[allow]s whose lints no longer fire, convert the
unconditionally-firing ones to #[expect], and keep the
arch/feature-conditional ones as #[allow]. Verified across kvm/mshv,
x86_64/aarch64, and --all-features.
Part of #8326.
Signed-off-by: Tushar Khatri <hello@tusharkhatri.in>
Currently ejecting a device leaks its mapping keeping the container fd
open. Remove the mapping so the fd can be closed.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Dylan Reid <dgreid@fb.com>
Validate the sender's migration protocol version when
handling the initial Start request.
Read the version from the Start command header, accept only
the supported version window n-1..=n, and reject unsupported
versions with Error. A rejected Start moves the receiver to
the aborted state.
This keeps compatibility one-way, from older protocol
versions to newer ones, and leaves later version-based
branching on the receiver side.
Log the protocol version on both sender and receiver to make
the active migration path visible.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
During migration send one final set of changed memory after capturing
the snapshot/state. This captures any memory changed as a side effect of
capturing that state. In particular with vhost-user capturing the device
state can lead to inflight requests being drained/flushed which could
change memory. As this is related to the snapshot account for this
memory transfer in the snapshot metrics.
No equivalent change is needed for snapshot as the memory is written
after the state is snapshotted.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Currently both kvm and mshv require an IGVM file to boot a SEV-SNP VM.
This is already configured in the top-level cloud-hypervisor
Cargo.toml where sev_snp depends on igvm.
Add a similar dependency in the vmm crate which helps simplify some of
the in-code cfg blocks by removing the ones that are within a sev_snp
cfg block.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Cloud Hypervisor maps guest RAM with MAP_NORESERVE, so the kernel never
reserves the backing pages at mmap time. On a host whose hugepage pool
cannot satisfy every guest, a VM is created successfully and then takes
a SIGBUS when the guest faults a page the pool can no longer back. This
is the failure mode reported in #5730 and #7387. As noted on #5730,
checking free pool headroom up front is not a reliable fix: another
process can consume pages between the check and the fault.
Add a reserve=on parameter to --memory and --memory-zone (default off,
preserving the current MAP_NORESERVE behaviour). When set, the region
is mapped without MAP_NORESERVE, so the kernel reserves the backing
pages (swap, or huge pages for hugepage-backed memory) at mmap time,
atomically with the mapping. An over-committed configuration then
fails cleanly at VM creation with an mmap ENOMEM instead of crashing
the guest later. Unlike prefault it does not fault the memory in, so
it does not slow down boot.
This mirrors QEMU's memory-backend reserve property, which has the same
name and meaning (reserve=off maps with MAP_NORESERVE). reserve is
threaded through the same mmap paths as the existing prefault option,
and is exposed in the OpenAPI schema, CLI help and docs. The top-level
--memory reserve=on path is unchanged: the default zone is synthesised
from MemoryConfig and inherits its reserve value.
Assisted-by: Claude Code (Opus 4.8)
Signed-off-by: Ian Klemm <hi@ianklemm.de>
remove_userspace_mapping tears down the KVM mapping but never returns
the slot id to the allocator's free list. Call `free_memory_slot` to
avoid the leak.
Signed-off-by: Dylan Reid <dgreid@fb.com>
FwCfgConfig already has a Default impl (e820/kernel/cmdline/initramfs/
acpi_tables = true, items = None), but deserialization did not use it:
without serde default every field was mandatory, so any caller building
a payload config over the API had to spell out the whole object even to
flip a single flag.
Add a container-level #[serde(default)] so missing fields fall back to
FwCfgConfig::default(). The container form is required here because the
defaults are all true; a per-field #[serde(default)] would resolve bool
to false and contradict the Default impl.
This lets callers send only the fields that differ from the defaults.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
The KVM SEV-SNP net-hotplug integration tests (supported added in
later commits) intermittently kill the VMM with SIGSYS on the
http-server thread. The thread is seen reading
/proc/sys/vm/overcommit_memory (openat + read).
Allow the syscalls there, gated on sev_snp+kvm. seccomp can't match
a path, so the open is restricted to O_RDONLY.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
On KVM, an SEV-SNP guest opens /dev/sev at VM creation and passes
its fd as kvm_sev_cmd.sev_fd. KVM uses it to issue the SNP_LAUNCH_*
commands. The open happens after the landlock ruleset is applied,
so grant rw access to /dev/sev when the sev_snp platform feature
is enabled.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
A confidential guest (e.g. SEV-SNP) requires every virtio device to
advertise VIRTIO_F_ACCESS_PLATFORM so the guest driver routes DMA
through the platform's bounce-buffer path; the driver refuses a device
that does not offer it.
Add VIRTIO_F_ACCESS_PLATFORM support to virtio-{watchdog,iommu} which
are exercised as part of the CVM integration tests.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
In Socket serial mode the device output sink was only installed once a
client connected, so output produced beforehand (kernel boot messages,
cloud-init) was dropped, and a client attaching after boot saw a blank
screen. Only PTY mode wrapped the sink in a SerialBuffer.
Install a persistent SerialBuffer as the Socket device's output sink at
SerialManager construction (discarding downstream via io::sink() until a
client connects), so output is captured into the 1 MiB ring even with no
client attached. On connect, retarget the buffer at the accepted client
and flush the backlog before live output resumes; on disconnect, keep
buffering so output produced while no client is attached is delivered to
the next one. The accepted socket is made non-blocking via
set_nonblocking() so a slow client cannot stall the vCPU thread
(SerialBuffer re-buffers on WouldBlock).
The serial-manager thread gains two syscalls under seccomp: sendto
(replaying the backlog is the first time it writes to the socket) and
ioctl restricted to FIONBIO, which is what set_nonblocking() issues.
Fixes: #7907
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
If `KVM_GET_CLOCK` already filled out the `realtime` field, it sets the
`KVM_CLOCK_REALTIME` flag, but if we instead preserve this flag, the
kernel will automatically adjust the kvmclock clock when calling
`KVM_SET_CLOCK` based on the elapsed wall-clock time between pause and
resume. This just requires removing the `reset_flags()` function, which
allows the `KVM_CLOCK_REALTIME` flag to persist in the serialized clock
state.
However, the kernel does not always fill the `realtime` field, depending
on clock source; in this case, fill `realtime` during pause based on the
system time. This is not as precise as the automatic `KVM_GET_CLOCK`
version, since we query the time slightly after the vCPU was paused, but
it allows the clock to be resumed mostly in sync instead of being wildly
off. In this case, we also set the `KVM_CLOCK_REALTIME` flag in the
saved `struct kvmclock` so `KVM_SET_CLOCK` will adjust the clock on
resume.
Basic test case:
1. Run a VM with a Linux guest.
2. Pause the guest via `vm.pause` API.
3. Wait several minutes.
4. Resume the guest via `vm.resume` API.
5. Verify the guest time (e.g. via `date` command) is valid.
6. Verify guest is still using `kvm-clock` timesource:
cat /sys/devices/system/clocksource/clocksource0/current_clocksource
Before applying the patch, the guest clock would be off by the delta
time between pause and resume; after the patch, the clock is (more or
less) in sync with the correct wall-clock time.
Old snapshots will not have the `KVM_CLOCK_REALTIME` flag populated, so
they will not be affected by the new behavior.
Signed-off-by: Daniel Verkamp <drv@meta.com>
When Cloud Hypervisor crashed or was killed, the API socket file was
left on disk, so the next start failed with EADDRINUSE ("Address already
in use") and the VMM could not restart. This affects any environment
where the socket directory survives across restarts (systemd services,
Kubernetes emptyDir volumes, and so on).
Before binding the path-based API socket, take an exclusive lock on a
sidecar "<socket>.lock" file using the block crate's OFD-lock helper.
Holding it proves no other instance is bound to this path, so a stale
socket left by a crashed run can be removed safely and race-free. If
the lock is already held, fail with a clear "API socket is already in
use" error instead of clobbering the live instance. The lock is held
for the process lifetime and released by the kernel on exit or crash.
The fd-based (socket-activation) path is left unchanged.
This implements the lock-file approach suggested by @DemiMarie.
Fixes: #7784
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Fix racy seccomp kill on shutdown. When a VM shuts down the
event-monitor thread's recv() loop ends and the thread exits. glibc's
thread teardown then runs __malloc_arena_thread_freeres, which trims the
per-thread malloc arena with madvise(MADV_DONTNEED).
Add madvise to the allowed calls to match other threads. The crash is
intermittent because it only fires when that thread's arena accumulated
trimmable memory by shutdown.
Signed-off-by: Dylan Reid <dgreid@fb.com>
VmConfig and its nested configuration structs, the VmInfoResponse
wrapper and DeviceNode serialize their Option<T> fields as JSON null
when unset. The OpenAPI specification types these fields as
non-nullable, so strict client generators (for example ogen for Go)
reject /vm.info responses and cannot generate a working API client.
Apply serde_with's skip_serializing_none to the affected structs so
that unset optional fields are omitted from the serialized JSON instead
of being emitted as null. API responses now validate against the
existing specification unchanged; no nullable annotations are required.
Fixes: #7775
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
The HTTP API mapped every ApiError to 500 Internal Server Error, so an
API client could not distinguish "the VM has not been created yet" from
a genuine server-side failure without parsing the error message text.
Derive the HTTP status code from the error itself in error_response():
errors whose root cause is VmError::VmNotCreated or VmMissingConfig are
now reported as 404 Not Found, regardless of which API action surfaced
them. The existing 400 (bad request) and 429 (too many requests)
mappings are preserved.
State-conflict errors such as VmNotRunning would ideally map to 409
Conflict, but micro_http's StatusCode has no Conflict variant, so they
remain 500 for now.
Fixes: #7774
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
When the guest writes DRIVER_OK and the device fails to activate, the
VMM previously bubbled the error up via VirtioActivate and never
released the activation barrier, leaving the vCPU that wrote DRIVER_OK
blocked on the barrier and effectively deadlocking the guest.
Per virtio 1.3 section 2.1.2, a device that has experienced an error
it cannot recover from should set DEVICE_NEEDS_RESET in its status and
notify the driver via a configuration change interrupt. Do that on
activation failure through the existing mark_device_needs_reset
helper, then release the activation barrier so the vCPU can resume.
DeviceManager::activate_virtio_devices now logs and continues instead
of aborting the whole pending list, so one failing device does not
take down the VMM or block pause and migration. The activator has
already reported the failure with the device id.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The new event allows management software to handle the migration better
via events. The `vm.migration-receive-ready` event tells that the VMM is
ready to accept connections whereas `vm.migration-receive-started` means
a migration is incoming.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Mark serial_number/uuid as deprecated in the OpenAPI schema and emit
warnings when those legacy --platform keys are used, while continuing to
accept them for compatibility.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Extend SMBIOS System Information with manufacturer, product,
version, family, sku, serial, and uuid fields, add a chassis
asset tag, and pass a structured SMBIOS config from --platform
into arch setup. Keep OEM strings and legacy serial_number/uuid
options working for compatibility. The platform option naming
follows `dmidecode -s <field>`.
Fields:
- system_manufacturer
- system_product_name
- system_version
- system_family
- system_serial_number
- system_uuid
- chassis_asset_tag
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Add a small SMBIOS config that carries serial_number, uuid,
and OEM strings, and pass it from platform config into
x86_64 setup.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Split the System Information write into helper functions and
reuse the string writer so the table layout and inputs are
unchanged.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Since enabling AMX tile state components affect the result returned by
`Hypervisor::get_supported_cpuid` we want this enabled prior to checking
CPUID compatibility between the source and destination VMs.
Although this is not required right now, it will be necessary once we
introduce CPU profiles and it will also be necessary if we decide to
make `check_cpuid_compatibility` (arguably) more thorough by also taking
state components into account.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We integrate the CPU profile into the various configs that
ultimately get set by the user.
This quickly ends up involving multiple files, luckily Rust
helps us find which ones via compilation errors.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The current OpenAPI schema format is wrong because uint16 and uint32 are
not valid top-level OpenAPI types. Describe queue_size and virtio_id as
integer types with explicit uint formats instead.
Signed-off-by: Roman <roman@vanesyan.com>
This change will allow us to get accurate time over ptp in guests
started from a MSHV-virtualized Linux host. Implementing it as a
virtio device is preferable to using the existing kvm_ptp because:
kvm_ptp relies on hypercalls that only exist on host kernels running
kvm. Virtio-rtc gives us more flexibility in what clock types we want
to provide. We can later extend the device to implement multiple clocks
(smeared UTC, TAI, monotonic, etc.). Virtio-rtc protocol supports
alarms. Alarms may later enable usecases where the guests can do their
own VM lifecycle management without relying on a host-side
orchestrator.
Implement device backend for virtio-rtc. Currently this implementation
encompasses:
1. CONFIG, CAP, READ, CROSSCAP (returns false)
2. One PTP clock is presented of type
VIRTIO_RTC_CLOCK_UTC_MAYBE_SMEARED with leap_second_smearing
VIRTIO_RTC_SMEAR_UNSPECIFIED
The device is disabled by default, requiring --rtc to be passed
Not implemented but theoretically supported by virtio-rtc is:
1. Cross-timestamping support
2. The alarm queue
Fixes#7730
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Several riscv64 paths are compiled by the KVM build but missed
imports or cfg coverage needed by the current code.
Import the vm-memory Bytes trait for the RISC-V UEFI loader, keep
Instant available for migration timing code, and enable the UEFI flash
error path for riscv64.
Assisted-by: OpenAI-Codex:GPT-5
Signed-off-by: wangyf0611 <wangyufeng@iscas.ac.cn>
riscv64 does not provide the legacy readlink and unlink syscalls, so
libc uses readlinkat and unlinkat for paths that are otherwise allowed
on other architectures.
Permit readlinkat and unlinkat for the VMM and vCPU seccomp filters on
riscv64, matching the existing aarch64 rules.
Assisted-by: OpenAI-Codex:GPT-5
Signed-off-by: wangyf0611 <wangyufeng@iscas.ac.cn>
Each VM reboot re-entered the VFIO/virtio-net add paths and re-appended
the same originating fds, leaving duplicates in preserved_fds and a
double-close hazard at final teardown. Switching to HashSet makes
add_preserved_fds idempotent.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Assisted-by: Claude:Opus-4.7
