On KVM, an SEV-SNP guest opens /dev/sev at VM creation and passes
its fd as kvm_sev_cmd.sev_fd. KVM uses it to issue the SNP_LAUNCH_*
commands. The open happens after the landlock ruleset is applied,
so grant rw access to /dev/sev when the sev_snp platform feature
is enabled.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
A confidential guest (e.g. SEV-SNP) requires every virtio device to
advertise VIRTIO_F_ACCESS_PLATFORM so the guest driver routes DMA
through the platform's bounce-buffer path; the driver refuses a device
that does not offer it.
Add VIRTIO_F_ACCESS_PLATFORM support to virtio-{watchdog,iommu} which
are exercised as part of the CVM integration tests.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
In Socket serial mode the device output sink was only installed once a
client connected, so output produced beforehand (kernel boot messages,
cloud-init) was dropped, and a client attaching after boot saw a blank
screen. Only PTY mode wrapped the sink in a SerialBuffer.
Install a persistent SerialBuffer as the Socket device's output sink at
SerialManager construction (discarding downstream via io::sink() until a
client connects), so output is captured into the 1 MiB ring even with no
client attached. On connect, retarget the buffer at the accepted client
and flush the backlog before live output resumes; on disconnect, keep
buffering so output produced while no client is attached is delivered to
the next one. The accepted socket is made non-blocking via
set_nonblocking() so a slow client cannot stall the vCPU thread
(SerialBuffer re-buffers on WouldBlock).
The serial-manager thread gains two syscalls under seccomp: sendto
(replaying the backlog is the first time it writes to the socket) and
ioctl restricted to FIONBIO, which is what set_nonblocking() issues.
Fixes: #7907
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
If `KVM_GET_CLOCK` already filled out the `realtime` field, it sets the
`KVM_CLOCK_REALTIME` flag, but if we instead preserve this flag, the
kernel will automatically adjust the kvmclock clock when calling
`KVM_SET_CLOCK` based on the elapsed wall-clock time between pause and
resume. This just requires removing the `reset_flags()` function, which
allows the `KVM_CLOCK_REALTIME` flag to persist in the serialized clock
state.
However, the kernel does not always fill the `realtime` field, depending
on clock source; in this case, fill `realtime` during pause based on the
system time. This is not as precise as the automatic `KVM_GET_CLOCK`
version, since we query the time slightly after the vCPU was paused, but
it allows the clock to be resumed mostly in sync instead of being wildly
off. In this case, we also set the `KVM_CLOCK_REALTIME` flag in the
saved `struct kvmclock` so `KVM_SET_CLOCK` will adjust the clock on
resume.
Basic test case:
1. Run a VM with a Linux guest.
2. Pause the guest via `vm.pause` API.
3. Wait several minutes.
4. Resume the guest via `vm.resume` API.
5. Verify the guest time (e.g. via `date` command) is valid.
6. Verify guest is still using `kvm-clock` timesource:
cat /sys/devices/system/clocksource/clocksource0/current_clocksource
Before applying the patch, the guest clock would be off by the delta
time between pause and resume; after the patch, the clock is (more or
less) in sync with the correct wall-clock time.
Old snapshots will not have the `KVM_CLOCK_REALTIME` flag populated, so
they will not be affected by the new behavior.
Signed-off-by: Daniel Verkamp <drv@meta.com>
When Cloud Hypervisor crashed or was killed, the API socket file was
left on disk, so the next start failed with EADDRINUSE ("Address already
in use") and the VMM could not restart. This affects any environment
where the socket directory survives across restarts (systemd services,
Kubernetes emptyDir volumes, and so on).
Before binding the path-based API socket, take an exclusive lock on a
sidecar "<socket>.lock" file using the block crate's OFD-lock helper.
Holding it proves no other instance is bound to this path, so a stale
socket left by a crashed run can be removed safely and race-free. If
the lock is already held, fail with a clear "API socket is already in
use" error instead of clobbering the live instance. The lock is held
for the process lifetime and released by the kernel on exit or crash.
The fd-based (socket-activation) path is left unchanged.
This implements the lock-file approach suggested by @DemiMarie.
Fixes: #7784
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Fix racy seccomp kill on shutdown. When a VM shuts down the
event-monitor thread's recv() loop ends and the thread exits. glibc's
thread teardown then runs __malloc_arena_thread_freeres, which trims the
per-thread malloc arena with madvise(MADV_DONTNEED).
Add madvise to the allowed calls to match other threads. The crash is
intermittent because it only fires when that thread's arena accumulated
trimmable memory by shutdown.
Signed-off-by: Dylan Reid <dgreid@fb.com>
VmConfig and its nested configuration structs, the VmInfoResponse
wrapper and DeviceNode serialize their Option<T> fields as JSON null
when unset. The OpenAPI specification types these fields as
non-nullable, so strict client generators (for example ogen for Go)
reject /vm.info responses and cannot generate a working API client.
Apply serde_with's skip_serializing_none to the affected structs so
that unset optional fields are omitted from the serialized JSON instead
of being emitted as null. API responses now validate against the
existing specification unchanged; no nullable annotations are required.
Fixes: #7775
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
The HTTP API mapped every ApiError to 500 Internal Server Error, so an
API client could not distinguish "the VM has not been created yet" from
a genuine server-side failure without parsing the error message text.
Derive the HTTP status code from the error itself in error_response():
errors whose root cause is VmError::VmNotCreated or VmMissingConfig are
now reported as 404 Not Found, regardless of which API action surfaced
them. The existing 400 (bad request) and 429 (too many requests)
mappings are preserved.
State-conflict errors such as VmNotRunning would ideally map to 409
Conflict, but micro_http's StatusCode has no Conflict variant, so they
remain 500 for now.
Fixes: #7774
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
When the guest writes DRIVER_OK and the device fails to activate, the
VMM previously bubbled the error up via VirtioActivate and never
released the activation barrier, leaving the vCPU that wrote DRIVER_OK
blocked on the barrier and effectively deadlocking the guest.
Per virtio 1.3 section 2.1.2, a device that has experienced an error
it cannot recover from should set DEVICE_NEEDS_RESET in its status and
notify the driver via a configuration change interrupt. Do that on
activation failure through the existing mark_device_needs_reset
helper, then release the activation barrier so the vCPU can resume.
DeviceManager::activate_virtio_devices now logs and continues instead
of aborting the whole pending list, so one failing device does not
take down the VMM or block pause and migration. The activator has
already reported the failure with the device id.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The new event allows management software to handle the migration better
via events. The `vm.migration-receive-ready` event tells that the VMM is
ready to accept connections whereas `vm.migration-receive-started` means
a migration is incoming.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Mark serial_number/uuid as deprecated in the OpenAPI schema and emit
warnings when those legacy --platform keys are used, while continuing to
accept them for compatibility.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Extend SMBIOS System Information with manufacturer, product,
version, family, sku, serial, and uuid fields, add a chassis
asset tag, and pass a structured SMBIOS config from --platform
into arch setup. Keep OEM strings and legacy serial_number/uuid
options working for compatibility. The platform option naming
follows `dmidecode -s <field>`.
Fields:
- system_manufacturer
- system_product_name
- system_version
- system_family
- system_serial_number
- system_uuid
- chassis_asset_tag
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Add a small SMBIOS config that carries serial_number, uuid,
and OEM strings, and pass it from platform config into
x86_64 setup.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Split the System Information write into helper functions and
reuse the string writer so the table layout and inputs are
unchanged.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Since enabling AMX tile state components affect the result returned by
`Hypervisor::get_supported_cpuid` we want this enabled prior to checking
CPUID compatibility between the source and destination VMs.
Although this is not required right now, it will be necessary once we
introduce CPU profiles and it will also be necessary if we decide to
make `check_cpuid_compatibility` (arguably) more thorough by also taking
state components into account.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We integrate the CPU profile into the various configs that
ultimately get set by the user.
This quickly ends up involving multiple files, luckily Rust
helps us find which ones via compilation errors.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The current OpenAPI schema format is wrong because uint16 and uint32 are
not valid top-level OpenAPI types. Describe queue_size and virtio_id as
integer types with explicit uint formats instead.
Signed-off-by: Roman <roman@vanesyan.com>
This change will allow us to get accurate time over ptp in guests
started from a MSHV-virtualized Linux host. Implementing it as a
virtio device is preferable to using the existing kvm_ptp because:
kvm_ptp relies on hypercalls that only exist on host kernels running
kvm. Virtio-rtc gives us more flexibility in what clock types we want
to provide. We can later extend the device to implement multiple clocks
(smeared UTC, TAI, monotonic, etc.). Virtio-rtc protocol supports
alarms. Alarms may later enable usecases where the guests can do their
own VM lifecycle management without relying on a host-side
orchestrator.
Implement device backend for virtio-rtc. Currently this implementation
encompasses:
1. CONFIG, CAP, READ, CROSSCAP (returns false)
2. One PTP clock is presented of type
VIRTIO_RTC_CLOCK_UTC_MAYBE_SMEARED with leap_second_smearing
VIRTIO_RTC_SMEAR_UNSPECIFIED
The device is disabled by default, requiring --rtc to be passed
Not implemented but theoretically supported by virtio-rtc is:
1. Cross-timestamping support
2. The alarm queue
Fixes#7730
Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Several riscv64 paths are compiled by the KVM build but missed
imports or cfg coverage needed by the current code.
Import the vm-memory Bytes trait for the RISC-V UEFI loader, keep
Instant available for migration timing code, and enable the UEFI flash
error path for riscv64.
Assisted-by: OpenAI-Codex:GPT-5
Signed-off-by: wangyf0611 <wangyufeng@iscas.ac.cn>
riscv64 does not provide the legacy readlink and unlink syscalls, so
libc uses readlinkat and unlinkat for paths that are otherwise allowed
on other architectures.
Permit readlinkat and unlinkat for the VMM and vCPU seccomp filters on
riscv64, matching the existing aarch64 rules.
Assisted-by: OpenAI-Codex:GPT-5
Signed-off-by: wangyf0611 <wangyufeng@iscas.ac.cn>
Each VM reboot re-entered the VFIO/virtio-net add paths and re-appended
the same originating fds, leaving duplicates in preserved_fds and a
double-close hazard at final teardown. Switching to HashSet makes
add_preserved_fds idempotent.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Assisted-by: Claude:Opus-4.7
When SVE is enabled, KVM replaces the FPSIMD V-registers with wider
SVE Z-registers. Attempting to access the old FPSIMD offsets returns
EINVAL.
Fix by classifying each register from KVM_GET_REG_LIST as core, system,
or extended. Extended registers (currently SVE only) are saved as
generic `ExtendedReg` entries split into `pre_finalize_regs` (registers
like SVE VLS that must be written before `vcpu_finalize`) and
`extended_regs`. FPSIMD registers are only accessed when SVE is absent.
Unrecognized register families error immediately so future extensions
like SME fail clearly rather than silently losing state.
The snapshot is deserialized before vCPU init to make pre-finalize
register state available for the init -> VLS -> finalize ordering
required by KVM.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Split the SVE detection and finalization logic out of `init()` into a
dedicated `finalize_sve()` method. This separates the concerns of vCPU
initialization (preferred target, processor features, KVM init) from SVE
finalization, and enables a subsequent commit to insert SVE VLS register
in the restore path without needing to add a restore state in `init()`.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Storing the snapshot causes issues when needing to do a subsequent
hotplug instead just pass it through on all the methods that need it
making the lifecycle cleaner.
Assisted-by: Claude:Opus-4.6
Signed-off-by: Rob Bradford <rbradford@meta.com>
The serial-manager thread was the only VMM-managed thread without a
seccomp filter. Add a Thread::SerialManager variant and whitelist the
31 syscalls needed for its epoll-based I/O loop (read, write, socket
ops, signal handling, memory allocation, glibc internals).
The filter is computed in start_thread() and applied before the epoll
loop, matching the pattern used by other VMM threads.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Assisted-by: Pi-agent:Claude-Opus-4.7
Only Thread::Vmm and Thread::Vcpu need to know the hypervisor type.
Make the type optional, and then simplify the users.
Assisted-by: Pi-agent:Claude-Opus-4.7
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Userfaultfd is a great mechanism for providing fast restore to Cloud
Hypervisor VMs. But that means the price to pay for bringing pages in
happens at runtime, which might slow down the guest when it's touching
pages which haven't been brought in yet.
By prefaulting the pages in the background, we're trying to get the best
of both worlds. That means we still get a very fast restore with the
uffd handler, but within a few seconds (depending on VM's RAM size), we
also get the pages fully faulted and we can stop the uffd handler thread
at that point.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-7
The JSON parser expects the field to be named
`iommu_address_width_bits`, but the code declared it as
`iommu_address_width`. This mismatch caused the field to be
unrecognized when deserializing configuration from JSON.
Rename the field to `iommu_address_width_bits` to match the
expected schema.
Signed-off-by: tchaton <thomas.chaton.ai@gmail.com>
This reverts commit ced3762a67.
This change lead to a serious memory regression when not using hugepages
or shared=on.
`MAP_PRIVATE` creates an anonymous memory allocation for every page
written when the backing store is a file. This CoW behaviour is useful
but leads to double allocations when the backing store is an empty file
created by `memfd_create()`. When the page is written to, the CoW
semantics require a real page to be created in the memory for the memfd
(previously before the page was touched they would all point to the zero
page). This real page is filled with zeroes because in theory this page
would be accessible via read/write syscalls on the FD even though in our
implementation it is only ever `mmap()`ed.
The intention of the commit was to enable `fallocate()` to be used to
punch holes but that would only affect the inaccessible backing page and
the page in the CoW anonymous memory would be unaffected. Leading it
likely not to have the desired effect.
Fixes: #8211
Signed-off-by: Rob Bradford <rbradford@meta.com>
When reading from an unregistered PIO address, pio_read() wasn't
initialising the buffer, so guests were reading stale bytes from the
previous PIO transaction rather than all 0xff bytes like master abort
on real hardware.
Fill data with 0xff on invalid reads.
Correct 'read to unregistered address' info message to 'read from
unregistered address' while we're touching this block.
Signed-off-by: Chris Webb <chris@arachsys.com>
When reading from an unregistered MMIO address, mmio_read() wasn't
initialising the buffer, so guests were reading stale bytes from the
previous MMIO transaction rather than all 0xff bytes like master abort
on real hardware.
Fill data with 0xff on invalid reads.
Correct 'read to unregistered address' info message to 'read from
unregistered address' while we're touching this block.
Signed-off-by: Chris Webb <chris@arachsys.com>
I started by looking at all `Option<Vec<T>>` values in config.rs and
vm_config.rs, and replaced them with `Option<Box<[T]>>`. This has the
advantage that one now can see at a glance if this field will ever
resize during operation or not, reducing cognitive load and increasing
maintainability. All fields that need the properties of a Ver or where
this change was not trivial are kept intact.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
When no image_type is specified, sector 0 writes are disabled as a
safety measure for autodetected raw images. Extend this protection
to autodetected fixed VHD images, which carry metadata in the last
sector and are equally susceptible to accidental overwrites of the
first sector when the format is not explicitly acknowledged.
Update the corresponding warning in the virtio block worker to be
format agnostic.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Explanatory comments for the rollback paths in both PCI BAR
relocation branches.
Assisted-by: Claude (Anthropic)
Signed-off-by: CMGS <ilskdw@gmail.com>
