Handle MSHV INS/OUTS port intercepts by translating the guest string
operand through MshvEmulatorContext and copying data between guest
memory and existing PIO callbacks.
Support REP counts, zero-count REP, and direction-flag based RSI/RDI
updates. Commit RIP plus RCX/RSI/RDI after the transfer completes.
This removes the fw_cfg/debug-port skip. OVMF can now use the real
string I/O path instead of relying on ignored ports.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Assisted-by: Copilot:GPT-5.5
Decode MSHV port access size, direction, string, and REP state through
small helpers instead of open-coded bitfield reads.
Keep scalar I/O behavior unchanged and continue rejecting string I/O in
this step. Put the x86_64 port I/O helpers in their own module so the
string emulation path can grow without bloating the top-level MSHV code.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Assisted-by: Copilot:GPT-5.5
Move common string instruction bookkeeping into the x86 instruction
emulator so MOVS, STOS, and MSHV port string I/O use one implementation
for REP counts, direction-flag handling, and index advancement.
This keeps existing MOVS/STOS behavior unchanged while removing the need
for MSHV to open-code the same string-operation details.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Assisted-by: Copilot:GPT-5.5
Move scalar HVMSG_X64_IO_PORT_INTERCEPT handling into a helper so the
string I/O implementation can build on the same dispatch path.
Keep the existing fw_cfg/debug-port skip plus string/REP assertions in
place. This is only code movement so later changes are easier to review.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Assisted-by: Copilot:GPT-5.5
Turn reserve=on for the hugepage-backed memory zone in the UFFD
snapshot/restore integration test. Hugepages are the most likely place
to want reserve (an over-committed huge page pool is exactly the case
that otherwise SIGBUSes the guest), so this is the natural test to give
the option real coverage, as suggested in review.
It exercises the reserve mmap path twice: once on the source VM boot and
once on the demand-paged restore. The existing skip guard already
requires the 256 free 2MiB pages this zone needs, and the source VM is
killed before the restore VM is started, so reserving from the pool
never has to back two VMs at once.
Assisted-by: Claude Code (Opus 4.8)
Signed-off-by: Ian Klemm <hi@ianklemm.de>
Cloud Hypervisor maps guest RAM with MAP_NORESERVE, so the kernel never
reserves the backing pages at mmap time. On a host whose hugepage pool
cannot satisfy every guest, a VM is created successfully and then takes
a SIGBUS when the guest faults a page the pool can no longer back. This
is the failure mode reported in #5730 and #7387. As noted on #5730,
checking free pool headroom up front is not a reliable fix: another
process can consume pages between the check and the fault.
Add a reserve=on parameter to --memory and --memory-zone (default off,
preserving the current MAP_NORESERVE behaviour). When set, the region
is mapped without MAP_NORESERVE, so the kernel reserves the backing
pages (swap, or huge pages for hugepage-backed memory) at mmap time,
atomically with the mapping. An over-committed configuration then
fails cleanly at VM creation with an mmap ENOMEM instead of crashing
the guest later. Unlike prefault it does not fault the memory in, so
it does not slow down boot.
This mirrors QEMU's memory-backend reserve property, which has the same
name and meaning (reserve=off maps with MAP_NORESERVE). reserve is
threaded through the same mmap paths as the existing prefault option,
and is exposed in the OpenAPI schema, CLI help and docs. The top-level
--memory reserve=on path is unchanged: the default zone is synthesised
from MemoryConfig and inherits its reserve value.
Assisted-by: Claude Code (Opus 4.8)
Signed-off-by: Ian Klemm <hi@ianklemm.de>
remove_userspace_mapping tears down the KVM mapping but never returns
the slot id to the allocator's free list. Call `free_memory_slot` to
avoid the leak.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Import the std modules used in the crate instead of spelling the full
paths at every use site.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Bring the confidential VM (CVM) integration tests up on the KVM SEV-SNP
backend in addition to MSHV. On KVM the IGVM is an Oak stage0 firmware
image and the guest kernel is supplied separately: stage0 reads the
kernel, cmdline and E820 over fw_cfg. The test harness selects this
model when a guest kernel is present at /igvm_files/bzImage, mirroring
how the stage0 IGVM is discovered; MSHV keeps using the monolithic IGVM
with the kernel baked in.
- test_infra: stage0 + direct-kernel + fw_cfg boot wiring (both the
command line and the HTTP/D-Bus API path) plus an on_kvm_sev_snp()
helper for tests to branch on.
- tests: the CVM tests that don't work on the KVM SEV-SNP path yet are
gated with #[cfg(not(feature = "kvm"))] inside the common_cvm module.
The MSHV build enables mshv,igvm,sev_snp (no kvm feature) while the
KVM build enables kvm,igvm,sev_snp,fw_cfg, so the cfg compiles these
tests into the MSHV binary only and drops them on KVM; both
hypervisors run the single common_cvm nextest profile. They all still
run on MSHV:
* test_pci_multiple_segments - stage0 places all 64-bit BARs in a
single global window, so a BAR allocated in a different
per-segment window is relocated cross-window and wedges boot.
* test_dmi_uuid / test_dmi_oem_strings /
test_dmi_system_and_chassis - SMBIOS is not delivered to SEV-SNP
guests on the KVM stage0 boot path, so the guest's DMI tables
read empty. VMM follow-up.
* test_vdpa_block - needs host vdpa_sim_blk setup, and vDPA DMA
into SEV-SNP-encrypted memory is unsupported (the guest hangs).
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
FwCfgConfig already has a Default impl (e820/kernel/cmdline/initramfs/
acpi_tables = true, items = None), but deserialization did not use it:
without serde default every field was mandatory, so any caller building
a payload config over the API had to spell out the whole object even to
flip a single flag.
Add a container-level #[serde(default)] so missing fields fall back to
FwCfgConfig::default(). The container form is required here because the
defaults are all true; a per-field #[serde(default)] would resolve bool
to false and contradict the Default impl.
This lets callers send only the fields that differ from the defaults.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
validate_memory() asserts that the guest's reported MemTotal exceeds the
expected size, but on failure printed nothing about either value.
Include both the actual and expected figures in the panic message so a
failing run is self-explanatory.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
The KVM SEV-SNP net-hotplug integration tests (supported added in
later commits) intermittently kill the VMM with SIGSYS on the
http-server thread. The thread is seen reading
/proc/sys/vm/overcommit_memory (openat + read).
Allow the syscalls there, gated on sev_snp+kvm. seccomp can't match
a path, so the open is restricted to O_RDONLY.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
On KVM, an SEV-SNP guest opens /dev/sev at VM creation and passes
its fd as kvm_sev_cmd.sev_fd. KVM uses it to issue the SNP_LAUNCH_*
commands. The open happens after the landlock ruleset is applied,
so grant rw access to /dev/sev when the sev_snp platform feature
is enabled.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
A confidential guest (e.g. SEV-SNP) requires every virtio device to
advertise VIRTIO_F_ACCESS_PLATFORM so the guest driver routes DMA
through the platform's bounce-buffer path; the driver refuses a device
that does not offer it.
Add VIRTIO_F_ACCESS_PLATFORM support to virtio-{watchdog,iommu} which
are exercised as part of the CVM integration tests.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Import the std modules used in the crate instead of spelling the full
paths at every use site.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Import std::fmt and std::result instead of spelling the full paths at
every use site.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
The guest kernel build instructions in the README referenced the
ch-6.12.8 branch, which was inconsistent with the version used by the
test scripts (scripts/test-util.sh uses ch-6.16.9). Update the README
to point at ch-6.16.9 so documentation and CI stay in sync.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Delete the now unused QcowFile implementation it was only used for
creating disk images for the tests and for the performance-metrics. It
was not used for the virtio-block device.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Pull rebuild_refcounts out of QcowFile so QcowFile can be removed in a
follow up commit. This function is still required by parse_qcow().
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Port the internal/mod.rs tests from QcowFile to QcowDisk so
the surviving tests exercise the code paths that are hit when used via
QcowDisk (and thus virtio-block).
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Fuzzing QcowFile, which is not used by virtio-block, is less helpful
than using the abstraction that is. Ensure we cover at least the same
functionality as before.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Switch from QcowFile to QcowDisk taking advantage of QcowTempDisk where
appropriate.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Port tests over to QcowTempDisk and also over to QcowDisk rather than
QcowFile where necessary.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
A common pattern in the test code is to create a temporary file, format
it as QCOW2 and then open it as a QcowDisk. Create a helper struct that
can be used in those tests. This is marked as #[cfg(test)] as initially
it will only be used by the test suite.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add a method to format a file as a QCOW2 file which will mainly be used
by the test infrastructure. This copies the logic from QcowFile. It
doesn't refactor it as the removal of QcowFile is planned.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Rob Bradford <rbradford@meta.com>
With socket serial output now buffered and replayed on connect, a
late-connecting client receives the whole boot backlog. The pty
interaction test had three problems with that:
- pty_read() slept a second between 512-byte reads and the loop consumed
one chunk per two-second tick, far too slow to drain the backlog. Read
in larger chunks without the per-read sleep and drain everything
available each round; bound the loop so a missing marker can't run to
the harness timeout.
- it wrote the login keystrokes before reading, so the unread backlog
back-pressured the sender and the keystrokes never reached the prompt.
Start reading concurrently with typing instead.
- the socat pty was created with echo on, so the replayed backlog was
echoed back to the guest as serial input, flooding it (UART input
overrun, login never completing). Create the pty with echo=0.
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
In Socket serial mode the device output sink was only installed once a
client connected, so output produced beforehand (kernel boot messages,
cloud-init) was dropped, and a client attaching after boot saw a blank
screen. Only PTY mode wrapped the sink in a SerialBuffer.
Install a persistent SerialBuffer as the Socket device's output sink at
SerialManager construction (discarding downstream via io::sink() until a
client connects), so output is captured into the 1 MiB ring even with no
client attached. On connect, retarget the buffer at the accepted client
and flush the backlog before live output resumes; on disconnect, keep
buffering so output produced while no client is attached is delivered to
the next one. The accepted socket is made non-blocking via
set_nonblocking() so a slow client cannot stall the vCPU thread
(SerialBuffer re-buffers on WouldBlock).
The serial-manager thread gains two syscalls under seccomp: sendto
(replaying the backlog is the first time it writes to the socket) and
ioctl restricted to FIONBIO, which is what set_nonblocking() issues.
Fixes: #7907
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
SerialBuffer owns its downstream writer privately, with no way to
replace it. Buffering the Socket console requires keeping one buffer
alive across client connects and disconnects and pointing it at each
newly accepted client (or a discarding sink when none is connected)
without dropping bytes buffered while no client was attached.
Add set_out(), which swaps the writer while leaving the buffered
contents intact, plus unit tests covering accumulate-while-detached,
replay on connect, live pass-through, delivery of while-detached output
to the next client, and that bytes already drained by one client are not
resent to the next.
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Import std::io instead of spelling the full paths at every use site.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Import the std modules used across the crate instead of spelling the
full paths at every use site.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
KVM will emulate XSAVE for us, so we need only to skip XCRS
setting/retrieval if the respective CPU feature is not available.
Signed-off-by: Nikolas Kyx <55556836+nyx191@users.noreply.github.com>
eject_device calls shutdown, but Block doesn't implement it, so the
worker thread was never joined. Drop for Block does not help either,
VirtioPciDevice keeps an Arc<Mutex<Block>> past eject, so Drop never
runs. The async worker keeps completing in-flight I/O into the guest RAM
that backed it when issued.
Implement shutdown() to call wait_for_epoll_threads(), which drops the
WorkerThreads handle and ensures that worker's io completes.
Signed-off-by: Dylan Reid <dgreid@fb.com>
server_api_error_display_modifier deserialized the JSON error response
into a `Vec<&str>`. However, if the error message contained escaped
characters, it could not deserialize it into a borrowed string `&str`
because unescaping requires allocation. This resulted in a
deserialization error and a failure to print the error chain.
This change switches the deserialization target to `Vec<String>` to
allow allocation.
Signed-off-by: Andrei Vagin <avagin@google.com>
The snapshot and restore subcommands in ch-remote had optional
snapshot_config and restore_config arguments, but the implementation was
unconditionally unwrapping them.
This change marks these arguments as required to handle the missing
argument validation and report a proper error message instead of letting
the application panic.
Signed-off-by: Andrei Vagin <avagin@google.com>
Prevent `SkuNotAvailable` errors for mshv workflow by checking capacity
restrictions for each location. Enhance the VM provisioning logic to
validate resource availibility before deployment.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
Describe --hypervisor as an optional override that defaults to
auto-detection from the host device node (/dev/mshv or /dev/kvm) for
both the build and tests commands and the shared test-script arguments.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Drop the explicit --hypervisor mshv from the mshv integration workflow.
The runner exposes /dev/mshv, so dev_cli.sh now selects MSHV through
auto-detection.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Make the --hypervisor argument optional for the build and tests
commands. Add detect_hypervisor_device() and resolve_hypervisor_device()
helpers that map an explicit kvm/mshv name to its device node, or fall
back to probing /dev/mshv and /dev/kvm on the host when the flag is
omitted. The resolved device is validated before it is mounted into the
container, and an explicit --hypervisor is still forwarded to the test
scripts.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Make the --hypervisor argument optional in process_common_args().
When it is not supplied, detect_hypervisor() now selects the backend
from the host device node: /dev/mshv for MSHV, /dev/kvm for KVM, and
errors out when neither is present. An explicit --hypervisor still
overrides the detection.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Local-time log fields called `jiff::Zoned::now()`, which resolves the
system timezone on every record by reading
`/etc/localtime`/`/etc/timezone` if it doesn't hit the cached version.
This cache miss could then cause a seccomp violation depending on the
thread it was run from.
Avoid this by capturing the value in `Logger`. This avoids opening the
seccomp filter for the whole process.
Signed-off-by: Dylan Reid <dgreid@fb.com>
If `KVM_GET_CLOCK` already filled out the `realtime` field, it sets the
`KVM_CLOCK_REALTIME` flag, but if we instead preserve this flag, the
kernel will automatically adjust the kvmclock clock when calling
`KVM_SET_CLOCK` based on the elapsed wall-clock time between pause and
resume. This just requires removing the `reset_flags()` function, which
allows the `KVM_CLOCK_REALTIME` flag to persist in the serialized clock
state.
However, the kernel does not always fill the `realtime` field, depending
on clock source; in this case, fill `realtime` during pause based on the
system time. This is not as precise as the automatic `KVM_GET_CLOCK`
version, since we query the time slightly after the vCPU was paused, but
it allows the clock to be resumed mostly in sync instead of being wildly
off. In this case, we also set the `KVM_CLOCK_REALTIME` flag in the
saved `struct kvmclock` so `KVM_SET_CLOCK` will adjust the clock on
resume.
Basic test case:
1. Run a VM with a Linux guest.
2. Pause the guest via `vm.pause` API.
3. Wait several minutes.
4. Resume the guest via `vm.resume` API.
5. Verify the guest time (e.g. via `date` command) is valid.
6. Verify guest is still using `kvm-clock` timesource:
cat /sys/devices/system/clocksource/clocksource0/current_clocksource
Before applying the patch, the guest clock would be off by the delta
time between pause and resume; after the patch, the clock is (more or
less) in sync with the correct wall-clock time.
Old snapshots will not have the `KVM_CLOCK_REALTIME` flag populated, so
they will not be affected by the new behavior.
Signed-off-by: Daniel Verkamp <drv@meta.com>
The API socket path was removed unconditionally when the process
exited, including on a failed start. On a failed start where another
running instance already held the path, that deleted the live
instance's socket.
Remove the socket only when start_vmm returned Ok, meaning this process
owned and bound it. A stale socket left by a crash is cleaned up under
the lock by the next start, so dropping the unconditional removal does
not leak sockets, and a failed start no longer clobbers a socket owned
by another instance.
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
When Cloud Hypervisor crashed or was killed, the API socket file was
left on disk, so the next start failed with EADDRINUSE ("Address already
in use") and the VMM could not restart. This affects any environment
where the socket directory survives across restarts (systemd services,
Kubernetes emptyDir volumes, and so on).
Before binding the path-based API socket, take an exclusive lock on a
sidecar "<socket>.lock" file using the block crate's OFD-lock helper.
Holding it proves no other instance is bound to this path, so a stale
socket left by a crashed run can be removed safely and race-free. If
the lock is already held, fail with a clear "API socket is already in
use" error instead of clobbering the live instance. The lock is held
for the process lifetime and released by the kernel on exit or crash.
The fd-based (socket-activation) path is left unchanged.
This implements the lock-file approach suggested by @DemiMarie.
Fixes: #7784
Signed-off-by: Max Makarov <maxpain@linux.com>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Fix racy seccomp kill on shutdown. When a VM shuts down the
event-monitor thread's recv() loop ends and the thread exits. glibc's
thread teardown then runs __malloc_arena_thread_freeres, which trims the
per-thread malloc arena with madvise(MADV_DONTNEED).
Add madvise to the allowed calls to match other threads. The crash is
intermittent because it only fires when that thread's arena accumulated
trimmable memory by shutdown.
Signed-off-by: Dylan Reid <dgreid@fb.com>
