Replicate the checks that are made against the top level MemoryConfig
into those for the per-zone configuration.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The introduced `TcpStream` accessor allows setting socket options on the
`TcpStream` without going through the file descriptor and unsafe
methods.
On-behalf-of: SAP julian.schindel@sap.com
Signed-off-by: Julian Schindel <julian.schindel@cyberus-technology.de>
Multiple QcowDisk/Qcow2Backing can share references to the same
QcowMetadata via an Arc. Unfortunately the .shutdown() which clears the
DIRTY bit was being called when the first of those was dropped. Instead
move this to the drop of the metadata itself. Now only once all
references to the metadata are dropped then we can safely set the DIRTY
bit.
Assisted-by: Codex:GPT-5.6
Signed-off-by: Rob Bradford <rbradford@meta.com>
The ivshmem device was remapping the host file every time the BAR was
reprogrammed. This occurs on the vCPU thread and so was falling foul of
our improved seccomp filters. Avoid this by maintaining the existing
region when the device BARs are changed and just adjust the guest GPA.
Assisted-by: Codex:GPT-5.6
Signed-off-by: Rob Bradford <rbradford@meta.com>
Unfortunately glibc can read the overcommit sysctl from any thread. This
has lead to us adding a patchwork of openat/read syscalls to our allow
list when those threads don't necessarily need openat for their actual
uses.
Only the VMM and migration worker thread have a strict requirement for
the openat syscall. The syscall was added to the other threads to deal
with this glibc behaviour.
As read() is itself harmless move it to the common syscalls, strip
full openat() from all but the threads that need it and add limited,
read only, openat to all threads.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This test flakes out regularly and never succeeds on retry (as it is
more likely to fail under the reduced load of the retry)
See: #8651
Signed-off-by: Rob Bradford <rbradford@meta.com>
Vcpus need to be configured according to the selected CPU profile.
We thus store the computed vCPU MSR config update in the CPU manager
and use it whenever a vCPU is created.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We introduce a function somewhat analogous to `generate_common_cpuid`,
except that it is only relevant for CPU profiles.
This function is more "high level" than the
CpuProfile::required_msr_updates method and is intended to be called
from the vmm crate.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
In order to safely apply CPU profiles we need to ensure that the host's
feature MSRs that are permitted by the CPU profile are also compatible
with the values the CPU profile dictates.
KVM_SET_MSRs takes care of checking compatibility for most feature MSRs
on both Intel and AMD CPUs that we will permit CPU profiles to have
(more on this in the upcoming CPU profile generation tool PRs),
but there is one exception.
Userspace may set whatever value for the Intel exclusive
IA32_ARCH_CAPABILITIES MSR without receiving any complaints from KVM.
We thus introduce our own compatibility check for
IA32_ARCH_CAPABILITIES.
One might even argue that KVM_SET_MSRs is called relatively late when
creating or receiving a VM and that it would be preferable to have
compatibility checks for all permitted feature MSRs run earlier. This
would also mean more informative debug logs.
We argue however that those additional checks would lead to too much
code that is not strictly necessary which is why we decided against
doing that in this patch set.
The code introduced here is only intended to be used with CPU profiles
at this point in time, but it is written in such a way that it could
also be used for MSR compatibility checks between a migration source
and destination (when the Host profile is in use) should that be
desired in the future.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We introduce a method on the CpuProfile enum that computes the required
MSR related updates in order to be compatible with the CPU profile.
The output of this function describes the feature MSRs that need to be
set and the snapshottable MSRs that are compatible with the chosen CPU
profile. The former provides the guest information about CPU features
(that are not obtainable from inspecting CPUID) while the latter is
used to ensure that MSRs that are incompatible with the CPU profile
are not set upon snapshot/restore.
These will be utilized in the vmm crate in a follow up commit.
We expect the hypervisor to take care of checking compatibility of the
feature MSRs when they are set. IA32_ARCH_CAPABILITIES is a known
exception however which we introduce our own checks for in a follow
up commit.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
When Cloud hypervisor has not been configured for (KVM) Hyper-V we want
to adapt the CPU profiles not to require existence of Hyper-V related
MSRs.
The first step introduced here is to create a list of such MSRs.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We introduce a type describing MSR adjustments associated with a
CPU profile.
The upcoming CPU profile generation tool will serialize instances of
this struct when generating a CPU profile.
A follow up PR will take care of filling out the currently stubbed
`msr_data` method.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
CPU Profiles will be serialized to JSON by the upcoming CPU Profile
generation tool and we want MSRs to be serialized as hex strings.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We create 64-bit analogues of the already existing hex (de-)serializer
helper functions for the 32-bit case.
These are necessary because the CPU profile needs associated data
describing how to adjust feature MSRs whose values are 64-bits.
In this case we prefer some small amount of code duplication over
macros and/or traits since we do not expect the need for further
variants of these helpers.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Feature MSRs must have consistent values across snapshot and restore,
otherwise we risk subtle time of check to time of use errors.
We thus adapt `Vcpu::set_state` to return a hard error if any feature
MSR cannot be restored.
It is enough to check that each MSR failing to be set does not have
an address corresponding to any of the feature MSRs stored in the
`KvmVcpu`. This is because the `msrs` in the `VcpuKvmState` contain all
the feature MSRs required by the selected CPU profile by construction.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Whenever a CPU profile is selected there will be feature MSRs that need
to be set when the CPU is configured.
We thus adapt the boot_msr_entries method to include the feature MSRs
that are passed to the vCPU upon construction.
Since KVM applies compatibility checks for all feature MSRs (except
IA32_ARCH_CAPABILITIES) we can only proceed if all MSR boot entries
are successfully set upon vCPU configuration.
This new check also applies to the regular boot entries when the host
profile is selected (the default), but that is arguably an improvement
in terms of correctness.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Although it is already possible to obtain the hypervisor type from the
hypervisor trait object, there is currently no way for functions that
only have a vCPU object to work with to know what the hypervisor
backend is.
In the rare case where there is significant divergence between the
supported hypervisor backends when working with vCPU objects the caller
must necessarily branch on the hypervisor type.
We thus introduce a hypervisor_type method on the Vcpu trait.
This will be utilized in a follow up commit where we need special
handling when configuring MSRs of vCPUs.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
When applying a CPU profile we need a way to change the configuration of
each vCPU to respect the requirements of the CPU profile.
This means that we need to set the feature MSRs in accordance with the
CPU profile upon configuring the vCPU and also ensuring that we do not
attempt to restore any MSRs that are not compatible with the profile
upon snapshot/restore.
The first step is to update `Vm::create_vcpu` to take an extra
parameter describing the necessary update.
In the case of KVM we modify the internal MSR state buffer when
constructing the vCPU whenever a VcpuMsrConfigUpdate is present.
The feature MSRs contained in the configuration will be treated in
follow up commits.
The changes to the vmm crate that are part of this commit are just
the minimum necessary to make the crate compile. We will update the
vmm crate to take CPU profiles into account in a follow up commit.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
CPU profiles will describe a list of MSRs they require when applying
the profile we want to check that the host has all the required MSRs
otherwise we have an incompatibility issue.
In order to check which MSRs the host supports we start by exposing the
hypervisor's get_msr_index_list method.
The MSRs required by a chosen CPU profile will be checked against the
output of the get_msr_index_list and get_feature_msrs on the hypervisor
in a follow up commit.
The get_msr_index_list method does not have an obvious implementation
for MSHV, because in that case one needs to obtain MSR indices from
the VM fd, after the VM has been initialized.
Since CPU profiles are only intended for the KVM hypervisor to begin
with, we leave the MSHV implementation as unimplemented for now. A
proper solution for MSHV should rather be found if/when CPU profiles
are also desired in that context.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
KVM defines feature MSRs as MSRs that expose host capabilities and
processor features.
CPU profiles will describe how to adjust such feature MSRs similarly to
how they describe CPUID modifications.
We thus introduce a method on the hypervisor trait that queries the
hypervisor for the supported feature MSRs.
This will later be used to obtain the feature MSRs that need to be
adjusted when a CPU profile is applied.
The method will also be used by the upcoming CPU profile generation
tool which will be introduced in a follow up PR.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Bump the pinned virtio-villain tag to v0.6.4. This release fixes
runtime behavior in the batch runner so it no longer reports spurious
WEDGED when a batch times out or a VMM writes to stderr.
With those fixes and the new retries flag the excludes for the flaky
tests are no longer needed and are dropped. A flaky failure is retried
a couple of times rather than suppressed, while a consistent failure
still surfaces. This mimics the nextest approach.
It also adds a large batch of block and net conformance tests, so the
run covers more of the device model.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
vmm_sys_util marks IoContext::submit as a safe function, so the unsafe
block was inert and only silenced by allow(unused_unsafe). Call submit
directly.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
AioDataIo and UringDataIo each carried their own eventfd, their own
completion queue, and the same enqueue and signal idiom. Route both
through the existing CompletionCommon and keep each backend's own in
flight map.
The aio drain enqueues fetched events with complete rather than a
silent push, so a drain can leave the eventfd signaled and cause one
extra harmless device wake. The eventfd is a counter, so the extra
signal is safe and the syscall batching stays the same.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The Sync prefix is too narrow because the queue is not tied to
synchronous execution. Rename it to CompletionCommon, following the
Common suffix the codebase already uses for shared helper types such
as VirtioCommon. The queue holds no engine specific state, so it can
be reused more widely across backends.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Map NoDeviceToRemove and DeviceManager(UnknownDeviceId) to 404 Not
Found, update the OpenAPI 404 descriptions, and add unit tests.
Signed-off-by: Stepan Rabotkin <epicstyt@gmail.com>
Assisted-by: Claude:Opus-4.8
The relocate-on-write path in update_cluster_addr() pushes the old L2
table cluster onto unref_clusters before allocating the relocation
target. When that allocation fails at allocator exhaustion, the error
unwinds with the still-referenced table on the free list, its deferred
refcount update dropped, the L1 entry unchanged and the cached table
still clean. The next metadata flush publishes the cluster to the
allocator, and a later allocation hands the live L2 table to a new
writer, which overwrites it with a refcount block, another L2 table or
guest data. Retried writes against the same L1 slot re-enter the
clean-table branch and push further duplicates, so one cluster can also
be handed out to two writers at once.
Images holding compressed clusters reach this constantly: writing to a
compressed cluster always takes the decompress -> append_data_cluster
-> update_cluster_addr path, so every such write relocates its L2
table.
Allocate the relocation target first and release the old table only
once the allocation has succeeded. A failed relocation now leaves the
old table referenced, off the free lists and intact.
Add deterministic regression coverage for both entry points: a plain
write that exhausts the allocator at the L2 relocation step, and a
compressed-cluster write that triggers the same relocation.
Signed-off-by: doge <me@crackerben.com>
The vsock muxer tracked host-allocated local ports in a set, and
`remove_connection()` released `key.local_port` from it for every
connection it removed. Only host-initiated connections take their local
port from `allocate_local_port()`, though: guest-initiated ones use the
guest-chosen `dst_port` verbatim, and never hold an allocation.
Removal was therefore able to release a port its connection did not own.
A guest-initiated connection whose destination port collides with a
host-allocated ephemeral port dropped that port from the allocator's
free list on teardown, while the host-initiated connection that owned it
was still live and using it. The allocator could then hand the same port
out twice.
Track the owning peer (guest) port alongside each allocation, and
release an allocation only when the connection being removed matches the
recorded owner. Guest-initiated connections never match, so they can no
longer disturb host-side port accounting.
Reaching this from the guest additionally requires a host process
listening on the AF_UNIX path for a port in the host ephemeral range,
which is not how host services are normally deployed. Hitting the
duplicate allocation would further require the allocator to wrap its
~2^30 port range. This is a correctness fix rather than a fix for a
practically exploitable issue.
Add a regression test that drives the muxer through the colliding-port
sequence and checks the host connection keeps its allocation.
Signed-off-by: Lily Goscha <lilygoscha@gmail.com>
Add a test for verifying that snapshotting a VM while preserving the
source VM works as expected.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-8
Advertise VHOST_USER_PROTOCOL_F_DEVICE_STATE and implement the
associated callbacks for the vhost-user-block backend.
Given there's no specific state that needs to be saved through these
operations, this a blank implementation.
This support will be useful for testing the snapshot case where the
source VM is preserved after the snapshot is complete.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-8
Extend the migration protocol with a `preserve_source` option so that a
source VM can be preserved. This benefits the snapshot case where the
offload daemon can now snapshot a VM without tearing it down.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-8
For vhost-user devices, a snapshot involved the vrings to be stopped,
but they couldn't be resumed. This commit aims at fixing this by saving
the vrings base so that it can be safely reset after the snapshot has
succeeded.
This will allow VMs where snapshots are taken from internal CH
implementation and from an offload daemon to be resumed once the
snapshot is complete.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-8
PR #8320 changed several endpoints to return 404 (Not Found) when the VM
has not been created yet, but the OpenAPI specification was not updated
to document these responses. Add the missing 404 entries for vm.info,
resize, resize-zone and the (cold) add-disk/device/pmem/net/vsock/vdpa
endpoints so the spec matches the implemented HTTP status codes.
Signed-off-by: Stepan Rabotkin <epicstyt@gmail.com>
Assisted-by: Claude:Opus-4.8
Bumps the rust-vmm group with 1 update in the / directory: [micro_http](https://github.com/firecracker-microvm/micro-http).
Bumps the rust-vmm group with 1 update in the /fuzz directory: [micro_http](https://github.com/firecracker-microvm/micro-http).
Updates `micro_http` from `5c2254d` to `876f3fe`
- [Commits](5c2254d6cf...876f3feccc)
Updates `micro_http` from `5c2254d` to `876f3fe`
- [Commits](5c2254d6cf...876f3feccc)
---
updated-dependencies:
- dependency-name: micro_http
dependency-version: 876f3feccc30e09225f2c77bf95a6b2d46a9259e
dependency-type: direct:production
dependency-group: rust-vmm
- dependency-name: micro_http
dependency-version: 876f3feccc30e09225f2c77bf95a6b2d46a9259e
dependency-type: direct:production
dependency-group: rust-vmm
...
Signed-off-by: dependabot[bot] <support@github.com>
Opening an issue and establishing agreement is now mandatory for
feature PRs. For bug fixes opening an issue if one does not already
exist may be beneficial.
Signed-off-by: Rob Bradford <rbradford@meta.com>
On `VM::boot` call, lock disks only after the state change has been
checked for validity. Otherwise, on invalid state change, boot is
aborted, but file locks aren't rolled back.
On-behalf-of: SAP julian.schindel@sap.com
Signed-off-by: Julian Schindel <julian.schindel@cyberus-technology.de>
Filtered activation queues can hide regressions where resume treats
their positions as transport queue indices. Such regressions route
interrupts to unrelated MSI-X vectors while eventfd notifications still
reach the correct workers.
Add a VirtioCommon regression test with queue indices one and three.
Verify resume signals both queue eventfds and records interrupts for the
original sparse indices.
Assisted-by: OpenAI:GPT-5.6-Sol
Signed-off-by: Yi Wang <foxywang@tencent.com>
VirtioCommon stores only activated queue eventfds, so filtered queues
are renumbered when resume injects interrupts. A sparse queue set can
therefore wake the wrong MSI-X vectors and leave completed requests
stalled after restore.
Retain each queue index alongside its eventfd during activate(). Use
the saved index for resume interrupts so each notification reaches the
queue's original vector.
Assisted-by: OpenAI:GPT-5.6-Sol
Signed-off-by: Yi Wang <foxywang@tencent.com>
QCOW metadata published a fully deallocated cluster before the
caller performed the host punch-hole. Under multi-queue load, a
concurrent metadata flush could make the cluster allocatable, and
another queue could reuse it before the delayed punch ran. If reused as
an L2 table, the stale punch erased live metadata and made guest data
unreachable.
This was observed in production as confirmed guest data loss, with
allocated guest clusters becoming refcounted but unreachable from the
QCOW mapping.
Keep punch-pending clusters out of both free lists. Publish a cluster
only after the host punch succeeds, so another metadata flush is
required before reuse.
Add deterministic coverage for the cross-queue schedule and for host
operation failures.
Signed-off-by: doge <me@crackerben.com>
apply_dealloc_action() discarded the result of the host punch-hole
and write-zeroes operations, so a guest DISCARD or WRITE ZEROES
request completed successfully even when the host operation failed.
Return the error to the per-queue engine and complete the request
with an error instead, in both the synchronous and io_uring QCOW
engines. A failure does not abort the remaining actions of the
request: they are still applied, and the first error is reported.
Signed-off-by: doge <me@crackerben.com>
A Unix stream connection can succeed while the process owning the
listener is exiting. The resulting connection can then fail during
SET_OWNER, GET_FEATURES, or later reinitialization.
Retry the complete connect and reinitialization transaction when it
returns a transport failure within a 60-second retry window. Each retry
uses a fresh frontend. Connection errors retain their existing handling,
non-transport errors fail immediately, and waits remain interruptible by
the worker kill event.
Signed-off-by: Peter Delevoryas <pdel@meta.com>
Assisted-by: Codex:GPT-5
This adds a threat model and security policy. It is loosely based on
the Xen Project Security Policy, but with many changes to fit Cloud
Hypervisor better.
The threat model is comprehensive and includes:
- Which entities are fully trusted.
- What untrusted entities are and are not allowed to do.
- What Cloud Hypervisor expects from its deployment environment.
- What trusted entities can safely do on behalf of untrusted entities.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
