GenericVhostUserConfig flattens PciDeviceCommonConfig, so the wire
format accepts an optional id like the other PCI device configs.
FsConfig, which flattens the same struct, already documents id.
Add it here as well so generated clients can name the device and
later target it with /vm.remove-device.
Signed-off-by: leo03164 <leo03164@gmail.com>
The field is declared as generic_vhost_user in vmm/src/vm_config.rs
without a serde rename, so the wire format uses the underscore form.
The OpenAPI document names the property generic-vhost-user, which is
also the only hyphenated key in the VmConfig schema.
VmConfig deserialization does not reject unknown fields, so a client
generated from the document sends a key that serde silently ignores:
the devices are dropped on vm.create without any error.
Signed-off-by: leo03164 <leo03164@gmail.com>
Increase timeout for MSHV integration tests as enabling
live_migration/upgrade tests asks for more time to finish.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The VmState enum in vmm/src/vm.rs has five variants (Created,
Running, Shutdown, Paused, BreakPoint) but the OpenAPI document
only lists four. BreakPoint is observable through vm.info when a
guest_debug build is paused by the GDB stub (Vm::debug_pause).
Clients generated with strict enum handling (e.g. via
openapi-generator) fail to deserialize the vm.info response in
that state. The document already covers guest_debug-gated surface
such as /vm.coredump, so the enum should be complete as well.
Signed-off-by: leo03164 <leo03164@gmail.com>
With landlock enabled, the vmdk disk path representing the
descriptor file is added to allowed path. However, this is a
plain text file which points to actual data blobs called extents.
The extent paths are not allowed and must be explicitly passed via
landlock-rules path.
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Adds integration tests for vmdk to cover i/o scenarios
for single extent & multi-extent span read writes. All tests
run disk consistency check in the end. The tests also extend
direct_io for VMDK backend.
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Adds support to detect and open flat vmdk images.
In particular, it updates the `detect_image_type` method to
account for unaligned & small sized descriptor file (which describes
the VMDK disk) reads using AlignedFile::read_at instead of
read_file_at to loop over for small reads.
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Implements synchronous I/O engine for flat VMDK backend.
It uses an extent aware worker to map each request
to one or more backing extents. The implementation supports extents
opened with O_DIRECT using AlignedFile.
Async backends of io_uring and AIO are unimplemented because
requests spanning extents cannot be submitted with one fd + offset.
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Defines and implements the layout of a VMDK extent, the region
of storage that is used by the virtual disk. Each line in the
descriptor file's extent section describes one extent.
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Map VmError::ConfigValidation in api_error_status_code() to client-error
codes: 409 Conflict for a duplicate identifier or path, 400 Bad Request
otherwise. Update the OpenAPI responses for the vm.add-* and vm.restore
endpoints, fix the InvalidIdentifier message, and add unit tests.
Signed-off-by: Stepan Rabotkin <epicstyt@gmail.com>
Assisted-by: Claude:Opus-4.8
Bumps the rust-vmm group with 1 update in the / directory: [micro_http](https://github.com/firecracker-microvm/micro-http).
Bumps the rust-vmm group with 1 update in the /fuzz directory: [micro_http](https://github.com/firecracker-microvm/micro-http).
Updates `micro_http` from `876f3fe` to `f2d9170`
- [Commits](876f3feccc...f2d91703a1)
Updates `micro_http` from `876f3fe` to `f2d9170`
- [Commits](876f3feccc...f2d91703a1)
---
updated-dependencies:
- dependency-name: micro_http
dependency-version: f2d91703a103521430209f48ddf5925be1e35ff8
dependency-type: direct:production
dependency-group: rust-vmm
- dependency-name: micro_http
dependency-version: f2d91703a103521430209f48ddf5925be1e35ff8
dependency-type: direct:production
dependency-group: rust-vmm
...
Signed-off-by: dependabot[bot] <support@github.com>
Register EDX of CPUID leaf 0x7 sub-leaf 0x2 (Structured Extended Feature
Enumeration Sub-leaf) is currently not checked, but it should be.
KVM supports several of the features enumerated by this value when they
are supported by the host such as Intel PSFD, IPRED_CTRL, RRSBA_CTRL
and more.
The migration destination needs to support any such feature exposed to
the guest from the migration source.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Register EDX of CPUID leaf 0x7 sub-leaf 0x1 (Structured Extended Feature
Enumeration Sub-leaf) is currently not checked, but it should be.
KVM supports several of the features enumerated by this value when they
are supported by the host, such as AVX-VNNI-INT8, AMX-COMPLEX, AVX10
and more.
The migration destination needs to support any such feature exposed to
the guest from the migration source.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Register ECX of CPUID leaf 0x7 sub-leaf 0x1 (Structured Extended Feature
Enumeration Sub-leaf) is currently not checked, but it should be.
KVM exposes bit 5 (MSR_IMM: the immediate forms of the RDMSR and WRMSR
instructions) when the host does.
Live migrating from a source that supports MSR_IMM to a destination
that does not is not safe.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
During a live migration, connections may get interrupted silently,
without being reset or closed. Interrupted idle connections may stay
alive indefinitely, for example if a connection dies during prefaulting
on the receiver side.
To detect dead idle connections, we enable
`SO_KEEPALIVE`. With `SO_KEEPALIVE`, the kernel will send keepalive
probes if a connection is idle and close the connection if the probes
remain unacknowledged.
To detect dead connections when actively sending data in a timely
manner, we enable `TCP_USER_TIMEOUT`, to reduce the timeout for closing
a connection where the peer doesn't acknowledge sent data.
On-behalf-of: SAP julian.schindel@sap.com
Signed-off-by: Julian Schindel <julian.schindel@cyberus-technology.de>
Both engines walked the same cluster mappings to serve a read from
allocated clusters, the backing file, or zero fill. In the uring
engine this walk was a separate helper, scatter_read_sync, reached
only when the read was not a single contiguous allocated extent. In
the sync engine the same loop sat directly in the read path. Move
scatter_read_sync into common.rs and call it from both. The uring
contiguous fast path stays in resolve_read, so those reads still
offload to the ring.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Both engines ran the same copy on write loop for a partial cluster
write. In the uring engine it was a separate helper, cow_write_sync.
In the sync engine the same loop sat directly in the write path. Move
cow_write_sync into common.rs and call it from both. The uring write
already runs synchronously, so its behavior does not change.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The sync and uring qcow engines each carried an identical
apply_dealloc_action plus punch_hole and write_zeroes bodies that
differed only by a flag and the completion sink. Move that shared
deallocation logic into common.rs as apply_dealloc_action and
deallocate_range_result. Each engine now calls it and injects the
completion through its own queue.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add a NAT variant for debuggers on a different subnet or reached only
through the host uplink. Source NAT suffices because KDNET is target
initiated, so replies return along the conntrack entry with no inbound
port forward.
Note that KDNET needs a routable tap address from DHCP or a static SAC
assignment, add SAC steps to enable it on a headless guest, and move
busparams pinning into an optional section.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When the restore event is missing, the restore latency test panics with
a generic RestoreTimeParse error that gives no clue about the cause.
Include the exit status of the restore child in the diagnostic. Status 1
means Cloud Hypervisor aborted the restore, while SIGKILL means the test
killed a still running child at its timeout.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
If a device is attached to the iommu individually (not behind an iommu
backed segment) then we shouldn't allow its removal as the static VIOT
ACPI table establishes which devices (or whole segments) are behind the
IOMMU. After a removal that would be incorrect.
An alternative that was considered was to "block" the BDF from being
used for another PCI device that was hotplugged unless it was also IOMMU
attached that however feels unncessary complex when the primary purpose
of the vIOMMU backing is for nested device testing.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Migration threads may be left orphaned and keep the socket bound after
a migration has failed. Prevent this by signaling termination via the
`kill_evt` in `ReceiveAdditionalConnections`'s `Drop` impl.
On-behalf-of: SAP julian.schindel@sap.com
Signed-off-by: Julian Schindel <julian.schindel@cyberus-technology.de>
Replicate the checks that are made against the top level MemoryConfig
into those for the per-zone configuration.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The introduced `TcpStream` accessor allows setting socket options on the
`TcpStream` without going through the file descriptor and unsafe
methods.
On-behalf-of: SAP julian.schindel@sap.com
Signed-off-by: Julian Schindel <julian.schindel@cyberus-technology.de>
Multiple QcowDisk/Qcow2Backing can share references to the same
QcowMetadata via an Arc. Unfortunately the .shutdown() which clears the
DIRTY bit was being called when the first of those was dropped. Instead
move this to the drop of the metadata itself. Now only once all
references to the metadata are dropped then we can safely set the DIRTY
bit.
Assisted-by: Codex:GPT-5.6
Signed-off-by: Rob Bradford <rbradford@meta.com>
The ivshmem device was remapping the host file every time the BAR was
reprogrammed. This occurs on the vCPU thread and so was falling foul of
our improved seccomp filters. Avoid this by maintaining the existing
region when the device BARs are changed and just adjust the guest GPA.
Assisted-by: Codex:GPT-5.6
Signed-off-by: Rob Bradford <rbradford@meta.com>
Unfortunately glibc can read the overcommit sysctl from any thread. This
has lead to us adding a patchwork of openat/read syscalls to our allow
list when those threads don't necessarily need openat for their actual
uses.
Only the VMM and migration worker thread have a strict requirement for
the openat syscall. The syscall was added to the other threads to deal
with this glibc behaviour.
As read() is itself harmless move it to the common syscalls, strip
full openat() from all but the threads that need it and add limited,
read only, openat to all threads.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This test flakes out regularly and never succeeds on retry (as it is
more likely to fail under the reduced load of the retry)
See: #8651
Signed-off-by: Rob Bradford <rbradford@meta.com>
Vcpus need to be configured according to the selected CPU profile.
We thus store the computed vCPU MSR config update in the CPU manager
and use it whenever a vCPU is created.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We introduce a function somewhat analogous to `generate_common_cpuid`,
except that it is only relevant for CPU profiles.
This function is more "high level" than the
CpuProfile::required_msr_updates method and is intended to be called
from the vmm crate.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
In order to safely apply CPU profiles we need to ensure that the host's
feature MSRs that are permitted by the CPU profile are also compatible
with the values the CPU profile dictates.
KVM_SET_MSRs takes care of checking compatibility for most feature MSRs
on both Intel and AMD CPUs that we will permit CPU profiles to have
(more on this in the upcoming CPU profile generation tool PRs),
but there is one exception.
Userspace may set whatever value for the Intel exclusive
IA32_ARCH_CAPABILITIES MSR without receiving any complaints from KVM.
We thus introduce our own compatibility check for
IA32_ARCH_CAPABILITIES.
One might even argue that KVM_SET_MSRs is called relatively late when
creating or receiving a VM and that it would be preferable to have
compatibility checks for all permitted feature MSRs run earlier. This
would also mean more informative debug logs.
We argue however that those additional checks would lead to too much
code that is not strictly necessary which is why we decided against
doing that in this patch set.
The code introduced here is only intended to be used with CPU profiles
at this point in time, but it is written in such a way that it could
also be used for MSR compatibility checks between a migration source
and destination (when the Host profile is in use) should that be
desired in the future.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We introduce a method on the CpuProfile enum that computes the required
MSR related updates in order to be compatible with the CPU profile.
The output of this function describes the feature MSRs that need to be
set and the snapshottable MSRs that are compatible with the chosen CPU
profile. The former provides the guest information about CPU features
(that are not obtainable from inspecting CPUID) while the latter is
used to ensure that MSRs that are incompatible with the CPU profile
are not set upon snapshot/restore.
These will be utilized in the vmm crate in a follow up commit.
We expect the hypervisor to take care of checking compatibility of the
feature MSRs when they are set. IA32_ARCH_CAPABILITIES is a known
exception however which we introduce our own checks for in a follow
up commit.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
When Cloud hypervisor has not been configured for (KVM) Hyper-V we want
to adapt the CPU profiles not to require existence of Hyper-V related
MSRs.
The first step introduced here is to create a list of such MSRs.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We introduce a type describing MSR adjustments associated with a
CPU profile.
The upcoming CPU profile generation tool will serialize instances of
this struct when generating a CPU profile.
A follow up PR will take care of filling out the currently stubbed
`msr_data` method.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
CPU Profiles will be serialized to JSON by the upcoming CPU Profile
generation tool and we want MSRs to be serialized as hex strings.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
We create 64-bit analogues of the already existing hex (de-)serializer
helper functions for the 32-bit case.
These are necessary because the CPU profile needs associated data
describing how to adjust feature MSRs whose values are 64-bits.
In this case we prefer some small amount of code duplication over
macros and/or traits since we do not expect the need for further
variants of these helpers.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Feature MSRs must have consistent values across snapshot and restore,
otherwise we risk subtle time of check to time of use errors.
We thus adapt `Vcpu::set_state` to return a hard error if any feature
MSR cannot be restored.
It is enough to check that each MSR failing to be set does not have
an address corresponding to any of the feature MSRs stored in the
`KvmVcpu`. This is because the `msrs` in the `VcpuKvmState` contain all
the feature MSRs required by the selected CPU profile by construction.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Whenever a CPU profile is selected there will be feature MSRs that need
to be set when the CPU is configured.
We thus adapt the boot_msr_entries method to include the feature MSRs
that are passed to the vCPU upon construction.
Since KVM applies compatibility checks for all feature MSRs (except
IA32_ARCH_CAPABILITIES) we can only proceed if all MSR boot entries
are successfully set upon vCPU configuration.
This new check also applies to the regular boot entries when the host
profile is selected (the default), but that is arguably an improvement
in terms of correctness.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Although it is already possible to obtain the hypervisor type from the
hypervisor trait object, there is currently no way for functions that
only have a vCPU object to work with to know what the hypervisor
backend is.
In the rare case where there is significant divergence between the
supported hypervisor backends when working with vCPU objects the caller
must necessarily branch on the hypervisor type.
We thus introduce a hypervisor_type method on the Vcpu trait.
This will be utilized in a follow up commit where we need special
handling when configuring MSRs of vCPUs.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
When applying a CPU profile we need a way to change the configuration of
each vCPU to respect the requirements of the CPU profile.
This means that we need to set the feature MSRs in accordance with the
CPU profile upon configuring the vCPU and also ensuring that we do not
attempt to restore any MSRs that are not compatible with the profile
upon snapshot/restore.
The first step is to update `Vm::create_vcpu` to take an extra
parameter describing the necessary update.
In the case of KVM we modify the internal MSR state buffer when
constructing the vCPU whenever a VcpuMsrConfigUpdate is present.
The feature MSRs contained in the configuration will be treated in
follow up commits.
The changes to the vmm crate that are part of this commit are just
the minimum necessary to make the crate compile. We will update the
vmm crate to take CPU profiles into account in a follow up commit.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
CPU profiles will describe a list of MSRs they require when applying
the profile we want to check that the host has all the required MSRs
otherwise we have an incompatibility issue.
In order to check which MSRs the host supports we start by exposing the
hypervisor's get_msr_index_list method.
The MSRs required by a chosen CPU profile will be checked against the
output of the get_msr_index_list and get_feature_msrs on the hypervisor
in a follow up commit.
The get_msr_index_list method does not have an obvious implementation
for MSHV, because in that case one needs to obtain MSR indices from
the VM fd, after the VM has been initialized.
Since CPU profiles are only intended for the KVM hypervisor to begin
with, we leave the MSHV implementation as unimplemented for now. A
proper solution for MSHV should rather be found if/when CPU profiles
are also desired in that context.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
