Vcpus need to be configured according to the selected CPU profile.
We thus store the computed vCPU MSR config update in the CPU manager
and use it whenever a vCPU is created.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
When applying a CPU profile we need a way to change the configuration of
each vCPU to respect the requirements of the CPU profile.
This means that we need to set the feature MSRs in accordance with the
CPU profile upon configuring the vCPU and also ensuring that we do not
attempt to restore any MSRs that are not compatible with the profile
upon snapshot/restore.
The first step is to update `Vm::create_vcpu` to take an extra
parameter describing the necessary update.
In the case of KVM we modify the internal MSR state buffer when
constructing the vCPU whenever a VcpuMsrConfigUpdate is present.
The feature MSRs contained in the configuration will be treated in
follow up commits.
The changes to the vmm crate that are part of this commit are just
the minimum necessary to make the crate compile. We will update the
vmm crate to take CPU profiles into account in a follow up commit.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
This removes a need to expose a method from the CpuManager back out to
the Vm which is called immediately after creating the CpuManager.
Signed-off-by: Rob Bradford <rbradford@meta.com>
CPU unplug requires co-operation between the guest and the VMM. Ensure
that the CPU the guest marks for eject is the one the VMM expects to be
removed. If the guest were to remove a different vCPU there would be
unexpected behaviour. Further ensure that the boot vCPU is never
unplugged.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Read the host cache info from sysfs and pass through the same to the
guest via the PPTT table. This is the same as the approach taken for
FDT. Similar to that, assume that the L3 cache is always shared and the
L2 cache is unique per CPU.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Use the helpers from the acpi_tables crate to construct the PPTT. This
is in preparation for adding cache hierarchy info to the PPTT which is
simpler using the helpers.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
A Windows guest that launches nested Hyper-V (for example to run WSL2)
fails to start its hypervisor on cloud-hypervisor's HW-reduced-ACPI
FADT. hvloader's hypervisor-launch path (0x18000f01c -> 0x180015628 ->
0x180015788) registers every legacy PM register block via 0x1800158dc
and rejects any block whose GAS address is 0 with status 8
(STATUS_INVALID_DEVICE_REQUEST). hvix64 then never launches and
HypervisorPresent stays False. The HW-reduced FADT leaves those blocks
zero.
Emit valid PM1a event/control blocks (I/O ports, lengths and X_GAS) in
the FADT and reserve those ports in the I/O allocator so nothing else
claims them. The HW-reduced guest OS ignores the legacy ports; only
hvloader's ACPI validation reads them.
These blocks are only useful to a guest that itself runs an enlightened
hypervisor, so emit them only when both guest nesting and the Hyper-V
enlightenments are enabled (--cpu nested=on,kvm_hyperv=on).
Signed-off-by: doge <me@crackerben.com>
Suggested by phip1611 on #8446.
This adds the repo's first clippy.toml, carving arch out of the
absolute_paths deny from #7670. Glob imports and trait imports that
must be in scope for method-call resolution (e.g. DeviceInfoForFdt for
.irq()) are left as-is.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
size_of is part of std::prelude as of Rust 1.80 (with size_of_val,
align_of, align_of_val), and the workspace MSRV is 1.89, so qualifying
it (mem::size_of, std::mem::size_of, core::mem::size_of) is unnecessary.
Convert every qualified size_of call-site to the bare prelude form and
drop the now-redundant `use std::mem::size_of;` imports, keeping
`use std::mem;` where it still serves non-prelude items (transmute,
swap, replace, take, zeroed, MaybeUninit, offset_of). size_of is the
only one of the four currently used in the tree.
Pure refactor, no behavioural change. Follow-up to the
clippy::absolute_paths cleanup (#7670), as discussed in #8444.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Import the modules used in the crate instead of spelling the full paths
at every use site, and drop the now-unnecessary crate-level
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
We have larger cloud deployments and analyzed the logs. Let's silence
some messages that generally provide little value on the `info!` level.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Restoring a snapshot (snapshot restore or live-migration receive) sets
each vCPU's TSC by writing MSR_IA32_TSC as the vCPU is created.
However because CpuManager creates and restores vCPUs one at a time, the
host TSC advances between the per-vCPU writes and KVM derives a slightly
different TSC offset for each vCPU.
KVM only engages its masterclock when every offset matches. This has a
side effect of breaking the HyperV TSC reference clock page resulting in
significantly reduced performance on Windows.
After restore synchronise all vCPU's TSC offset to the boot vCPU's via
the KVM_VCPU_TSC_CTRL device attribute group (Linux 5.16+) this allows
the KVM TSC masterclock to engage and mitigates performance issues with
the KVM HyperV emulation.
See: #8383
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude <claude-opus-4-8>
Unlike x86, ARM64 has no kvmclock support to sync guest time upon
required. However, the guest reads the architected virtual timer
(CNTVCT_EL0) directly which can be modified by the VMM to update the
time after snapshot restore. Since the CNTVCT is in ticks, we also need
to read CNTFRQ (via mrs due to lack of ONEREG interface) to compute the
ticks from wall clock difference.
Because the counter is a vCPU register, the capture must run with the
vCPUs quiesced, so the VMM now captures the clock just after
cpu_manager.pause() through the boot vCPU. This is behaviorally
identical for x86, whose clock is VM-wide. There is no restore/advance
yet, so aarch64 guests still resume behind real time until the following
commit.
Signed-off-by: Atish Patra <atishp@meta.com>
Reject without asserting that the ACPI CPU hotplug register accesses
match those that are specified by the ACPI definitions.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Assisted-by: Claude:Opus-4.7
The RISC-V device tree's timebase-frequency was hardcoded to 10 MHz
(0x989680). Actual hardware uses different frequencies.
Read the timebase frequency from KVM_GET_ONE_REG via
KVM_REG_RISCV_TIMER (offset 0, kvm_riscv_timer.frequency),
thread it through the VMM to arch to FDT layers, and fall back to
the 10 MHz default when KVM returns no value.
Signed-off-by: Meng Zhuo <mengzhuo@iscas.ac.cn>
Remove stale #[allow]s whose lints no longer fire, convert the
unconditionally-firing ones to #[expect], and keep the
arch/feature-conditional ones as #[allow]. Verified across kvm/mshv,
x86_64/aarch64, and --all-features.
Part of #8326.
Signed-off-by: Tushar Khatri <hello@tusharkhatri.in>
We integrate the CPU profile into the various configs that
ultimately get set by the user.
This quickly ends up involving multiple files, luckily Rust
helps us find which ones via compilation errors.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
When SVE is enabled, KVM replaces the FPSIMD V-registers with wider
SVE Z-registers. Attempting to access the old FPSIMD offsets returns
EINVAL.
Fix by classifying each register from KVM_GET_REG_LIST as core, system,
or extended. Extended registers (currently SVE only) are saved as
generic `ExtendedReg` entries split into `pre_finalize_regs` (registers
like SVE VLS that must be written before `vcpu_finalize`) and
`extended_regs`. FPSIMD registers are only accessed when SVE is absent.
Unrecognized register families error immediately so future extensions
like SME fail clearly rather than silently losing state.
The snapshot is deserialized before vCPU init to make pre-finalize
register state available for the init -> VLS -> finalize ordering
required by KVM.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Split the SVE detection and finalization logic out of `init()` into a
dedicated `finalize_sve()` method. This separates the concerns of vCPU
initialization (preferred target, processor features, KVM init) from SVE
finalization, and enables a subsequent commit to insert SVE VLS register
in the restore path without needing to add a restore state in `init()`.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Only Thread::Vmm and Thread::Vcpu need to know the hypervisor type.
Make the type optional, and then simplify the users.
Assisted-by: Pi-agent:Claude-Opus-4.7
Signed-off-by: Wei Liu <liuwe@microsoft.com>
# TL;DR
In https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7990 we
replaced the old deadlock with another deadlock. This commit finally
resolves (hopefully) all dead locks on that code path by not holding
`CpuManager::vcpu_states` while waiting for vCPU pause acknowledgements.
A vCPU can receive the pause kick while servicing the ACPI CPU hotplug
MMIO device, and that MMIO path also needs `vcpu_states`. Holding the
mutex across the wait phase deadlocks pause against that MMIO access.
# Problem
`signal_vcpus()` used to lock `CpuManager::vcpu_states` for the whole
function, signal every vCPU, and then wait for each vCPU to acknowledge
the kick.
That lock scope is too wide. A vCPU is allowed to observe the kick in
userspace rather than returning directly from `KVM_RUN`. During boot,
`vcpu0` can be in an MMIO access on the ACPI CPU hotplug device when
pause arrives. `AcpiCpuHotplugController::read()` and `write()` both
lock `vcpu_states` to inspect or update the selected vCPU state.
The deadlock looks like this:
VMM thread vCPU thread
---------- ----------
lock(vcpu_states)
signal_vcpus()
wait for ack ---------------------> receives pause kick
enters ACPI CPU hotplug MMIO
lock(vcpu_states) [blocks]
wait for ack <--------------------- cannot set vcpu_run_interrupted
The VMM thread waits for `vcpu_run_interrupted` to flip, but the vCPU
cannot reach the pause acknowledgement path because it is sleeping on
the same mutex.
The debug logs matched that cycle exactly: signal delivery kept
working, `vcpu0` stayed in one unmatched `run()` invocation, the stuck
thread sampled in `futex_do_wait`, and the backtrace pointed at
`AcpiCpuHotplugController::read()`.
# Reproducer
This was reproducible by continuously issuing `pause()` / `resume()`
from while a Linux guest was still booting. That boot-tim window
reliably exercises the ACPI CPU hotplug MMIO access that participates in
the deadlock. Once the guest had finished booting, the problem became
much harder to trigger (as there is no MMIO operation without explicit
CPU plugging).
# Solution
Keep the existing two-phase behavior so all vCPUs are still signalled
before the wait phase, but narrow the lifetime of the `vcpu_states`
mutex. Reacquire it only long enough to access one `VcpuState` at a
time in each phase.
That preserves the original pause semantics and the fast signal-all /
wait-all structure, while removing the lock inversion with the ACPI CPU
hotplug MMIO path.
This also remains safe if a vCPU is hot-removed while pause is in
progress. Hot-remove does not shrink `vcpu_states`; it stops the thread
and clears the `VcpuState` handle in place. `signal_vcpus()` can
therefore snapshot the vector length up front, and if a vCPU disappears
between the signal and wait phases,
`wait_until_signal_acknowledged()` will observe `handle.is_none()` and
return successfully.
The interruption handshake itself lives in atomics inside each
`VcpuState`. The outer mutex is only needed to reach the state objects,
not to keep the acknowledgement protocol correct. Dropping the mutex
between iterations therefore does not weaken the pause protocol, but it
does allow MMIO handlers and other `vcpu_states` users to make forward
progress while the VMM waits for the kick to be observed.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Adapt the IGVM loader to work with both MSHV and KVM backends, which
differ in page type constants, CPUID page layout, and VMSA handling.
Abstract page types into a PageTypeConfig struct populated at runtime
from the detected hypervisor, replacing hardcoded mshv_bindings constants.
Apply the VMSA register state to each vCPU via setup_sev_snp_regs(),
translating SevSelector attributes to KVM segment format using a bitfield
decoder.
KVM's SNP launch path sanitizes certain CPUID bits that could lead to
an insecure guest. If the VMM sets these bits, KVM rejects the CPUID
page import on the first attempt, requiring a retry with the
firmware-corrected values.
Pre-clear the known problematic bits before import to avoid the
reject-and-retry cycle:
- Leaf 0x1, ECX bit 24: TSC_DEADLINE (filtered by KVM)
- Leaf 0x7, EBX bit 1: SGX (filtered by KVM)
- Leaf 0x7, EDX: clear entirely (contains speculative features)
- Leaf 0x80000008, EBX bit 25: filtered by KVM
- Leaf 0x80000021, ECX: clear entirely
This keeps the CPUID page stable across launch updates and avoids
noisy error logs from the retry path.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Co-authored-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Extract AcpiCpuHotplugController from CpuManager and move the BusDevice
implementation to the new type. This separates VMM-internal vCPU
management from the guest-visible ACPI CPU hotplug MMIO interface.
Besides clarifying responsibilities and reducing technical debt, this
fixes a rare deadlock involving pause handling and MMIO access.
New responsibilities:
- CpuManager manages VMM-internal vCPU lifecycle and coordination
- AcpiCpuHotplugController implements the guest-visible ACPI CPU hotplug
MMIO interface
A vCPU thread may exit KVM_RUN to perform an MMIO access previously
handled by CpuManager. If the VMM thread begins processing a `pause`
event before that MMIO operation acquires access to CpuManager,
CpuManager::pause() will block waiting for the vCPU thread to ACK
the pause, while the vCPU thread is blocked waiting to complete the MMIO
operation through the same CpuManager - which it can never lock - the
VMM is deadlocked.
This can occur during early boot or CPU hotplug when pause events race
with MMIO accesses. The issue is rare and timing-dependent, but real.
For reproducing: run `ch-remote pause|resume` in a loop while booting
a Linux VM (via direct kernel boot).
With the new design, these MMIO operations no longer depend on
CpuManager, which removes the deadlock path entirely.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Previously, KVM_KVMCLOCK_CTRL was skipped when kvm_hyperv=on because
Windows does not use pvclock directly. However, KVM internally uses
pvclock data structures as the basis for computing the Hyper-V
Reference TSC page parameters. Not calling KVM_KVMCLOCK_CTRL means
there is no mechanism to signal time discontinuity to Windows guests
after pause/resume, contributing to multi-minute hangs.
Remove the kvm_hyperv guard so all guests receive the clock-paused
notification.
Signed-off-by: CMGS <ilskdw@gmail.com>
Fix broken URL reported by Lychee. The binutils-gdb github repo no
longer exists. Use the equivalent sourceware.org link.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
As far as I can tell, PIDTYPE_PID is a kernel-internal name, and
PR_SCHED_CORE_SCOPE_THREAD is the UAPI name. There's no PIDTYPE_PID
in the UAPI headers, and the core scheduling documentation says that
the fourth prctl argument should be a "PR_SCHED_CORE_SCOPE_-prefixed
macro constant".
Link: https://www.kernel.org/doc/html/v6.19/admin-guide/hw-vuln/core-scheduling.html#usage
Fixes: 3f800d2bb ("vmm: Add core scheduling support for vCPU threads")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Avoid a potential infinite loop where if the leader fails to create a
cookie due to an unexpected error (not one of the SMT/no kernel support
errors) then the other vcpu threads will continue around their
spinloops.
This change also clarifies the state machine for the leader election
with an explicit enum.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add a core_scheduling option to --cpus with three modes of operation.
This feature takes advantage of a kernel feature that restricts
scheduling of processes on the SMT threads on the same core. This is
useful for mitigating certain classes of side-channel attacks and has
better performance that disabling SMT on the CPU.
- vm (default): All vCPU threads share one core scheduling cookie.
They may be co-scheduled on SMT siblings while host threads are
excluded - this has minimal performance impact and can even
potentially improve performance from co-location.
- vcpu: Each vCPU gets a unique cookie preventing any two vCPUs from
sharing SMT siblings. This has the strongest isolation but at some
compromise of performance.
- off: No core scheduling applied (old behaviour).
This isolation is done by the kernel maintaining a "cookie" - threads
with the same cookie can share the same core.
In vCPU mode each vCPU thread the cookie is created when the thread
starts and each gets a unique cookie. For VM mode the first vCPU thread
(the leader) will create the cookie. All other vCPU threads started (via
hotplug or during boot) will have that cookie shared to it.
EINVAL/ENODEV from prctl is silently ignored so this works transparently
on kernels older than 5.14 that lack PR_SCHED_CORE or when SMT disabled.
Full details of this kernel feature can be found at:
https://docs.kernel.org/admin-guide/hw-vuln/core-scheduling.html
This implementation was inspired by crosvm's implementation - in
particular the enable_core_scheduling() function.
This is challenging to test via integration testing but the logging of
the received cookie shows it working:
VM case:
cloud-hypervisor: 0.243102s: <vcpu1> INFO:vmm/src/cpu.rs:1247 -- vCPU 1: core scheduling cookie = 0x33e4c167
cloud-hypervisor: 0.243102s: <vcpu0> INFO:vmm/src/cpu.rs:1247 -- vCPU 0: core scheduling cookie = 0x33e4c167
vCPU case:
cloud-hypervisor: 0.089356s: <vcpu0> INFO:vmm/src/cpu.rs:1247 -- vCPU 0: core scheduling cookie = 0x13993ad6
cloud-hypervisor: 0.089380s: <vcpu1> INFO:vmm/src/cpu.rs:1247 -- vCPU 1: core scheduling cookie = 0xd48e86e
Signed-off-by: Rob Bradford <rbradford@meta.com>
On MSHV, exposing multithreaded CPU topologies requires setting the
PROCESSORS_PER_SOCKET partition property so that CPUID.0xB reports
correct logical processor counts and topology levels to the guest.
This property must be set after all vCPUs are configured, as the
hypervisor uses the complete vCPU layout to derive and report CPU
topology information.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
In [0] we refactored some Arc<Mutex<T>> parameters to &Mutex<T>> to
satisfy clippy's needless_pass_by_value lint. Nevertheless, this is also
not so idiomatic, so as a follow-up, we put the responsibility to lock
objects to the caller side (only where this is not strictly needed by
the callee).
While on it, I also tried to pass vm_config directly into
pre_create_console_devices() which would clean up some code, but then
we have interleaving mutable and immutable borrows of the Vmm, which
are denied by the borrow checker.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
We enable AMX tile state components via the hypervisor (as introduced
in the previous commit) instead of doing this inline in the body of
`CpuManager::new`.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The MSRs are constant at boot time so rather than creating a vector in
the boot_msr_entries() method instead reaturn a reference to static MSR
array data.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is a follow-up of [0].
# Advantages
- This saves dozens of unneeded clone()s across the whole code base
- Makes it much easier to reason about how parameters are used
(often we passed owned Arc/Rc versions without actually needing
ownership)
# Exceptions
For certain code paths, the alternatives would require awkward or overly
complex code, and in some cases the functions are the logical owners of
the values they take. In those cases, I've added
#[allow(clippy::needless_pass_by_value)].
This does not mean that one should not improve this in the future.
[0] 6a86c157af
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This helps to uncover expensive and needless clones in the code base.
For example, I prevented extensive clones in the snapshot path where
(nested) BTreeMap's have been cloned over and over again. Further,
the lint helps devs to much better reason about the ownership of
parameters.
All of these changes have been done manually with the necessary
caution. A few structs that are cheap to clone are now `copy` so that
this lint won't trigger for them.
I didn't enable the lint so far as it is a massive rabbit hole and
needs much more fixes. Nevertheless, it is very useful.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This better aligns with the rest of the code and makes it clearer
that these tests can run "as is" in a normal hosted environments
without the special test environment.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Resignal every 10ms the thread if it has not acknowledged the signal via
setting the atomic when the vCPU thread was acknowledged. Further, avoid
an infinite loop by generating an error if it takes more than 1000ms to
interrupt the thread.
The retry helps mitigate a race condition where the signal is received
between checking the pause atomic and entering KVM_RUN ioctl when
pausing. Hitting this race condition would leave the
wait_untial_signal_acknowledged() method spinning indefinitely.
The timeout error avoids the VMM process being blocked indefinitely.
See: #7427
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Consuming `&Arc<T>` as argument is almost always an antipattern as it
hides whether the callee is going to take over (shared) ownership
(by .clone()) or not. Instead, it is better to consume `&dyn T` or
`Arc<dyn T>` to be more explicit. This commit cleans up the code.
The change is very mechanic and was very easy to implement across the
code base.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
