Add KvmSevSnpIdBlock and KvmSevSnpIdAuth structs matching the AMD
SEV-SNP Firmware ABI Spec (Rev 1.58), and build them from the IGVM
SNP ID block directive during launch finish. This properly populates
id_block_uaddr/id_auth_uaddr in KVM_SEV_SNP_LAUNCH_FINISH and derives
auth_key_en from the assembled author key, matching QEMU's behavior.
Thread the guest policy from sev_snp_init to launch_finish via an
atomic on KvmVm so the ID block gets the correct policy value.
Also track has_snp_id_block in IgvmLoadedInfo to enable the ID block
based on whether the IGVM file actually contains one, rather than
hardcoding it for KVM.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Use the VP register page to read and write emulation-related
special registers directly, avoiding expensive IOCTLs for
registers that instruction emulation never touches.
In cpu_state(), read only segments, cr0, and efer from the VP
register page instead of calling get_sregs() which issues
IOCTLs for tr, ldt, gdt, idt, cr2, apic_base, and
pending_interruption.
In update_cpu_state(), when segments change, write only the 6
segment registers to the VP register page and set the segment
dirty bit, instead of calling set_sregs() which issues IOCTLs
for tr, ldt, gdt, idt, cr0-cr4, cr8, efer, and apic_base.
Both paths fall back to the IOCTL-based methods when the VP
register page is not available.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The GvaGpaValid flag in the intercept message indicates whether
the provided GPA corresponds to the decoded GVA. Without checking
this flag, the emulator may incorrectly use a stale GPA mapping
when the hypervisor invalidates it.
Add a check for the GvaGpaValid flag before using the cached
(GVA, GPA) mapping. If the flag is clear, use a sentinel value
to force translate() to perform a proper hypercall-based
translation instead of using an invalid cached mapping.
Signed-off-by: Pedro Barbuda <pbarbuda@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Adapt the IGVM loader to work with both MSHV and KVM backends, which
differ in page type constants, CPUID page layout, and VMSA handling.
Abstract page types into a PageTypeConfig struct populated at runtime
from the detected hypervisor, replacing hardcoded mshv_bindings constants.
Apply the VMSA register state to each vCPU via setup_sev_snp_regs(),
translating SevSelector attributes to KVM segment format using a bitfield
decoder.
KVM's SNP launch path sanitizes certain CPUID bits that could lead to
an insecure guest. If the VMM sets these bits, KVM rejects the CPUID
page import on the first attempt, requiring a retry with the
firmware-corrected values.
Pre-clear the known problematic bits before import to avoid the
reject-and-retry cycle:
- Leaf 0x1, ECX bit 24: TSC_DEADLINE (filtered by KVM)
- Leaf 0x7, EBX bit 1: SGX (filtered by KVM)
- Leaf 0x7, EDX: clear entirely (contains speculative features)
- Leaf 0x80000008, EBX bit 25: filtered by KVM
- Leaf 0x80000021, ECX: clear entirely
This keeps the CPUID page stable across launch updates and avoids
noisy error logs from the retry path.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Co-authored-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
During SNP boot all guest RAM is initially marked
KVM_MEMORY_ATTRIBUTE_PRIVATE. Pages imported via SNP_LAUNCH_UPDATE are
properly accepted by the guest, but generic RAM pages (e.g. the AP
trampoline at GPA 0xD000) are not. When stage0 on the BSP starts
secondary vCPUs via x2APIC, the APs try to execute from the trampoline
page through the shared mapping while KVM still has it marked private,
causing a KVM_EXIT_MEMORY_FAULT (flags=KVM_MEMORY_EXIT_FLAG_PRIVATE)
that previously fell through to the catch-all error, killing the VM.
Handle VcpuExit::MemoryFault by toggling the page's memory attribute
between private and shared based on the exit flags, allowing the vCPU
to retry the access.
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
SEV-SNP guests will issue this hypercall to signal a change in the page
encryption status to the hypervisor.
Handle VcpuExit::Hypercall in the KVM vCPU run loop: decode the GPA,
page count, and private/shared attribute from the hypercall arguments,
then call KVM_SET_MEMORY_ATTRIBUTES to update the page state.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Add the KVM_SEV_SNP_LAUNCH_FINISH ioctl, which finalizes the SNP
launch sequence and transitions the VM into a runnable encrypted
state.
Additionally, add KVM_SEV_SNP_LAUNCH_FINISH to the seccomp allowlist.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Implement the KVM_SEV_SNP_LAUNCH_UPDATE ioctl.
Extend Vm::import_isolated_pages() with a uaddrs parameter carrying
host virtual addresses, which KVM needs, unlike MSHV. Compute uaddrs
from guest memory mappings in the IGVM loader.
Add KVM_SEV_SNP_LAUNCH_UPDATE to the seccomp allowlist.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Introduce the SevFd abstraction that wraps /dev/sev and implements the
KVM_SEV_INIT2 and KVM_SEV_SNP_LAUNCH_START ioctls for SEV-SNP VM
initialization on KVM.
Key changes:
- Add sev.rs with KvmSevInit and KvmSevSnpLaunchStart ioctl structs
matching the kernel layout (linux/arch/x86/include/uapi/asm/kvm.h)
- Implement KVM_SEV_INIT2 and KVM_SEV_SNP_LAUNCH_START ioctls
- Set KVM_MEMORY_ATTRIBUTE_PRIVATE on newly created memory regions
when guest_memfd is supported
- Widen SevSnpPageAccessProxy cfg gates from mshv-only to all
sev_snp-enabled builds
- Make sev_snp_init a required trait method (remove default impl)
- Include KVM_SEV_SNP_LAUNCH_START in the seccomp allowlist
- Parse VMSA SEV features from IGVM and include them in the
KVM_SEV_INIT2 ioctl
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Co-authored-by: Rob Bradford <rbradford@meta.com>
Signed-off-by: Rob Bradford <rbradford@meta.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
The SNP guest policy (AMD SEV-SNP ABI bits controlling SMT, migration,
debug, etc.) was previously hardcoded inside the MSHV implementation.
Widen Vm::sev_snp_init() to accept an SnpPolicy parameter so each
hypervisor backend receives the policy at init time.
Add get_default_sev_snp_guest_policy() in the VMM to construct the
default policy.
Co-authored-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
Add support for guest_memfd (available in Linux kernel v6.8+), which
enables private memory for confidential VMs.
Key changes:
- Introduce UserMemoryRegion abstraction with guest_memfd fields
- Add From impls between kvm_userspace_memory_region2 and UserMemoryRegion
- Convert all KVM memory region operations from kvm_userspace_memory_region
to kvm_userspace_memory_region2, with automatic fallback to v1 when
guest_memfd is not supported
- Add set_user_memory_region() wrapper that dispatches to v1/v2 based on
kvm_guest_memfd_supported capability
- Create guest_memfd via KVM_CREATE_GUEST_MEMFD ioctl when supported
- Extend KvmDirtyLogSlot to preserve region2 fields across dirty log
start/stop cycles
This is prerequisite infrastructure for KVM-based confidential computing
that requires private guest memory backed by guest_memfd.
Co-authored-by: Alex Orozco <aorozco@google.com>
Signed-off-by: Keith Adler <kadler@cloudflare.com>
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
KVM_GET_MSR_INDEX_LIST does not consistently include all
guest-programmable MTRR MSRs.
During save/restore while booting, the VMM initially sets only
MSR_MTRRdefType, then guest firmware or other early boot code can
program additional MTRR state before the snapshot is taken. If those
MSRs are missing from the vCPU MSR buffer, snapshot omits part of the
guest's MTRR configuration and restore resumes with an incomplete
MTRR map.
Add the guest-programmable MTRR MSRs to the KVM MSR index list used
to build the vCPU MSR buffer so the existing snapshot/restore path
preserves the guest's MTRR state.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
When booting an SEV-SNP guest VM using IGVM with -pvalidate_opt 1 (lazy
page acceptance), the guest kernel's #VC exception handler may issue
VMGEXIT with SVM_EXIT_CPUID (0x72) or SVM_EXIT_MSR (0x7c) exit codes
via the GHCB page protocol. The hypervisor had no handlers for these
exit codes, causing the guest's #VC handler to fail and trigger
sev_es_terminate(), which sends GHCB_MSR_TERM_REQ (0x100). The
hypervisor then panicked on the unhandled 0x100 operation.
Add the following handlers to the GHCB VMGEXIT processing:
- SVM_EXIT_CPUID (0x72): Read function/index/xcr0/xss from the GHCB
page and return CPUID results via get_cpuid_values().
- SVM_EXIT_MSR (0x7c): Handle MSR read (RDMSR) and write (WRMSR)
requests from the guest via the GHCB page protocol.
- GHCB_MSR_TERM_REQ (0x100): Decode reason_set and reason_val from the
GHCB MSR and return an error instead of panicking, allowing graceful
error propagation.
Testing:
Reproducer (on Azure DC16as_cc_v5, /dev/mshv):
cloud-hypervisor --cpus boot=1,nested=off --memory size=512M \
--disk path=osdisk.img path=cloudinit \
--net "tap=,mac=12:34:56:78:90:06,ip=192.168.6.1,mask=255.255.255.128" \
--serial null --console pty \
--api-socket /tmp/ch.sock \
--igvm /igvm_files/linux-ttyS0.bin \
--host-data <hash> --platform sev_snp=on -v
Before fix:
thread 'vcpu0' panicked at hypervisor/src/mshv/mod.rs:1207:30:
Unsupported VMGEXIT operation: 100
After fix: VM boots successfully to login prompt with no panics.
All virtio devices (console, rng, disks) activate normally.
No regression risk for non-SEV-SNP: all new code is within the
HVMSG_X64_SEV_VMGEXIT_INTERCEPT handler which is only reached for
SEV-SNP guests.
Signed-off-by: Souradeep Chakrabarti <schakrabarti@microsoft.com>
Bump to the released versions that are compatible wherever possible but
for the vhost and vfio crates they are git hashes as no releases with
compatible versions have yet been made.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Nightly rustfmt now prefers `self` re-exports inline rather
than a separate 'pub use {kvm_bindings, kvm_ioctls}' line.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The 0.6.7 version of the mshv crates introduced a new version of
make_default_partition_create_arg inside `struct Mshv`. This version
queries the available processor features on the host and gives the same
feature set to the guests.
Move Cloud Hypervisor to this new function.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Add handling for GHCB_INFO_SPECIAL_DBGPRINT VMG exit in the SEV-SNP
guest exit handler. This exit occurs when the guest sends debug print
requests through the GHCB interface.
Without this handler, SEV-SNP guests fail to boot when debug output
is triggered, such as when a debugger is attached to the guest image.
The handler acknowledges the exit without printing to avoid performance
degradation from frequent debug print requests.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Provide a stub implementation for save_data_tables() to unblock pause
functionality. Without this, pausing a VM causes Cloud Hypervisor to
panic due to the unimplemented!() macro. This unblocks the
test_api_http_pause_resume testcase. We don't need to save any state
just to pause and resume.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Set HV_PARTITION_CREATION_FLAG_SMT_ENABLED_GUEST when the guest
topology has more than one thread per core. This allows the
hypervisor to schedule guest VPs correctly on SMT-enabled hosts.
Without this flag, the hypervisor schedules guest VPs incorrectly,
causing SMT unusable.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
In most of the cases, special registers don't change after emulations,
but current code sets them back unconditionally, and although some of
them are set over the register page, others require a system call and a
hypervisor to be updated, which is a waste it there were not changes.
Introduce and CPU update method for Microsoft Hypervisor emulator and
set special registers only when they were changed. This change reduces
guest boot time by 4% for a single VP guest boot (in L1VH partition) in
my experiments.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
This is a precursor change to overall ioctl and hypercall reduction
effort. The old (current) CPU state can be compared to the new to
determine what has changed and avoid unnecessary register updates.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
x86::__cpuid is safe on Rust ≥1.94 but unsafe on older versions. This
causes unused_unsafe warnings when compiling with Rust ≥1.94. However,
on earlier Rust versions, the code won’t compile if the unsafe blocks
are absent.
Work around this by adding #[allow(unused_unsafe)] where needed to
suppress the warnings.
See #7588 for more discussion.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
On MSHV, exposing multithreaded CPU topologies requires setting the
PROCESSORS_PER_SOCKET partition property so that CPUID.0xB reports
correct logical processor counts and topology levels to the guest.
This property must be set after all vCPUs are configured, as the
hypervisor uses the complete vCPU layout to derive and report CPU
topology information.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The TILE data state of AMX may require 8KB+ space, calling the legacy
KVM_GET_XSAVE will encounter an error since KVM_GET_XSAVE only can get
4KB space. This patch adds KVM_GET_XSAVE2 support to allow snapping more
data.
Fixes: #7533
Signed-off-by: Songqian Li <sionli@tencent.com>
The MSR enumeration logic in MSHV does not accurately
account for the capabilities of the underlying
hardware and hypervisor. Previously, integration tests
passed because only the default MSRs—universally supported
across platforms—were involved. However, recent feature
additions in MSHV have introduced MSRs that are not
supported on Intel hardware, causing failures.
This patch retrieves the MSRs from the VM specific
API call that filters out the unsupported MSRs.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Currently when the user configures AMX the corresponding state
components get dynamically enabled directly inside the function body of
vmm::cpu::CpuManager::new.
With our ongoing work on CPU templates/profiles, there will (likely) be
one more binary crate for producing CPU profiles that also needs to do
this (without creating a CpuManager) and it may also be the case that
we will need to call this function prior to `CpuManager::new` during
live migrations.
We thus add a method for enabling the AMX tile state components on the
hypervisor trait that may be called wherever necessary. We argue that
this is beneficial for code clarity independently of the upcoming CPU
templates/profiles PR that we are working on.
The astute reader will notice that the logic introduced here is not 100%
the same as what is done inside the vmm::cpu::Cpumanager::new method. We
claim that our approach is more in-line with the official documentation.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The MSRs are constant at boot time so rather than creating a vector in
the boot_msr_entries() method instead reaturn a reference to static MSR
array data.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
During the lifecycle of a confidential VM, the guest may revoke
previously shared pages via an attribute-intercept VM exit.
When this happens, the host-side cache must be updated so that any
subsequent access by the VMM triggers a fresh request for the guest
to re-share the page.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This is a follow-up of [0].
# Advantages
- This saves dozens of unneeded clone()s across the whole code base
- Makes it much easier to reason about how parameters are used
(often we passed owned Arc/Rc versions without actually needing
ownership)
# Exceptions
For certain code paths, the alternatives would require awkward or overly
complex code, and in some cases the functions are the logical owners of
the values they take. In those cases, I've added
#[allow(clippy::needless_pass_by_value)].
This does not mean that one should not improve this in the future.
[0] 6a86c157af
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
As well as saving the MSRs as it is currently does ensure that the KVM
capability is enabled along with keeping the internal state updated.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Co-authored-by: Chengyu Fu <chengyu.fu@linux.alibaba.com>
It takes a pointer to a userspace address that it accesses, so it should
be marked unsafe. This was missed earlier.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Also drop support for building the TDX code for 32-bit targets. All
CPUs with TDX support are 64-bit so supporting 32-bit targets is not
needed.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
To ensure that struct sizes are the same on 32-bit and 64-bit, various
kernel APIs use __u64 (Rust u64) to represent userspace pointers.
Userspace is expected to cast pointers to __u64 before passing them to
the kernel, and cast kernel-provided __u64 to a pointer before using
them. However, various safe APIs in Cloud Hypervisor took
caller-provided u64 values and passed them to syscalls that interpret
them as userspace addresses. Therefore, passing bad u64 values would
cause memory disclosure or corruption.
Fix the bug by using usize and pointer types as appropriate. To make
soundness of the code easier to reason about, the PCI code gains a new
MmapRegion abstraction that ensures the validity of pointers. The rest
of the code already has an MmapRegion abstraction it can use. To avoid
having to reason about whether something is keeping the MmapRegion
alive, reference counting is added. MmapRegion cannot hold references
to other objects, so the reference counting cannot introduce cycles.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
To ensure that struct sizes are the same on 32-bit and 64-bit, various
kernel APIs use __u64 (Rust u64) to represent userspace pointers.
Userspace is expected to cast pointers to __u64 before passing them to
the kernel, and cast kernel-provided __u64 to a pointer before using
them. However, various safe APIs in Cloud Hypervisor took
caller-provided u64 values and passed them to syscalls that treat them
as userspace addresses. Therefore, passing bad u64 values would cause
memory disclosure or corruption. The memory region APIs are one example
of this, so mark them as unsafe.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
This better aligns with the rest of the code and makes it clearer
that these tests can run "as is" in a normal hosted environments
without the special test environment.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
