As well as rejecting writes to sector 0 in the case of raw files where
the user hasn't specified the image_type also reject virtio requests of
type discard and write_zeroes.
Signed-off-by: Rob Bradford <rbradford@meta.com>
If the disk image was autodetected to raw (not specified with image_type
= 0) then in the virtio-block subsystem generate errors for writes to
block 0 (treat as if read-only). This gives an immediate error vs using
the image implementations in the block subsystem.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add an image_type to DiskConfig to specify the image type. If none is
specified autodetect the image type but disable potentially unsafe
behaviour in the QCOW2 backend by disabling the backing file support.
If the image type is autodetected then fix it in the config so that it
will be persistant across reboots and migrations/snapshot & restores.
This also handles the case where the image type was not specified as
part of the disk configuration.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Verify that opening a QCOW2 image with a backing file reference
through QcowDiskSync with backing_files=off produces the user-facing
BackingFilesDisabled error rather than MaxNestingDepthExceeded.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When a QCOW2 image has a backing file but backing_files=on is not set,
the error was MaxNestingDepthExceeded which gives no indication that
this is a policy decision or how to resolve it.
Add a BackingFilesDisabled error variant whose message indicates that
backing file support is disabled and references the backing_files
option. The translation from MaxNestingDepthExceeded to
BackingFilesDisabled happens at the QcowDiskSync boundary where the
policy decision is made, preserving the original error for genuine
recursive depth exhaustion.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The 0.6.7 version of the mshv crates introduced a new version of
make_default_partition_create_arg inside `struct Mshv`. This version
queries the available processor features on the host and gives the same
feature set to the guests.
Move Cloud Hypervisor to this new function.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
test_guest_numa_generic_initiator was missing the #[test] attribute and
did not match the VFIO CI filter pattern "vfio::test_nvidia"
Add #[test] and rename to test_nvidia_guest_numa_generic_initiator so
the existing CI infrastructure picks it up on the vfio-nvidia runner.
Fixes: #7718
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
The device_id field was added to both NumaConfig and NumaNode as part
of the Generic Initiator support, but create_numa_nodes() change
was missed when the commits were reorganized.
As a result, node.device_id is never propogated from the config to
the runtime node and the ACPI SRAT Type 5 (Generic Initiator Affinity)
entries were never emitted.
Add the missing propogation so that create_srat_table() can resolve
the device and emit the correct affinity structure
Fixes: #7717
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Add performance tests for standalone qcow2 images without backing
files - uncompressed, zlib and zstd compressed. Each variant
includes single queue and multiqueue tests for sequential
read, random read and warmed up sequential read.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add multiqueue num_queues=4 performance tests for qcow2 overlay
images with both qcow2 and raw backing files - sequential read,
random read, and warm read variants.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The backing_files option defaults to false, so qcow2 overlay
tests fail with MaxNestingDepthExceeded. Pass backing_files=on
when the test file is an overlay.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The focal image checksums have been moved in the -common
sha1sums file. Use the correct file for metrics.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Unlike most virtio feature bits, VIRTIO_BLK_F_RO is not optional.
It indicates that the host is refusing to permit write operations, and
the guest must not be allowed to override it.
However, the block device currently does not enforce this. If the guest
does not negotiate VIRTIO_BLK_F_RO, the block device will think the
device is writable and forward write requests to the backend.
This is not a security problem right now because the backing device of a
read-only device is always opened read-only. The kernel will thus
reject the write operations with EBADF. If support is added for
receiving the backing device file descriptor via SCM_RIGHTS (#7704),
it will be possible to have a read-only block device backed by a
writable file descriptor. This would make the bug a genuine security
vulnerability.
Fix the bug by explicitly checking if VIRTIO_BLK_F_RO was offered but
not negotiated. In this case, log a warning and proceed as if the guest
did acknowledge the feature. This always indicates a guest driver bug.
Fixes: #7697
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
OVMF sends FLUSH requests to read-only virtio-block devices. Refusing
these requests prevents OVMF from accessing the EFI System Partition and
therefore makes VMs unable to boot. Accept these requests instead.
them.
Ignoring these requests is possible, but inconsistent with fsync(2)
which honors them.
Fixes: #7698
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Add handling for GHCB_INFO_SPECIAL_DBGPRINT VMG exit in the SEV-SNP
guest exit handler. This exit occurs when the guest sends debug print
requests through the GHCB interface.
Without this handler, SEV-SNP guests fail to boot when debug output
is triggered, such as when a debugger is attached to the guest image.
The handler acknowledges the exit without printing to avoid performance
degradation from frequent debug print requests.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The --net help text documented fd as fd=<fd1,fd2...>, but
comma-separated FD lists in option values must be bracketed to avoid
top-level option splitting.
Update NetConfig::SYNTAX to use fd=<[fd1,fd2,...]>, matching parser
behavior and existing net parsing tests:
`cargo test -p vmm test_net_parsing`
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Add comprehensive integration tests for DISCARD and WRITE_ZEROES:
Multiqueue stress tests verify concurrent operations across queues,
testing scattered writes with simultaneous fstrim, and write/discard
races that stress refcount table locking.
Format specific tests verify QCOW2 deallocates clusters after DISCARD,
raw files create holes using fallocate, and unsupported formats VHD
and VHDX correctly reject DISCARD requests.
Tests for sparse=off verify raw files preallocate full disk size and
QCOW2 uses zero flag instead of deallocating clusters.
Add helper functions to verify sparse files, count QCOW2 zero flagged
regions using qemu-img map, and verify guest reads zeros from
discarded regions.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add comprehensive tests for DISCARD and WRITE_ZEROES operations:
QCOW2 zero flag test validates the complete workflow: allocate
cluster, DISCARD it, verify reads return zeros, write new data,
verify cluster reallocated.
QcowSync tests verify punch_hole and write_zeroes with Arc<Mutex<>>
sharing, including tests for cache consistency with multiple async
I/O operations.
RawFileSync tests verify punch_hole and write_zeroes using
fallocate.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Implement punch_hole() and write_zeroes() for raw file backends using
io_uring and fallocate.
punch_hole() uses FALLOC_FL_PUNCH_HOLE to deallocate storage.
write_zeroes() uses FALLOC_FL_ZERO_RANGE to write zeros efficiently.
Both use FALLOC_FL_KEEP_SIZE to maintain file size.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Implement punch_hole and write_zeroes for QcowSync backend by
delegating to QcowFile::punch_hole which triggers cluster
deallocation. write_zeroes delegates to punch_hole as unallocated
clusters read as zeros in QCOW2.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add VIRTIO_BLK_T_DISCARD and VIRTIO_BLK_T_WRITE_ZEROES request types.
Parse discard/write_zeroes descriptors (sector, num_sectors, flags),
convert to byte offsets, and call punch_hole/write_zeroes on the disk
backend. Mark as unsupported in sync mode.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Implement DISCARD using QCOW2 zero flag (bit 0 of L2 entries) with
sparse aware behavior.
When sparse=true - fully deallocate clusters by decrementing
refcount, clearing L2 entry, and reclaiming storage via punch_hole
when refcount reaches zero.
When sparse=false - use zero flag to keep storage allocated while
marking as reading zeros. Only works when cluster is not shared.
Shared clusters are fully deallocated.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When sparse=false is configured, preallocate the entire raw disk file
at startup using fallocate(). This provides space reservation and
reduces fragmentation.
Only applies to raw disks. QCOW2/VHD/VHDX formats manage their own
allocation.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add sparse parameter to QcowFile constructors and propagate it from
device_manager through QcowDiskSync. This makes the sparse configuration
available throughout the QCOW2 implementation for controlling allocation
and deallocation behavior.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add supports_zero_flag() to DiskFile trait to indicate whether a disk
format can mark clusters/blocks as reading zeros without deallocating
storage.
QCOW2 supports this via the zero flag in L2 entries. VHDX also has
PAYLOAD_BLOCK_ZERO state for this, though it's not yet implemented in
cloud-hypervisor.
This enables DISCARD to be advertised even with sparse=false for formats
with zero-flag support, since they can mark regions as zeros (keeps
storage allocated) instead of requiring full deallocation.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add sparse boolean configuration option to DiskConfig with a default
value of true to control disk space allocation behavior.
When sparse is true, the disk uses sparse allocation where deallocated
blocks are returned to the filesystem, and the DISCARD feature is
advertised to the guest.
When sparse is false, disk space is kept fully allocated and DISCARD
is not advertised.
WRITE_ZEROES is always advertised when the backend supports it,
regardless of the sparse setting.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add capability query to DiskFile trait to check backend
support for sparse operations (punch hole, write zeroes,
discard). Only advertise VIRTIO_BLK_F_DISCARD and
VIRTIO_BLK_F_WRITE_ZEROES when the backend supports these
operations.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add functions to probe whether a file or block device actually
supports PUNCH_HOLE and ZERO_RANGE operations at runtime. The
probe is performed at file open time by testing the operations
at EOF with a zero-length range, which is a safe no-op.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add punch_hole() and write_zeroes() methods to the AsyncIo trait
with stub implementations for all backends. These will be used to
support DISCARD and WRITE_ZEROES operations.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Decompose the monolithic `new_from_memory_manager` function into
smaller, focused helper methods to improve code readability,
maintainability, and testability.
Changes:
- Extract `should_force_iommu()` to determine IOMMU requirements for
confidential computing (TDX/SEV-SNP)
- Extract `should_stop_on_boot()` to check debug pause configuration
- Extract `create_cpu_manager()` to encapsulate CPU manager creation
and CPUID population
- Extract `init_tdx_if_enabled()` for TDX-specific VM initialization
- Extract `create_device_manager()` to encapsulate device manager setup
- Extract `hypervisor_specific_init()` to orchestrate initialization
sequences for different hypervisors (KVM, MSHV, SEV-SNP)
- Extract `init_sev_snp()` for SEV-SNP confidential VM setup
- Extract `init_mshv()` for MSHV hypervisor initialization
- Extract `init_kvm()` for KVM hypervisor initialization
- Extract `create_fw_cfg_if_enabled()` for fw_cfg device creation
This refactoring replaces complex nested `cfg_if!` blocks with cleaner
conditional method calls, providing clear separation between hypervisor-
specific initialization paths while preserving existing functionality.
No functional changes intended.
Issue: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7598
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Update .lychee.toml to exclude the following patterns:
- ARM domains (developer.arm.com, infocenter.arm.com) which return 403
Forbidden due to anti-bot protections in CI.
- Local TCP addresses (192.168.1.10) which are unsupported by the
link-checker tool.
- The .lychee.toml file itself, to prevent the tool from recursively
checking its own regex exclusion patterns as valid URLs.
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Document device_id parameter in NumaConfig, automatic
guest_numa_id assignment, default NUMA distances and
restrictions on Generoc Initiator NUMA nodes
Add numa configuration examples with GPU device and distance
relationships.
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Add test_guest_numa_generic_initiator to validate ACPI Generic
Initiator Affinity (SRAT Type 5) support for VFIO device.
The test verifies the following :
- Guest VM boots with a VFIO device associated with a {cpu,
memort}-less NUMA node
- Guest Kernel correctly detects Generic Initiator through
ACPI tables SRAT, SLIT
- NUMA topology in the guest includes the device-only node
with correct distances
Invoked via :
./scripts/dev_cli.sh tests --integration -- --hypervisor kvm \
--test-filter test_guest_numa_generic_initiator
The test requires a real VFIO device bound to vfio-pci driver and
skips gracefully if hardware is unavailable.
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Update FDT generation to skip NUMA properties when Generic Initiator
nodes are present, preventing conflicts between FDT and ACPI NUMA
information. FDT cannot represent Generic Initiator nodes, so ACPI
(via SRAT Type 5) becomes the authoritative source for the entire
NUMA topology when Generic Initiators exist.
Skip FDT numa-node-id properties in CPU and memory nodes
when Generic Initiator is present
Distance map bug fix : iterate over actual NUMA node IDs instead
of 0..len()
Use distance symmetry to derive distance when forward config is
missing
Default to distance cost 20 when neither direction specified
Only create memory nodes if NUMA node has memory region
Added unit tests
ARM64 boot protocol:
https://docs.kernel.org/arch/arm64/booting.html
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Support ACPI Generic Initiator Affinity to associate
PCI devices with NUMA proximity domains
Add GenericInitiatorAffinity struct
Add from_pci_bdf() to encode PCI Segment:Bus:Device.Function
Add from_acpi_device() for ACPI device handles (future use)
Generate SRAT Type 5 entries for nodes with device_id
Improve create_slit_table() to check distance symmetry when
forward distance is missing
Track device ID to BDF mappings in DeviceManager
Includes comprehensive unit tests
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Validate device_id in numa config is mutually
exclusive with cpus and memory_zones
Add NumaConfig::validate() and modify NumaConfig::parse()
Add ValidationError::InvalidNumaConfig for detailed error
messages
Include unit tests covering valid and invalid configs
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Add an optional device_id string field to NumaConfig for identifying
PCI devices associated with a NUMA node. This is used by the Generic
Initiator support to map devices to their proximity domain.
Update OpenAPI spec (cloud-hypervisor.yaml) to include the
new device_id field in the NumaConfig schema.
The device_id is optional and parsed from the --numa parameter:
--numa "device_id=<device_id>,distances=[...],..."
The optional field is accepted but not used.
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
The documentation says guest_numa_id is required to be unique and
therefore the parser() giving default value for non-existing
guest_numa_id with .unwrap_or(0) is dangerous.
Return a validation error if guest_numa_id is not provided instead
of silently defaulting to 0.
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Verify live resize of QCOW2 disks works via the API, including
resizing that requires L1 table growth.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add support for live resizing QCOW2 images. This enables growing
the virtual size of a QCOW2 disk while the VM is running.
Key features:
- Growing the image automatically expands the L1 table if needed
- Shrinking is not supported
- Resizing for images with backing files is not supported
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Provide a stub implementation for save_data_tables() to unblock pause
functionality. Without this, pausing a VM causes Cloud Hypervisor to
panic due to the unimplemented!() macro. This unblocks the
test_api_http_pause_resume testcase. We don't need to save any state
just to pause and resume.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Instead of closing a file descriptor that belongs to the vhost-user
frontend, drop the vu_common_ctrl::VhostUserHandle and the
vhost::vhost_user::Frontend it contains. This causes the destructor to
drop the file descriptor.
This breaks the last DPDK test, so disable it. See #7689.
Fixes: #7163
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
When a VFIO device with multiple MMIO regions is hot-unplugged, each
region must be individually matched and removed from the DeviceManager's
mmio_regions list. Compare per-region rather than building an aggregate
across all regions, which would never match any individual entry.
Also remove the now-unused HashSet import.
Signed-off-by: Damian Barabonkov <dbctl@pm.me>
Change has_matching_slots() to compare two MmioRegion instances
directly rather than requiring callers to construct an intermediate
HashSet of slot numbers. Remove the now-unused
user_memory_region_slots() method and HashSet import.
Signed-off-by: Damian Barabonkov <dbctl@pm.me>
Replace `clone()` with `take()` when retrieving device configurations
from `DeviceManager.config`.
This avoids unnecessarily copying the device configuration lists (e.g.,
`disks`, `net`, `fs`) when they are being processed and subsequently
moved out of the configuration. This optimization improves performance
by reducing memory allocations and cloning overhead.
Signed-off-by: Changyuan Lyu <changyuanl@google.com>
Since kernel commit 6693731487a8 ("vsock/virtio: Allocate nonlinear SKBs
for handling large transmit buffers"), a large vsock packet can be split
into multiple descriptors.
If we encounter such TX packets, pull the content into an owned buffer.
Fixes: #7672
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When running manual tests locally, it is sometimes necessary to
generate a cloud-init file at a custom path instead of defaulting
to /tmp. This is useful for developers and higher-level management
layers where files in /tmp may be cleaned up automatically.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Backing files (e.g. for QCOW2) interact badly with landlock since they
are not obvious from the initial VM configuration. Only enable their use
with an explicit option.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Test reading from overlay at offsets beyond backing file returns
zeros. Covers reads within backing range, beyond backing, and
boundary spanning.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When an overlay QCOW2 image is larger than its backing file, reads
from offsets beyond the backing file virtual size would previously
fail with an I/O error.
The backing file virtual size is determined at open time and stored
for bounds checking during read operations:
- If the entire read is beyond the backing size, return all zeros
- If the read spans the boundary, read available data from backing and
fill the remainder with zeros
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Set HV_PARTITION_CREATION_FLAG_SMT_ENABLED_GUEST when the guest
topology has more than one thread per core. This allows the
hypervisor to schedule guest VPs correctly on SMT-enabled hosts.
Without this flag, the hypervisor schedules guest VPs incorrectly,
causing SMT unusable.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The kernel allows madvise on shared memory if
/sys/kernel/mm/transparent_hugepage/shmem_enabled is set.
Always try and configure THP via madvise when
the user requests THP be enabled.
If this fails, only a warning log is emitted and THP won't be enabled.
Signed-off-by: Champ-Goblem <cameron@northflank.com>
When restoring from snapshot with shared=false, write access to the
backing file is not required. Opening it read-only allows restore to
succeed on read-only media and overlay lower layers while preserving
MAP_PRIVATE semantics.
Signed-off-by: Rowen-Ye <rowenye1@gmail.com>
This patch adds the skeleton of the CVM test
support and modify existing scripts and test framework
to enable such scenario. Split the sha1sum to support both
regular and CVM guest. Add one test case for CVM. Will further
add more test cases.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
X64_64 image download steps is being used for both
regular and CVM guest. Keeping the steps withing a function
in the test-util.sh
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Now Guest struct has an option to set timeout.
No need to pass timeout while booting the guest.
If no timeout is set, default is used.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Modify Guest struct to keep some test specific
data so that test cases could be shared between
regular guest and CVM.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Testing generic vhost-user devices will require virtiofsd to support the
--tag option, which v1.8.0 does not support.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Replace generic WritingHeader error with specific SyncingHeader
error for header fsync operations. This provides more precise
error reporting when syncing QCOW2 header changes to disk fails.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Wrap QcowFile in Arc<Mutex<>> to ensure thread safety when multiple
virtio queues access the same QCOW2 image concurrently.
Previously, each queue received its own QcowSync instance via
new_async_io() that shared the underlying QcowFile through Clone.
However, cloned QcowFile instances share internal mutable state
(L2 cache, reference counts, file seek position) without
synchronization, leading to data corruption under concurrent I/O.
This change serializes all QCOW2 operations through a mutex, which
ensures correctness at the cost of parallelism. A more performant
solution would require separating metadata locking from actual I/O
operations, tracked in #7560.
Related: #7560
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
TL;DR: Would reduce CI pressure by cancelling more "unnecessary" runs
but I can't verify without running a merge queue.
A common development pattern is to push a change and then immediately
check CI results. Follow-up fix pushes are quite common, which leads to
multiple CI runs being queued for the same pull request.
In Cloud Hypervisor, the size and cost of the CI matrix means that
several consecutive pushes (for example 3-4 in a short time) put
significant pressure on CI runners and noticeably increase feedback
latency.
In practice, concurrency handling is especially tricky for the merge
queue. From personal experience: If one does not take special care, CI
runs triggered by a `merge_group` can cancel each other, as in a merge
queue there are two runs for each job by default: one for the normal PR
and one for the merge commit. This is easy to run into, also because the
available documentation and best practices for this feature are not very
good.
At the same time, our workflows do not run on `push` events, but only
on `pull_request` and `merge_group`. Because of this, using
`${{ github.ref }}` alone as a concurrency key is not very meaningful,
and in practice only few runs are actually cancelled for successive PR
updates. Therefore, we should improve the usage of this feature.
This change tries to improve the situation by refining the concurrency
group key. The goal is to keep cancellation for multiple PR pushes,
while at the same time preventing unintended cancellations in the merge
queue by separating `merge_group` runs from regular PR runs.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
In kata-containers we use the api_client crate, but it's currently
failing our cargo deny check due to missing license, and there aren't
any license files within the crate, so I haven't found a good way to
work around this.
Alternatively I'd be happy to add the license to the workspace crate
and then reference it here, but that seems to clash with the direction
of the project in #7525.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
QCOW2 v3 autoclear_features field contains bits for features whose
metadata becomes invalid when the image is modified by software that
doesn't understand them. Defined bits:
- Bit 0: Bitmaps extension
- Bit 1: Raw external data
Cloud-hypervisor doesn't support bitmaps or external data files, so
all autoclear bits are cleared on writable open. This signals other
tools that these features' data may be stale.
Readonly opens preserve autoclear bits unchanged.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This adds some missing features that are useful. In particular it adds
VIRTIO_F_RING_INDIRECT_DESC which gives a performance improvement.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Reported-by: Daniel Farina <daniel@ubicloud.com>
TDX builds its own ACPI tables in `create_acpi_tables_tdx` so it will
return None in the standard `create_acpi_tables` function and the
assertion for `rsdp_addr` will fail.
Signed-off-by: Zhibin Li <banlu.lzb@antgroup.com>
Add tests for corrupt bit behavior during I/O operations.
- Unaligned L2 table address triggers corrupt bit on read
- Unaligned cluster address triggers corrupt bit on read and write
- Normal operations do not set the corrupt bit
- V2 images work correctly without feature bits
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
MSHV doesn't present an ITS to guests. So, /proc/interrupts would never
have "ITS-PCI-MSIX".
Instead, a Gicv2m frame is presented to guests. So expect
"GICv2m-PCI-MSIX" in testcases.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Add integration tests for QCOW2 corrupt bit handling. Verify that
images with the corrupt bit set are rejected for writable access but
allowed for read-only access with a warning.
Helper functions are added to read and modify the corrupt flag in the
QCOW2 v3 header.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Validate that L2 table offsets and refcount block offsets are cluster
aligned. Set the corrupt bit when unaligned offsets are detected, as
this indicates corrupted L1 or refcount table entries.
Validate that data cluster offsets from L2 entries are cluster aligned
during both reads and writes to existing clusters. Set the corrupt bit
when unaligned data cluster offsets are detected.
Prevent allocation of clusters at offset 0, which contains the QCOW2
header and should never be allocated. This catches corruption in the
available clusters list. Set the corrupt bit when this condition is
detected.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Set the QCOW2 corrupt bit when internal inconsistencies are detected
that indicate image metadata may be corrupted:
- Decompression decode failure, meaning compressed cluster data is
invalid
- Decompression size mismatch, where decompressed data doesn't match
expected cluster size
- Partial write after decompression, where L2 table was updated but
data cluster not fully written, leaving metadata inconsistent
- Invalid refcount index, where cluster address is outside valid
refcount table range, indicating a corrupted L2 entry
- Dirty L2 with zero L1 address, where L2 table is marked dirty but
L1 has no address for it
Note: Marking decompression failures as corrupt is more conservative
than QEMU, which returns EIO without setting the corrupt bit. This is
debatable since corrupted compressed data doesn't necessarily indicate
metadata corruption, but it provides a stronger safety guarantee by
preventing further writes to potentially damaged images.
Once set, the image can only be opened read-only until repaired with
qemu-img check -r.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add comprehensive tests for the corrupt bit handling. Cover writable
rejection, read-only access, persistence, and dirty bit
coexistence.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Implement proper handling of the QCOW2 corrupt bit (incompatible feature
bit 1) according to the specification:
- Add Error::CorruptImage for rejecting writable opens of corrupt images
- Add CORRUPT to SUPPORTED features (handled specially, not rejected)
- Add QcowHeader::set_corrupt_bit() to mark images as corrupt
- Add QcowHeader::is_corrupt() helper method
- Reject writable opens of corrupt images with Error::CorruptImage
- Allow readonly opens of corrupt images with a warning
The corrupt bit indicates that image metadata may be inconsistent. Per
spec, such images must not be written to until repaired by external
tools like qemu-img. Read-only access is permitted to allow data
recovery.
Users can open corrupt images read-only using:
--disk path=/path/to/image.qcow2,readonly=on
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
VsockPacket::hdr holds a raw pointer to the address of the VSock packet
header, which is in guest memory. It opens the door to double-fetch
(or TOCTOU) race conditions. Therefore, VSockPacket::hdr content can't
be trusted since it can be arbitrarily changed by the guest, at any
time.
To mitigate this, we can copy the header content to an array in VMM's
memory that the guest can't modify.
Signed-off-by: Thomas Leroy <thomas.leroy.mp@gmail.com>
Update QcowHeader and other related places to use BeUint methods
internally for reading/writing header fields.
This removes the byteorder dependency from mod.rs and consolidates
all big-endian file I/O through the shared BeUint trait.
Suggested-by: Rob Bradford <rbradford@rivosinc.com>
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add a read_be() method to the BeUint trait and make it pub(super)
so it can be used across the qcow module. Change BeUint::write_be()
to take Self instead of u64, providing type safety through TryFrom
conversion.
Suggested-by: Rob Bradford <rbradford@rivosinc.com>
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add tests to verify dirty bit is set while VM runs and cleared on
clean shutdown. As part of it, ensure graceful shutdown when OS
disk verification requires consistent image state.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Verify dirty bit is set on open and cleared on close for v3 images.
Ensure v2 and read-only files are not affected. Update existing
tests.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add support for the dirty bit (bit 0 of incompatible_features) which
indicates the image was not closed cleanly. This improves data
integrity by allowing detection of potentially corrupted images.
On open:
- If dirty bit is already set, log a warning and trigger
refcount rebuild
- Set the dirty bit and write it to disk immediately
- Sync to ensure persistence before any writes
- Skip dirty bit and refcount rebuild for readonly files
On clean close:
- Clear the dirty bit in the header
- Write it to disk and sync
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Test all refcount_order values (0-6):
- Basic open for each width
- Write/read roundtrip
- Overwrite and multi-cluster allocation
- L2 cache eviction under memory pressure
- Sub-byte and byte-aligned max value handling
- Overflow error detection
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Reject refcount values exceeding the maximum for the image's
refcount_order. This prevents silent truncation when storing
refcounts in narrow widths (e.g., 1-bit max is 1, 4-bit max is 15,
etc.).
Returns RefcountOverflow error with the attempted value, maximum,
and bit width. Propagates as EINVAL to the guest.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
QCOW2 v3 specifies refcount_order 0-6 with
refcount_bits = 1 << refcount_order. Previously only 16-bit (order 4)
was supported.
Changes:
- RefcountBytes trait handles byte-aligned types (8/16/32/64-bit)
- Generic pack/unpack for sub-byte widths (1/2/4-bit)
- Function pointers for read/write selected at open time
- Internal refcount type widened from u16 to u64
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Parse the feature name table header extension to provide descriptive
error messages when unsupported incompatible features are detected.
Currently only the compression bit (bit 3, zstd) is supported.
This prevents opening qcow2 images with features that would cause
incorrect behavior or data corruption (e.g., dirty bit, corrupt bit,
external data file, extended L2 entries).
Feature names are defined as follows:
1. The image's feature name table header extension (if present)
2. Hardcoded fallback names for known features
3. Generic "unknown feature bit N" for undefined features
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Co-developed-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Implement read support for bit 0 in QCOW2 L2 table entries.
When this flag is set, the cluster reads as zeros without accessing
disk. This improves compatibility with QCOW2 images that use this
optimization.
According to the QCOW2 specification, bit 0 of the standard cluster
descriptor indicates that the cluster reads as zeros. Unlike
l2_entry == 0 indicating a completely unallocated entry, bit 0 can
be set on an allocated cluster.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
build_edk2 was leaving behind .built markers
even when compilation failed.
Gate creation of.built marker to occur only on
successful build
Modify build_edk2() to exit with error code
when arm64 firmware artifact : CLOUDHV_EFI.fd
is not produced
Fixes#7608
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
build_edk2() module in scripts/common-aarch64.sh
does not produce the UEFI firmware for aarch64 as
the commits used to assemble sources for acpica,
edk2-platforms and edk2 do not compile after GCC
version upgraded from 11.4.0 to 13.3.0 in the
developer container (ubuntu 22.04 to 24.04)
Apply minimum upgrade to EDK2_REPO and ACPICA_REPO
required to compile with GCC 13.3.0
while still assuring guest VM boot for all
integration tests
BaseTools: Brotli compression submodule that was
previously failing has been fixed following commit
bump
Developers can now produce UEFI firmware for
aarch64 using the following commands
```
./scripts/dev_cli.sh shell
source scripts/test-util.sh
source scripts/common-aarch64.sh
build_edk2
```
Update docs/uefi.md
Fixes#7608
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
Recent changes related to arm64 support in MSHV exposed
inconsistencies in the VM initialization and CVM boot paths.
The VM creation flow currently diverges across multiple scenarios,
including regular MSHV, CVM, and arm64, with each path performing
guest initialization steps in a different order.
Certain platform-specific requirements further constrain the ordering
of operations, such as the timing of address space creation,
IGVM loading, interrupt controller setup, and payload loading. For
CVM case address-space creation must be done after IGVM loading, and
PSP measurement. For Regular and arm64 this memory initialization
must be done early. For MSHV, vm.init() and sev_snp.init() are called in
different order which is run time and build time conditionally checked.
Additionally, while the KVM initialization path differs slightly
from MSHV, it shares common logic that is currently split across
separate conditional and build-time code paths, contributing to
fragmentation of the overall flow.
This change restructures the VM creation and initialization sequence
to better align shared logic, enforce scenario-specific ordering
constraints, and ensure consistent and correct behavior across all
supported configurations. In doing so, it restores proper CVM boot
behavior and improves the maintainability of the initialization code.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The spec says simply that that an empty payload should be returned on
error. Be slightly more helpful by adding a warning.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Based upon the discussion and in
https://github.com/rust-vmm/vhost/issues/29#issue-830820820 and the QEMU
behaviour the get_config offset should be zero. This was not caught by
our integration tests as the vhost-user-blk backend as implemented in
this repository does not use the offset.
Fixes: #7615
Signed-off-by: Rob Bradford <rbradford@meta.com>
Since the mshv integration workflow has been stable for a long time,
make the workflows no longer optional.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
Add warmup_iterations field to run iterations before measuring
performance. This complements existing cold start tests
by separating cache effects from steady state throughput.
New tests with 2 warmup iterations:
- block_qcow2_backing_qcow2_read_warm_MiBps
- block_qcow2_backing_raw_read_warm_MiBps
Results show warm cache is much faster and more consistent:
- QCOW2: 1766 MiB/s (4% variance) vs cold 960 MiB/s (73% variance)
- RAW: 1822 MiB/s (6% variance) vs cold 1300 MiB/s (55% variance)
RAW backing is 3% faster than QCOW2 in steady state.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The write_to() function is used by test code to create qcow2 files for
testing. For v3 headers with extended header_size (>104), it needs to:
1. Write the mandatory compression_type field at bytes 104-111
2. Write the header extension end marker at the header_size offset
3. Seek to backing_file_offset before writing the backing file path
Additionally, create_for_size_and_path() must set backing_file_offset
to account for the 8 byte extension end marker in v3 files, so the
backing file path doesn't overwrite the extension area.
Add unit tests for read_header_extensions() covering backing format
parsing (raw/qcow2), unknown extensions, and error cases (invalid
formats, invalid UTF-8). These tests depend on the header writing fixes
to create properly formatted v3 test files.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add support for parsing QCOW v3 header extensions to read the
backing file format. The QCOW v3 spec allows optional header
extensions between the fixed header and the backing file name.
Implement read_header_extensions() to parse the extension area,
which starts at the header_size offset. At the moment it is
used to read the backing file format. Further extension
processing is open in folow up implementations.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add support for raw backing files in addition to qcow2 backing
files. This enables QCOW2 overlays to use raw images as their
backing store.
The backing file format is auto-detected when not specified,
using the existing detect_image_type() function.
Add backing_file_format field to QcowHeader to store the format
type, which will be populated from header extensions by a
subsequent patch.
Modify new_from_backing() to accept a backing_format parameter,
consolidating support for both raw and qcow2 backing files in a
single function. The backing_file_size parameter allows overlay
creation without opening the backing file multiple times.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
In most of the cases, special registers don't change after emulations,
but current code sets them back unconditionally, and although some of
them are set over the register page, others require a system call and a
hypervisor to be updated, which is a waste it there were not changes.
Introduce and CPU update method for Microsoft Hypervisor emulator and
set special registers only when they were changed. This change reduces
guest boot time by 4% for a single VP guest boot (in L1VH partition) in
my experiments.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
This is a precursor change to overall ioctl and hypercall reduction
effort. The old (current) CPU state can be compared to the new to
determine what has changed and avoid unnecessary register updates.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
Minor modifications were made to make the sentences sound more natural.
Also fixed some parameter usage issues in bash code block.
Signed-off-by: Yi Wang <foxywang@tencent.com>
There are some syntax and format issues in tdx/sev documents.
Make some modification to make the description more natural.
And the link of SEV-SNP is invalid, fix it.
Signed-off-by: Yi Wang <foxywang@tencent.com>
There are some minor syntax and command issues in debug-port document.
As commit 5febdec81a (vmm: Enable `gdbstub` on AArch64) supported
aarch64, the docs should keep consistent.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Fix some minor syntax issues in api/building document to make
the sentences more fluent and easy to read.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Some description in the device document were inconsistent with the
source code. Also fix some syntax issues to make the sentences more
fluent.
Signed-off-by: Yi Wang <foxywang@tencent.com>
CI reports:
In scripts/test-util.sh line 216:
cleanup() {
^-- SC2329 (info): This function is never invoked. Check usage (or ignored if invoked indirectly).
The shellcheck can't trace calling in trap, so we need add hint
to make it happy.
Signed-off-by: Yi Wang <foxywang@tencent.com>
The IORT table's ID mapping uses a 256-ID offset per PCI segment to
ensure unique device IDs across all segments. This partitioning scheme
(output_base = 256 * segment_id) must match the device ID encoding used
in KVM MSI routing configuration [1].
This mapping assumes one bus per PCI segment, and supports up to 256 PCI
segments in the system.
[1] c9374d87ac
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The IORT specification (Revision E.b, Table 12) defines the ITS Group
Node structure with an ITS Identifiers array following the node header.
Although the IORT table is zero-initialized, this commit adds an
explicit write of the ITS identifier value (0) for clarity and spec
compliance.
This ITS identifier must match the `translation_id` field in the MADT
GIC ITS structure to ensure proper interrupt routing on ARM platforms.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The current IORT table implementation is based on IORT Spec revision E.b
[1], as evidenced by:
* The PCI root complex node revision being set to `3`
* The code being updated in late 2021 [2] when revision E.b was the
latest version
This patch ensures the IORT table is properly generated according to
this specification revision, fixing three issues:
1. The IORT table revision should be `3` rather than `2` (see Table 2 in
the spec [1])
2. The GIC ITS group node revision should be `1` rather than `0`
(see Table 12 in the spec [1])
3. The "Memory access properties" and "ATS Attribute" fields of the PCI
root complex node was set incorrectly - specifically the MAF (Memory
Access Flags) including CPM and DACS bits (see Tables 14, 15, and 17
in the spec [1])
[1] https://developer.arm.com/documentation/den0049/eb/?lang=en
[2] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3356
Signed-off-by: Bo Chen <bchen@crusoe.ai>
It should always succeed and is apparently implicitly called by libc or
some dependency somewhere.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
x86::__cpuid is safe on Rust ≥1.94 but unsafe on older versions. This
causes unused_unsafe warnings when compiling with Rust ≥1.94. However,
on earlier Rust versions, the code won’t compile if the unsafe blocks
are absent.
Work around this by adding #[allow(unused_unsafe)] where needed to
suppress the warnings.
See #7588 for more discussion.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
On MSHV, exposing multithreaded CPU topologies requires setting the
PROCESSORS_PER_SOCKET partition property so that CPUID.0xB reports
correct logical processor counts and topology levels to the guest.
This property must be set after all vCPUs are configured, as the
hypervisor uses the complete vCPU layout to derive and report CPU
topology information.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This change is a prerequisite for live disk resizing. Before this
commit, the epoll-handler threads just got a copy of the sector
size which we cannot update during runtime.
On-behalf-of: SAP thomas.prescher@sap.com
Signed-off-by: Thomas Prescher <thomas.prescher@cyberus-technology.de>
Add sequential and random read performance tests for QCOW2 overlays
with QCOW2 backing files.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce a new BlockControl struct to encapsulate fio operation
parameters. This replaces the tuple-based fio_control with a more
extensible structure that includes:
- fio_ops: The FIO operation type
- bandwidth: Whether to measure bandwidth or IOPS
- test_file: The file path to test against
This refactoring enables reusing performance_block_io with different
test files.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Virtio PCI devices are created in a set of nested functions. In each
of this functions a vector is created to add created devices to, only
to be appended to the vector of the higher nesting level. Those nested
vectors are unnecessary as we can directly write to the member of
`DeviceManager`.
Signed-off-by: Pascal Scholz <pascal.scholz@cyberus-technology.de>
On-behalf-of: SAP pascal.scholz@sap.com
The TILE data state of AMX may require 8KB+ space, calling the legacy
KVM_GET_XSAVE will encounter an error since KVM_GET_XSAVE only can get
4KB space. This patch adds KVM_GET_XSAVE2 support to allow snapping more
data.
Fixes: #7533
Signed-off-by: Songqian Li <sionli@tencent.com>
Refactor write_pointer_table to accept iterators instead of requiring
materialized vectors, eliminating temporary allocations in L1 table
sync operations.
Changes:
- Modified write_pointer_table() to take Iterator<Item = &T> and
dereference internally before passing owned values to the callback
- Added write_pointer_table_direct() convenience wrapper for cases
without value transformation
- Updated sync_caches() to use l1_table.iter() directly instead of
.get_values().iter().copied()
- Implemented Deref<Target = [T]> for VecCache to enable direct .iter()
Performance impact:
- Eliminates L1 table allocation during sync (~2KB per 100GB disk)
- L2 and refcount table writes already used slices, no change there
- Zero performance overhead: iterator dereferencing is equivalent to
.copied() and optimizes identically
The L1 sync previously collected entries into a Vec to apply the
OFLAG_COPIED flag. The new iterator+callback pattern computes this
on-the-fly, avoiding the allocation entirely.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add Deref<Target = [T]> implementation for VecCache<T> to allow direct
slice operations without explicitly calling get_values(). This enables
cleaner code patterns like cache.iter() instead
of cache.get_values().iter().
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add fcntl to virtio_block_thread_rules to allow try_clone() on file
descriptors. The try_clone() method uses fcntl(fd, F_DUPFD_CLOEXEC)
to duplicate file descriptors, which is needed for efficient QCOW2
L1 table sync that avoids temporary allocations.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Creating a vdpa_net with an associated MAC address and setting the right
amount of queues in order to fix the integration test related to
vdpa_net.
Fixes: #5756
Signed-off-by: Sebastien Boeuf <seb@rivosinc.com>
In [0] we refactored some Arc<Mutex<T>> parameters to &Mutex<T>> to
satisfy clippy's needless_pass_by_value lint. Nevertheless, this is also
not so idiomatic, so as a follow-up, we put the responsibility to lock
objects to the caller side (only where this is not strictly needed by
the callee).
While on it, I also tried to pass vm_config directly into
pre_create_console_devices() which would clean up some code, but then
we have interleaving mutable and immutable borrows of the Vmm, which
are denied by the borrow checker.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Our CI test execution system finds some mergeable pages in the system
directly on boot since a recent update. Remove the broken assumption in
the test_memory_mergeable integration test and only check if the guest
VMs influence the shared pages count.
Signed-off-by: Stefan Nürnberger <stefan.nuernberger@cyberus-technology.de>
https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7294 adjusted
the checks for read-only requests made to virtio-blk devices and started
rejecting VIRTIO_BLK_T_GET_ID requests. These requests do not perform
any writes and are needed in order to access device serials from within
the guest.
Signed-off-by: Connor Brewster <cbrewster@hey.com>
`cargo rustc` is incompatible with virtual manifests, so the CI needs to
use cargo build instead. However, passing `RUSTFLAGS="-D warnings"` via
the environment would propagate to all dependencies, and some of them
currently fail to build under ``-D warnings` due to issues like [0]:
```
error: creating a mutable reference to mutable static
--> src/temp.rs:97:5
|
97 | DIRS.pop()
| ^^^^^^^^^^ mutable reference to mutable static
```
To resolve this, apply ``-D warnings` only to the `cargo clippy`
commands (which apply to our workspace only) and avoid enforcing it for
the entire cargo build.
[0]: https://github.com/cloud-hypervisor/cloud-hypervisor/actions/runs/19962283528/job/57245376263?pr=7525
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
TL;DR: cargo clippy|check|... now runs on whole workspace by default.
## Steps
- add new workspace member `cloud-hypervisor`
- move `./src` to new workspace member
- move `./tests` to new workspace member
- move relevant parts from Cargo.toml to new workspace member
- kept necessary parts in main Cargo.toml, such as profile
configurations
## About
The main Cargo.toml historically mixes workspace and crate definitions
for cloud-hypervisor and ch-remote. This makes it hard to read and
requires `--workspace` to run cargo clippy or cargo test on all
workspace members, which is counter-intuitive.
This patch separates the workspace from the crate definition in the main
Cargo.toml file. After this, cargo clippy, cargo test, etc., work on the
whole workspace naturally, giving a smoother developer experience. The
Cargo.toml without a package definition is also called a virtual
workspace or virtual manifest by Cargo [0].
Backporting is not a concern: CHV no longer backports, but the affected
files are rarely modified anyway.
[0] https://doc.rust-lang.org/cargo/reference/workspaces.html#virtual-workspace
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This will also prevent some useless rebuilds. Using `--verbose` we can
observe that the build.rs causes frequent useless rebuilds - having
less is a good thing. They come from the dependency of `build.rs` to
the local git repository.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The BufWriter must be flushed explicitly to handle errors
properly. Without explicit flush, errors during the implicit
drop flush are ignored.
This is the same issue fixed for write_pointer_table
in commit 85556951a.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The MSR enumeration logic in MSHV does not accurately
account for the capabilities of the underlying
hardware and hypervisor. Previously, integration tests
passed because only the default MSRs—universally supported
across platforms—were involved. However, recent feature
additions in MSHV have introduced MSRs that are not
supported on Intel hardware, causing failures.
This patch retrieves the MSRs from the VM specific
API call that filters out the unsupported MSRs.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add set_cluster_refcount_track_freed() helper to consolidate the
common pattern of setting a cluster refcount and tracking freed
refblocks. This reduces code duplication and improves readability.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The image passed for the guest construction is copied. Previously,
check-img has been checking the unchanged image from the workspace dir,
which is supposed to be error free.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Freed clusters correctly have refcount=0. Remove the assertion that
expected no clusters with zero refcount, as it was validating the
buggy behavior.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When converting a compressed cluster to standard during write
operations, the old compressed cluster's refcount was never
decremented, causing leak warnings by `qemu-img check ..`
`Leaked cluster X refcount=N reference=M`
Additionally, compressed data can span multiple physical clusters,
not just one. The compressed cluster address and size are encoded
in the L2 entry, and the data may cross cluster boundaries.
The proper handling is implemented as follows:
- Extract compressed cluster address and size before overwriting
L2 entry
- Identify all clusters occupied by the compressed data
- Decrement refcount for each cluster in the range
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When a refcount block is evicted from cache and replaced with a new
one, the old refcount block cluster was added to unref_clusters but
its refcount was never decremented to 0 on disk. This left the cluster
with refcount=1 while no metadata referenced it, causing errors in
qemu-img check
`Leaked cluster X refcount=1 reference=0`
This fix recursively calls set_cluster_refcount(freed_cluster, 0) to
properly decrement the freed refcount block's refcount on disk. The
recursion handles cascading replacements where freeing one refcount
block may trigger the replacement of another.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
"Signed-off-by" is the only variant that is accepted. So we
should remove the inconsistency to prevent:
- user forgets this at all
- CI complains
- user adds "Signed-Off-By"
- CI still complains because of the wrong format
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
We enable AMX tile state components via the hypervisor (as introduced
in the previous commit) instead of doing this inline in the body of
`CpuManager::new`.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Currently when the user configures AMX the corresponding state
components get dynamically enabled directly inside the function body of
vmm::cpu::CpuManager::new.
With our ongoing work on CPU templates/profiles, there will (likely) be
one more binary crate for producing CPU profiles that also needs to do
this (without creating a CpuManager) and it may also be the case that
we will need to call this function prior to `CpuManager::new` during
live migrations.
We thus add a method for enabling the AMX tile state components on the
hypervisor trait that may be called wherever necessary. We argue that
this is beneficial for code clarity independently of the upcoming CPU
templates/profiles PR that we are working on.
The astute reader will notice that the logic introduced here is not 100%
the same as what is done inside the vmm::cpu::Cpumanager::new method. We
claim that our approach is more in-line with the official documentation.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
This patch updates the documentation to reflect the newly added
nested CPU feature option in the CLI.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add an option in the CLI to enable nested support.
Adding an option --cpu nested=on|off to the CPU
argument to enable nested support for Microsoft
Hypervisor. By default it is enabled for both KVM
and MSHV.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Create HypervisorVmConfig early and pass the
struct to VM creation API in the vmm crate. Getting
rid of multiple conditional parameter.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The OFLAG_COPIED bit (bit 63) indicates a cluster's refcount is exactly
1 and doesn't need copy-on-write. This bit must be set in L1 entries
when their referenced L2 clusters have refcount=1.
Previously, L1 entries were always written as raw addresses without the
OFLAG_COPIED bit, violating the QCOW2 specification and causing qemu-img
check to report errors like
`ERROR OFLAG_COPIED L2 cluster: l1_index=X .... refcount=1`
The implementation queries each L2 cluster's refcount in sync_caches()
and sets OFLAG_COPIED appropriately when writing the L1 table. This
ensures QCOW2 images are specification compliant and maintain correct
COW semantics to avoid data corruption.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Previously the code relies on the implicit flush when BufWriter is
dropped. That's not safe.
Per BufWriter's document:
```
It is critical to call flush before BufWriter<W> is dropped. Though
dropping will attempt to flush the contents of the buffer, any errors
that happen in the process of dropping will be ignored. Calling flush
ensures that the buffer is empty and thus dropping will not even attempt
file operations.
```
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The MSRs are constant at boot time so rather than creating a vector in
the boot_msr_entries() method instead reaturn a reference to static MSR
array data.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Previously, state that we accumulated during the migration process in
the receiver was kept in `mut Option` variables or HashMaps. The
problem is that it is unclear in the code when these variables can be
safely used. It's also difficult to add new state, such as the state
we need to handle the upcoming feature for performing the migration
using multiple connections.
To solve this, I've refactored the code to use the state machine
pattern. Each state carries the data it needs to. Any state that
didn't arrive yet (memory_files, memory_manager) cannot be accessed
until we are in the proper state.
Some benefits that fall out of this:
- We now respond to all requests, even invalid ones, at least with an
error message.
- Any error handling a request will result in an error message being
sent.
- There is only a single place where responses are sent and thus it's
very hard to mess this up in the code.
- The main protocol state machine fits on a screen.
I would argue that especially the error cases are now much more
consistent. There is still a lot to be done. There is still state
transfer via self.vm and similar. In an ideal world, this would also
be carried by the state machine. I also want to see better handling of
payloads, which are still handled all over the place, but this change
is already too big. :)
Co-authored-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP julian.stecklina@sap.com
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
... and nuke some Option<> while I was there. Given that HashMap has a
usable default and we end up passing an empty HashMap anyway, just get
rid of the Option.
On-behalf-of: SAP julian.stecklina@sap.com
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
During the lifecycle of a confidential VM, the guest may revoke
previously shared pages via an attribute-intercept VM exit.
When this happens, the host-side cache must be updated so that any
subsequent access by the VMM triggers a fresh request for the guest
to re-share the page.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This is a follow-up of [0].
# Advantages
- This saves dozens of unneeded clone()s across the whole code base
- Makes it much easier to reason about how parameters are used
(often we passed owned Arc/Rc versions without actually needing
ownership)
# Exceptions
For certain code paths, the alternatives would require awkward or overly
complex code, and in some cases the functions are the logical owners of
the values they take. In those cases, I've added
#[allow(clippy::needless_pass_by_value)].
This does not mean that one should not improve this in the future.
[0] 6a86c157af
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Fix the condition to correctly identify a new PTY connection.
Fixes: 287887c99 (vmm: fix console IO safety)
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Previously, `into_raw_fd()` transferred fd ownership to epoll while
keeping separate clones in `reader` and `writer`, causing leaks when
the stream was closed. Now, `reader` owns the fd and epoll borrows it.
When the FD is closed in the kernel, reader will be reset to None.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The Intel Granite Rapids processors include more AMX related features
that are advertised in leaf 0x7 subleaf 0x1. If the VM is not
configured to support AMX (the default) then these feature bits need
to be masked out.
Furthermore Tile information and TMUL information
in leaves 0x1d and 0x1e respectively are also purely related to AMX
and should also be zeroed whenever AMX support is disabled.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP <oliver.anderson@sap.com>
When enabling amx feature, we should call arch_prctl to request
permission to use tile data for guest. The permission should be
requested before the first vcpu is created, so we need to call
arch_prctl in vmm thread. This patch adds the arch_prctl syscall for
vmm_thread_rules.
Fixes: #7516
Signed-off-by: Songqian Li <sionli@tencent.com>
This helps to uncover expensive and needless clones in the code base.
For example, I prevented extensive clones in the snapshot path where
(nested) BTreeMap's have been cloned over and over again. Further,
the lint helps devs to much better reason about the ownership of
parameters.
All of these changes have been done manually with the necessary
caution. A few structs that are cheap to clone are now `copy` so that
this lint won't trigger for them.
I didn't enable the lint so far as it is a massive rabbit hole and
needs much more fixes. Nevertheless, it is very useful.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
We've encountered issues that Cloud Hypervisor corrupts disk images
after use. Those issues may not be immediately obvious until the
corrupted images are used again.
Run consistency checks over the disk images in the test cases to catch
issues as early as possible.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
It is a common use case to run a subset of tests locally to verify
certain functionalities.
The default behaviour for nextest is to error out if no tests are run.
That causes the test scripts to return a non-zero value (failure). Pass
`--no-tests=pass` to nextest to match what `cargo test` does if no tests
are run.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This is continuing the work from [0]. Some places in CHV
"misuse" Result::map_err for side-effects. inspect_err is
a better suited alternative for exactly that use-case.
Unfortunately, there is no clippy lint for this yet.
[0] f02745a7ed
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This commit is the first in a series of similar commits to clean up
obsolete `extern crate` statements
Since Rust 1.30, normal macros can be imported via `use`, and with Rust
1.31 and edition 2018 this has become the preferred approach.
`extern crate` is only needed for `alloc` in `no_std` crates, which does
not apply here.
By dropping these (often redundant or odd) `extern crate` lines, we
expose the actual dependencies more clearly and reduce technical debt.
## Auto-generation of the series
Most of this series was produced automatically:
1. Removed all "extern crate" references
2. Run the script [0] to add missing `use` statements
3. Run `cargo +nightly fmt --all`
4. Fix the remaining problems manually
The treewide changes were then split into per-folder commits.
[0]
```python
import os
import re
# Mapping of macro/function usage to imports
MACRO_IMPORTS = {
"info!": "use log::info;\n",
"debug!": "use log::debug;\n",
"error!": "use log::error;\n",
"trace!": "use log::trace;\n",
"warn!": "use log::warn;\n",
"event!": "use event_monitor::event;\n",
"anyhow!(": "use anyhow::anyhow;\n",
"bitflags!(": "use bitflags::bitflags;\n",
"ioctl_ior_nr!": "use vmm_sys_util::{ioctl_ior_nr};\n",
"ioctl_iow_nr!": "use vmm_sys_util::{ioctl_iow_nr};\n",
}
# Regex for finding the first use statement
USE_REGEX = re.compile(r"^\s*(use|pub use) .+?;")
def process_file(path):
with open(path, "r", encoding="utf-8") as f:
lines = f.readlines()
content = "".join(lines)
existing_imports = set(lines)
needed_imports = set()
# Check macros/functions against mapping, only add if not already present
for key, import_stmt in MACRO_IMPORTS.items():
if key in content and import_stmt not in existing_imports:
needed_imports.add(import_stmt)
if not needed_imports:
print(f"Unmodified {path} (no new imports needed)")
return # Nothing to do
# Find first use or pub use statement
for i, line in enumerate(lines):
if USE_REGEX.match(line):
insertion_index = i + 1
break
else:
print(f"Unmodified {path} (no use or pub use statement found)")
return # No use statement found, skip file
# Insert imports
lines[insertion_index:insertion_index] = list(needed_imports)
# Write back file
with open(path, "w", encoding="utf-8") as f:
f.writelines(lines)
print(f"Modified {path}, added imports: {''.join(needed_imports).strip()}")
for root, _, files in os.walk("."):
for file in files:
if file.endswith(".rs"):
process_file(os.path.join(root, file))
```
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Add tests:
- zlib: test_virtio_block_qcow2_zlib()
- zstd: test_virtio_block_qcow2_zstd()
Both these tests use zlib- and zstd-compressed images as OS image.
Modify test_virtio_block_qcow2_backing_file() test: it is practical
to test qcow2 file-backing with compression, so use zlib-compressed
image as a backing file.
Signed-off-by: Eugene Korenevsky <ekorenevsky@aliyun.com>
Add support of reading and writing compressed clusters.
Support zlib and zstd compressions.
L2 cache: store entire L2 entries, not only standard cluster addresses.
Read path. Offsets of compressed clusters cannot be determined,
therefore replace QcowFile.file_offset_read() with QcowFile.file_read().
This method reads the cluster, decompresses it if necessary and returns
the data to the caller.
Write path. QcowFile.file_offset_write(): since writing to compressed
clusters is not generally possible, allocate a new standard
(non-compressed) cluster if compressed L2 entry is encountered; then
decompress compressed cluster into new cluster; then return offset
inside new cluster to the caller. Processing of standard clusters is
not changed.
Signed-off-by: Eugene Korenevsky <ekorenevsky@aliyun.com>
As well as saving the MSRs as it is currently does ensure that the KVM
capability is enabled along with keeping the internal state updated.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Co-authored-by: Chengyu Fu <chengyu.fu@linux.alibaba.com>
The granularity has significant implications in typical cloud
deployments with network storage. The Linux kernel will sync advisory
locks to network file systems, but these backends may have different
policies and handle locks differently. For example, Netapp speaks a NFS
API but will treat advisory OFD locks for the whole file as mandatory
locks, whereas byte-range locks for the whole file will remain
advisory [0].
As it is a valid use case to prevent multiple CHV instances from
accessing the same disk but disk management software (e.g., Cinder in
OpenStack) should be able to snapshot disks while VMs are running, we
need special control over the lock granularity. Therefore, it is a valid
use case to lock the whole byte range of a disk image without
technically locking the whole file - to get the best of both worlds.
This also brings CHVs behavior in line with QEMU [1].
Whole-file locks remain a valid use case and could be supported later.
This patch only provides the necessary groundwork; making it
configurable is out of scope for now.
[0] https://kb.netapp.com/on-prem/ontap/da/NAS/NAS-KBs/How_is_Mandatory_Locking_supported_for_NFSv4_on_ONTAP_9
[1] <qemu>/util/osdep.c::qemu_lock_fcntl()
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This should be guaranteed by GuestMemory and GuestMemoryRegion, but
those traits are currently safe, so add checks to guard against
incorrect implementations of them.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
It takes a pointer to a userspace address that it accesses, so it should
be marked unsafe. This was missed earlier.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Callers of get_host_address_range() rely on it returning a pointer to at
least size bytes of memory. mem.get_host_address() is an overrideable
method of a safe trait, so it is better for safe code to not rely on its
correctness for safety. Instead, use mem.get_slice(), which returns a
VolatileSlice whose invariants guarantee that it points to a sufficient
amount of memory. If mem.check_range() succeeds but mem.get_slice()
returns a slice that is too small, this means that there is either a
logic error or a situation the code cannot support yet, so panic.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
The kernel will validate that the size is page-aligned. The file offset
is always zero, so the kernel will also validate that the offset is
page-aligned.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
This API passes a u64 to a kernel API that treats the u64 as a userspace
address. Therefore, it should be marked unsafe, but it currently is not
[1]. Wrap the call in an unsafe block to document that invariants must
be upheld to avoid undefined behavior. This causes a compiler warning,
so suppress the warning with #[allow(unused_unsafe)].
[1]: https://github.com/rust-vmm/vfio/issues/100
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
I believe that its only caller used it safely, but it is still better to
mark the code as unsafe. Also add additional validity checks.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Also drop support for building the TDX code for 32-bit targets. All
CPUs with TDX support are 64-bit so supporting 32-bit targets is not
needed.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
To ensure that struct sizes are the same on 32-bit and 64-bit, various
kernel APIs use __u64 (Rust u64) to represent userspace pointers.
Userspace is expected to cast pointers to __u64 before passing them to
the kernel, and cast kernel-provided __u64 to a pointer before using
them. However, various safe APIs in Cloud Hypervisor took
caller-provided u64 values and passed them to syscalls that interpret
them as userspace addresses. Therefore, passing bad u64 values would
cause memory disclosure or corruption.
Fix the bug by using usize and pointer types as appropriate. To make
soundness of the code easier to reason about, the PCI code gains a new
MmapRegion abstraction that ensures the validity of pointers. The rest
of the code already has an MmapRegion abstraction it can use. To avoid
having to reason about whether something is keeping the MmapRegion
alive, reference counting is added. MmapRegion cannot hold references
to other objects, so the reference counting cannot introduce cycles.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
To ensure that struct sizes are the same on 32-bit and 64-bit, various
kernel APIs use __u64 (Rust u64) to represent userspace pointers.
Userspace is expected to cast pointers to __u64 before passing them to
the kernel, and cast kernel-provided __u64 to a pointer before using
them. However, various safe APIs in Cloud Hypervisor took
caller-provided u64 values and passed them to syscalls that treat them
as userspace addresses. Therefore, passing bad u64 values would cause
memory disclosure or corruption. The memory region APIs are one example
of this, so mark them as unsafe.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
When booting a Linux guest in SMP configuration, on sapphire rapids
and granite rapids the following kernel warning can be observed:
[Firmware Bug]: CPUID leaf 0x1f subleaf 1 APIC ID mismatch 1 != 0
[Firmware Bug]: CPUID leaf 0x1f subleaf 2 APIC ID mismatch 1 != 0
The reason is that we announce the presence of the extended topology
leaf, but fail to announce the x2apic ID in EDX for each subleaf.
Signed-off-by: Thomas Prescher <thomas.prescher@cyberus-technology.de>
On-behalf-of: SAP thomas.prescher@sap.com
as of rust 1.90, writes to unix sockets use the sendto syscall. This
affects the vcpu threads when vfio_user devices are accessed.
Signed-off-by: Stefan Nürnberger <stefan.nuernberger@cyberus-technology.de>
This is the first commit in a series of commits to improve the Code
Quality in Cloud Hypervisor in a sustainable way. These are the
default rules from `clippy::all` but written here to be more explicit.
`clippy::all` refers to all "default sensible" lints, not all
existing lints.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This better aligns with the rest of the code and makes it clearer
that these tests can run "as is" in a normal hosted environments
without the special test environment.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
TL;DR: Massive quality of life improvement for devs
Cloud Hypervisor uses the Cargo test framework for multiple tests:
- normal unit tests
- unit tests requiring special environment (the Tap device tests)
- integration tests requiring a special environment
This prevented the execution of `cargo test --workspace`, which results
in a very poor developer experience. Although
`./scripts/run_unit_tests.sh` exists, there are valid reasons why devs
cannot or even don't want to use it.
By adding a new `chv_testenv` rustc config, we can conditionally only
activate tests when the `./scripts/` magic runs them. This improves
the general developer experience by a lot.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Commit 5ec47d4883 was intended to patch ebx bits 23-16 in cpuid leaf
0x1, but it was not working as expected, as in rust, operator << has a
stronger precedence than & [1]. Later commit b6667f948e fixed the
operator precedence clippy warning, but did not fix the actual issue. As
a result, the current code is not changing ebx,
```
cpu_ebx |= ((dies_per_package as u32) * (cores_per_die as u32) * (threads_per_core as u32))
& (0xff << 16);
```
Since the total number of logical processors is generally less than
65536, the right hand side of the expression is 0 in most cases.
[1] https://doc.rust-lang.org/reference/expressions.html#expression-precedence
Fixes: 5ec47d4883 ("arch: x86_64: enable HTT flag")
Signed-off-by: Changyuan Lyu <changyuanl@google.com>
A major improvement to the developer experience of clippy in
Cloud Hypervisor.
1. Make `cargo clippy` just work with the same lints we use in CI
2. Simplify adding new lints
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
... by just passing the iterator along. For large VMs this bitmap is
gigantic. A 12TB VM has 384MB of dirty bitmap.
With all these optimizations from the previous commits in place, we
see quite the improvement when it comes to scanning the dirty bitmap.
For a bitmap with 1% bits (randomly) set, dirty_log() takes:
Original code: 2166ms (100.0%)
New code: 382ms ( 17.6%)
on my system. The sparser the dirty bitmap the faster. Scanning an
empty bitmap is 100x faster. For a 5% populated bitmap we are still 3x
faster.
If someone wants to play with this, there is a benchmark harness here:
https://github.com/blitz/chv-bitmap-bench
On-behalf-of: SAP julian.stecklina@sap.com
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
This would be a good opportunity to optimize another pointless vector
away, but I don't have a good way to test this at the moment. But
maybe someone else gives it a shot.
On-behalf-of: SAP julian.stecklina@sap.com
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Adding itertools as dependency improves the iteration code in the
following significantly.
With this change, we don't need a copy of the vector. Just something
that can be coerced into an iterator. We also use the bit position
iterator to make the code somewhat clearer. The new code is much
faster, because it will not iterate over every bit, just each 1 bit in
the input.
The next commit will complete this optimization and have some concrete
numbers.
On-behalf-of: SAP julian.stecklina@sap.com
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Due to the fw_cfg test being disabled on MSHV this results in no tests
being runnable which results in an error in nextest. Reduce that error
to a warning use --no-tests=warn
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
cargo nextest is an improved test runner that allows retries as well a
reporting the times for the test runs.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
When using nextest for running tests each test is run in its own process
so the old solution of using a static variable for the guest ID (used to
determine the network segment) no longer works.
Instead use a text file on the filesystem protected with an exclusive
lock. The test process will read from it and then write back the next ID
that can be used. It wraps around at the limit of u8 and skips ID 0.
This function intentionally panics rather than propagate errors as it
should only be called for testing purposes and there the panic handler
will give a useful backtrace and cleanup.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is required to support exclusive locking on files which is needed
for safe test ID generation when using nextest (since it runs each test
as a separate process.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
as of rust 1.90, writes to unix socket streams use send_with_flags
instead of write, so it uses a sendto syscall instead of write.
Signed-off-by: Matt Moriarity <matt@mattmoriarity.com>
To get that list, I've used
```
git log | grep --fixed-strings -- "-by:" | head -n 100000 | sort | less
```
on the Linux kernel's git repository.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The set of extensions supported by a RISC-V system needs to be exposed
to the guest - currently that is a fixed, minimal set of extensions.
These extensions are not sufficient to boot Ubuntu 25.10 which now has a
mininimum requirement of RVA23S64 (which is a minimum set of extensions
that make sense for server use cases.)
The easiest way to convey the extensions that the guest should use is to
copy those that the host kernel understands (and thus includes in the
/proc/cpuinfo) data.
However since nested virtualisation is not currently possible - exclude
the "H" (Hypervisor) extension from the list of short (single letter)
extensions.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
After updating the Linux kernel to 6.19.6 the second segment (segment=1)
is now under the 2nd IOMMU group (which it a more logical setup) and as
such the added device which is on that segment is in that second IOMMU
group.
The same check is made in test_vdpa_block so also test there.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Resignal every 10ms the thread if it has not acknowledged the signal via
setting the atomic when the vCPU thread was acknowledged. Further, avoid
an infinite loop by generating an error if it takes more than 1000ms to
interrupt the thread.
The retry helps mitigate a race condition where the signal is received
between checking the pause atomic and entering KVM_RUN ioctl when
pausing. Hitting this race condition would leave the
wait_untial_signal_acknowledged() method spinning indefinitely.
The timeout error avoids the VMM process being blocked indefinitely.
See: #7427
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Dump kernel logs after running the tests in the MSHV workflow to help
debug failures.
In addition to getting the kernel logs using `dmesg` also use AzCli to
retrieve the serial console logs. If the VM is hung or panicked, the
workflow would be unable to SSH into it and execute `dmesg`. In this
case the serial console logs would be helpful.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Replace the use of `guestmount and `guestunmount` with standard tools
(losetup, mount) to modify cloud disk image. This eliminates
dependency on guestmount which requires /dev/kvm and is not available
in MSHV root partition.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
Refactor MSHV hypervisor selection logic to enable aarch64 support
instead of exiting. Use `--features mshv` to conditionally build &
test when hypervisor is mshv replacing hard exit for aarch64 on mshv.
Signed-off-by: Aastha Rawat <aastharawat@microsoft.com>
There are several copy-pasted code fragments in impl QcowFile. All of
them add L2 entry to the cache and one of them (in file_offset_write())
does also allocating new L2 entry if necessary.
Fold all these code fragments (except of one in l2_table() which does
error handling in special way) into cache_l2_cluster() method without
changing the logic.
This will make the code more compact and clean.
Signed-off-by: Eugene Korenevsky <ekorenevsky@aliyun.com>
With the current syntax docker gives 'docker: invalid reference format'
error. Also during parsing /xxx:/yyy in process_volumes_args
with \" inside variable i.e arr_vols=("${arg_vols//#/ }")
gives wrong output.
Example:
scripts/dev_cli.sh tests --integration --volumes /mshv:/mshv
Error: The volume /mshv /mshv does not exist.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The API documentation tells users to expect a message when Cloud
Hypervisor is launched, this message was removed in commit 13724db
This change updates the documentation to reflect how the program
actually functions, which is to say no message.
Signed-off-by: Ariel Chenet <achenet@fastmail.com>
Consuming `&Arc<T>` as argument is almost always an antipattern as it
hides whether the callee is going to take over (shared) ownership
(by .clone()) or not. Instead, it is better to consume `&dyn T` or
`Arc<dyn T>` to be more explicit. This commit cleans up the code.
The change is very mechanic and was very easy to implement across the
code base.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Disable test_snapshot_restore_hotplug_virtiomem for mshv. It is failing
frequently in the CI. It needs to be stabilized before enabling again.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The ivshmem tests involving snapshot/restore & live migration are
failing in the MSHV CI with this error:
Could not get vCPU state GetMsrEntries(
Hypercall 80 failed with 0x5 : InvalidParameter ...<snip>)
This needs more investigation. It is worth noting that the general live
migration tests are also not run in the CI for MSHV.
Disable these tests for MSHV for the time being.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The ivshmem tests are all failing in the CI for MSHV because Cloud
Hypervisor is built without the mshv feature.
Error: Cloud Hypervisor exited with the following chain of errors:
0: Failed to open hypervisor interface (is hypervisor interface
available?)
1: Failed to create the hypervisor
2: no supported hypervisor
Modify the build command to include the mshv feature.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
The existing concurrency configuration causes workflow runs for one Pull
Request to cancel in-progress runs for another Pull Request. When the
workflow is triggered by the `pull_request_target event`, `github.ref`
is always set to the base branch, regardless of which branch raised the
PR. This causes every new pull request to share the same concurrency key
and thus cancel the running CI of any other open pull request.
This commit updates the `concurrency.group` to use
`github.event.pull_request.number` for unique isolation when running on
PR, falling back to `github.ref` for non-PR push (e.g., merge group).
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Disable the following common parallel and common sequential tests for
mshv since these are failing consistently in the CI.
Common parallel:
- test_tpm
- test_cpu_topology_421
- test_cpu_topology_142
- test_cpu_topology_262
- test_cpu_hotplug
Common sequential:
- test_snapshot_restore_basic
- test_snapshot_restore_with_fd
- test_snapshot_restore_pvpanic
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
test_fw_cfg is frequently failing in the CI for MSHV. Exclude it for
now. It needs further investigation. See issue #7434 for details.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Move the GICv2M MSI frame SPI base from 32 to 128 to avoid conflicts
with platform device interrupts.
While at it, rename the constants to make it explicit that they are
associated with the GICv2m MSI Frame.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
This was added in 7be69edf51 to deal with
changes to the KVM bindings that made run() and set_immediate_exit()
take &mut self. Instead adopt a Box<> value in Vcpu allowing the removal
of this internal Mutex.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The virtio spec defines the feature bit range and 4 x 32-bit pages.
However no features are currently defined with bits > 63. However Linux
has now started acking features in those higher pages. Since this is
harmless (we drop the acks to those feature pages) and quietly return 0
for available features in those higher pages this warning can be
removed.
Fixes: #7414
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Fix a race condition that happens in resume()-pause() cycles.
It is odd that for pause(), the CpuManager waited via `state.paused`
for the vCPU thread to ACK the state change but not for `resume()`.
In the `resume()` case, oddly CpuManager "owned" the state change
in `state.paused`. This commit changes this so that the vCPU ACKs
its state change itself in `state.paused` when it transitions from
pause->run. Further, `CpuManager::resume()` now gracefully waits for
the vCPU to be resumed.
More technical:
This change ensures proper synchronization and prevents situations
in that park() follows right after unpark(), causing deadlocks and
other weird behavior due to race conditions.
Calling resume() now takes slightly longer, very similar to pause().
This is, however, even for 254 vCPUs in the range of less than 10ms,
and ultimately we now have correct behaviour.
## Reproducer
Since [0] is merged, the underlying problem can be tested without this
commit by modifying the pause() API call to run `CpuManager::pause()`
and `CpuManager::resume()` in a loop a thousand times.
`ch-remote --api-socket ... pause`
```patch
diff --git a/vmm/src/vm.rs b/vmm/src/vm.rs
index d7bba25cc..35557d58f 100644
--- a/vmm/src/vm.rs
+++ b/vmm/src/vm.rs
@@ -2687,6 +2687,10 @@ impl Pausable for Vm {
MigratableError::Pause(anyhow!("Error activating pending virtio devices: {:?}", e))
})?;
+ for _ in 0..1000 {
+ self.cpu_manager.lock().unwrap().pause()?;
+ self.cpu_manager.lock().unwrap().resume()?;
+ }
self.cpu_manager.lock().unwrap().pause()?;
self.device_manager.lock().unwrap().pause()?;
```
## Outlook
Decades of experience in VMM development showed us that using
many AtomicBools is a footgun. They are not synchronized with each
other at all. On the long term, we might want to refactor things to
have a single shared AtomicU64 with different bits having different
meanings.
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7290
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
These bindings actually refer to atomic bool shared across all
vCPUs to instruct single vCPUs with their next action. As there
are already enough Arc<AtomicBool>, this helps while debugging
things to see that different bindings refer to the same atomic
bool.
In other words: This naming really confused us while debugging.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This gives a reliable way of identifying if the VM has booted as well as
a reliable way to validate the addition of the network interface.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Add a second L1 network interface definition to the guest Cloud Init
configuration, including an additional host IP. Do this by splitting the
network range into two /25s. For clarity the network struct members have
also been renamed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Previous RISC-V ACPI related change introduced MSIS which will be
rejected by typo check, let's extend the typos to accept it.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add necessary definitions and RISC-V ACPI tables to enable ACPI feature.
More specifically, this commit add MADT definitions for RISC-V AIA
interrupt chips.
Signed-off-by: Can Zhang <icloud9957@gmail.com>
Rename the MSHV CI workflow to better align with the existing naming
scheme and also make the arch explicit.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Add step-level timeout for running integration tests so that the
workflow doesn't get stuck on any test.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
We are actually using v0.3.0 of vm-fdt since main branch of that crate
already includes everything. Let's pin vm-fdt to v0.3.0 instead.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Pull requests from forked repos cannot access GitHub secrets which
results in failure of MSHV CI. Switching to pull_request_target
resolves this. It allows the workflow to run with access to repo secrets
and ensures that code from the base branch is used instead of forked
code, preventing potential security risks.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Add workflow to run integration tests on mshv. It calls the azure
infra setup workflow and executes integration tests in the
provisioned environment.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
Add workflow to setup Azure infra to validate MSHV.
This is used to provision the environment before executing tests
on it.
Signed-off-by: AASTHA RAWAT <aastharawat@microsoft.com>
This provides clearer life-cycle management of resources around vfio,
and aligns better with the kernel behavior as reported below with vfio
legacy mode (with vfio container/group).
Fixes: #7328
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Yi Wang <foxywang@tencent.com>
We recently moved many of our tests to use focal to jammy as the guest
images, including rate-limiter tests (#7367). We forgot to update the
rate-limiter scripts to reflect such change.
Our CI pipeline failed to report such error because our self-host runner
happened to be not working at the time we land the changes (see #7405).
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Only use 75% of the available threads - this will reduce dislk and
memory pressure. Reducing the chance of flaky tests.
See: #7405
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
While an IGVM is provided validation fails as there is
no kernel or firmware. This patch fixes the sev_snp boot
failure.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
MSHV (Microsoft Hypervisor) doesn't expose an ITS to guests. Instead it
exposes a GICv2m MSI Frame for mapping MSIs to SPIs.
Update the MADT generation code to create the GIC MSI Frame structure
instead of the ITS structure whenever the guest vGIC is using a GICv2m
frame. This allows booting guests via firmware on MSHV.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
In preparation for removing focal support - start by using a more
generic "disk_config" variable name for the DiskConfig struct vs
embedding the image type.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Rust's aliasing rules do not permit mutating a `usize` through a shared
reference. Unsafe blocks and FFI are no exception.
This PR fixes such a violation detected in the AMX support checks by
passing a raw mutable pointer instead of a shared reference in the
syscall.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP <oliver.anderson@sap.com>
For a graceful resource management of externally provided FDs in Cloud
Hypervisor, corresponding FDs need to be closed on a device removal.
This is the case for virtio-net devices using external FDs, for example.
With the fix introduced in this commit, we allow management software to
properly clean up resources, e.g., libvirt can clean up tap devices.
PS: CHV uses "added" and "removed", which has the same meaning as
hot device attach/hotplug and hot device detach/unplug.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Fix clippy warning `uninlined_format_args` reported by rustc rustc
1.89.0 (29483883e 2025-08-04).
```console
warning: variables can be used directly in the `format!` string
--> block/src/lib.rs:649:17
|
649 | info!("{} failed to create io_uring instance: {}", error_msg, e);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
= note: `#[warn(clippy::uninlined_format_args)]` on by default
help: change this to
|
649 - info!("{} failed to create io_uring instance: {}", error_msg, e);
649 + info!("{error_msg} failed to create io_uring instance: {e}");
|
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
"Update configs must have a unique combination of 'package-ecosystem',
'directory', and 'target-branch'. Ecosystem 'cargo' has overlapping
directories for target branch 'main'."
Drop `target-branch` in rust-vmm group to make those two unique.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
## TL;DR
This unifies error handling, implementation, and logging of config
objects that are populated with additional FDs received as part of
the HTTP API request from the management software. This prevents current
and future code in repetition.
In the future, if we want to support let's say virtio-blk with external
FDs, the AddBlk API handler simply needs a call to `attach_fds_to_cfg()`
and gets all the magic for free.
## Motivation
The motivation of the new abstraction is to provide a verbose, solid,
and bulletproof solution for a complex domain. The interaction between
- the management layer,
- the passing of file descriptors over UNIX domain sockets via
SCM_RIGHTS,
- the attachment of configurations to those FDs,
- and the ability to give new developers clear insights into what
happens under the hood
is not trivial. These factors justify encapsulating the complexity
behind a convenient and well-documented abstraction, making the system
both robust in production and approachable for new developers.
In addition, it allows us to perform unit testing. Further, We get rid
of existing partial code duplication and inconsistencies.
Finally, while this approach may initially result in more code, every
new handler that accepts FDs benefits from reduced duplication and a
correct implementation by relying on the shared abstraction. This will
also enable future functionality, such as virtio-blk devices backed by
FDs which can then be integrated with ease.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Deserializing values as `-1` makes sense to prevent errors, so let's
keep it. However, serializing them differently adds confusion. For
example, a `ch-remote info` call should not report `-1` but the actual
FDs.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
(1) The old messages are missing the "why" part. With this change, users
of Cloud Hypervisor have somehow more context and people looking at
the code perfectly know what's going on.
(2) Using warn! implies that the user should take action, but in this
case, there’s nothing the user can do. If the API is used correctly
and file descriptors are passed via an SCM_RIGHTS message over a
UNIX domain socket, everything works as intended. In that case,
there's no need to issue a warning — debug! is sufficient.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Live migration, state save/resume, and hotplug are not trivial when it
comes to virtio-net devices backed by externally provided FDs. As the
mechanism behind it can be considered as quite "multi-step magic" even
for experienced programmers, it makes sense to thoroughly document this
to ease debugging and to improve the mental model of developers working
on this in the future.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This change enables easier integration with third-party
tools by removing the requirement for a dummy IP address
when configuring tap devices. The modification applies to
both CLI and API interactions.
Previously, cloud-hypervisor would automatically set a
default static IP address (192.168.249.1) if none was provided.
This could lead to:
* multiple devices without explicit IP configurations
would end up with the same default IP
* unnecessary inclusion of this IP in firewall rules
* the IP address could clash with host networking and
routing
This introduces a new constraint:
When providing an IP, the mask must also be provided.
Removes warnings introduced in #7179.
Closes issue #7083.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
The function `read_local_stream_port` had no proper handling for
unexpected or incomplete input.
When the control socket of the VSOCK device was closed without sending
the expected `CONNECT <PORT>\n` statement completely, the thread
got stuck in an infinite loop as it attempted to read from a closed
socket over and over again which never returned any data.
This resulted in the thread responsible for `epoll` being completely
blocked. New VSOCK connections could not be established and existing
ones became defunct, effectively leading to a Denial of Service of
the entire VSOCK device.
The issue can be reproduced by opening a socket and immediately
closing it.
```
socat - UNIX-CONNECT:/socket.vsock
<Ctrl-C>
```
Instead of applying a quick fix by handling the `EPOLLHUP` event before
reading, the function is refactored to remove the error-prone `while`
loop and multiple `read`s.
Notably, we now check if the number of bytes read is zero, which occurs
when `event_set == EPOLLHUP | EPOLLIN`, indicating that the socket has
been closed by the client.
Additionally, the actual parsing code is now extracted into a dedicated
function that is tested.
Fixes: #6798
Signed-off-by: Maximilian Güntner <code@mguentner.de>
For MSHV customers don't want to make everything
default during partition creation. For example
nested support, some synthetic features could be
controlled from CLI through platform argument.
Create_vm API getting messy after adding more flags.
This patch introduces common data struct to be passed
from vmm crate to hypervisor crate during partition creation.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Handle MSHV reset intercepts on arm64 such that guests can gracefully
shutdown/reboot instead of panicking due to unhandled intercept.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Reference link to set_device_attribut for aarch64
is invalid. Looks like the code browsing does not
have the reference anymore.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This message makes it easier to confirm a successful shutdown when
scanning logs during development and in production. If the message is
missing and the last log entry is not an error, the most likely cause
is a livelock (e.g. contention on a lock) introduced while developing
new Cloud Hypervisor features.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This commit includes all simple clippy fixes excluding the
collapsing of nested ifs using the let-chains feature. This
follows in the next commit.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Bump mshv-ioctls and mshv-bindings to 0.6.0. Most notably, this version
contains fixes and new bindings for arm64 guests.
Bump the vfio crates too so that they point to the latest mshv crates.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
Virtio console is activated much later in boot. The output it spits out
lacks the initial CPU configuration of the guest.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
These can differ between platforms, so it's better to use centralized
definitions of them. We can't currently do this for the KVM and VFIO
ioctls, because the corresponding crates don't publicly expose the
ioctl numbers.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The previously linked file was converted to ReStructuredText, then
later removed from the kernel, with part of the justification being
"Most of what booting-without-of.rst contains is now in the DT
specification", so point to that instead.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Since the user has to explicitly ask for Landlock to be enabled in
Cloud Hypervisor, it's surprising that, even when they've done that,
Landlock will silently not be enabled if the kernel doesn't support
it.
To prevent accidental absence of a desired security protection, exit
with an error if Landlock, or the one feature of it we use in Cloud
Hypervisor (file access) is not supported.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
arch::aarch64::fdt::create_cpu_nodes will always look at this if it
exists. (If it doesn't exist, this is a no-op —
add_rule_with_access() won't add rules for paths that don't exist.)
Fixes: b3e5738b4 ("vmm: Introduce ApplyLandlock trait")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
MSHV now supports movable pages i.e VA backed guest.
Also with more features and stability in the MSHV
Kernel drives MSHV now supports more test scenario.
This patch enables more integration test cases.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Without write access to the directory the socket will go in, it's not
possible to create the socket.
I've tested outgoing connections from the VM, and they don't seem to
need read permissions on that directory to connect to a socket on the
host.
Fixes: b3e5738b4 ("vmm: Introduce ApplyLandlock trait")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
path_beneath_rules() just needs the paths given to it to be
AsRef<Path>, so there's no need to create new PathBufs for it.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Raise the max number of supported (v)CPUs on kvm x86_64 hosts
to 8192 (the max allowed value of CONFIG_NR_CPUS in the Linux kernel).
Other platfroms keep their existing CPU limits pending further
development and testing.
The change has been tested on Intel and AMD hosts.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
These are now used by Cloud Hypervisor when linked with Glibc 2.42.
These values should be correct for all currently supported Cloud
Hypervisor platforms, although they are not for all Linux platforms.
Closes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7276
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This commit removes the SGX support from cloud hypervisor. SGX support
was deprecated in May as part of #7090.
Signed-off-by: Shubham Chakrawar <schakrawar@crusoe.ai>
This patch gives user an option to override the
default migratable version to any later release.
This option makes MSHV specific tests suitable for
tests since MSHV is stable after some breaking changes.
This patch is also necessary for MSHV CI.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This step was done manually by searching for "unsafe fn" in
the code base and adding corresponding Safety sections.
`clippy::missing_safety_doc` only works for public functions
but none of the corresponding functions is public.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Implement the batch submission function for raw disk, default it is
enabled. After parsing the requests this method is
called for better IO latency and bandwidth.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Cache and batch IO requests after parsing all
items in the queue, improving performance—especially
for small block sizes—by reducing per-request overhead.
Introduced two methods in the AsyncIo trait for batch
submission, with implementation in the raw disk backend.
This method should be called during/after parsing all block IO requests
in the available queue. If the batch submission is not enabled, by
default it does the old way of submitting requests.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Instead of returning boolean return an struct of completion status
so that it can be cached for batch submission.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch changes the read-only check using acked features bit, which
will help to check more features.
Signed-off-by: Songqian Li <sionli@tencent.com>
Also tweak extended interrupt handlings, as needed.
Most credit should go to Neel Natu, who figured out the magic
bits needed to make things work and provided detailed comments.
This patch is still NOOP, as VM config allows only up to
254 vCPUs on x86_64.
Note: changes in this and related previous patches/PRs have
only been tested on Linux hosts running on Intel x86_64 hardware.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
test_direct_kernel_boot_bzimage runs only on x86, so the cfg!() branch
for selecting grep_cmd is unnecessary. Remove it for clarity.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
One can call `to_vec()` anyway if one needs an owned copy. This change
further helps to prevent needless copies in upcoming changes.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
On aarch64 and RISC-V, calling load_firmware() through load_kernel()
provides no benefit and only duplicates checks already performed in
load_payload(). load_payload() now directly invokes load_firmware() or
load_kernel(), removing unnecessary indirection and redundancy.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Both functions are defined separately for the two architecture with
minor differences.
* `load_firmware()`: call `arch::uefi::load_uefi` which are available on
both architecture;
* `load_kernel()`: manually align to `arch::layout::KERNEL_START` 2MB
for both architecture (e.g. no-op for `aarch64`);
Signed-off-by: Bo Chen <bchen@crusoe.ai>
With 254 vCPUs, pausing now takes ~4ms instead of >254ms. This
improvement is visible when running `ch-remote pause` and is
particularly important for live migration, where every millisecond
of downtime matters.
For the wait logic, it is fine to stick to the approach of
sleeping 1ms on the first missed ACK as:
1) we have to wait anyway
2) we give time to the OS, enabling it to schedule a vCPU thread next
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Enable kvm build test and clippy test on RISC-V 64-bit platform to
ensure whole projects builds properly.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
MP table is a legacy device that is incompatible
with x2apic CPU IDs exceeding 254. The Linux kernel
is perfectly happy without MP table in these cases.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
Currently, the following scenarios are supported by Cloud Hypervisor to
bootstrap a VM:
1. provide firmware
2. provide kernel
3. provide kernel + cmdline
4. provide kernel + initrd
5. provide kernel + cmdline + initrd
As the difference between `--firmware` and `--kernel` is not very clear
currently, especially as both use/support a Xen PVH entry, adding this
helps to identify the cause of misconfiguration.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This is necessary to use the let-chains feature in a
follow-up. After upgrading to Rust edition 2024, clippy
wants to collapse various if's with let-chains.
Update image to 20250815-0 since MSRV in Dockerfile is updated.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This patch introduces the inter-vm shared memory(ivshmem) device
to share a memory region between multiple processes running
different guests and the host.
This patch supports the basic ivshmem functions like ivshmem-plain
in QEMU[1].
[1] https://www.qemu.org/docs/master/specs/ivshmem-spec.html
Signed-off-by: Yi Wang <foxywang@tencent.com>
Signed-off-by: Songqian Li <sionli@tencent.com>
Move UserspaceMapping to vm-device to avoid redefinition since
UserspaceMapping is used by both `virtio-devices` and `device`
crate.
Signed-off-by: Songqian Li <sionli@tencent.com>
It is always called with topology provided, so there is no
need to pass topology as an Option. Simplifying the signature
makes further topology-related changes to arc/src/x86_64 module
simpler.
Signed-off-by: Peter Oskolkov <posk@google.com>
This is the second patch in a series intended to let Cloud Hypervisor
support more than 255 vCPUs in guest VMs; the first patch/commit is
https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7231
At the moment, CPU topology in Cloud Hypervisor is using
u8 for components, and somewhat inconsistently:
- struct CpuTopology in vmm/src/vm_config.rs uses four components
(threads_per_core, cores_per_die, dies_per_package, packages);
- when passed around as a tuple, it is a 3-tuple of u8, with
some inconsistency:
- in get_x2apic_id in arch/src/x86_64/mod.rs the three u8
are assumed to be (correctly)
threads_per_core, cores_per_die, and dies_per_package, but
- in get_vcpu_topology() in vmm/src/cpu.rs the three-tuple is
threads_per_core, cores_per_die, and packages (dies_per_package
is assumed to always be one? not clear).
So for consistency, a 4-tuple is always passed around.
In addition, the types of the tuple components is changed from u8 to
u16, as on x86_64 subcomponents can consume up to 16 bits.
Again, config constraints have not been changed, so this patch
is mostly NOOP.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
This is the first change to Cloud Hypervisor in a series of changes
intended to increase the max number of supported vCPUs in guest VMs,
which is currently limited to 255 (254 on x86_64).
No user-visible/behavior changes are expected as a result of
applying this patch, as the type of boot_cpus and related
fields in config structs remains u8 for now, and all configuration
validations remain the same.
Signed-off-by: Barret Rhoden <brho@google.com>
Signed-off-by: Neel Natu <neelnatu@google.com>
Signed-off-by: Ofir Weisse <oweisse@google.com>
Signed-off-by: Peter Oskolkov <posk@google.com>
This allows us to enable/disable the fw_cfg device via the cli
We can also now upload files into the guest vm using fw_cfg_items
via the cli
Signed-off-by: Alex Orozco <alexorozco@google.com>
This allows the fw_cfg device to be recognized by the guest linux
kernel. This becomes more relavnt in the following cl where I add
the option to load files into the guest via fw_cfg. The Linux kernel
already has a fw_cfg driver that will automatically load these files
under /sys when CONFIG_FW_CFG_SYSFS is enabled in the kernel config
For arm we must add fw_cfg to the devices tree
Signed-off-by: Alex Orozco <alexorozco@google.com>
We pass a reference to the guest memory when we create the device
in DeviceManager. This allows us to access the guest memory for DMA.
Signed-off-by: Alex Orozco <alexorozco@google.com>
The acpi tables are created in the same place the acpi tables would be
created for the regular bootflow, except here we add them to the
fw_cfg device to be measured by the fw and then the fw will put the
acpi tables into memory.
Signed-off-by: Alex Orozco <alexorozco@google.com>
The kernel and initramfs are passed to the fw_cfg device as
file references. The cmdline is passed directly.
Signed-off-by: Alex Orozco <alexorozco@google.com>
Here we add the fw_cfg device as a legacy device to the device manager.
It is guarded behind a fw_cfg flag in vmm at creation of the
DeviceManager. In this cl we implement the fw_cfg device with one
function (signature).
Signed-off-by: Alex Orozco <alexorozco@google.com>
This partially reverts
ed8f347fe6 from #7183 and
6277d7d5f2 from #7201.
# Output how it was merged for v47 (#7066)
```
Error: Cloud Hypervisor exited with the following chain of errors:
0: Error booting VM
1: The VM could not boot
2: Error manipulating firmware file
3: No such file or directory (os error 2)
Debug Info: VmBoot(VmBoot(FirmwareFile(Os { code: 2, kind: NotFound, message: "No such file or directory" })))
```
# Output after #7183 and #7201
```
cloud-hypervisor: 31.385730ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:27 -- Error: Cloud Hypervisor exited with the following chain of errors:
cloud-hypervisor: 31.417961ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 0: Error booting VM
cloud-hypervisor: 31.448078ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 1: The VM could not boot
cloud-hypervisor: 31.486711ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 2: Error manipulating firmware file
cloud-hypervisor: 31.513331ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:39 -- 3: No such file or directory (os error 2)
cloud-hypervisor: 31.548037ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:44 --
cloud-hypervisor: 31.568045ms: <main> ERROR:/home/pschuster/dev/cloud-hypervisor/src/lib.rs:45 -- Debug Info: VmBoot(VmBoot(FirmwareFile(Os { code: 2, kind: NotFound, message: "No such file or directory" })))
```
The "proper logger" has indeed the advantage that messages can
be gracefully redirected to log files etc. However, this makes the
error message hardly readable.
Therefore, I propose to use error!() only for runtime errors messages
but not a pretty-printed version of those.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Error::UefiLoad is required for load_firmware to propagate errors
encountered, define it for riscv64.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
uefi_flash field in memory_manager is required for uefi loading and
booting, enable it for riscv64 architecture.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The option parser did not allow quoting any special characters, which
means that some strings could not be passed via command-line options.
Fix this by allowing double quotes to quote all the special characters,
such as '@', ',', '[', and ']'. '"' can be escaped by doubling it. The
implementation uses some tricks to stay modular.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
rustc 1.90.0-beta.1 (788da80fc 2025-08-04) suggests using library
feature `unsigned_is_multiple_of`. It is stabled in Rust 1.87.0.
Update image to 20250807-0 since MSRV in Dockerfile is updated.
Signed-off-by: Songqian Li <sionli@tencent.com>
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Now 'create_acpi_tables_internal()' can generate ACPI tables with
different dsdt offset, so that it can be reused for generating ACPI
tables for `fw_cfg`.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Alex Orozco <alexorozco@google.com>
Optimize lychee workflow to check only changed
files in pull requests, avoiding excessive API
calls to prevent rate limits.
Fixes#7056
Signed-off-by: Shubham Chakrawar <schakrawar@crusoe.ai>
In case of CVM guest rsdp is set to none. Unwrapping it
make the vmm crashed. Don't call configure system if the
rsdb address is none.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Many of the workspace members in the Cloud-hypervisor workspace share
common dependencies. Making these workspace dependencies reduces
duplication and improves maintainability.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The current documentation only mentions Windows
server support. Since only minimal adjustments
are needed for Windows 11 support, these were
added where applicable.
Also contains whitespace fixes, adds missing `\`
in example code and adds more cross references.
Signed-off-by: Lucas Grosche <lucas.grosche@cyberus-technology.de>
The other workspace members in the Cloud-hypervisor workspace
currently declare log version 0.4.22, but the tpm crate
has an older version. This inconsistency is addressed by this PR
which opens the door for declaring log as a workspace dependency.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
Running one or two tests in a tight loop can cause the download
functions to quickly hit GitHub's API rate limit. That causes the test
script to fail for no apparent reason.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Generate the data disk under a temporary directory so that multiple
instances of the test suites can run at the same time.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The other workspace members in the Cloud-hypervisor workspace
currently declare libc crate version 0.2.167, but the tpm crate
has an older version. This inconsistency is addressed by this PR
which opens the door for declaring libc as a workspace dependency.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
The other workspace members in the Cloud-hypervisor workspace
currently declare anyhow version 1.0.94, but the tpm crate
has an older version. This inconsistency is addressed by this PR
which opens the door for declaring anyhow as a workspace dependency.
Signed-off-by: Oliver Anderson <oliver.anderson@cyberus-technology.de>
On-behalf-of: SAP oliver.anderson@sap.com
It shouldn't be necessary to lock the file for the adaptor. This removes
two layers of indirection for QcowDiskSync and VhdxDiskSync.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The PL031 RTC provides two features: a real-time counter and an alarm
interrupt. To use the alarm, the driver normally writes a time value
into the match register RTCMR, and when the counter reaches that value
the device triggers the interrupt.
At the moment the implementation ignores programming of the alarm, as
the feature seems rarely used in VMs. However the interrupt is still
triggered arbitrarily when the guest writes to registers, and the line
is never cleared. This really confuses the Linux driver, which loops in
the interrupt handler until Linux realizes that no one is dealing with
the interrupt (200000 unanswered calls) and disables the handler.
One way to fix this would be implementing the alarm function properly,
which isn't too difficult but requires adding some async timer logic
which probably won't ever get used. In addition the device's interrupt
is level-triggered and we don't support level interrupts at the moment,
though we could probably get away with changing this interrupt to edge.
The simplest fix, though, is to just disable the interrupt logic
entirely, so that the alarm function still doesn't work but the guest
doesn't see spurious interrupts.
Add a default() implementation to satisfy clippy's new_without_default
check, since Rtc::new() doesn't take a parameter after this change.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
The read_exact() call was introduced in 82ac114b8 ("virtio-devices:
vsock: handle short read in muxer") to solve a crash when a connection
disconnected without sending any data, but it introduced a problem of
its own: because the socket is non-blocking, read_exact() may read
some data, then return ErrorKind::WouldBlock. In that case, the data
it read will be discarded. So for example if it read "CONNECT ",
and then nothing else was available to read yet, "CONNECT " would be
discarded, and so the next time this function was called, when epoll
triggered again for the socket, only the following data would end up
in command.buf, causing an error due to just a port number being an
invalid command.
Contrary to that commit message, this code was actually designed to
handle short reads just fine — in the case of a short read, it stores
the data it has read in command, and returns
Error::UnixRead(ErrorKind::WouldBlock), which is ignored by the
caller, and the function gets called again when there is more data to
read, building up command potentially over the course of several
reads. The only thing it didn't handle correctly, as far as I can
tell, was a 0-byte read, which happens when a client disconnects from
the socket without writing anything. All that's needed to fix this is
to avoid an invalid subtraction in that case, so this change reverts
82ac114b8, fixing the issue with partial commands being discarded, and
instead handles the 0-byte read by using slice::get, and treating an
empty command as an incomplete command, which of course it is.
Fixes: 82ac114b8 ("virtio-devices: vsock: handle short read in muxer")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This makes it possible to run cargo test just for the virtio-devices
crate (as long as either KVM or MSHV is specified).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
`std::mem::offset_of` could be used for calculating nested fields, use
this feature to shorten aarch64 reg offset calculation.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`std::mem::offset_of` is stabilized since Rust 1.77, let's use
implementation provided by std instead of manual implementation.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually implemented `_offset_of` and `offset_of` in
`arch/riscv64/mod.rs` are not used now, remove them.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`std::mem::offset_of` supports calculating offset of nested structures,
let's use implementation provided by std instead of manual
implementation.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Other lines are already logged using `log::error!` and
`env_logger` is initialized before calling
`start_net_backend` in `main()`.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
Unify log formatting and printing as `eprintln!` and `log::error!`
would be used alongside each other.
When using e.g. `env_logger` lines printed with `eprintln!` would
lack formatting / colors.
Currently only relevant in `ch-remote` + `cli_print_error_chain`.
Note that the replaced messages now also end up in the logfile of
`cloud-hypervisor` when configured and not any longer in stderr.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
Until now all messages generated using `log::level!`
(e.g., `warn!`) have not been printed as `ch-remote` did not
register a logger.
Furthermore, replace all `eprintln!` with `error!`
to align formatting for consistency.
Signed-off-by: Maximilian Güntner <code@mguentner.de>
Recently vfio crates have moved to crates.io, thus we should start
consuming the crate from crates.io instead git url.
This results in better versioning instead of tracking some git commit
sha.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since action-rs/cross is deprecrated, thus move to
houseabsolute/actions-rust-cross.
We should pin the cross-version to the latest version to fix the build
issues with virtio-bindings crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
When invoking the script chown shows a warning.
chown: warning: '.' should be ':': ‘1000.1000’
From `info coreutils 'chown invocation'`.
Some older scripts may still use ‘.’ in place of the ‘:’ separator.
POSIX 1003.1-2001 (*note Standards conformance::) does not require
support for that, but for backward compatibility GNU ‘chown’ supports
‘.’ so long as no ambiguity results, although it issues a warning and
support may be removed in future versions. New scripts should avoid the
use of ‘.’ because it is not portable, and because it has undesirable
results if the entire OWNER‘.’GROUP happens to identify a user whose
name contains ‘.’.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
That feature was dropped when consolidating the UUID dependency because
somehow building the whole project worked. The CI system was happy.
However, building the block crate alone is broken. The vhdx code uses
Uuid::new_v4, which requires `v4` to be enabled.
Add the feature back.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When a proper Rust backtrace is printed, the Rust std wants to use the
SYS_getcwd(79) system call to prettify some paths while printing. In
Cloud Hypervisor, this is at least relevant for printing panics or if
a `anyhow::Error` value is printed using `{e:?}` (but not `{e:#?}`).
The syscall cause can be found in `impl fmt::Display for Backtrace {}`
in `library/std/src/backtrace.rs`.
Without this addition, the seccomp violation of the SYS_getcwd (79)
hinders the proper error message including a full backtrace from showing
up. This annoying behaviour already delayed many debugging efforts. With
this fix, things just work. The new syscall itself should be pretty
harmless for normal operation.
```
thread 'vmm' panicked at virtio-devices/src/rng.rs:224:9:
Yikes, things went horribly wrong!
==== Possible seccomp violation ====
Try running with `strace -ff` to identify the cause and open an issue: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/new
[1] 287683 invalid system call (core dumped) RUST_BACKTRACE=full cargo run --bin cloud-hypervisor -- --api-socket --kerne
```
```
thread 'vmm' panicked at virtio-devices/src/rng.rs:224:9:
Yikes, things went horribly wrong!
stack backtrace:
0: 0x557d91286b62 - std::backtrace_rs::backtrace::libunwind::trace::hc20b48b31ee52608
at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
1: 0x557d91286b62 - std::backtrace_rs::backtrace::trace_unsynchronized::h5d207cd20f193d88
at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
...
67: 0x0 - <unknown>
Error: Cloud Hypervisor exited with the following error:
Failed to join on VMM thread: Any { .. }
Debug Info: ThreadJoin(Any { .. })
```
- add any panic, for example into the create or drop function of a
device
- add --seccomp=true|log to analyze the situation
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The virtio_features_to_tap_offload() defined in ctrl_queue.rs
is duplicated. Remove it and use the one defined in lib.rs
instead.
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Checksumming a type that has padding would use the padding in the
checksum, which is definitely wrong and is often unsound.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Cargo fuzz build report an error:
error: lifetime flowing from input to output with different syntax can be confusing
--> /home/runner/work/cloud-hypervisor/cloud-hypervisor/block/src/lib.rs:747:13
|
747 | fn file(&mut self) -> MutexGuard<F>;
| ^^^^^^^^^ ------------- the lifetime gets resolved as `'_`
| |
| this lifetime flows to the output
error: lifetime flowing from input to output with different syntax can be confusing
--> /home/runner/work/cloud-hypervisor/cloud-hypervisor/block/src/async_io.rs:68:11
|
68 | fn fd(&mut self) -> BorrowedDiskFd;
| ^^^^^^^^^ -------------- the lifetime gets resolved as `'_`
| |
| this lifetime flows to the output
Signed-off-by: Yi Wang <foxywang@tencent.com>
Remote server errors are transferred as raw HTTP body. This way,
we lose the nested structured error information.
This is an attempt to retrieve the errors from the HTTP response
and to align the output with the normal error output.
For example, this produces the following chain of errors. Note
that everything after level 0 was retrieved from the HTTP server
response:
```
Error: ch-remote exited with the following chain of errors:
0: http client error
1: Server responded with InternalServerError
2: Error from API
3: The disk could not be added to the VM
4: Failed to validate config
5: Identifier disk1 is not unique
Debug Info: HttpApiClient(ServerResponse(InternalServerError, Some("Error from API<br>The disk could not be added to the VM<br>Failed to validate config<br>Identifier disk1 is not unique")))
```
In case the JSON can't be parsed properly, ch-remote will print:
```
Error: ch-remote exited with the following chain of errors:
0: http client error
X: Can't get remote's error messages from JSON response: EOF while parsing a value at line 1 column 0: body=''
Debug Info: HttpApiClient(ServerResponse(InternalServerError, Some("")))
```
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
With the foundations of each error type implementing std::error::Error,
we can now nicely walk the `.source()` chain and print an error trace.
This commit introduces improved user-facing error printing when:
- Cloud Hypervisor fails with an error
- ch-remote fails (client error)
- ch-remote fails (remote error)
The additional context is a clear improvement in UX for both users and
developers. In the following example, the new behaviour is shown for
a direct invocation of Cloud Hypervisor leading to a failure. This
looks similar for ch-remote.
```
Old Style
`target/release/cloud-hypervisor --api-socket /tmp/chv2.sock --kernel /etc/bootitems/linux/kernel_minimal/stable.bzImage --cmdline console=ttyS0 --serial tty --console off --disk path=img.raw --initramfs /etc/bootitems/linux/initrd_minimal/default`
Error booting VM: VmBoot(LockingError(BlockError(LockDiskImage(AlreadyLocked)))
```
```
`target/release/cloud-hypervisor --api-socket /tmp/chv2.sock --kernel /etc/bootitems/linux/kernel_minimal/stable.bzImage --cmdline console=ttyS0 --serial tty --console off --disk path=img.raw --initramfs
/etc/bootitems/linux/initrd_minimal/default`
Error: Cloud Hypervisor exited with the following chain of errors:
0: Error booting VM
1: The VM could not boot
2: Error locking disk images: Another instance likely holds a lock
3: Cannot lock images of all block devices
4: Failed to get Write lock for disk image: ./img.raw
5: The file is already locked
Debug Info: VmBoot(VmBoot(LockingError(DiskLockError(LockDiskImage { error: AlreadyLocked, lock_type: Write, path: "./raw_disk.bin" })))
```
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The changes were mostly automatically applied using the following
Python script:
```python
import os, re
for root, _, files in os.walk("."):
for f in files:
if not f.endswith(".rs"):
continue
p = os.path.join(root, f)
with open(p, "r", encoding="utf-8") as file:
lines = file.readlines()
changed = False
for i in range(len(lines) - 1):
if re.search(r'#\[error\(".*: \{0[^}]*\}"\)\]', lines[i]) and "#[source]" in lines[i + 1].strip():
lines[i] = re.sub(r': \{0[^}]*\}"\)\]', '")]', lines[i])
changed = True
if changed:
with open(p, "w", encoding="utf-8") as file:
file.writelines(lines)
print("Fixed:", p)
```
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
# Conflicts:
# vmm/src/api/http/mod.rs
Having them sorted alphabetically makes more sense since there are
already many, and the list is growing. This improves the UX for users
and developers.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Instead of exiting on IO errors, report the errors to the guest with
VIRTIO_BLK_S_IOERR. For example, the guest kernel will log something
similar to this if the nbd behind /dev/vdc is unexpectedly disconnected:
[ 166.033957] I/O error, dev vdc, sector 264 op 0x1:(WRITE) flags 0x9800 phys_seg 1 prio class 2
[ 166.035083] Aborting journal on device vdc-8.
[ 166.037307] Buffer I/O error on dev vdc, logical block 9, lost sync page write
[ 166.038471] JBD2: I/O error when updating journal superblock for vdc-8.
[...]
[ 174.234470] EXT4-fs (vdc): I/O error while writing superblock
In case the rootfs is not located on the affected block device, this
will not crash the guest.
Fixes: #6995
Signed-off-by: Gauthier Jolly <contact@gjolly.fr>
This was caught by the nightly compiler during cargo fuzz build.
error: lifetime flowing from input to output with different syntax can be confusing
--> /home/runner/work/cloud-hypervisor/cloud-hypervisor/hypervisor/src/arch/x86/emulator/mod.rs:493:26
|
493 | pub fn new(platform: &mut dyn PlatformEmulator<CpuState = T>) -> Emulator<T> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ----------- the lifetime gets resolved as `'_`
| |
| this lifetime flows to the output
|
= note: `-D mismatched-lifetime-syntaxes` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(mismatched_lifetime_syntaxes)]`
help: one option is to remove the lifetime for references and use the anonymous lifetime for paths
|
493 | pub fn new(platform: &mut dyn PlatformEmulator<CpuState = T>) -> Emulator<'_, T> {
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
cargo fuzz build complaints about some un-used function in the
instruction emultator. Silence the warning by allowing dead code
generation.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since it is used by multiple components at this point, it is better to
move it to workspace level dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are a little many cpuid logs now. When starting a vm with
64 vcpu, we can get more than four thousand INFO messages:
cat vm1.log |grep 'arch/src/x86_64/mod.rs:891' |wc -l
4352
Signed-off-by: Yi Wang <foxywang@tencent.com>
This commit adds the warning to deprecate the SGX support with the
intention to remove the support from code base in two release cycles.
See: #6960
Signed-off-by: Bo Chen <bchen@crusoe.ai>
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the Error implementation in the Cloud Hypervisor code
base to match the remaining parts so that everything follows the agreed
conventions. These are leftovers missed in the previous commits.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
When booting a Linux guest in SMP configuration,
the following kernel warning can be observed:
[Firmware Bug]: CPUID leaf 0xb subleaf 1 APIC ID mismatch 1 != 0
The reason is that we announce the presence of the extended topology
leaf, but fail to announce the x2apic ID in EDX.
Signed-off-by: Thomas Prescher <thomas.prescher@cyberus-technology.de>
On-behalf-of: SAP thomas.prescher@sap.com
No sane guest OS will allow hotplugging all cpus.
However, the REST API currently allows specifying
`ch-remote resize --cpus 0`.
On Linux, we can then observe the following error in the kernel log:
processor cpu0: Offline failed.
Subsequent resize commands via ch-remote will then fail with
VcpuPendingRemovedVcpu because the removal of cpu0 was never
successful.
Fix this by disallowing resizing to zero vCPUs.
Signed-off-by: Thomas Prescher <thomas.prescher@cyberus-technology.de>
On-behalf-of: SAP thomas.prescher@sap.com
For use in QEMU, I would like GuestMemoryRegion to return a BitmapSlice
instead of a &Bitmap. This adds some flexibility that QEMU needs in
order to support a single global dirty bitmap that is sliced by the
various GuestMemoryRegions.
However, this removes access to the methods of AtomicBitmap, and in
particular reset() and get_and_reset(). Fortunately, cloud-hypervisor
always uses GuestMemoryMmap, and therefore `region` is known to be a
&GuestRegionMmap. Dereferencing it returns the MmapRegion to which the
bitmap is attached, thus calling MmapRegion::bitmap(); this has the
same effect as `<GuestRegionMmap as GuestRegion>::bitmap()`, and works
both with or without https://github.com/rust-vmm/vm-memory/pull/324.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This streamlines the code base to follow best practices for
error handling in Rust: Each error struct implements
std::error::Error (most due via thiserror::Error derive macro)
and sets its source accordingly.
This allows future work that nicely prints the error chains,
for example.
So far, the convention is that each error prints its
sub error as part of its Display::fmt() impl.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Allow tap interfaces to be configured with an IPv6 address. The change
is fairly straightforward: we need to update the API types and CLI
parsing to accept either an IPv6 or IPv4 and then match on the IP
address type when the tap device is configured.
For IPv6 addresses, the netmask (prefix) must be provided at the same
time as the address itself (in the SIOCSIFADDR ioctl). They cannot be
configured separately. So we remove the separate "set_netmask" function
and convert "set_ip_addr" to also accept a netmask. For IPv4 addresses,
the IP address and netmask were already always set together, so this
should have no functional impact for users of IPv4 addresses.
Signed-off-by: Gregory Anders <ganders@cloudflare.com>
The DeviceManagerError type is among the types missing the Error trait
so far. To streamline the code and to simplify usage on higher levels
of this error type, this type now implements Display and Error.
As not all variant values are Error yet, `#[source]` is not everywhere
where it could be. This is done in the next commits.
The high level goal is: Enable future work to improve the error output
of cloud hypervisor.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The new locking behavior uncovered that unfortunate test situation:
Many tests running in parallel access the same disk image with
rw permissions. Luckily, none of the tests actually writes to
the disk. Therefore, we can set it to readonly=true. In case this
changes, the test needs to be moved to the sequential test module.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
# What
This commit introduces file-based advisory locking for the files backing
up the block devices by using the fcntl() syscall with OFD locks. The
per-open-file-descriptor (OFD) locks are more robust than traditional
POSIX locks (F_SETLK) as they are not tied to process IDs and avoid
common issues in multithreaded or multi-fd scenarios [1]. Therefore,
we don't use `std::fs::File::try_lock()`, which is backed by F_SETLKW.
The locking mechanism is aware of the `readonly` property and allows
`n` readers or `1` writer (exclusive mode).
As the locks are advisory, multiple cloud-hypervisor processes can
prevent themselves from writing to the same file. However, this is not
a system-wide file-system level locking mechanism preventing to open()
a file.
The introduced new locking mechanism does not cover vhost-user devices.
# Why
To prevent misconfiguration and improve safety, it is good practice to
protect disk image files with a locking mechanism. Experience and common
best practices suggest that advisory locks are preferable over mandatory
locks due to better compatibility and fewer pitfalls (in fs space).
The introduced functionality is aligned with the approach taken by
QEMU [0], and is also recommended in [1].
# Implementation Details
We need to ensure that not only normal operation keeps working but also
state save/resume and live-migration. Especially for live migration,
it is crucial that the sender VMM releases the locks when the VM stops
so the receiver VMM can acquire them right after that.
Therefore, the locking and releasing happen directly on the block
device struct. The device manager knows all block devices and can
forward requests to these types.
Last but not least, this commit uses on explicit lock acquiring
but implicit lock releasing (FD close). It only explicitly releases
the locks where this integrates more smoothly into the existing
code.
# Testing
I tested
- normal operation
- state save/resume,
- device hot plugging,
- and live-migration
with read/shared and write/exclusive locks.
One can use the `fcntl-tool` to test if locks are actually acquired
or released [2].
# Links
[0] 825b96dbce/util/osdep.c (L266)
[1] https://apenwarr.ca/log/20101213
[2] https://crates.io/crates/fcntl-tool
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
To support PCIe P2P between VFIO devices, we populate IOMMU mappings for
the non-emulated MMIO regions of all VFIO devices via
`VFIO_IOMMU_MAP_DMA` (f0c1f8d), but the patch did not properly update
the IOMMU mappings with BAR reprogramming.
Fixes: #7027
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The way how we handle PCI configuration space for vfio and vfio-user
devices are different from the rest of PCI devices. Besides accesses to
BAR registers (trapped to access the shadowing PCI config space we
maintained), accesses to other registers (including the COMMAND
register) are handled directly by the underline vfio or vfio-user
device.
This patch adds the proper handling of pending BAR reprogramming for
vfio and vfio-user devices.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The Memory Space Enable (MSE) bit from the COMMAND register in the
PCI configuration space controls whether a PCI device responds to memory
space accesses, e.g. read and write cycles to the device MMIO regions
defined by its BARs. The MSE bit is used by the device drivers to ensure
the correctness of BAR reprogramming. A common workflow is, the driver
first clears the MSE bit, then writes new values to the BAR registers,
and finally set the MSE bit to finish the BAR reprogramming.
This patch changes how we handle BAR reprogramming for all PCI
devices (e.g. virtio-pci, vfio, vfio-user, etc.), so that we follow the
same convention, e.g. moving PCI BARs when its MSE bit is set.
Note that some device drivers (such as edk2) only clear and set MSE once
while reprogramming multiple BARs of a single device. To support such
behavior, this patch adds support for multiple pending BAR reprogramming.
See: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7027#issuecomment-2853642959
Signed-off-by: Bo Chen <bchen@crusoe.ai>
A BAR reprogramming of a PCI device will only happen when the (guest)
kernel write to its PCI config space, e.g. the detection of bar
reprogramming (`detect_bar_repgraomming()`) can be embedded to the PCI
config space write (`write_config_register()`). It simplifies APIs
exposed by the `struct PciConfiguration` and `trait PciDevice`. It also
prepares for easier handling of pending bar reprogramming when the MSE
bit of the COMMAND register is not enabled at the time of changing BAR
registers.
See: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7027#issuecomment-2853642959
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Fixing the following clippy issue using `cargo clippy --fix`:
error: variables can be used directly in the `format!` string
--> build.rs:25:27
|
25 | version.push_str(&format!("-{}", extra_version));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Currently a lot of functions are stubbed out with unimplemented feature
tag. Add the missing implementation to successfully boot ARM64 guests on
MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Num SPI and Base SPI nodes should be added before the end node to be
included in the device tree generation.
Fixes: eac44e6 (arch: Extend FDT for GICv2M device for ARM64 on MSHV)
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
It seems like address allocation has been spread into different files
and different location for x86 vs ARM. This makes it hard to follow the
code. Thus, unify it a single location which satisfies all the
requirement.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
It just simplifies code and improves the code read-ability without much
affecting the boot performance.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This will satisfy the requirement of MSHV i.e., setting the GICD base
address before initializing the VM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For MSHV guests, we would need interrupt controller to be initialized
before the VM gets initialized. This is because we are registering the
base address of GIC distributor with MSHV as part of interrupt
controller initialization workflow. And MSHV mandates that this property
is set before we initialize the VM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For ARM64 guests we need to set three important partition property:
1) PPI interrupt ID for timer interrupt
2) PPI interrupt ID for PMU interrupts.
3) Hiding LPI support from the guest because MSHV does emulate ITS for
the guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since this would be used in other places inside the hypervisor and
hypervisor crate cannot take a dependency on arch crate, as that creates
cyclic dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently PPI interrupt ID are hardcoded as numbers, it would be ideal
to define them as constants and could be reused in other parts of the
hypervisor crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Along with also bump the vfio-bindings crates to use the latest
mshv-bindings.
There is a breaking change in the new mshv crate which requires an
additional step to initialize vm after creating it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
On x86 MSHV guests only used to support MSI based interrupts via IOAPIC
but ARM64 guests uses legacy interrupt for its functioning. Thus, extend
the logic to create routing entry to support legacy interrupts for ARM64
guests on MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For MSHV arm64 guest, there is an in-hypervisor GICv2M emulation and for
that to work, it needs to be enlightened with the base address of GIC
redistributor exposed to guest via FDT.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
GICv2M requires two additional properties to be exposed via FDT:
1) Base SPI number and 2) Total number of SPIs. SPIs in general starts
from 32 and goes upto 1019. But currently we are limiting the range to
96 as that should be good enough for any normal Linux guest to function.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Since there are not a wide availability of RISC-V boards available at
the moment, it is easiest to develop with a QEMU virtual machine. I had
a hard time setting one up, but with the assistance of Ruoqing, I was
able to get one running. These are the steps I took to do so.
Signed-off-by: abm-77 <andrewmiller77@protonmail.com>
Ubuntu 24.04 LTS (Noble Numbat) is tested against cloud-hypervisor v45.0
release and it is working, document status in README.md.
Signed-off-by: Bingxin Li <bl497@cam.ac.uk>
ARM64 system register constants are not 1:1 mapped to MSHV definition of
those registers so we need a small helper function to translate that
mapping before retrieving those system registers.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are other potential users of these registers definitions in the
hypervisor crate. And hypervisor crate cannot use definitions from arch
crate because it creates cyclic dependency.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
In order to be on-pair with what's we're using from micro-http, let's
also add the proper status code here as well (as it will be used by
`ch-remote`).
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
Although the CPU manager gives us a quite descriptive error, on the
application side (the part calling Cloud Hypervisor) we have absolutely
no way to distinguish such error from any other error that may happen
when resizing a VM.
With this in mind, let's be more specific and return a TooManyRequests
(429) error, allowing the caller to have a chance to decide whether they
want to retry the operation or not.
https://datatracker.ietf.org/doc/html/rfc6585#section-4
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
As the coming patches in this series will take advantage of a status
code that was recently added there.
Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
New MSHV version updates the get_msr_list output as
vector instead of fam-wrapper. It avoids unnecessary
conversions.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Latest mshv crates contains some IOCTL changes that
enhances VM creation and configures the features in correct
way. Also adds some features that improves register access.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
MSHV allows VMM to map the VP register page into root.
This feature helps VMM to faster process most of the frequent
used registers. This patch uses the VP register page for port
handling in CPU run method.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
As part of this configure the program counter, pstate and X0 registers.
Program counter will point to the start address of the kernel/firmware
in the guest memory. X0 will point to start of the FDT.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
As part of this configuration, two things are being done:
1. Setting up the base address of GIC Distributor
2. Setting up the base address of GIC Interrupt Translator
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Initial PSTATE value would be same for both KVM and MSHV. Thus, move it
to common register definition pool.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are a lot of internal functions that are not and probably should
not be called from other places.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
If a device returns 0xff as a capability pointer bad things happen.
The code before the previous commits would crash in debug builds due
to integer overflow. With the two lowest bits masked out, it sends the
code into an endless loop.
Be more robust by at least handling the case where the capability
appears to point to itself.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
The PCI standard mandates that the lower bits of the capability
pointer are masked out before using the pointer. See PCI Local Bus
Specification 3.0 Chapter 6.7 "Capabilities List".
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Currently, the code tries to follow the PCI capabilities list in
offset 0x34 in the config space regardless of whether the status
registers says this is valid. Fix by adding the appropriate check.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
OSDev has cranked up its bot protection. The following link works for
me locally after clicking the "I'm a human" button. I guess the CI
fails this check...
Without this exception the CI fails the link check stage:
* [403] [https://wiki.osdev.org/IOAPIC](https://wiki.osdev.org/IOAPIC) | Network error: Forbidden
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Use the context from Unmapped Gpa exit from the hypervisor to initialize
the MshvEmulatorContext and later call the emulator to decode the
instruction.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently it would be using the syndrome register for instruction
decoding which is what KVM has been using in-kernel to decode
instructions for ARM64 guests. In future, it could be extended with an
actual instruction emulator if required. But most Linux guests works
well with the instruction decoder using syndrome register.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This helps in implementing an instruction decoder for MSVH which does
not support in-kernel instruction decoding like KVM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Some tests are specifically designed for KVM hypervisor platform. Thus,
guard them using appropriate feature flags.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There are a bunch of unused variables as of now on the MSHV side and
compiler warns about them. Thus, mark them as unused for the time being.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
MSHV does not emulate a GICv3-ITS for guests to support MSI interrupts,
instead it exposes a GICv2m device. Currently adding a skeleton code
which would be modified later on with complete implementation.
With this we can start compiling cloud-hypervisor for MSHV on ARM64.
This will make sure that we don't regress in future in terms of basic
compilation test.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
`zerocopy` is bumped to 0.8.x after 0.3.4 of igvm crates, bump to rev
d062818 to capture `zerocopy` upgrade, but we should bump to 0.3.5
later.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Manually bump zerocopy to 0.8.24 since our dependabot could not perform
the upgrade properly.
Manually bump acpi_tabls as well since it's depending on zerocopy.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The dependency `bitfield-struct` 0.10.x of `igvm` 0.3.5 requires MSRV
1.83.0, bump to catch up.
Update image to 20250412-0 because MSRV in Dockerfile is updated.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This aligns with our CLI syntax. The correctness of `DiskConfig` will be
ensured via `VmConfig::validate()`, e.g. `path` and `socket` are
mutually exclusive.
Fixes: #7016
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Use the builtin function instead of using `==` operator.
Warning from the beta compiler:
error: use `std::ptr::eq` when comparing raw pointers
--> pci/src/vfio.rs:1616:24
if host_addr == libc::MAP_FAILED {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: try: `std::ptr::eq(host_addr, libc::MAP_FAILED)`
= help: for further information visit
= https://rust-lang.github.io/rust-clippy/master/index.html#ptr_eq
= note: `-D clippy::ptr-eq` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::ptr_eq)]`
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Use the builtin function to improve the readability of the code.
Warning from beta compiler:
error: manual arithmetic check found
--> vm-allocator/src/address.rs:151:30
|
|let adjust = if alignment > 1 { alignment - 1 } else { 0 };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| help: replace it with: `alignment.saturating_sub(1)`
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#implicit_saturating_sub
= note: `-D clippy::implicit-saturating-sub` implied by `-D warnings`
= help: to override `-D warnings` add`#[allow(clippy::implicit_saturating_sub)]`
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Rust has a new way of constructing other error and clippy complains if
we are still using the older way to construct error message. Thus,
migrate to the new approach suggested by the clippy.
Warning from beta compiler:
error: this can be `std::io::Error::other(_)`
--> block/src/vhdx/mod.rs:142:17
|
| / std::io::Error::new(
| | std::io::ErrorKind::Other,
| | format!("Failed to update VHDx header: {e}"),
| | )
| |_________________^
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#io_other_error
help: use `std::io::Error::other`
std::io::Error::other(
format!("Failed to update VHDx header: {e}"),
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
`serde_json` crate is referenced by multiple components, centralize it
to workspace to better manage this crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
KVM supports GICv3-ITS emulation and the current GicState is modelled
around the KVM implementation. We should refactor this to accomodate
other hypervisor requirements. For example, MSHV only support GICv2M
emulation for guests for delivering MSI interrupts instead of GICv3-ITS.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Add `riscv.md` to guide developers/users on preparing guest kernel,
image, and direct boot VM on riscv64 platforms; document support status
and known limitations.
Co-authored-by: Bo Chen <bchen@crusoe.ai>
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The larger the VM memory, the greater the memory pressure, and the
greater the stop_dirty_log() overhead. Moving stop_dirty_log() outside
the downtime period can reduce downtime.
Signed-off-by: Jinrong Liang <cloudliang@tencent.com>
`riscv64_set_one_reg_to_vcpu` macro is used to set value of specific
RISC-V `$reg_name` stored in `state` to KVM Vcpu.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`riscv64_get_one_reg_from_vcpu` macro is used to extract RISC-V register
data from KVM Vcpu according to `$reg_name` provided to `state`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`igvm` crates are referenced by multiple components, centralize them to
workspace to better manage those crates.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Certain MSHV ioctls are only available on x86 architecture. Thus,
conditionally compile seccomp filter for x86 and in general enable
seccomp filters when compiling for MSHV on aarch64.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The CLI has grown to a big variety of options. clap prints them in the
help message (--help) in the order they were defined. We now are at a
point where grouping things logically together doesn't work well.
Further, there is no support by clap for logical grouping and the
current code base wasn't consistent. Therefore, this commit introduces
two changes:
- a new structure to define arguments (all in an array)
- an alphabetical ordering of the arguments
No other changes have been made. No options have been altered.
This significantly improves:
- code maintainability and extensibility
- readability of the --help output
A unit test ensures they stay sorted. A better approach to check if the
list of arguments (known at build time) is sorted would be a compile
time check (`const`), but this currently isn't possible in stable Rust.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Use the definitions from the rust-vmm/mshv crate for various
datastructures such as StandardRegisters, RegList, VcpuInit etc.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
We are moving the metrics and rate-limiter workers to a bare-metal
system, so that we can have more consistent results (particularly for
the block device metrics)
Signed-off-by: Bo Chen <bchen@crusoe.ai>
v0.5.0 of `seccompiler` has supported riscv64, let's bump from v0.4.0 to
v0.5.0 to capture that support.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
In addition to the BDF information on the guest, reporting the device
path of the VFIO device on the host is more useful when it comes to
debug DMA map/unmap errors particularly ones caused by failing hardware
on the host.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
`test_snapshot_restore_with_fd` uses unsafe file descriptors and with
rust 1.82.0 it errors with:
```
fatal runtime error: IO Safety violation: owned file descriptor already
closed
```
so has been skipped for now.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We are having complains from Rust 1.86.0-beta.1 (f0cb41030 2025-02-17)'
clippy, which suggests us to replace `repeat().take()` with
`repeat_n()`. While `repeat_n()` is stablized in Rust 1.82.0.
Update image to 20250307-2 because MSRV in Dockerfile is updated.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
While executing `rustup toolchain add --profile minimal
1.82.0-x86_64-unknown-linux-musl`, it errors out with:
```
error: toolchain '1.82.0-x86_64-unknown-linux-musl' may not be able to
run on this system
note: add the `--force-non-host` flag to install the toolchain anyway
```
Add `--force-non-host` to install minimal musl toolchain for x86_64.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Currently we are just storing the StandardRegisters in the Vcpu state
which would be required for saving and restoring the ARM64 guest on
MSHV.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The reason `test_vfio_user` fails is as @likebreath pointed: our ARM
host does not support SVE, while the nvme_tgt binary built from the
container image requires it. As a result, we encountered a SIGILL when
running the nvme_tgt binary. This also explains why this is not
happening when the container is built on the same host itself.
And quote from @rbradford:
When a job is run on one of the workers it looks to see if there is a
container locally matching the name as specified in the dev_cli.sh
script - if there is then it uses it. Otherwise it will try and download
it from the container registry - if that fails then it will built
locally. For the x86-64 workers started dynamically it will never have a
local version as they are a fresh VM. But on the ARM64 builder is a
local container image cache.
This can lead to an issue where if the image is build with one version
(a handcrafted datestamp) and then the Dockerfile is changed without
changing the timestamp then an old version may be fetched from the cache
or server. It is there for essential to always bump the datestamp (there
is a number after the - that can be used for this.)
However there is also the added complexity that image that is build and
uploaded to the container registry is not the same as the built locally
and thus used for the initial testing of the Dockerfile change. This
leads to the issue we have seen where different CPU compiler flags (from
-march=native) from the QEMU cross build in the hosted GHA action and
the local ARM64 build. Resulting in a binary in the remotely built
container not working locally.
We end up specifying TARGET_ARCHITECTURE="armv8.2-a" for building spdk,
and put built `python/spdk/` folder into `/usr/local/bin/spdk-nvme`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`arm64` build in ubuntu:22.04 errors out with `error processing package
libc-bin`. This issue is a known issue between the binfmt (running
different architectures via QEMU) and the libc ldconfig binary running
in container. We're "suddenly" having issues as ubuntu-latest (which is
the OS version we run the GH action container with) was recently changed
from 22.04 to 24.04 and hence why upgrading the container userspace from
22.04 to 24.04 solves the problem.
Removed deprecated package `python3-distutils`.
Update image name from `20250111-0` to `20250222-0`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Currently a bunch of KVM specific interfaces are leaked into the vmm
crate which should ideally does not contain any hypervisor specific data
structures.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Use `lychee` to check availability of links in cloud-hypervisor.
The urls explictly excluded in config file are manually checked.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
We have lost track to releases before v27.0 since these projects no
longer exists. Delete links to those projects.
Update links to a detailed view specific to each group of release.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Previous repo hosts details of `Rust Style` is removed, point `Rust
Style` to `style-guide` in `rust-lang/rust` repo.
Link to provide illustration on `signed-off-by language` is also
removed, use a snapshot found in web archive [1] instead.
Format `CONTRIBUTING.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
[1] https://web.archive.org
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
TDX homepage was moved to elsewhere, and `sgx` support is upstreamed
since v5.11 kernel.
Format `intel_sgx.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
TDX homepage was moved to elsewhere, and `tdx-tools` repo was removed.
Provide a valid link of TDX homepage and change all reference to
`tdx-tools` to `tdx-linux`.
Format `intel_tdx.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
There is a link referencing `rate-limiter` module of `firecracker`, but
that module no longer exsits.
Point the link to a commit with the same date in `firecracker` when this
commit was merged to `cloud-hypervisor`.
Format `io_throttling.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`docs/arm64.md` was removed and splited into `README.md`, `building.md`
and `uefi.md` in #4991.
Let's point it to
`8ab15b9a98/docs/arm64.md`
the commit right before `docs/arm64.md` was removed in main branch.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Previous link `docs/logging` is not valid, replacing `docs/logging` with
`docs/logging.md`.
Format `logging.md` using `mdformat` with GitHub Flavored Markdown
(GFM).
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Use this to add any extra versioning information to the binary. It is
useful when packaging Cloud Hypervisor.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This is a typo here, `interrupt-controller` attribute of each CPU node
should not have a `#` prepended.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Replace the use of a reference kernel configuration file from this
repository with the use of a defconfig from the linux fork.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The IMSIC attr of RISC-V AIA is wrongly configured to start from 0, which
would error out with `os error 22` (invalid argument).
```console
Error booting VM: VmBoot(DeviceManager(CreateInterruptController(CreateAia(CreateVaia(Vaia error SetDeviceAttribute(SetDeviceAttribute(Invalid argument (os error 22))))))))
```
`riscv_imsic_attr_of` should shift `cpu_index` by 1 here to produce
correct IMSIC attr.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
For more control over updating the guest kernel use a fixed tag name
rather than fetching the latest.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Currently, Cloud Hypervisor does not set a VIRTIO_IOMMU_F_INPUT_RANGE
feature bit for the VirtIO IOMMU device, which, according to spec[1],
means that the guest may use the whole 64-bit address space is for
IOMMU purposes:
>If the feature is not offered, virtual mappings span over the whole
>64-bit address space (start = 0, end = 0xffffffff ffffffff)
As far as I am aware, there are currently no host platforms on
the market capable of addressing the whole 64-bit address space.
For example, I am currently working with a host platform that reports
39-bit address space for IOMMU purposes:
>DMAR: Host address width 39
When running a VFIO pass-through guest on such a platform, NVIDIA
driver in guest gets DMA mapping failures when working with large data,
and this results in Cloud Hypervisor exiting with the following error:
>cloud-hypervisor: 1501.220535s: <__iommu>
>ERROR:virtio-devices/src/thread_helper.rs:53 -- Error running worker:
>HandleEvent(Failed to process request queue : ExternalMapping(Custom
>{ kind: Other, error: "failed to map memory for VFIO container, iova
>0x7fff00000000, gpa 0x24ce25000, size 0x1000: IommuDmaMap(Error(22))"
>}))
Passing "--platform iommu_address_width=39" to Cloud Hypervisor built
with this change fixes this.
[1]: https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/
virtio-v1.3-csd01.html#x1-5420006
Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>
When VM restore fails, the VMM state is left with some side-effects,
such as a VM being created. It would prevent the VMM from creating and
booting a new VM or restoring from a VM snapshot.
To fix this issue, this patch explicitly handles the side effects to the
VMM state when VM restore fails, e.g. clear the VmConfig and shutdown
the VM being created.
Fixes: #6869
Signed-off-by: Bo Chen <bo.arvin.chen@gmail.com>
With VM restore, the VMM is always re-creating a VM based on the
restored `VmConfig`. We should always re-generate the 'console_info'
from the `Vmm` struct to stay consistent with the new VM being created.
Signed-off-by: Bo Chen <bo.arvin.chen@gmail.com>
Instead of reinventing this mock infrastructure in the upcoming fuzzer,
reuse the one that is already available.
However this change makes Clippy complain that TestBackend and
TestContext don't implement Default. This is just test code, we can
suppress Clippy in this case.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Updating the kernel to v6.12 has shown up a flaw in the workflow for our
binary kernel releases. The CI job that builds the binary kernel in the
cloud-hypervisor/linux repository fetches the config from the main
branch of the cloud-hypervisor/cloud-hypervisor repository. However the
CI job to update the kernel version to use is in the cloud-hypervisor
repository.
As a workaround - update the kernel config and version in the
cloud-hypervisor repository to point to v6.12 and use the ability to
build the kernel during the CI run. Once merged to main a new release
can be made in the linux respository which will build a binary asset
using the new config. After that release the CI jobs on the
cloud-hypervisor repository can changed back to using the binary kernel
assets.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Remove an erroneous optimisation that used the page size mask to reduce
the range to iterate through on the set of mappings. This doesn't work
as the virtio-iommu ranges are larger than a single page. This may have
worked in the past when the mappings were limited to a single page.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
No change to the Dockerfile but I observed that the 20251022-0 image was
not available in the repository.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
MSHV's SEV-SNP implementation calls ioeventfds whenever there is an
event.
This change removes the need frequent allocation and deallocation of a
vector, while at the same time makes sure other call sites are
unaffected.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This avoids ambiguity of parameters:
error: ambiguous reference to positional arguments by number in a tuple variant; change this to a named argument
--> block/src/qcow/mod.rs:48:48
|
48 | #[error("File larger than max of {}: {0}", MAX_QCOW_FILE_SIZE)]
| ^^^^^^^^^^^^^^^^^^
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This will become useful when we build the fuzzing target for the
instruction emulator, because there is no need to pull in the rest of
the hypervisor crate in that situation.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The fastfmt feature and VEX support use techniques that appear to leak
memory in the eye of LLVM's address sanitizer.
While at it, disable a bunch of instruction set decoding support we
never intend to support.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The size was set to one because without VIRTIO_BLK_F_SEG_MAX, the guest
only used one data descriptor per request.
The value 32 is empirically derived from booting a guest. This value
eliminates all SmallVec allocations observable by DHAT.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This allows the guest to put in more than one segment per request. It
can improve the throughput of the system.
Introduce a new check to make sure the queue size configured by the user
is large enough to hold at least one segment.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When the main fuzzer function returns (), it is equivalent to
returning Corpus::Keep.
In some of the return paths, we want to reject the input so that the
libfuzzer won't spend more time mutating them.
The should make fuzzing more efficient. No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The checksum field in the original buffer should be zeroed.
The code was zeroing a temporary buffer. That's wrong.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The original code was buggy. It always attempted to update the header,
even when the file was opened as read-only. That led to an error.
The specification states that the headers should be updated when the
first user visible write happens. We can just drop the incorrect code.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The ASYNC flag forces requests to go to worker threads. Worker threads
are expensive. Let the kernel decide what to do.
With this change, I no longer see an excessive amount of io_uring worker
threads.
Quote from the manual for io_uring_sqe_set_flags(3):
```
IOSQE_ASYNC
Normal operation for io_uring is to try and issue an sqe
as non-blocking first, and if that fails, execute it in an
async manner. To support more efficient overlapped
operation of requests that the application knows/assumes
will always (or most of the time) block, the application
can ask for an sqe to be issued async from the start. Note
that this flag immediately causes the SQE to be offloaded
to an async helper thread with no initial non-blocking
attempt. This may be less efficient and should not be
used liberally or without understanding the performance
and efficiency tradeoffs.
```
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Instead of silently ignoring the error, return an error to the callers.
This in practice should never happen, because the submission queue size
(ring depth) is the same as the virtio queue size. Virtio queue won't
push more requests than there are submission queue entries.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The original code relied on the default `read_vectored` or
`write_vectored` implementations from the standard library.
The default implementation of those functions only uses the first
non-empty buffer. That's not correct when there are more than one
buffers.
Fixes: #6876
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This system is erroring out on jobs due to insufficient memory - reduce
parallelism to allow CI jobs to complete.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Updated the live migration documentation to include instructions for
performing cross-host live migrations over TCP connections.
Signed-off-by: Jinrong Liang <cloudliang@tencent.com>
Add support for cross-host live migration over TCP, which
significantly improves performance in cross-host live migration
scenarios compared to the Unix socket forwarding method.
Signed-off-by: Jinrong Liang <cloudliang@tencent.com>
Bump vm-memory from 0.16.0 to 0.16.1 to include the implementations of
ReadVolatile and WriteVolatile for TcpStream.
Signed-off-by: Jinrong Liang <cloudliang@tencent.com>
While non-Intel CPU architectures don't have a special concept of IO
address space, support for PCI I/O regions is still needed to be able
to handle PCI devices that use them.
With this change, I'm able to pass through an e1000e device from QEMU
to a cloud-hypervisor VM on aarch64 and use it in the cloud-hypervisor
guest. Previously, it would hit the unimplemented!().
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Add a bitmap to MshvVM struct for caching the pages
that the VMM got shared access from the guest.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Pass memory size from vmm crate to hypervisor VM
creation API so use for creating Bitmap for SEV-SNP
guest.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
For SEV-SNP VM on MSHV we need to request page access during
IO, we want to avoid such request for the page that have already
been requested. In order to maintain the bitmap we need the memory size
during bitmap creation.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
VMM needs to handle VMG exit for guest request. This patch
removes the old method that uses gpa_read/write(IOCTL and hypercall),
which is expensive and update the GHCB page using
mapped(root partition) struct.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
VMM needs to handle VMG exit for MMIO. This patch
removes the old method that uses gpa_read/write(IOCTL and hypercall),
which is expensive and update the GHCB page using
mapped(root partition) struct.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
VMM needs to handle VMG exit for IO Port. This patch
removes the old method that uses gpa_read/write(IOCTL and hypercall),
which is expensive and update the GHCB page using
mapped(root partition) struct.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
VMM needs to handle VMG exit for doorbell page. This patch
removes the old method that uses gpa_read/write(IOCTL and hypercall),
which is expensive and update the GHCB page using
mapped(root partition) struct.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
After handling the VMG exit vmm needs to clear the exitinfo1
into the GHCB page. This patch replaces the old
method(gpa_write) and clear the exitinfo1 using mapped GHCB struct.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
For confidential VM on MSHV, GHCB page is the communication
method between guest and host. All the CVM exits, VMM
needs to read and write to the GHCB page. MSHV provides
an option to remap the page to the root partition. This
way VMM could directly read and write to the page and
skip extra IOCTL and hypercall. This improvement makes
the IO 10% faster.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
TPM support on RISC-V is not scheduled in this stage, we will come back
when it's ready and we have workloads to verify it on RISC-V.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Currently `devices` module cannot be solely built, by specifying
`arch/kvm` to turn on the features required for its dependency - `arch`
crate to build. Thus enabling `devices` crate to be built and tested
with command:
```sh
cargo build -p devices --features kvm
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`arch` module are required for devices module to build. Enabling `arch`
module in target specific dependencies section.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Provide implementation for GSI allocator to work on riscv64
architecture, and doc comment for riscv64 as well.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Currently `vm-allocator` module cannot be solely built, by add
`features` section and specifying `arch/kvm` to turn on the features
required for its dependency - `arch` module to build. Thus enabling
`vm-allocator` crate to be built and tested with command:
```sh
cargo build -p vm-allocator --features kvm
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`arch` module are required to provide definitions for riscv64 system irq
and memroy allocation, enabling `arch` module in target specific
dependencies section.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Integrate fdt setup and layout for `riscv64` to `arch` crate, provide
definitions of `riscv64` platform specific error and reuse
`MmioDeviceInfo`, `PciSpaceInfo` structures and `DeviceInfoForFdt` impl
block for `riscv64`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Reused layout set for `aarch64` as much as possible to provide a concise
and compatible (for devices) layout for RISC-V 64-bit.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`fdt_parser` and `vm-fdt` crates are required to setup fdt used to boot
riscv64 virtual machines, enabling the two crates in target specific
dependencies section.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
It turns out we need to setup `a0`, `pc` and `a1` for all vcpus before
we run them, remove predicates used to set `pc` and `a1` for `vcpu0`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Previous `Vaia` implementation uses types as it is, thus the property
string generated for fdt setup requires additional type conversion.
Change the types used in the methods of `Vaia` trait to provide a
concise fdt setup process.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The rate-limiter worker was moved to use small Azure VMs (#6731) and now
requires more time to complete.
Increasing its timeout to stablize this worker.
Signed-off-by: Bo Chen <bo.arvin.chen@gmail.com>
With 0.5.0 release of `rust-hypervisor-firmware`, `aarch64` binary were
added to assets, which causes the `FW_URL` to have multiple download url
separated by a white space, thus our integration tests would fail.
Constrain `FW_URL` to `hypervisor-fw` to resolve this.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The previous docs were very developer centric and have led several
people to believe that cross-machine migration is not supported at
all.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
I've added newlines between paragraphs and code blocks for easier
reading. I've also changed the code blocks to use the correct
highlighting.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Replace `map_or()` on false condition with `is_some_and` to provide
better readability, as suggestted by v1.84.0-beta.1 `cargo clippy`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Adapt the existing code to transparently the MemorySlotAllocator. The
MemoryManager is the canonical holder of the these values with them
turned into a MemoryAllocator on demand.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Add an allocator for KVM memory slots, a finite resource in the kernel
- these need to be manipulated across different crates with a common
dependency on this crate.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
`prepare_linux` is capable of determining whether we need to invoke
`build_custom_linux` for building linux from source or `download_linux`
for downloading pre-built.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`prepare_linux` checks if a `--build-guest-kernel` option is present,
and build kernel from `cloud-hypervisor/linux.git`. Otherwise, it will
invoke `download_linux` to use pre-built kernel.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
If any VM argument (e.g. --disk) is provided require some payload (e.g.
--kernel or --firmware) when parsing the command line arguments.
See: #6831
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Add unit-test to make sure get_regs and set_regs on riscv64 architecture
work as expected, effectively avoiding typos in register names.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Complement `create_standard_regs` implementation on RISC-V platform to
work with `From` trait of `kvm_riscv_core`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Incorporates riscv64 register interaction and AIA creation to kvm
module. Complete `Vcpu` trait on RISC-V platform.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Integrate `aia` module into `riscv64` module, and enable `riscv64`
module if target architecture is RISC-V 64-bit.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`fuzz` is a separate workspace, enable consistency check on rust-vmm
crates in `fuzz` to keep stuffs in sync with root workspace.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
`get_device_attr` in 0.19.0 kvm-ioctls is marked as unsafe, wrap
invokation of `get_device_attr` with `unsafe` block.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
- Bump kvm-bindings from 0.9.1 to 0.10.0
- Bump kvm-ioctls from 0.18.0 to 0.19.0
- Bump vm-memory from 0.15.0 to 0.16.0
- Bump linux-loader from 0.12.0 to 0.13.0
- Bump virtio-bindings from 0.2.1 to 0.2.4
- Bump virtio-queue from 0.13.0 to 0.14.0
- Pin mshv-bindings to 0.3.1
- Pin mshv-ioctls to 0.3.1
- Pin vhost to rev "d983ae0"
- Pin vhost-user-backend to rev "d983ae0"
Since vhost 0.12.0 and vhost-user-backend 0.16.0 are going to be yanked,
temporarily pin these two to "d983ae0", which are expected to be
replaced by 0.13.0 vhost and 0.17.0 vhost-user-backend after released.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Add `package-consistency-check.py` script to prevent #6809 and #6815
from happening. This script takes a string present in the repository
field of packages to identify pacakges from a specific source.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This reverts commit b264aebcde.
Pulls in new vm-memory so should be done as part of a global bump of
Rust-VMM crates.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: the following explicit lifetimes could be elided: 'a
--> hypervisor/src/arch/x86/emulator/mod.rs:492:6
|
492 | impl<'a, T: CpuStateManager> Emulator<'a, T> {
| ^^ ^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_lifetimes
= note: `#[warn(clippy::needless_lifetimes)]` on by default
help: elide the lifetimes
|
492 - impl<'a, T: CpuStateManager> Emulator<'a, T> {
492 + impl<T: CpuStateManager> Emulator<'_, T> {
|
warning: the following explicit lifetimes could be elided: 'a
--> hypervisor/src/mshv/x86_64/emulator.rs:19:6
|
19 | impl<'a> MshvEmulatorContext<'a> {
| ^^ ^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_lifetimes
help: elide the lifetimes
|
19 - impl<'a> MshvEmulatorContext<'a> {
19 + impl MshvEmulatorContext<'_> {
|
warning: the following explicit lifetimes could be elided: 'a
--> hypervisor/src/mshv/x86_64/emulator.rs:65:6
|
65 | impl<'a> PlatformEmulator for MshvEmulatorContext<'a> {
| ^^ ^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_lifetimes
help: elide the lifetimes
|
65 - impl<'a> PlatformEmulator for MshvEmulatorContext<'a> {
65 + impl PlatformEmulator for MshvEmulatorContext<'_> {
|
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
--> test_infra/src/lib.rs:1307:25
|
1307 | let child = self
| _________________________^
1308 | | .command
1309 | | .stderr(Stdio::piped())
1310 | | .stdout(Stdio::piped())
1311 | | .spawn()
1312 | | .unwrap();
| |_________________________^
|
= note: consider calling `.wait()`
= note: not doing so might leave behind zombie processes
= note: see https://doc.rust-lang.org/stable/std/process/struct.Child.html#warning
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#zombie_processes
= note: `#[warn(clippy::zombie_processes)]` on by default
The API caller should ensure that they call .wait() or equivalent on the
spawned child to reap it.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The snapshot/restore test didn't wait on the child being spawned:
warning: spawned process is never `wait()`ed on
--> performance-metrics/src/performance_tests.rs:495:25
|
495 | let mut child = GuestCommand::new(&guest)
| _________________________^
496 | | .args(["--api-socket", &api_socket_source])
497 | | .args([
498 | | "--cpus",
... |
507 | | .spawn()
508 | | .unwrap();
| |_____________________^
|
= note: consider calling `.wait()`
= note: not doing so might leave behind zombie processes
= note: see https://doc.rust-lang.org/stable/std/process/struct.Child.html#warning
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#zombie_processes
= note: `#[warn(clippy::zombie_processes)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
We know what we are doing within `tracer` crate, disable lint on these
shared reference to mutable static.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Functions marked as `pub` in test module need to be documented. Since
these are literally helper functions and there is no need to be `pub`,
remove these `pub` keywords.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
As clippy of rust-toolchain version 1.83.0-beta.1 suggests, remove
redundant `return` statement to keep code clean.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
As clippy of rust-toolchain version 1.83.0-beta.1 suggests, use `c""` to
construct nul-terminated `CStr`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
As clippy of rust-toolchain version 1.83.0-beta.1 suggests, remove
manual implementation of `is_power_of_two` to improve readability.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
As clippy of rust-toolchain version 1.83.0-beta.1 suggests, replace
manually implemented `div_round_up!` and the like with `div_ceil` from
std.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This system has a lot of cores (80) resulting in all the tests being
spawned simultaneously and leading to exhaustion of the available
memory. Instead limit the number of threads.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
After updating the OS image to the latest jammy version this test now
requires more memory to run
Fixes: #6782
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
When running with the newer jammy OS image error messages from Rust
Hypervisor Firmware about lack of ram when rebooting where observed.
Increase the RAM allocation to allow it to boot after a reboot.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Currently `arch` crate cannot be built, by specifying `hypervisor/kvm`
to turn on the features required for its dependency - `hypervisor` crate
to build. Thus enabling `arch` crate to be built with command:
```sh
cargo build -p arch --features kvm
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
AddressManager::move_bar() acquires the device_tree mutex.
The function is called from PciConfigIo::config_space_write()/
PciConfigMmio::config_space_write() while the pci_bus mutex
is acquired.
The functions DeviceManager::pci_resources()/eject_device()
acquire these mutexes in reverse order, which leads to a deadlock.
Fixes: #6775
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
This lint disallows asserttions on is_ok()/is_err() in favor of either
using unwrap (so that at least if the test fails, we the failure message
will contain the actual failure reason instead of just "was not
ok/err"), or actually matching the specific variant.
Inspired by and quoted from @roypat.
Enable this lint for `build.yaml` and `quality.yaml`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Asserting on .is_ok()/.is_err() leads to hard to debug failures (as if
the test fails, it will only say "assertion failed: false". We replace
these with `.unwrap()`, which also prints the exact error variant that
was unexpectedly encountered (we can to this these days thanks to
efforts to implement Display and Debug for our error types). If the
assert!((...).is_ok()) was followed by an .unwrap() anyway, we just drop
the assert.
Inspired by and quoted from @roypat.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
This is based on a revert of commit
ce49a6f4b8 with some modifications for a
different worker configuration.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Only download the kernel binaries from the github release if the remote
file is newer (avoids multiple copies accumulating in the download
directory.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Cargo's features are additive. Disabling default features in individual
packages but not on the workspace level makes no sense.
This fixes the several build warnings.
Fixes: 5a70d7ec69 (build: Centralize rust-vmm crates to workspace)
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Since cloning Option<Arc<T>> will clone for Arc<T>, this patch fixes the
follow warning:
warning: this call to `as_ref.map(...)` does nothing
--> vmm/src/lib.rs:872:13
|
872 | self.console_resize_pipe.as_ref().map(Arc::clone),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `self.console_resize_pipe.clone()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_asref
= note: `#[warn(clippy::useless_asref)]` on by default
Signed-off-by: Songqian Li <sionli@tencent.com>
This patch removes pub import vm_config in config.rs to eliminate
the ambiguity of vm_comfig reference.
Signed-off-by: Songqian Li <sionli@tencent.com>
By introducing `imports_granularity="Module"` format strategy,
effectively groups imports from the same module into one line or block,
improving maintainability and readability.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Enforcing group_imports="StdExternalCrate" requires using cargo fmt from
the nightly toolchain. Create a new workflow that runs on nightly to run
cargo fmt.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Historically the Cloud Hypervisor coding style has been to ensure that
all imports are ordered and placed in a single group. Unfortunately
cargo fmt has no support for ensuring that all imports are in a single
group so if whitespace lines were added as part of the import statements
then they would only be odered correctly in the group.
By adopting "group_imports="StdExternalCrate" we can enforce a style
where imports are placed in at most three groups for std, external
crates and the crate itself. Choosing a style enforceable by the tooling
reduces the reviewer burden.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Serial device doesnt support resize semantics. Setting up the
console resize pipe in the Serial device setup path, overwrites
the setup done as part of virtio-console.
Signed-off-by: BharatNarasimman <bharatn@microsoft.com>
This patch removes locks in VmCreate request and VmInfo response
since we needn't use a lock here and should ensure that internal
implementation is transparent to the runtime.
Signed-off-by: Songqian Li <sionli@tencent.com>
Killing process with SIGKILL will miss the information since CLH process
needs to end normally to export code coverage information.
Signed-off-by: Songqian Li <sionli@tencent.com>
Add release and target params to `cargo test` since we collect
the code coverage reports from `xx/$BUILD_TARGET/release/`.
Signed-off-by: Songqian Li <sionli@tencent.com>
The construction of `FilePair` in `virtio_devices` component has changed
in 287887c, wrapping the parameters with `Arc` to fix fuzz build.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Modify `Cargo.toml` in each member crate to follow the dependencies
specified in root `Cargo.toml` file.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
For a VM with virt-console enabled, when a reboot is requested, the
console devices are closed during the shutdown path. As part of this
the sigwinch listener process and the console resizer pipe are closed.
For the new incarnation of the VM, fresh set of console devices are
setup and a new console resizer pipe is created. The new VM should
be setup to use the newly created console devices including the console
resizer pipe.
Reading from the older console resizer pipe results in unexpected eof
error and terminates the cloud hypervisor process.
Signed-off-by: BharatNarasimman <bharatn@microsoft.com>
Rebooting a VM fails with the following error when debug assertions
are enabled:
fatal runtime error: IO Safety violation: owned file descriptor already closed
This happens because FromRawFd::from_raw_fd is used on RawFds stored
in ConsoleInfo every time a VM begins to boot, so the second
time (after a reboot, or if the first attempt to boot via the API
failed), the fd will be closed. Until this assertion is hit, the code
is operating on either closed file descriptors, or new file
descriptors for something completely different. If debug assertions
are disabled, it will just continue doing this with unpredictable
results.
To fix this, and prevent the problem reocurring, ownership of the
console file descriptors needs to be properly tracked, using Rust's
type system, so this commit refactors the console code to do that.
The file descriptors are now passed around with reference counts, so
they won't be closed prematurely. The obvious way to do this would be
to just have each member of ConsoleInfo be an Arc<File>, but we need
to accomodate that serial console file descriptors can also be
sockets. We can't just store an OwnedFd and convert it when it's
used, because we only get a reference from the Arc, so we need to
store the descriptors as their concrete types in an enum. Since this
basically duplicates the ConsoleOutputMode enum from the config, the
ConsoleOutputMode enum is now not used past constructing the
ConsoleInfo.
So that ownership can be represented consistently, the debug console's
tty mode now uses its own stdout descriptor.
I'm still using .try_clone().unwrap() (i.e. dup()) to clone file
descriptors for Endpoint::FilePair and Endpoint::TtyPair, because I
assume there's a reason for them not just to hold a single file
descriptor.
I've also retained the existing behaviour of having serial manager
ignore the tty file descriptor passed to it (which is stdout), and
instead using stdin. It looks a lot weirder now, because it has to
explicitly indicate it's ignoring the fd with an underscore binding.
Fixes: 52eebaf6 ("vmm: refactor DeviceManager to use console_info")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The assignment of console_resize_pipe in the TTY case seems to have
been accidentally deleted. I've put it back, but since this is adding
code, I used the new safe API for checking whether a file is a
terminal, introduced in Rust 1.70.0. We should probably use that
everywhere, but that's out of scope of this bug fix.
Fixes: 52eebaf6 ("vmm: refactor DeviceManager to use console_info")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
I've moved this so that it's just after the enum definition, which
will hopefully make it less easy to miss if events are added/removed
again in future.
Fixes: 6d1077fc ("vmm: Unix socket backend for serial port")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Complete the removal of the DAX support by removing the use of
non-standard messages. These messages have since been removed from the
vhost_user crate (rust-vmm/vhost#246) and so need to be removed from our
implementation since that would otherwise block updating to a newer
version of the crate.
The ability to enable DAX support in Cloud Hypervisor has been disabled
some time ago but this code was residual with no way to enable it.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
And modify to code to use the updated interfaces.
Arguments for map_guest_memory, get_dirty_bitmap, vp.run(),
import_isolated_pages, modify_gpa_host_access have changed.
Update these to use the new interfaces, including new MSHV_*
definitions, and remove some redundant arguments.
Update seccomp IOCTLs to reflect interface changes.
Fix irq-related definitions naming.
Bump vfio-ioctls to support mshv v0.3.0.
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
`GetRegList` variant will be referenced on both Aarch and RISC-V. Fixed
comment to generalize this error variant.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Since RISC-V has its own definition of `CoreRegister`, expand the Aarch
variant to avoid collision of `HypervisorCpuError`.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The error message of `SetRegister` and `GetRegister` fail to describe
the reason as the error variant suggests. Fixed error message
accordingly.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
The rate-limiter worker now is running on Azure VMs whose performance
are more prone to be fluctuate, particularly on block performances. This
commit relaxes the limit check for group rate limiter tests to make them
less flaky.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Port I/O is only supported on x86_64 - use inverted conditional logic to
match the other architectures rather than calling them out specifically.
This will be relevant when RISC-V support is added.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The steps with predicates of `x86_64` targets would never turn on
`use-cross`, removing them from quality check.
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
Bound checks for virtio-mem and ACPI memory hotplug are off by
one and two, respectively. This prevents users to fully use the reserved
memory hotplug size.
For ACPI, if we specific `--memory size=2G,hotplug_size=4G` and run
`ch-remote resize --memory 6G`, cloud-hypervisor will report the
following error because of the incorrect bound check:
`<vmm> ERROR:vmm/src/lib.rs:1631 -- Error when resizing VM:
MemoryManager(InsufficientHotplugRam)`
Similarly, for virtio-mem, cloud-hypervisor will fail the incorrect
bound check and abort the resize. The VM will see the following error
in dmesg:
`virtio_mem virtio3: unknown error, marking device broken: -22`
This patch has fixed both bound checks and ensure that users can
hot add memory up to the reserved hotplug size.
Signed-off-by: Yuhong Zhong <yz@cs.columbia.edu>
Since the underline bare-metal system is going to retire, we are moving
the metrics and rate-limiter workers to use Azure VMs.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Otherwise, you just see output like "Error running command: Server
responded with an error: InternalServerError", which isn't very
helpful. The response body used to be include with the message, but
was removed when the Error enum was converted to thiserror.
Fixes: 5d0d56f5 ("api_client: Use thiserror for errors")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
With cd0cdac ("virtio-devices: Fix seccomp rules for SevSnp guest"), the
sys_ioctl rule that was applied in virtio_thread_common, would override
previously specified sys_ioctl rules for individual thread type. This
causes the SevSnp guest to crash with seccomp violation.
Fixes: cd0cdac ("virtio-devices: Fix seccomp rules for SevSnp guest")
Signed-off-by: Purna Pavan Chandra <paekkaladevi@microsoft.com>
The Linux kernel fork repository for Cloud Hypervisor now produces
prebuilt x86-64 and aarch64 binaries. Speed up the CI by using those
binaries.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
An example warning output is:
error: first doc comment paragraph is too long
--> virtio-devices/src/lib.rs:158:1
|
158 | / /// Convert an absolute address into an address space (GuestMemory)
159 | | /// to a host pointer and verify that the provided size define a valid
160 | | /// range within a single memory region.
161 | | /// Return None if it is out of bounds or if addr+size overlaps a single region.
| |_
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#too_long_first_doc_paragraph
= note: `-D clippy::too-long-first-doc-paragraph` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::too_long_first_doc_paragraph)]`
Signed-off-by: Bo Chen <chen.bo@intel.com>
In case of SEV-SNP guests on MSHV, any host component including VMM
trying to access any guest memory region, needs to call gain_access API
to acquire access to that particular memory region. This applies to all
the virtqueue buffers as well which the VMM is using to use to perform
DMA into the guest and in order to facilitate device emulation.
While creating various virtqueues we are already using access platform
hook to translate the addresses but currently we are missing the size
arguments which would be required by the SevSnpAccessPlatformProxy to
gain access to the memory of these queues.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This would be used in case of SEV-SNP guest because we need to accquire
access to these virtqueue segments before accessing them.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
DeviceManager::add_virtio_console_device used to create the console
resize pipe and assign it to self.console_resize_pipe, but when this
was changed to use console_info, that was deleted without replacement.
This meant that, even though the console resize pipe was created by
pre_create_console_devices, the DeviceManager never found out about
it, so console resize didn't work (at least for pty consoles).
To fix this, the console resize pipe needs to be passed to the Vm
initializer, which is already supported, it was just previously not
used for new VMs.
Since DeviceManager already stores the console resize pipe in an Arc,
and Vmm also needs a copy of it, the sensible thing to do is change
DeviceManager::new to take Arc, and then we don't need to dup the file
descriptor, which could fail.
Fixes: 52eebaf6 ("vmm: refactor DeviceManager to use console_info")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
When the rate limit was reached it was possible for the notification to
the guest to be lost since the logic to handle the notification was
tightly coupled with processing the queue. The notification would
eventually be triggered when the rate limit pool was refilled but this
could add significant latency.
Address this by refactoring the code to separate processing queue and
signalling - the processing of the queue is suspended when the rate
limit is reached but the signalling will still be attempted if needed
(i.e. VIRTIO_F_EVENT_IDX is still considered.)
Signed-off-by: wuxinyue <wuxinyue.wxy@antgroup.com>
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Previously we didn't handle extended guest requests at all and always
returned an error. This lead to issues with some guests that expected
extended requests to succeed. Instead, handle extended requests like
normal requests and write zeros to the extended area to signal to the
guest that we don't want to supply any additional certificate data.
Signed-off-by: Tom Dohrmann <erbse.13@gmx.de>
In case of ARM64 kvm_regs are considered as StandardRegister which is no
longer required since we have defined architecture independent
StandardRegisters in hypervisor crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Use the StandardRegisters defined in the hypervisor crate instead of
re-defining it from MSHV/KVM crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This change is along the lines of x86 platform which defined similar
convinence macros to access the StandardRegisters.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
With commit 1e967697c ("vmm: pass AccessPlatform implementation for
SEV-SNP guest"), we started performing one additional ioctl to gain
access to the guest memory before accessing those regions inside
virtio-device emulation code path. This additional ioctl is not part of
the current seccomp filter, which is causing the SevSnp guest to crash
in this scenario with seccomp violation.
Fixes: 1e967697c ("vmm: pass AccessPlatform implementation for SEV-SNP
guest")
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This fixes a issue of running vm compiled in debug with Rust
1.80.0 or later, where this check was introduced.
Signed-off-by: Wenyu Huang <huangwenyuu@outlook.com>
Implement get_guest_debug_hw_bps() for mshv and simply return 0 for now.
This unblocks the usage of the crashdump feature with mshv. If left
unimplemented, Cloud Hypervisor built with mshv and guest_debug features
crashes immediately upon start due to unimplemented!() macro.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
With this we are removing the CloudHypervisor definition of
StandardRegisters instead using an enum which contains different
variants of StandardRegisters coming from their bindigs crate.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently we are redefining StandardRegisters instead of using the ones
coming from bindings. With this we can remove the unnecessary
construction of global structure which contains fields from different
hypervisor dependent structs.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The index is derived from the access offset, so it is controlled by the
guest. Check it before accessing internal data structures.
Since Rust enforces strict bound check even in release builds, the VMM
process will crash if the guest misbehaves. There is no security issue
since the guest can only DoS itself.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Use the IOCTL numbers directly from mshv-ioctls instead of hardcoding
them in the seccomp filters.
Remove seccomp rules for unused ioctls:
MSHV_GET_VERSION_INFO,
MSHV_ASSERT_INTERRUPT.
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
CVM guests can configure GHCB page multiple times during it's
lifetime depending on it's requirement. For example a Linux CVM guest
configures a different GHCB page during compressed kernel boot and sets
up a new one after decompressing the kernel. As a cleanup step, VMM
should unset the previous GHCB page before registering a new one for
a particular vcpu thread.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
DeviceFd::get_device_attr should be marked as unsafe, because it
allows writing to an arbitrary address. I have opened a kvm-ioctls
PR[1] to fix this. The hypervisor crate was using the function
unsafely by passing it addresses of immutable variables. I noticed
this because an optimisation change[2] in Rust 1.80.0 caused the
kvm::aarch64::gic::tests::test_get_set_icc_regs test to start failing
when built in release mode.
To fix this, I've broken up the _access functions into _set and _get
variants, with the _get variant using a pointer to a mutable variable.
This has the side effect of making these functions a bit nicer to use,
because the caller now has no need to use references at all, for
either getting or setting.
[1]: https://github.com/rust-vmm/kvm-ioctls/pull/273
[2]: d2d24e395a
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This fixes the vsock::device::tests::test_virtio_device test with Rust
1.80.0 or later, where this check was introduced.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Passing AccessPlatform trait to virtio-device for requesting
restricting page access during IO for SEV-SNP guest.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Implement AccessPlatform for SEV-SNP guest to access
restricted page using IO. VMM calls MSHV api to get access
of the pages, MSHV requests guest to release the access.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add a structure to hold the reference of the Vm trait
from Hypervisor crate to access of restricted page
from SEV-SNP guest.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Pvmemcontrol provides a way for the guest to control its physical memory
properties, and enables optimizations and security features. For
example, the guest can provide information to the host where parts of a
hugepage may be unbacked, or sensitive data may not be swapped out, etc.
Pvmemcontrol allows guests to manipulate its gPTE entries in the SLAT,
and also some other properties of the memory map the back's host memory.
This is achieved by using the KVM_CAP_SYNC_MMU capability. When this
capability is available, the changes in the backing of the memory region
on the host are automatically reflected into the guest. For example, an
mmap() or madvise() that affects the region will be made visible
immediately.
There are two components of the implementation: the guest Linux driver
and Virtual Machine Monitor (VMM) device. A guest-allocated shared
buffer is negotiated per-cpu through a few PCI MMIO registers, the VMM
device assigns a unique command for each per-cpu buffer. The guest
writes its pvmemcontrol request in the per-cpu buffer, then writes the
corresponding command into the command register, calling into the VMM
device to perform the pvmemcontrol request.
The synchronous per-cpu shared buffer approach avoids the kick and busy
waiting that the guest would have to do with virtio virtqueue transport.
The Cloud Hypervisor component can be enabled with --pvmemcontrol.
Co-developed-by: Stanko Novakovic <stanko@google.com>
Co-developed-by: Pasha Tatashin <tatashin@google.com>
Signed-off-by: Yuanchu Xie <yuanchu@google.com>
BusDevice trait functions currently holds a mutable reference to self,
and exclusive access is guaranteed by taking a Mutex when dispatched by
the Bus object. However, this prevents individual devices from serving
accesses that do not require an mutable reference or is better served
with different synchronization primitives. We switch Bus to dispatch via
BusDeviceSync, which holds a shared reference, and delegate locking to
the BusDeviceSync trait implementation for Mutex<BusDevice>.
Other changes are made to make use of the dyn BusDeviceSync
trait object.
Signed-off-by: Yuanchu Xie <yuanchu@google.com>
After moving landlock config to VMConfig, there is no need to start
destination VM with landlock cmdline options in
test_live_migration_with_landlock test.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
There were some scenarios where we are not clearing SW_EXIT_INFO1 to
indicate that there were no error while handling the GHCB exit.
Recently, new Linux guests got stricter with checking the value of
SW_EXIT_INFO1 after coming back from VMGEXIT and started crashing. Fix
this behavior by clearing out SW_EXIT_INFO1 in case of no error.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Support event idx feature for virtio-blk device.
This feature could improve disk IO performance by suppressing
notifications from guest to host and interrupts from host to
guest, which has been already supported in virtio-net and
vhost-user devices.
To achieve this, virtqueue's event-idx-related API is
leveraged for avail_event field update and needs_notification
check.
Fixes: #6580
Signed-off-by: wuxinyue <wuxinyue.wxy@antgroup.com>
HV APIC(i.e., synthetic APIC controller exposed by Microsoft Hypervisor)
does not support one-shot operation using a TSC deadline value. Due to
which we see the following backtrace inside the guest when running with
hypervisor-fw/OVMF:
[ 0.560765] unchecked MSR access error: WRMSR to 0x832 (tried to
write 0x00000000000400ec) at rIP: 0xffffffff8f473594
(native_write_msr+0x4/0x30)
[ 0.560765] Call Trace:
[ 0.560765] ? native_apic_msr_write+0x2b/0x30
[ 0.560765] __setup_APIC_LVTT+0xbc/0xe0
[ 0.560765] lapic_timer_set_oneshot+0x27/0x30
[ 0.560765] clockevents_switch_state+0xaf/0xf0
[ 0.560765] tick_setup_periodic+0x47/0x90
[ 0.560765] tick_setup_device.isra.0+0x7c/0x110
[ 0.560765] tick_check_new_device+0xce/0xf0
[ 0.560765] clockevents_register_device+0x82/0x170
[ 0.560765] clockevents_config_and_register+0x2f/0x40
[ 0.560765] setup_APIC_timer+0xe1/0xf0
[ 0.560765] setup_boot_APIC_clock+0x5f/0x66
[ 0.560765] native_smp_prepare_cpus+0x1d6/0x286
[ 0.560765] kernel_init_freeable+0xcf/0x255
[ 0.560765] ? rest_init+0xb0/0xb0
[ 0.560765] kernel_init+0xe/0x110
[ 0.560765] ret_from_fork+0x22/0x40
Also, if this feature is exposed guest would not finish booting and get
stuck right before unpacking the root filesystem.
Fixes: 06e8d1c40 ("hypervisor: mshv: fix topology for Intel HW on MSHV")
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This is a preparatory work to make space for ARM64 emulator.
There is no functional change in this commit, just moving code from one
file to another.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
As per VirtIO spec 1.2 section 5.2.6, the `status` field is a byte, not
u32. cloud-hypervisor writes an `u32` to guest memory, which
accidentally zeros out the following 3 bytes, and may corrupt guest OS
internal state.
Signed-off-by: Changyuan Lyu <changyuanl@google.com>
The assertion that only code emulation requires GVA to GPA translation
is wrong.
Allow the caller of `translate` to pass in permission flags directly.
Provide a new method `read_memory_flags` so that we can add the EXECUTE
permission flag where necessary.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This method will return the existing Response if the status is
successful (Status::Ok) otherwise issue a command to abandon the
migration and return the desired error.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This requires stashing the config values in `struct Vmm`. The configs
should be validated before before creating the VMM thread. Refactor the
code and update documentation where necessary.
The place where the rules are applied remain the same.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Add file/dir paths from landlock-rules arguments to ruleset. Invoke
apply_landlock on VmConfig to apply config specific rules to ruleset.
Once done, any threads spawned by vmm thread will be automatically
sandboxed with the ruleset in vmm thread.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Introduce ApplyLandlock trait and add implementations to VmConfig
elements with PathBufs. This trait adds config specific rules to
landlock ruleset.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Users can use this parameter to pass extra paths that 'vmm' and its
child threads can use at runtime. Hotplug is the primary usecase for
this parameter.
In order to hotplug devices that use local files: disks, memory zones,
pmem devices etc, users can use this option to pass the path/s that will
be used during hotplug while starting cloud-hypervisor. Doing this will
allow landlock to add required rules to grant access to these paths when
cloud-hypervisor process starts.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Users can use this cmdline option to enable/disable Landlock based
sandboxing while running cloud-hypervisor.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
landlock syscalls are required by event_monitor, signal_handler,
http-server and vmm threads. Rest of the threads are spawned by the vmm
thread and they automatically inherit the ruleset from the vmm thread.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
We will start receiving GPA intercepts. For our use cases they are
handled the same way as UNMAPPED GPA intercepts.
Put in some logging to distinguish the two cases.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Previously we required the hypervisor to give us a valid instruction
stream. That worked well enough because we never hit any edge conditions
(such as when the instruction stream crosses page boundary).
Now that MSHV can deal with partial or empty instruction stream, we can
remove that requirement.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Original we checked for R and W, but that code path never got executed.
It is now understood that we can only get here when we execute code. Fix
the permission flags.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The default value of IP is zero. If the decoder's state not set
properly, then the guest state is going to be wrong.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
TIOCGWINSZ modifies its argument, so it needs to mutably borrow it.
Unfortunately, ioctl()'s signature is not able to enforce this, and
the write happens in the kernel, so I don't think anything like miri,
valgrind, UBSan, etc. would have been able to catch this.
The UB passing an immutable reference caused resulted, for me, in
get_win_size() returning (0, 0) since LLVM commit
9a09c737a052 ("[BasicAA] Make isNotCapturedBeforeOrAt() check for
calls more precise (#69931)").
I've had a look through the other ioctl() calls in Cloud Hypervisor,
and I don't think any others have the same problem.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Comparing RAX with RIP makes no logical sense other than RIP happens to
be the correct value. Use `target_rax` instead.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
cargo-cyclonedx rejects the current license key in Cargo.toml, because
it doesn't comply with the SPDX standard:
Package cloud-hypervisor has an invalid license expression (LICENSE-APACHE & LICENSE-BSD-3-Clause), using as named license: Invalid Lax SPDX expression: unknown term
Fix by using the names from the SPDX list:
https://spdx.github.io/license-list-data/
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
In 42e9632c53 a fix was made to address a
typo in the taplo configuration file. Fixing this typo indicated that
many Cargo.toml files were no longer adhering to the formatting rules.
Fix the formatting by running `taplo fmt`.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Killing CLH by SIGKILL will cause inaccurate code coverage
information. This patch changes the signal to SIGTERM.
Fixes: #6507
Signed-off-by: Songqian Li <sionli@tencent.com>
Since vdpa device does not support pause/resume [1], it does not make
sense to restore on paused state.
[1] 099cdd2af8
Signed-off-by: Bo Chen <chen.bo@intel.com>
Impose a limit on the maximum nesting of file formats that can open more
files. For example, a qcow2 file can have a backing file, which could be
another qcow2 file with a backing file (or even the same file as the
original), potentially causing unbounded recursion.
This commit is based on crosvm implementation:
eb1640e301Fixes: #6472
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
While checking if the console device is a tty use the cloned fd instead
of libc::STDOUT_FILENO.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Console devices are created after vm_config is received and the created
devices are passed Vm during vm_receive_state.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
During vm_shutdown or vm_snapshot, all the console devices will be
closed. When this happens stdout (FD #2) will also be closed as the
console device using these FD is closed. If the VM were to be started
later, FD#2 can be assigned to a different file. But
pre_create_console_devices looks for FD#2 while opening tty device,
which could point to any file.
To avoid this problem, the STDOUT FD is duplicated when being
assigned to a console device. Even if the console devices were to be
closed, the duplicated FD will be closed and FD#2 will continue to
point to STDOUT.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
While adding console devices, DeviceManager will now use the FDs in
console_info instead of creating them.
To reduce the size of this commit, I marked some variables are unused
with '_' prefix. All those variables are cleaned up in next commit.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Use pre_create_console_devices method to create and populate console
device FDs into console_info in Vmm Object.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
With this change all the information to manage console devices is now
available within Vmm Object.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Introduce ConsoleInfo struct. This struct will be used to store FDs of
console devices created in pre_create_console_devices and passed to
vm_boot.
Move set_raw_mode, create_pty methods to console_devices.rs to
consolidate console management methods into a single module.
Lastly, copy the logic to create and configure console devices into
pre_create_console_devices method.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
If the KVM version is older than v6.6, KVM_GET_SUPPORTED_CPUID will omit
the L1 cache information in CPUID function 0x8000_0005. Fall back to
the host L1 cache information if it is omitted by KVM.
Signed-off-by: Sean Banko <sbanko@crusoe.ai>
Currently by default each core is allocated it's own socket. Basically
it is n socket 1 core 1 thread/core kind of a structure as witnessed
from within the guest.
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 8
NUMA node(s): 1
This is not a good default topology because resources are distributed
across multiple sockets. For example, a Linux guest with multi socket
configuration will have to calibrate TSC per socket due to which it
might observe a higher amount of boot time than usual.
A better idea for default topology would be 1 socket n core 1
thread/core which ensure better resource locality.
After this change topology would change to:
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Fixes: #6497
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Populating these cpuid with default values so that
CLH can patch with topology information. Otherwise it gets
skipped while setting some topology information.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Rather than calling cpuid and then updating the APIC ID field - use the
existing common CPUID data which already includes CPUID data for eax=1
(aka function = 1). This removes the need to call cpuid per vCPU thread
created. This has a positive impact on boot time with multiple vCPUs as
the cpuid instruction is serialising.
Fixes: #5646
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The cloud-hypervisor package does not have a lib target, so it is
invalid. Cargo just ignores that line.
Dropping it removes a warning.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The original code gave an owned fd to UnixListener. That made the same
fd wrapped into two owned files.
When the files were dropped, the same fd would be closed more than once.
A newly introduced check in Rust's stdlib caught that error.
A newly cloned fd should be given to UnixListener.
Fixes: #6485
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Whenever the file descriptor table is full, Linux expands it by doubling
it's size.
The filesystem code that does this uses RCU synchronization to ensure
all pre-existing RCU read-side critical sections have completed. The
latency induced by this synchronization is a big part of the total time
required to restore a snapshot.
The kernel has an optimization in code, where it doesn't call
synchronize_rcu() if there is only one thread in the process. We can
take advantage of this optimization by expanding the descriptor table at
the application start, when it has only one thread.
This commit tries to expand the table to 4096 entries, this way we avoid
any expansion that could take place later.
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Certain VMEXITs can only happen for x86 guests, thus reduce the scope to
x86_64 at the compilation stage.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
For MSHV we always create frozen partition, so we
resume the VM during boot. Also during pause and resume
VM events we call hypervisor specific API.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Create a partition frozen always, then unfreeze the partition
during boot phase or resume phase. We also freeze the
partition during pause event. Time is freeze during the
time between freeze and unfreeze.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add Pause/Resume functions for VM trait. For KVM it
will be empty implementations. For MSHV it needs to freeze
and unfreeze the partition.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Currently, the watchdog configuration in systemd is reloaded by
reboot. We can cancel this reboot operation by restarting the
systemd daemon process.
Signed-off-by: Songqian Li <sionli@tencent.com>
tests_snapshot_restore* have been earlier removed from common_parallel
due to the falkiness they add testsuite. Running them sequentially would
eliminate the flakiness. Hence, add the tests back to testsuite but into
common_sequential module.
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
test_snapshot_restore_* tests often have transient failures and add to
overall flakiness of the integration testsuite. Hence, remove them from
common_parallel. However, these tests need to be added back to
common_sequential
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
VM is created with FDs explicitly passed to CH via --net parameter
and snapshotted. New net FDs are passed in turn during restore.
Boilerplate code from _test_snapshot_restore().
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
Add a section about restoring VM with new Net FDs explicitly passed to
ch-remote via 'net_fds' parameter
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
Enable restore command the ability to send file descriptors along with
HTTP request. This is useful when restoring a VM with explicit FDs
passed to NetConfig(s).
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
Consume FDs passed via SCM_RIGHTs to VmRestore API and assign them
appropriately to RestoredNetConfig's fds field.
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
'NetConfig' FDs, when explicitly passed via SCM_RIGHTS during VM
creation, are marked as invalid during snapshot. See: #6332.
So, Restore should support input for the new net FDs. This patch adds
new field 'net_fds' to 'RestoreConfig'. The FDs passed using this new
field are replaced into the 'fds' field of NetConfig appropriately.
The 'validate()' function ensures all net devices from 'VmConfig' backed
by FDs have a corresponding 'RestoreNetConfig' with a matched 'id' and
expected number of FDs.
The unit tests provide different inputs to parse and validate functions
to make sure parsing and error handling is as per expectation.
Fixes#6286
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
Co-authored-by: Bo Chen <chen.bo@intel.com>
The list is gathered from going through various code paths in the code
base.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Per the KVM API document, that capability is only valid with in-kernel
irqchip that handles MSIs.
Through out the code base, there is no call to KVM_IOCTL_SIGNAL_MSI.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The compiler is now able to warn if an invalid attribute (e.g like a
feature) is not available.
See https://blog.rust-lang.org/2024/05/06/check-cfg.html for more
details.
Add build.rs files in the crates that use #cfg(fuzzing) to add fuzzing
to the list of valid cfg attributes.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The ability to control the rustc flags (required for adding new
attributes to the allowed list of #[cfg(..)]) requires bumping the MSRV
to 1.77.0
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Code in this crate is conditional on this feature so it necessary to
expose as a new feature and use that feature as a dependency when the
feature is enabled on the vmm crate.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Enable the use of the vmm crate with the "guest_debug" feature and make
the code that exercises that in the fuzzer unconditional on
"guest_debug" as a feature (as that is not specified as a feature in the
fuzz workspace itself.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The "dhat-heap" feature needs to be enabled inside the vmm crate as a
depenency from the top-level as there is build time check for that
feature inside the vmm crate.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
These tests use relatively large memory allocations and if they are
allowed to run in parallel can result in the container encountering an
OOM situation.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Update the vhost-user-backend crate version used along with related
crates (vhost and virtio-queue.) This requires minor changes to the
types used for the memory in the backends with the use of the
BitmapMmapRegion type for the Bitmap implementation.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Replace the specific Bitmap implementation from the type signature used
for functions that take memory. This allows more flexibility when using
these functions in particular when these functions are used by the
vhost-user-block backend. An updated vhost-user-backend crate requires
extra constraints on the Bitmap implementation used (it must support
BitmapReplace which AtomicBitmap does not.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Replace the specific Bitmap implementation from the type signature used
for functions that take memory. This allows more flexibility when using
these functions in particular when these functions are used by the
vhost-user-net backend. An updated vhost-user-backend crate requires
extra constraints on the Bitmap implementation used (it must support
BitmapReplace which AtomicBitmap does not.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is to resolve the inconsistencies from our openapi specification,
as default values do not make sense for required fields.
Reported-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
Change from default 4G to 2G to reduce the memory usage. This will help
avoid "No disk space left" errors when all snapshot tests run at once.
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
The snapshot_dir is not needed anymore fter the guest is restored.
Hence, remove it. This will potentially avoid transient failures caused
due to insufficient disk space.
Signed-off-by: Purna Pavan Chandra <paekkaladevi@linux.microsoft.com>
The mem_size field is not needed in TestContext. Drop it.
Make sure guest_evvq is read once. Clippy cannot figured out that it was
used.
While at it, add an extra assert for the spurious rxvq event test, too.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Clippy complained one use of the char_array macro.
Instead of fixing that use case, the observation is that the macro was
needed only because MP table types mixed c_char and c_uchar for no
particular reason.
Only use c_uchar in those types, and drop char_array.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The maintainer can then replace the plaintext body (derived from the
tag) with a markdown version and then publish.
As part of 6c4144e943 creating a draft
release was accidentally dropped.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
As MSHV also implements set/get_clock data, this patch
removes the KVM feature guard and make it x86_64 only and
both for KVM and MSHV.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The contents of this crate may change and cause conflicts - re-exporting
the contents is unnecessary.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is required as the VcpuFd::run and VcpuFd::set_immediate_exit
methods now take a &mut self. I explored alternative solutions:
1. Using RefCell for runtime interior mutability - the Vcpu trait is
Sync and it's not possible to use RefCell with types that are Sync.
2. Using UnsafeCell - the mutable reference nature of ::run and and
::set_kvm_immediate_exit was added for a reason so it unwise to
bypass this.
3. Adjusting the trait interface to expose the &mut self - this requires
an Arc<Mutex<>> around the hypervisor::vcpu::Vcpu object and was very
intrusive.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The members for {Io, Mmio}{Read, Write} are unused as instead exits of
those types are handled through the VmOps interface. Removing these is
also a prerequisite due to changes in the mutability of the
VcpuFd::run() method.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
On Microsoft Hypervisor, we need to save/restore five
VP state components which are as follows:
1. Local APIC
2. Xsave
3. Synthetic Message Page
4. Synthetic Event Flags Page
5. Synthetic Timers
In the MSHV crate we created a single struct for all the
components and API to get/set the states.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
When using multiple PCI segments, the 32-bit and 64-bit mmio
aperture is split equally between each segment. Add an option
to configure the 'weight'. For example, a PCI segment with a
`mmio32_aperture_weight` of 2 will be allocated twice as much
32-bit mmio space as a normal PCI segment.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
In accordance with reuse requirements:
- Place each license file in the LICENSES/ directory
- Add missing SPDX-License-Identifier to files.
- Add .reuse/dep5 to bulk-license files
Fixes: #5887
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Replace actions: actions-rs/toolchain, upload-release-asset.
actions-rs/toolchain is unmaintained, for cross-compilation replace it
with houseabsolute/actions-rust-cross
Action upload-release-asset is unmaintained and has been archived since
Mar 4, 2021. An issue with upload-release-asset is that it creates
separate release for each job of matrix strategy. Thus, to solve these
issues, replace unmaintained upload-release-asset with upload-artifact
See: #5929
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
This removes the requirement to ensure that we land PRs that update the
Dockerfile (and the appropraite dev_cli.sh change) in a specific time
frame.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Assigning KVM_IRQFD (when unmasking a guest IRQ) after
KVM_SET_GSI_ROUTING can avoid kernel panic on the guest that is not
patched with commit a80ced6ea514 (KVM: SVM: fix panic on out-of-bounds
guest IRQ) on AMD systems.
Meanwhile, it is required to deassign KVM_IRQFD (when masking a guest
IRQ) before KVM_SET_GSI_ROUTING (see #3827).
Fixes: #6353
Signed-off-by: Yi Wang <foxywang@tencent.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
The caller shouldn't pass in an &str that's too long. This is a
precaution if something goes wrong in the caller.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Properly detach a device from a domain if that device is already
attached to another domain on an attach request (following section
5.13.6.3.2 of the virtio-iommu spec). Resolves nested virtualization
reboot.
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
Ensures that any endpoints already attached to the domain are properly
mapped to a new endpoint on said endpoint's attach request. This is done
by search for all previous mappings in the domain and then issuing map
requests for the newly attached endpoint.
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
Add infrastructure to lookup the host address for mmio regions on
external dma mapping requests. This specifically resolves vfio
passthrough for virtio-iommu, allowing for nested virtualization to pass
external devices through.
Fixes#6110
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
VfioUserDmaMapping is already in the pci crate, this moves
VfioDmaMapping to match the behavior. This is a necessary change to
allow the VfioDmaMapping trait to have access to MmioRegion memory
without creating a circular dependency. The VfioDmaMapping trait
needs to have access to mmio regions to map external devices over
mmio (a follow-up commit).
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
The memory region that is associated with the hotpluggable part of
a virtio-mem zone isn't backed by the file specified in the
MemoryZoneConfig. The file is used only for the fixed part of the
zone. When you try to restore a snapshot with virtio-mem, the
backing file is used for all its regions. This results in the
following error:
VmRestore(MemoryManager(GuestMemoryRegion(MappingPastEof)))
This patch sets backing_file only for the fixed part of a virtio-mem
zone.
Fixes: #6337
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
HV_PAGE_SIZE is defined as a usize in mshv_bindings.
Remove the redefinition, and perform casts where necessary.
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
Add IOCTL number for generic hypercall ioctl (MSHV_ROOT_HVCALL).
Update IOCTL numbers for set/get vp state.
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
This reverts commit fcf229a33a.
The virtio-queue version needs to stay the same as the rest of the
source tree.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The 'NetConfig' may contain FDs which can't be serialized correctly, as
FDs can only be donated from another process via a Unix domain socket
with `SCM_RIGHTS`. To avoid false use of the serialized FDs, this patch
explicitly set 'NetConfig' FDs as invalid for (de)serialization.
See: #6286
Signed-off-by: Bo Chen <chen.bo@intel.com>
Prefix field with an _ to indicate that this is intentionally unused.
This resolved a nightly compiler check issue due to the unusued field.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
When restoring a VM, the VirtioPciCfgCapInfo struct is not properly
initialized. All fields are 0, including the offset where the
capabibility starts. Hence, when you read a PCI configuration register
in the range [0..length(VirtioPciCfgCap)] you get the value 0 instead of
the actual register contents.
Linux rescans the whole PCI bus when adding a new device. It reads the
values vendor_id and device_id for every device. Because these are
stored at offset 0 in pci configuration space, their value is 0 for
existing devices. As such, Linux considers that the devices have been
unplugged and it removes them from the system.
Fixes: #6265
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Current MSHV emulator only works for x86 instruction decoder. So, let's
restrict its usage to x86.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
According to the virtio iommu spec (section 5.13.6.6), all mappings
within the entire range from virt_start to virt_end in an unmap
request must be removed. This change adds this functionality,
iterating through all mappings that fall within an unmap request
for that domain and removing them.
Signed-off-by: Andrew Carp <acarp@crusoeenergy.com>
Current instruction emulator can only decode x86 instructions. Thus,
restrict the exit handling for just x86 guests.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
APIC controller is only available on x86 machine. ARM uses a different
interrupt controller so those exit messages won't happen for ARM guests.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
MSR and CPUID are limited to x86 architecture so, reduce the visbility
of these two members inside struct MshvVcpu to just x86 architecture.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
error: the item `io` is imported redundantly
Error: --> devices/src/legacy/uart_pl011.rs:468:9
|
467 | use super::*;
| -------- the item `io` is already imported here
468 | use std::io;
| ^^^^^^^
error: the item `Arc` is imported redundantly
Error: --> devices/src/legacy/uart_pl011.rs:469:21
|
467 | use super::*;
| -------- the item `Arc` is already imported here
468 | use std::io;
469 | use std::sync::{Arc, Mutex};
| ^^^
error: could not compile `devices` (lib test) due to 8 previous errors
Error: warning: build failed, waiting for other jobs to finish...
Error: The process 'cross' failed with exit code 101
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
error: the item `read_le_u32` is imported redundantly
Error: --> devices/src/legacy/gpio_pl061.rs:342:17
|
341 | use super::*;
| -------- the item `read_le_u32` is already imported here
342 | use crate::{read_le_u32, write_le_u32};
| ^^^^^^^^^^^
|
= note: `-D unused-imports` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_imports)]`
error: the item `write_le_u32` is imported redundantly
Error: --> devices/src/legacy/gpio_pl061.rs:342:30
|
341 | use super::*;
| -------- the item `write_le_u32` is already imported here
342 | use crate::{read_le_u32, write_le_u32};
| ^^^^^^^^^^^^
error: the item `Arc` is imported redundantly
Error: --> devices/src/legacy/gpio_pl061.rs:343:9
|
341 | use super::*;
| -------- the item `Arc` is already imported here
342 | use crate::{read_le_u32, write_le_u32};
343 | use std::sync::Arc;
| ^^^^^^^^^^^^^^
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
clippy was flagging this up as a mixture of mixed attributes but that
attribute is no longer necessary
warning: item has both inner and outer attributes
--> hypervisor/src/arch/x86/emulator/mod.rs:769:1
|
769 | / #[cfg(test)]
770 | | mod tests {
771 | | #![allow(unused_mut)]
| |_________________________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#mixed_attributes_style
= note: `#[warn(clippy::mixed_attributes_style)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
error: the item `kvm_bindings` is imported redundantly
Error: --> hypervisor/src/kvm/aarch64/gic/mod.rs:9:18
|
9 | use crate::kvm::{kvm_bindings, KvmVm};
| ^^^^^^^^^^^^ the item `kvm_bindings` is already defined by prelude
|
= note: `-D unused-imports` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_imports)]`
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: assigning the result of `Clone::clone()` may be inefficient
--> vmm/src/device_manager.rs:4188:17
|
4188 | id = child_id.clone();
| ^^^^^^^^^^^^^^^^^^^^^ help: use `clone_from()`: `id.clone_from(child_id)`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#assigning_clones
= note: `#[warn(clippy::assigning_clones)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: `devices` (lib) generated 1 warning (run `cargo clippy --fix --lib -p devices` to apply 1 suggestion)
warning: assigning the result of `Clone::clone()` may be inefficient
--> virtio-devices/src/transport/pci_device.rs:1073:9
|
1073 | self.bar_regions = bars.clone();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use `clone_from()`: `self.bar_regions.clone_from(&bars)`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#assigning_clones
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: assigning the result of `Clone::clone()` may be inefficient
--> devices/src/pvpanic.rs:213:9
|
213 | self.bar_regions = bars.clone();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use `clone_from()`: `self.bar_regions.clone_from(&bars)`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#assigning_clones
= note: `#[warn(clippy::assigning_clones)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Model Specific Registers (MSRs) are usually available on x86
architecture. So, let's reduce the visibility of MSRs to just x86
architecture.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
A deadlock can happen from the destination VM of live upgrade or
migration due to waiting on paused device worker threads. For example,
when a serialization error happens after the `DeviceManager` struct is
restored (where all virtio device worker threads are spawned but in
paused/parked state), a deadlock will happen from
`DeviceManager::drop()`, as it blocks for waiting worker threads to
join.
This patch ensures that we wake up all device (mostly virtio) worker
threads before we block for them to join.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Current version can't boot a SNP guest while using
more than one VCPU. It turns out that there is an
issue in the Snp AP creation process. We should be writing
to Software exitinfo1 instead of exitinfo2. This patch fixes
the issue and we can boot multi vcpu SNP guest.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Currently, we only support injecting NMI for KVM guests but we can do
the same for MSHV guests as well to have feature parity.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Microsoft Hypervisor raises this request to change the guest page
visibility in case of SEV-SNP guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
Kernel can panic when get a unknown nmi when set unknown_nmi_panic,
we can use this feature to test if nmi injected to guest.
Signed-off-by: Yi Wang <foxywang@tencent.com>
When running on the merge group this workflow is run twice - once for
the create event (merge queue creates a new branch) and once for the
merge_group event. Unfortunately the second event would cause the first
to be cancelled - unfortunately sometimes that second event is the
create event where the job in the workflow only runs if it is also a
tag.
By creating distinct concurrency groups for each event type then the
cross cancellation can be avoided.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The workers share a common public IP address and often GitHub will
reject attempts to access the API due to exceeding the anonymous rate
limit threshold.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
When a bare-metal worker is canceled, its workspace can be left with
files owned by the root user as a result of running tests from our
container. This patch add a step to fix workspace permissions for such
case before checking out code.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This commit ensures that the HttpApi thread flushes all the responses
before the application shuts down. Without this step, in case of a
VmmShutdown request the application might terminate before the
thread sends a response.
Fixes: #6247
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Because of double quotes the current value is passed
as a single argument with a space in it to cargo.
This commit changes it to an array so each element
is passed as a different arguments.
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
On platforms where PCIe P2P is supported, inject a PCI capability into
NVIDIA GPU to indicate support.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
The DCO tool doesn't understand merge_groups but we still need to have a
valid status check to allow the merge group to proceed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
And clean up some of the whitespace formatting so that the "name" and
"on" are grouped away from the "jobs".
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Most of our CI workers are now running form GitHub actions, so we are
ready to disable Jenkins CI workers.
See: #6231
Signed-off-by: Bo Chen <chen.bo@intel.com>
cloud-hypervisor, ch-remote, vhost-user-block and vhost-user-net all
need at least one argument to do anything useful, so printing command
help is helpful when they are run without arguments or a subcommand.
Use clap::Command::arg_required_else_help(true) to do this.
Signed-off-by: Chris Webb <chris@arachsys.com>
ch-remote crashes when run with --api-socket but no subcommand:
$ target/release/ch-remote --api-socket /tmp/api
thread 'main' panicked at src/bin/ch-remote.rs:509:14:
internal error: entered unreachable code
Use clap::Command::subcommand_required(true) to yield a more friendly
error in this case.
Signed-off-by: Chris Webb <chris@arachsys.com>
Host data that is passed to the hypervisor. Then
the firmware includes the data in the attestation report.
The data might include any key or secret that the SevSnp guest
might need later.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The host data provided at launch. Data is passed
to the hypervisor during the completion of the
isolated import.
Host Data provided by the hypervisor during guest launch.
The firmware includes this value in all attestation
reports for the guest.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
To reduce issues caused by flaky tests split the musl and glibc jobs
into separate jobs. This means fewer jobs will need to be restarted for
flaky tests. This will also increase CI throughput since the musl builds
account for ~40% of the total CI time when run together with glibc.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Since we don't register ioevents in case of SEV-SNP guests. Thus, we
should not unregister it as well.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
With the nightly toolchain (2024-02-18) cargo check will flag up
redundant imports either because they are pulled in by the prelude on
earlier match.
Remove those redundant imports.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
When a guest running on a terminal reboots, the sigwinch_listener
subprocess exits and a new one restarts. The parent never wait()s
for children, so the old subprocess remains as a zombie. With further
reboots, more and more zombies build up.
As there are no other children for which we want the exit status,
the easiest fix is to take advantage of the implicit reaping specified
by POSIX when we set the disposition of SIGCHLD to SIG_IGN.
For this to work, we also need to set the correct default exit signal
of SIGCHLD when using clone3() CLONE_CLEAR_SIGHAND. Unlike the fallback
fork() path, clone_args::default() initialises the exit signal to zero,
which results in a child with non-standard reaping behaviour.
Signed-off-by: Chris Webb <chris@arachsys.com>
Allow cloud-hypervisor to direct boot the bzImage kernel format using
the regular 32 bit entry point. This can share the memory and vcpu
setup with the regular PVH boot code, but requires the setup of the
'zero page'.
Signed-off-by: Stefan Nuernberger <stefan.nuernberger@cyberus-technology.de>
For SevSnp guest IO events are handled by GHCB protocol.
While we get the notification we have to notify via eventfd.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2024-02-16 11:28:32 -08:00
373 changed files with 45960 additions and 25941 deletions
You will need to `chmod +x .git/hooks/pre-commit` to have it run on every
@@ -36,7 +58,7 @@ commit you make.
## Certificate of Origin
In order to get a clear contribution chain of trust we use the [signed-off-by language](https://01.org/community/signed-process)
In order to get a clear contribution chain of trust we use the [signed-off-by language](https://web.archive.org/web/20230406041855/https://01.org/community/signed-process)
used by the Linux kernel project.
## Patch format
@@ -79,12 +101,12 @@ you want to merge your changes to `cloud-hypervisor`:
1. Fork the [cloud-hypervisor](https://github.com/cloud-hypervisor/cloud-hypervisor) project
into your github organization.
2. Within your fork, create a branch for your contribution.
3. [Create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/)
1. Within your fork, create a branch for your contribution.
1. [Create a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/)
against the main branch of the Cloud Hypervisor repository.
4. To update your pull request amend existing commits whenever applicable and
1. To update your pull request amend existing commits whenever applicable and
then push the new changes to your pull request branch.
5. Once the pull request is approved it can be integrated.
1. Once the pull request is approved it can be integrated.
sh 'scripts/dev_cli.sh tests --metrics -- -- --report-file /root/workloads/metrics.json'
}
}
stage('Upload metrics report') {
steps {
sh 'curl -X PUT https://cloud-hypervisor-metrics.azurewebsites.net/api/publishmetrics -H "x-functions-key: $METRICS_PUBLISH_KEY" -T ~/workloads/metrics.json'
}
}
}
}
stage('Worker build - Rate Limiter') {
agent { node { label 'focal-metrics' } }
when {
branch 'main'
beforeAgent true
expression {
return runWorkers
}
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Run rate-limiter integration tests') {
options {
timeout(time: 10, unit: 'MINUTES')
}
steps {
sh 'scripts/dev_cli.sh tests --integration-rate-limiter'
The following scenarios are supported by Cloud Hypervisor to bootstrap a VM, i.e.,
to load a payload/bootitem(s):
- Provide firmware
- Provide kernel \[+ cmdline\]\ [+ initrd\]
Please note that our Cloud Hypervisor firmware (`hypervisor-fw`) has a Xen PVH
boot entry, therefore it can also be booted via the `--kernel` parameter, as
seen in some examples.
### Custom Kernel and Disk Image
#### Building your Kernel
Cloud Hypervisor also supports direct kernel boot. For x86-64, a `vmlinux` ELF kernel (compiled with PVH support) is needed. In order to support development there is a custom branch; however provided the required options are enabled any recent kernel will suffice.
Cloud Hypervisor also supports direct kernel boot. For x86-64, a `vmlinux` ELF kernel (compiled with PVH support) or a regular bzImage are supported. In order to support development there is a custom branch; however provided the required options are enabled any recent kernel will suffice.
// Introduce trait InterruptController to uniform the interrupt
// service provided for devices.
// Device manager uses this trait without caring whether it is a
// IOAPIC (X86) or GIC (Arm).
// IOAPIC (X86), GIC (Arm) or AIA (RISC-V).
pubtraitInterruptController: Send{
fnservice_irq(&mutself,irq: usize)-> Result<()>;
#[cfg(target_arch = "x86_64")]
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
