Drop the borrowed iovec AsyncIo entry points now that all callers use
owned operations. Rename the transitional owned batch and completion
methods to the final trait names and remove the borrowed submission
helpers from the queue wrappers.
This removes a bunch of known safety foot-guns so future-us don't
accidentally use them.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Dylan Reid <dgreid@fb.com>
Some block devices (ZFS volume) may require BLKDISCARD and BLKZEROOUT
ioctls for discard and write_zeroes operations respectively.
There is no good way to probe whether fallocate is supported on a block
device. Arguably, punch_hole and write_zeroes are rare. Instead of
having a complex scheme for the IO uring backend, we force it to always
use ioctls. The code can be changed if the synchronized ioctls become a
performance issue.
Changes:
- Detect block devices at construction time
- Use BLKDISCARD ioctl for punch_hole (discard) on block devices
- Use BLKZEROOUT ioctl for write_zeroes on block devices
- Add BLKDISCARD/BLKZEROOUT to VirtioBlock seccomp whitelist
- Keep fallocate() path for regular files (no behavior change)
- Consolidate some helper functions to the new sparse module
Signed-off-by: Wei Liu <liuwe@microsoft.com>
probe_sparse_support() and DiskTopology::is_block_device() each carry
their own copy of the same fstat()+S_IFMT dance to ask "is this fd a
block device?". Hoist a single pub helper
pub(crate) fn is_block_device(fd: RawFd) -> bool
into block::lib and route both call sites through it. Drop the
MaybeUninit gymnastics in favour of mem::zeroed() since libc::stat is
POD.
Drop DiskTopology::is_block_device since it is now just a one line
wrapper around the new helper function.
Pure refactor in preparation for the BLKDISCARD/BLKZEROOUT support,
which needs the same probe in three more backends.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The bounce buffer for an unaligned descriptor was allocated in
execute_async and leaked on error paths, even though, for the sync case
the kernel already had a pointer to the buffer.
Clean this up by moving ownership of the buffer to the AlignedOperation
type. To make it actually safe, stop stashing a guest memory pointer for
the duration of the op. Instead, save the guest address and pass guest
memory back to the complete function.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Delete QcowDiskSync and QcowDiskAsync wrapper structs along with
their DiskFile trait impls. Only the AsyncIo worker structs
QcowSync and QcowAsync remain. Reduce module visibility of
qcow_sync and qcow_async to pub(crate).
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce QcowDisk, a unified DiskFile implementation for QCOW2
disk images that handles backend selection at runtime via a
use_io_uring flag, matching the pattern used by FixedVhdDisk.
The wrapper delegates to QcowSync or QcowAsync based on the flag
and includes a compile time guard that returns an error when
io_uring is requested but the feature is not enabled.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce FixedVhdDisk as a single DiskFile wrapper around FixedVhd.
It accepts a use_io_uring flag at construction time and dispatches to
FixedVhdSync or FixedVhdAsync inside create_async_io() accordingly.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Update open_raw to construct RawDisk instead of choosing between
RawFileDisk, RawFileDiskSync and RawFileDiskAio. The backend decision
is now made inside RawDisk::create_async_io.
Remove the DiskFile wrapper structs from raw_sync.rs, raw_async.rs
and raw_async_aio.rs. Only the AsyncIo worker structs remain in those
files. Reduce their module visibility to pub(crate).
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce the unified DiskFile wrapper for raw disk images. The
RawBackend enum selects between sync, io_uring, and AIO backends
at construction time.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce block::factory with open_disk() as the single entry point
for opening disk images. It handles file opening, format detection,
async I/O probing, and backend construction.
Per format helpers (open_fixed_vhd, open_raw, open_qcow2, open_vhdx)
prefer io_uring over AIO over synchronous fallback. Warnings only
fire when a backend was eligible but its runtime probe failed, not
when the user intentionally disabled it.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
There is no reason for most of the Request struct to be writable from
anywhere in the codebase. Encapsulate it.
Use getter functions for access outside the request module. Replace the
trivial setter for the writeback field with direct assignment.
No functional change intended.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
The address that is passed from the guest should be treated as
untrusted. Currently an invalid address will panic the VMM. This only
allows the guest to hurt itself, but we shouldn't have the VMM crashing.
Instead let's return an error if possible or invalidate the queue if it
happen during setup.
The data flow from guest to translate_gva/translate_gpa is:
1. Guest writes a raw u64 address into a virtio descriptor in the
shared descriptor table (guest memory).
2. The virtio-queue crate reads this descriptor via read_obj() and
returns the addr field as-is in a GuestAddress — no validation.
3. Device code calls .translate_gva(access_platform, len) on the
GuestAddress.
4. With IOMMU (access_platform is Some): the address is an IOVA that
must be translated to a GPA via the IOMMU mapping table. If the
guest provides an unmapped IOVA, translation returns Err.
Previously, .unwrap() here panicked the VMM.
5. Without IOMMU (access_platform is None): translate_gva is a no-op
(returns self). The raw address flows to GuestMemory::read_obj()
which validates it — out-of-range addresses return
Err(InvalidGuestAddress), so no host memory corruption is possible.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Per queue I/O worker that uses io_uring for asynchronous reads
against fully allocated clusters. The struct holds the shared
metadata, data file, optional backing reader, the io_uring
instance and a synthetic completion list.
Feature gated on io_uring in lib.rs.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce the device level handle for the async QCOW2 backend.
QcowDiskAsync mirrors QcowDiskSync. It parses the image, resolves
the backing chain and wraps QcowMetadata in an Arc for sharing
across virtio queues. No trait impls yet, just the struct,
constructor, Drop and Debug.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
These position independent I/O helpers use pread64/pwrite64 to avoid
races on the shared file position when multiple queues operate on
duplicated file descriptors. Extracting them prepares for reuse by
the upcoming qcow_async backend.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce ExecuteError::UnsupportedFlags to carry both the
request type and the rejected flags value, replacing the
generic ExecuteError::Unsupported at discard and write zeroes
flag validation sites. This provides structured context for
debugging without changing the returned VIRTIO_BLK_S_UNSUPP
status.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The virtio spec v1.2 in 5.2.6.2 requires that the device
MUST return VIRTIO_BLK_S_UNSUPP for write zeroes commands
if any unknown flag is set.
Add an early check that rejects requests with reserved flag
bits set by returning VIRTIO_BLK_S_UNSUPP via the existing
ExecuteError::Unsupported variant.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The virtio spec v1.2 in 5.2.6.2 requires that the device
MUST return VIRTIO_BLK_S_UNSUPP for discard commands if the
unmap flag is set or if any unknown flag is set.
The discard handler was not reading the flags field at all,
silently accepting requests with arbitrary flags. Read and
validate the flags, rejecting any non-zero value with
VIRTIO_BLK_S_UNSUPP.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add a shared helper that returns the logical size in bytes for both
regular files and block devices using an immutable &File reference.
Regular files use metadata().len(). Block devices use the BLKGETSIZE64
ioctl. Any other file type returns an InvalidInput error.
This avoids seek(SeekFrom::End(0)) which requires &mut self and can
return incorrect results for block devices when the file position is
in an unexpected state.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Include the number of segments found in the request payload in
the TooManySegments error variant so the logged message shows
both the actual and maximum values.
Suggested-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce DISCARD_WZ_MAX_PAYLOAD as the precomputed product of
DISCARD_WZ_SEG_SIZE and MAX_DISCARD_WRITE_ZEROES_SEG. Use it in the
DISCARD and WRITE_ZEROES segment count checks instead of repeating
the multiplication inline.
Suggested-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add range validation for DISCARD and WRITE_ZEROES, matching the
existing check in the read/write path. Per virtio spec section
5.2.6.1, a driver must not submit a request which would cause a
read or write beyond capacity. Use checked_add to guard against
overflow, then compare against disk_nsectors.
Without this, requests beyond device capacity pass through to the
host punch_hole/write_zeroes calls, relying on backend specific
behavior rather than returning VIRTIO_BLK_S_IOERR consistently.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Split the data length check into two conditions:
- reject descriptors shorter than one
virtio_blk_discard_write_zeroes segment, and
- reject payloads exceeding MAX_DISCARD_WRITE_ZEROES_SEG segments
Previously only the minimum length was checked and extra segments
were silently dropped.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Composable disk capability traits with DiskFile as a supertrait
bundling DiskSize and Geometry. Optional capabilities are
separate traits: PhysicalSize, DiskFd, SparseCapable, Resizable.
AsyncDiskFile extends DiskFile with async I/O construction.
Empty module with doc comment, trait definitions follow.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Replace magic numeric offsets with mem::offset_of!() referencing the
virtio_blk_discard_write_zeroes struct from the virtio-bindings crate
when reading the sector, num_sectors and flags fields in the DISCARD
and WRITE_ZEROES request handlers.
No functional change.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The write zeroes segment descriptor (struct
virtio_blk_discard_write_zeroes, virtio spec v1.2 section 5.2.6)
includes a flags field with an unmap bit. Per section 5.2.6.2, if
unmap is set, the device MAY deallocate the specified range of
sectors in the device backend storage, as if the discard command
had been sent.
Read the flags field and when the unmap bit is set, use punch_hole
to deallocate the range. Otherwise continue using write_zeroes via
ZERO_RANGE which preserves allocation.
This allows the guest to reclaim host disk space through write
zeroes requests on thin provisioned images.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
probe_file_sparse_support() defined FALLOC_FL_KEEP_SIZE,
FALLOC_FL_PUNCH_HOLE, and FALLOC_FL_ZERO_RANGE as local constants.
These are available from the libc crate directly.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add raw_async_io_tests.rs with punch_hole, write_zeroes, and
multiple_operations helpers that take &mut dyn AsyncIo + &mut File.
These are raw-backend-specific. They verify data by reading the
underlying file directly, which only works for plain file backends
without container format metadata.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add a small helper in the block crate that opens a disk image
file and wraps any failure in a BlockError carrying the file
path and operation context. Use it from the vmm device manager
so that a failed open now reports which path couldn't be opened.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Convert detect_image_type() from io::Result to BlockResult so
that I/O failures carry the operation name in the error context.
Update the corresponding vmm error variant to wrap BlockError.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Introduce error.rs as the home for a unified error hierarchy that
will replace the per format error types at the public crate
boundary. This commit is intentionally empty beyond the copyright
header and module declaration in lib.rs.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Test valid power of two alignment, layout compatibility,
direct helper coverage, and O_DIRECT write/read roundtrip.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
DiskTopology::probe() returned a hardcoded 512 for regular files,
causing O_DIRECT failures on volumes with larger block sizes
(e.g. 4K).
Use statx(STATX_DIOALIGN) (Linux >= 6.1) to query the real per file
DIO memory and offset alignment. Unlike fstatvfs().f_bsize, which
only returns the filesystem preferred I/O block size,
STATX_DIOALIGN reports the true DIO constraints accounting for the
filesystem, underlying block device, and any stacking (loop, dm,
etc.).
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
O_DIRECT requires buffer addresses to be aligned to the backend
device's logical block size. The existing bounce buffer logic in
execute_async() hardcodes SECTOR_SIZE (512) for the alignment check
and bounce buffer allocation. This is insufficient for devices with
a 4096-byte logical block size, where misaligned buffers cause
-EINVAL from the host kernel.
Add an alignment() method to the AsyncIo trait that returns the
backend's logical block size, defaulting to SECTOR_SIZE. The three
raw I/O backends (io_uring, AIO, synchronous) probe the device
topology via DiskTopology::probe() at creation time and return the
actual logical block size. All image format backends would simply
use the default value of 512 bytes since their underlying are
not block devices.
execute_async() now queries disk_image.alignment() instead of using
the hardcoded SECTOR_SIZE
Fixes: #7720
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
There is no non destructive readonly ioctl to query block device
discard or write zeroes capabilities. BLKZEROOUT is guaranteed to
succeed via kernel software fallback. BLKDISCARD may fail at runtime
with EOPNOTSUPP on devices that lack trim support, but the error
propagates to the guest as VIRTIO_BLK_S_IOERR and well behaved
guests handle it gracefully.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
As well as rejecting writes to sector 0 in the case of raw files where
the user hasn't specified the image_type also reject virtio requests of
type discard and write_zeroes.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add an image_type to DiskConfig to specify the image type. If none is
specified autodetect the image type but disable potentially unsafe
behaviour in the QCOW2 backend by disabling the backing file support.
If the image type is autodetected then fix it in the config so that it
will be persistant across reboots and migrations/snapshot & restores.
This also handles the case where the image type was not specified as
part of the disk configuration.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add VIRTIO_BLK_T_DISCARD and VIRTIO_BLK_T_WRITE_ZEROES request types.
Parse discard/write_zeroes descriptors (sector, num_sectors, flags),
convert to byte offsets, and call punch_hole/write_zeroes on the disk
backend. Mark as unsupported in sync mode.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When sparse=false is configured, preallocate the entire raw disk file
at startup using fallocate(). This provides space reservation and
reduces fragmentation.
Only applies to raw disks. QCOW2/VHD/VHDX formats manage their own
allocation.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add functions to probe whether a file or block device actually
supports PUNCH_HOLE and ZERO_RANGE operations at runtime. The
probe is performed at file open time by testing the operations
at EOF with a zero-length range, which is a safe no-op.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Backing files (e.g. for QCOW2) interact badly with landlock since they
are not obvious from the initial VM configuration. Only enable their use
with an explicit option.
Signed-off-by: Rob Bradford <rbradford@meta.com>
https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7294 adjusted
the checks for read-only requests made to virtio-blk devices and started
rejecting VIRTIO_BLK_T_GET_ID requests. These requests do not perform
any writes and are needed in order to access device serials from within
the guest.
Signed-off-by: Connor Brewster <cbrewster@hey.com>
This helps to uncover expensive and needless clones in the code base.
For example, I prevented extensive clones in the snapshot path where
(nested) BTreeMap's have been cloned over and over again. Further,
the lint helps devs to much better reason about the ownership of
parameters.
All of these changes have been done manually with the necessary
caution. A few structs that are cheap to clone are now `copy` so that
this lint won't trigger for them.
I didn't enable the lint so far as it is a massive rabbit hole and
needs much more fixes. Nevertheless, it is very useful.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This should be guaranteed by GuestMemory and GuestMemoryRegion, but
those traits are currently safe, so add checks to guard against
incorrect implementations of them.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
