Virtio v1.2 says that if CONFIG_WCE is negotiated
but FLUSH is not, the device must initialize writeback to 0.
It also says that if CONFIG_WCE was not negotiated but FLUSH
was, the driver should assume presence of a writeback cache.
Introduce a pure is_writeback_enabled helper and a
set_writeback_mode helper. This makes the two call flows
explicit:
* write_config resolves the guest requested mode against the
negotiated features before storing it back
* activate starts from the default writeback preference and then
resolves it against the negotiated features
* reset restores the initial writeback state
This keeps the config space value and the runtime writeback flag
in sync and makes the spec driven fallback easier to follow.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Refactor PciDevice::allocate_bars trait and all implementations
to take &mut SystemAllocator instead of &Arc<Mutex<SystemAllocator>>,
removing double indirection.
The caller in device_manager.rs now acquires the lock before
calling allocate_bars.
Signed-off-by: Chinmoy <daschinmoyy21@gmail.com>
Now that the VhostUserCommon::shutdown implementation has been filled
out to support migration it can also be used for the drop
implementations in the vhost-user devices.
It's worth noting that the call to wait_for_epoll_threads() was a no-op
as those threads are only configured on conventional virtio devices.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Advertising support for this virtio feature is required to enable
support for migration. (Along with the LOG_SHMFD protocol feature.)
Signed-off-by: Rob Bradford <rbradford@meta.com>
If the epoll thread is paused, which would be expected as a part of live
migration/snapshot-restore unpause the thread so that it can receive the
kill event. This mirrors the reset() behaviour of virtio devices. It is
important here so as to close the connection with the vhost-user-backend
to allow same host and --local migration and since after getting the
device state the vhost-user backend should no longer be used.
As a result of this change we can do --local and same-host migration
with virtio-fs.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This is used by all devices so it can be part of the common state.
Moving it simplifies the code and simplifies some future improvements
around shutdown for migration.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Since vhost-user devices are always virtio devices it makes sense to
structure this struct inside the VhostUserCommon struct. This then also
makes some of the methods on VhostUserCommon cleaner since they can now
act directly on the common virtio bits (e.g. for kill_evt)
Signed-off-by: Rob Bradford <rbradford@meta.com>
PR #7852 fixed the missing VirtioBlockConfig fields but did not
change the feature advertisement logic. The condition
`sparse || disk_image.supports_zero_flag()` causes qcow2 to
advertise DISCARD even with sparse=false, because qcow2 can
mark clusters as zero (supports_zero_flag() returns true).
Windows viostor BSODs (DRIVER_IRQL_NOT_LESS_OR_EQUAL) when
DISCARD is advertised on qcow2 backends, making sparse=off
ineffective as a workaround for qcow2 images.
Restrict DISCARD to explicit sparse=true only. WRITE_ZEROES
remains available for all sparse-capable backends.
Fixes#7849
Signed-off-by: CMGS <ilskdw@gmail.com>
The queue_msix_vector register (offset 0x1a in virtio PCI common
config) was indexed into the msix_queues Vec using the guest-controlled
queue_select value without bounds checking. A malicious guest can set
queue_select to any u16 value via offset 0x16, then read or write
offset 0x1a to trigger an out-of-bounds panic, crashing the VMM.
Replace direct Vec indexing with .get()/.get_mut() for bounds-checked
access, returning VIRTQ_MSI_NO_VECTOR (0xFFFF) on OOB reads to match
the virtio "no vector" sentinel. Add a regression test that sets
queue_select to 0xFFFF and exercises both the read and write paths.
AI/LLM disclosure: this patch was co-authored with Claude Code.
Fixes#7917
Signed-off-by: Tobias Kässer <t.kaesser@gmail.com>
The sync fallback path in process_queue_submit() hardcoded
VIRTIO_BLK_S_IOERR for all errors. This caused unsupported
request errors to report IOERR to the guest instead of the
correct VIRTIO_BLK_S_UNSUPP.
Use ExecuteError::status() to return the appropriate status
code for each error variant.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Enable the DEVICE_STATE protocol feature negotiation for all vhost-user
devices (block, fs, net, and generic). Restoring the state (including
the backend state if present) and vrings.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Add a common method for validating the state (checking vrings &
device_state) and then restoring the backend state if present.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Fetch the opaque device state from the backend and store it along with
the last vring used in the state used for the snapshot.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The vhost-user protocol now has support for the backend to provide an
opaque blob of data (read or written through a pipe) that the VMM can
use to save/restore state after snapshot/restore or live migration.
It also adds a command for checking the backend accepts the uploaded
device state. One quirk of saving the state is that GET_VRING_BASE must
be used first to quiesce the state of the backend and flush any
in-flight requests. This then also requires saving that index for use on
the restore.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Rename update_supports_migration() to update_supported_features() as
this method will be extended to track additional capability flags beyond
just migration support (e.g. DEVICE_STATE for snapshot/restore.)
Signed-off-by: Rob Bradford <rbradford@meta.com>
This is a refactoring step in preparation for fetching backend device
state via SET_DEVICE_STATE_FD which can fail.
Signed-off-by: Rob Bradford <rbradford@meta.com>
In GenericVhostUser's read_config and write_config, access
vu_common.acked_protocol_features directly instead of going through
the state() method which creates a struct. This removes creating the
struct just to access two fields that are already directly accessible.
Signed-off-by: Rob Bradford <rbradford@meta.com>
With a common state structure for all vhost-user devices the state()
methods can also be refactored for reuse. This will make it easier to
add new common fields in the future for snapshot/restore.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Each vhost-user device type had near identical State structs.
Deduplicate those by introducing a new common struct (and parameterising
it when it needs to embed a config member.) This will make it easier to
reuse more code in the future and to add more struct members to handle
the requirements of snapshot/restore.
These changes have been designed to have no impact on the existing
snapshot/restore state.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Use logical_size() instead of physical_size() for the byte-range
advisory lock. physical_size() returns st_blocks*512 which is the
actual host allocation and can be smaller than the guest visible
extent on sparse files, leaving part of the range unprotected.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Previously this code could lead to the device being trying to be
activated multiple times as the code to trigger the activation was based
on the state of the device (not yet activated and device being ready).
This could occur if anothe vCPU wrote to a PCI BAR on this device before
the device activation was completed by the VMM thread. Now we only
trigger the activation if the device readiness has changed as a result
of this BAR write (by checking that the readiness was originally
unready.)
Signed-off-by: Rob Bradford <rbradford@meta.com>
When configuring multiple queues for a virtio device, the guest can
activate between 1 and the configured amount of queues. The firmware,
for example, may activate only one queue, while a Linux guest would
likely activate all available queues.
The constructor of virtio-blk initializes the `paused_sync` barrier
using the configured queue count (plus one for the main thread). This
can be wrong if the guest enable a different number of queues at
activation time, which can make pause hang. Thus, we now recompute the
barrier size from the queues that are actually activated.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
When configuring multiple queues for a virtio device, the guest can
activate between 1 and the configured amout of queues. The firmware,
for example, may activate only one queue, while a Linux guest would
likely activate all available queues.
The constructor of virtio-net initializes the `paused_sync` barrier
using the configured queue count (plus one for the main thread). This
can be wrong if the guest enables a different number of queues at
activation time, which can make pause hang. Thus, we now recompute the
barrier size from the queues that are actually activated.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Bump to the released versions that are compatible wherever possible but
for the vhost and vfio crates they are git hashes as no releases with
compatible versions have yet been made.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This now required after Rust-VMM crate bumps. Also reorder some syscalls
so that they are now in alphabetical order.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Split the data length check into two conditions:
- reject descriptors shorter than one
virtio_blk_discard_write_zeroes segment, and
- reject payloads exceeding MAX_DISCARD_WRITE_ZEROES_SEG segments
Previously only the minimum length was checked and extra segments
were silently dropped.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Change Block to hold DiskBackend instead of
Box<dyn async_io::DiskFile>. In device_manager, existing formats
(raw, vhd, vhdx) are wrapped in DiskBackend::Legacy while
QcowDiskSync uses DiskBackend::Next. The fuzz target is updated
accordingly.
The Error::DiskResize variant now carries BlockError instead of
DiskFileError, matching the BlockResult return type of
DiskBackend::resize().
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Set discard_sector_alignment from the logical block size
reported by the backend topology instead of hardcoding it
to 1 sector. This gives the guest accurate alignment hints
so it can avoid sub block discards that the filesystem
might silently ignore.
For example, on a 4K block filesystem the alignment is now
8 sectors (4096/512) instead of 1.
For image formats with their own allocation units (QCOW2
clusters, VHD/VHDX block sizes), the ideal alignment would
be derived from the format cluster/block size. This is
left for a followup that surfaces allocation granularity
through DiskTopology.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The virtio vhost-user device backend prefers to use externally-provided
eventfds as irqfds. This allows the frontend VM to notify the backend
VM directly, without the need for a userspace proxy process. Since the
frontend can provide irqfds at any time, the backend needs to register
and unregister irqfds dynamically.
This is tricky because the functions that access the irqfd table all
take `&self`, not `&mut self`. The obvious solution to this problem is
to wrap the table in a mutex. Most of these functions are not called on
hot paths, but `.notifier()` is called whenever Cloud Hypervisor needs
to inject an interrupt into a guest. Most devices don't need to
register irqfds at runtime, and for them, slowing down interrupt
injection would be wasteful.
Instead, require devices to opt-in to irqfd registration. The irqfd
table now comes in two forms: one that contains a mutex and one that
does not. The one containing a mutex can be mutated freely, while
attempting to mutate the one that does not will panic.
Right now, no code registeres irqfds at runtime, but this will change in
subsequent commits.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
It is currently left as unimplemented!().
No functional change intended as there are no callers.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Trigger the interrupts in the guest for the virtio device queues behind
the vhost-user devices when resuming. This avoids a situation where
interrupts from the backend get lost when they are dispatched from the
backend when then guest is paused leading to the guest/backend
effectively waiting for each other to move forward. This is more
reproducible with longer durations between pause and resume as there is
more opportunity for the backend to completely process it's queue and
fire all the interrupts.
It's perfectly safe and allowed by the virtio spec to generate these
interrupts and the performance impact is negligible and is a safe way to
ensure forward progress after a resume.
See: #7850
Signed-off-by: Rob Bradford <rbradford@meta.com>
When VIRTIO_BLK_F_DISCARD or VIRTIO_BLK_F_WRITE_ZEROES features
are advertised, the virtio spec v1.2, sections 5.2.4 and
5.2.6.1, requires the corresponding VirtioBlockConfig fields
to contain valid, non zero values. Leaving them at zero causes
strictly behaved drivers to either reject the features or crash.
Populate max_discard_sectors, max_discard_seg,
discard_sector_alignment, max_write_zeroes_sectors,
max_write_zeroes_seg and write_zeroes_may_unmap after
feature advertisement so drivers can safely negotiate
these features.
Fixes: #7849
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This device is not called virtio-vhost-user; that's something else.
I don't think the comment really clarifies anything anyway, so just
remove it.
Fixes: 8c618ff5e ("virtio-devices: generic-vhost-user: implement device")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
A buggy or malicious guest may write an inappropriate value into
virtqueue's next_avail field. This will result in an error
when iterating over the queue:
863837ef86/virtio-queue/src/queue.rs (L708)
but this error is (logged and) ignored if pop_descriptor_chain()
is used:
863837ef86/virtio-queue/src/queue.rs (L583)
A reasonable approach, implemented here, is to mark the device as
NEEDS_RESET and ignore further queue events until the guest
reinitializes the device.
How this patch was tested:
Linux kernel was patched to trigger a bad next_avail when the
virtqueue queue counter reaches 5000:
--------------- START OF LINUX KERNEL PATCH ----------
$ git diff
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index b784aab668670..989f2a0c64a77 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -15,6 +15,9 @@
#include <linux/spinlock.h>
#include <xen/xen.h>
+
+void virtqueue_kick_always(struct virtqueue *vq);
+
#ifdef DEBUG
/* For development, we want to crash whenever the ring is screwed. */
#define BAD_RING(_vq, fmt, args...) \
@@ -677,6 +680,12 @@ static inline int virtqueue_add_split(
struct virtqueue *_vq,
* new available array entries. */
virtio_wmb(vq->weak_barriers);
vq->split.avail_idx_shadow++;
+ {
+ if ((vq->split.avail_idx_shadow % 100) == 0)
+ printk(KERN_ERR "avail idx: %d",
+ (int)vq->split.avail_idx_shadow);
+ if (vq->split.avail_idx_shadow == 5000)
+ vq->split.avail_idx_shadow = 0;
+ }
vq->split.vring.avail->idx = cpu_to_virtio16(_vq->vdev,
vq->split.avail_idx_shadow);
vq->num_added++;
@@ -689,6 +698,11 @@ static inline int virtqueue_add_split(
struct virtqueue *_vq,
if (unlikely(vq->num_added == (1 << 16) - 1))
virtqueue_kick(_vq);
+ {
+ if (unlikely(vq->split.avail_idx_shadow == 0))
+ virtqueue_kick_always(_vq);
+ }
+
return 0;
unmap_release:
@@ -2515,6 +2529,11 @@ bool virtqueue_kick(struct virtqueue *vq)
}
EXPORT_SYMBOL_GPL(virtqueue_kick);
+void virtqueue_kick_always(struct virtqueue *vq)
+{
+ virtqueue_kick_prepare(vq);
+ virtqueue_notify(vq);
+}
/**
* virtqueue_get_buf_ctx - get the next used buffer
* @_vq: the struct virtqueue we're talking about.
--------------- END OF LINUX KERNEL PATCH ----------
Then the kernel was booted, and the host pinged until the
nic became unresponsive:
ping -i 0.002 192.168.4.1
Device status was confirmed using
cat /sys/class/net/eth0/device/status
(it was 0x4f).
Then the device was re-initialized:
DEV_NAME=$(basename $(readlink -f /sys/class/net/eth0/device))
echo $DEV_NAME | tee /sys/bus/virtio/drivers/virtio_net/unbind
echo $DEV_NAME | tee /sys/bus/virtio/drivers/virtio_net/bind
ip link set eth0 up
At this point networking became healthly again.
Signed-off-by: Peter Oskolkov <posk@google.com>
Moves the MSRV requirement to the workspace and expands it to all
cloud-hypervisor dependencies and dev-dependencies.
This improves discoverability for new contributors working on crates
other than the cloud-hypervisor itself and creates consistency regarding
the MSRV of cloud-hypervisor dependencies.
Functionally, this doesn't change anything for dependencies of the
cloud-hypervisor crate as the MSRV requirement is already enforced by CI
when building the cloud-hypervisor with the MSRV versioned compiler.
On-behalf-of: SAP julian.schindel@sap.com
Signed-off-by: Julian Schindel <julian.schindel@cyberus-technology.de>
We ran the Microsoft Windows installer in CHV (via network + VNC) and
the installation always failed when the installer wanted to write the
partition table. Since recently, for very good reason, sector 0 writes
are disabled if the image type is not set explicitly and only
implicitly auto-detected as raw [0].
To ease troubleshooting, I've added a descriptive log message. It is a
little spammy, but it is what is required to help users to troubleshoot.
[0] 6ecdf90e22
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Signal the guest before processing queue submissions to enable
earlier guest side completion event handling, reducing end-to-end
latency for block device operations.
FIO benchmarks show up to 7.4% bandwidth improvement at 16
iodepth and 4k block size with NVMe devices.
Signed-off-by: wuxinyue <wuxinyue.wxy@antgroup.com>
Both process_input_queue and process_output_queue were only processing
the first descriptor of a chain, leading to data loss if the driver used
chained descriptors.
This change iterates through all descriptors in a chain. It also moves
the flush call out of the descriptor loops to improve performance.
Signed-off-by: Andrei Vagin <avagin@google.com>
For a transmit queue (guest to host), the host only reads from the
guest-provided buffers and does not write to them. According to the
virtio specification (e.g., Section 2.6.8 in Virtio 1.1), the 'len'
field in the used ring has to be set to the number of bytes written to
the buffers. Therefore, it should be 0 for the console transmit queue.
Signed-off-by: Andrei Vagin <avagin@google.com>
The update_console_size method was using fetch_and on the acked_features
atomic, which modified the atomic and cleared other feature bits.
Changed it to use a non-destructive load and bitwise AND.
Signed-off-by: Andrei Vagin <avagin@google.com>
Add a per-disk lock_granularity parameter that lets users choose
between byte-range OFD locks and whole-file OFD locks:
--disk path=/foo.img,lock_granularity=byte-range
--disk path=/bar.img,lock_granularity=full
Byte-range is the default and matches QEMU behavior, working
best with storage backends where whole-file OFD locks are treated
as mandatory. The full option restores the original whole-file
locking for environments that depend on it.
The LockGranularityChoice enum and its FromStr impl live in the
block crate alongside the existing LockGranularity type. The
Block device converts the user-facing choice to the internal
LockGranularity at lock time, keeping device_manager.rs simple.
Closes: #7553
Signed-off-by: Victor Vieux <vieux@repl.it>
Rename the transport feature bit constant from
VIRTIO_F_IOMMU_PLATFORM to VIRTIO_F_ACCESS_PLATFORM across
the entire virtio-devices crate.
The virtio specification as of v1.1 carries bit 33 as
VIRTIO_F_ACCESS_PLATFORM. The Linux kernel UAPI header
<linux/virtio_config.h> carries VIRTIO_F_IOMMU_PLATFORM
only as a backward-compatible alias.
This is a pure rename with no functional or behavioral
change.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
