A virtqueue notification (doorbell) is normally delivered to the device
through an ioeventfd registered on the notify address, so a plain MMIO
write to the notify register is consumed by the hypervisor and never
reaches write_bar().
It does reach write_bar() when the driver rings the doorbell through the
VIRTIO_PCI_CAP_PCI_CFG window (write_cap_pci_cfg -> write_bar) instead
of a mapped BAR, or on backends that deliver the write to the VMM such
as SEV-SNP.
The notification arm of write_bar() only re-signalled the matching
queue eventfd under the sev_snp feature and logged an error
otherwise, so a doorbell delivered through the PCI_CFG window was
silently dropped on standard builds and the queue was never
processed.
The virtio spec allows driving the device purely through the PCI_CFG
window, so signal the matching queue eventfd for any doorbell that
reaches write_bar() on every build.
Add unit tests that ring a queue's doorbell via write_bar() and
assert only the addressed queue's eventfd is signalled.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Assisted-by: Copilot:Opus-4.8
Rather than instantiating a vector for parsing the descriptor chain in
advance instead use a SmallVec bounded by the expected length of the
descriptor chain. This removes vector allocations from those paths.
As smallvec was already a block dependency move it to a workspace
dependency and use it from there.
Fixes: #5079
Signed-off-by: Rob Bradford <rbradford@meta.com>
The vhost-user (fs, net, block, generic) and vsock worker threads allow
socket() unconditionally in their seccomp rules. These threads only ever
open AF_UNIX sockets: the vhost-user transport connects/binds a Unix
socket (via the vhost crate's Endpoint/Listener), and the vsock device's
host side is a Unix socket. None of them open AF_INET/AF_INET6 sockets.
Restrict their socket() rule to AF_UNIX. Because these threads are
spawned by the VMM thread and inherit its filter, socket() was already
limited to the VMM's set (AF_UNIX/AF_INET/AF_INET6); this narrows it
further to just AF_UNIX, so each worker is confined to what it actually
uses rather than the broader inherited set.
Related to #8490 (giving each thread a more restrictive filter than the
VMM thread).
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Only set the VIRTIO_NET_S_ANNOUNCE status bit if the feature
VIRTIO_NET_F_GUEST_ANNOUNCE was acknowledged.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The virtio-iommu translation required the whole [addr, addr+size) span
to be covered by a single mapping. A guest is free to describe one
contiguous buffer with several adjacent mappings: the specification lets
the driver map at page granularity and the Linux IOMMU core splits a
single mapping request at page size boundaries. A descriptor buffer
backed this way was rejected as an invalid translation even though every
page was mapped, wedging the device.
Walk consecutive mappings when no single one covers the span, accepting
the translation once the mappings are adjacent in IOVA space and
contiguous in guest-physical space. A non-contiguous span cannot be
represented by the single returned address and is still rejected.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Rob Bradford <rbradford@meta.com>
The virtio-net guest announce feature bit was accidentally dropped
during upstreaming, so the device never advertised it to guests. Restore
the feature offer for virtio-net.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Advertise `VIRTIO_NET_F_GUEST_ANNOUNCE` on virtio-net devices, surface
`VIRTIO_NET_S_ANNOUNCE` through config status, and handle
`VIRTIO_NET_CTRL_ANNOUNCE_ACK` on the control queue.
This adds the guest-visible state needed for post-migration or
post-restore announce requests; the VMM side triggering is added in
follow-up commits.
The motivation is to reduce post-migration and post-restore
connectivity gap. After a live migration or after restoring, it can
take the guest several seconds to be reachable again over the network.
With these announcements, the network path should be refreshed within a
few milliseconds.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Expose `VIRTIO_NET_S_LINK_UP` through the virtio-net config status field
when `VIRTIO_NET_F_STATUS` was negotiated.
This makes the guest-visible status bits reflect the device runtime
state and prepares the config status path used by later post-migration
announce handling.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Add unit tests for add_pci_capabilities covering the configless
device path. The device config capability is present when the
config region is sized and absent when the size is zero.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The watchdog, rng, and rtc devices expose no device specific
configuration fields. Each now reports a config size of zero so the
transport omits the device configuration capability instead of
advertising an unbacked region that the device cannot service.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add a config_size method to VirtioDevice and use it when building the
PCI device configuration capability. The transport advertises the size
reported by the device and omits the capability entirely when the size
is zero, because the virtio driver rejects a zero length capability.
The method defaults to None, so every device keeps its current
capability size.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
size_of is part of std::prelude as of Rust 1.80 (with size_of_val,
align_of, align_of_val), and the workspace MSRV is 1.89, so qualifying
it (mem::size_of, std::mem::size_of, core::mem::size_of) is unnecessary.
Convert every qualified size_of call-site to the bare prelude form and
drop the now-redundant `use std::mem::size_of;` imports, keeping
`use std::mem;` where it still serves non-prelude items (transmute,
swap, replace, take, zeroed, MaybeUninit, offset_of). size_of is the
only one of the four currently used in the tree.
Pure refactor, no behavioural change. Follow-up to the
clippy::absolute_paths cleanup (#7670), as discussed in #8444.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
We have larger cloud deployments and analyzed the logs. Let's silence
some messages that generally provide little value on the `info!` level.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
The two synchronous completion paths add the head to the used ring with
desc_chain.memory() but reload self.mem.memory() to enable
notifications. Keep both on the snapshot the chain was parsed from so
the used ring update and the notification enable always act on one
guest memory view rather than two independent atomic loads.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Import the modules used in the crate instead of spelling the full paths
at every use site, and drop the now-unnecessary crate-level
#![expect(clippy::absolute_paths)].
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Removal of absolute paths is currently in progress. To avoid regressing
those changes add a clippy deny at the workspace level and at the crate
level override with #[expect(clippy::absolute_paths)]
See: #7670
Signed-off-by: Rob Bradford <rbradford@meta.com>
Submit a head only virtio-blk chain, confirm Request::parse rejects
it, and verify that add_used with len 0 returns the head to the
guest by advancing the used ring index.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When Request::parse failed, for example for a chain containing only
the head descriptor, process_queue_submit returned the error via
`?`. The caller process_queue_submit_and_signal swallowed
Error::RequestParsing with a warn! and returned Ok(()), but
queue.iter().next() had already consumed the head from the avail
ring. The head was never written to the used ring, so the descriptor
slot leaked and the queue could be stalled by a guest that keeps
submitting malformed chains.
Handle the parse error in line. Log a warning, add the head to the
used ring with len 0, reenable notifications, and continue draining
the queue. A VIRTIO_BLK_S_IOERR status cannot be written because the
status descriptor address is exactly what failed to parse.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Reuse the functionality in the AlignedFile wrapper for the QCOW RawFile
wrapper. This makes alignment handling more transparent.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Rob Bradford <rbradford@meta.com>
During pause the block backend's async I/O path can have unfinished I/O
requests. A snapshot or migration RAM copy taken after pause returns can
then race with kernel writes and capture torn pages.
Since vCPUs are already paused, the VMM thread can stop new block
submissions and wait for the worker to drain before parking the worker
threads.
Assisted-by: Codex:GPT-5
Signed-off-by: Dylan Reid <dgreid@fb.com>
When the host half-closed the socket this was wrongly interpreted as a
full shutdown preventing the guest from sending any more data. Instead
propagate the half-close by setting just `VSOCK_FLAGS_SHUTDOWN_SEND`,
leaving the connection alive so that guest-to-host writes are still
forwarded. The connection is only torn down once the guest also shuts
down its send side or a host write fails.
Fixes: #8300
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Rob Bradford <rbradford@meta.com>
When the guest did a half-close (shutting down only its send side) the
connection state was updated but the write half of the host Unix socket
was never closed so the host peer never saw an EOF. This caused issues
with newer systemd (v256+) as it now half closes its socket and waits
for the host side to react and fully close the connection.
Propagate the guest's half-close to the host by shutting down the write
half of the backing stream. This is deferred until any buffered guest
data has been flushed so that no data is lost, and the connection is
left open so that host-to-guest data keeps flowing.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Rob Bradford <rbradford@meta.com>
Convert the still-needed #[allow]s to #[expect] so they warn if the
lints stop firing.
Part of #8326.
Signed-off-by: Tushar Khatri <hello@tusharkhatri.in>
Currently ejecting a device leaks its mapping keeping the container fd
open. Remove the mapping so the fd can be closed.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Dylan Reid <dgreid@fb.com>
Query and save the dirty log before shutting down the vhost-user device.
This allows any final dirty memory ranges to be recorded before it
becomes impossible to do that as the vhost-user handle has been closed.
This is required to ensure that all memory writes have been correctly
recorded that may be triggered by inflight I/O drains from vhost-user
device state capture.
One small implementation wrinkle: with local migrations there is no
dirty logging (since we just pass the memory FD over the socket) so
calling dirty_log() would generate an error. As there is no clean way to
query if dirty logging has been started add a boolean to track if its
active.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Rather than each device implementation holding a reference to the guest
memory move this to VhostUserCommon. This refactoring simplifies the
function signatures but also allows for methods that act on
VhostUserCommon that don't have the memory available to them.
Signed-off-by: Rob Bradford <rbradford@meta.com>
A confidential guest (e.g. SEV-SNP) requires every virtio device to
advertise VIRTIO_F_ACCESS_PLATFORM so the guest driver routes DMA
through the platform's bounce-buffer path; the driver refuses a device
that does not offer it.
Add VIRTIO_F_ACCESS_PLATFORM support to virtio-{watchdog,iommu} which
are exercised as part of the CVM integration tests.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Ruben Hakobyan <hruben@meta.com>
eject_device calls shutdown, but Block doesn't implement it, so the
worker thread was never joined. Drop for Block does not help either,
VirtioPciDevice keeps an Arc<Mutex<Block>> past eject, so Drop never
runs. The async worker keeps completing in-flight I/O into the guest RAM
that backed it when issued.
Implement shutdown() to call wait_for_epoll_threads(), which drops the
WorkerThreads handle and ensures that worker's io completes.
Signed-off-by: Dylan Reid <dgreid@fb.com>
Feed Ok into VirtioPciDeviceActivator and assert that activate returns
Ok, device_activated becomes true, DEVICE_NEEDS_RESET is not set,
status is otherwise unchanged, no Config interrupt is delivered, and
the barrier waiter unblocks normally.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Feed BadActivate into VirtioPciDeviceActivator and assert that the
error propagates, device_activated stays false, DEVICE_NEEDS_RESET is
set in status, a single Config interrupt is delivered, and a thread
waiting on the activation barrier unblocks. The barrier release is
the deadlock fixed by the NEEDS_RESET on activation failure change.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add TestVirtioDevice with a controllable ActivateResult,
TestVirtioInterrupt that records delivered interrupt types, and a
make_activator helper that builds a complete VirtioPciDeviceActivator
with observable status, activated flag, interrupt log, and barrier.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
When the guest writes DRIVER_OK and the device fails to activate, the
VMM previously bubbled the error up via VirtioActivate and never
released the activation barrier, leaving the vCPU that wrote DRIVER_OK
blocked on the barrier and effectively deadlocking the guest.
Per virtio 1.3 section 2.1.2, a device that has experienced an error
it cannot recover from should set DEVICE_NEEDS_RESET in its status and
notify the driver via a configuration change interrupt. Do that on
activation failure through the existing mark_device_needs_reset
helper, then release the activation barrier so the vCPU can resume.
DeviceManager::activate_virtio_devices now logs and continues instead
of aborting the whole pending list, so one failing device does not
take down the VMM or block pause and migration. The activator has
already reported the failure with the device id.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
RX packet assembly checks descriptor capacity before the backend sees
the packet. The backend then updates hdr.len before the header is
written back.
Validate that final length before committing the header, so we never
tell the guest that more bytes were written than fit in the RX buffer.
Assisted-by: Codex:GPT-5
Signed-off-by: Dylan Reid <dgreid@fb.com>
Remove the need for unsafely materializing slices from guest memory
pointers which is, by definition, undefined behavior.
Achieved by introducing a TxBufSource trait that is implemented for both
types of sources (Guest Memory or local copy) and by using the volatile
read/write primities for moving data from a readable or writable to
guest memory.
Assisted-by: Codex:GPT-5
Signed-off-by: Dylan Reid <dgreid@fb.com>
Keep packet data as a checked guest memory range and add helpers for
volatile reads and writes. Arguably VsockPacket should hold a
VolatileSlice for the guest memory usecase, but the lifetime tracking
involved wasn't worth it.
Keep the old slice accessors for now so existing callers still build.
The next commit switches them over.
Assisted-by: Codex:GPT-5
Signed-off-by: Dylan Reid <dgreid@fb.com>
The helper only needs to update the packet len field. Use write_slice()
instead of rebuilding a mutable slice from a raw host pointer.
Assisted-by: Codex:GPT-5
Signed-off-by: Dylan Reid <dgreid@fb.com>
Take the recent thread refactor further. Bundle a device's worker
JoinHandles together with the kill event that stops them into a single
WorkerThreads value, owned by VirtioCommon. Its Drop signals the workers
to exit, unparks any parked for migration, and joins them.
This makes a detached/leaked worker unrepresentable. reset(),
wait_for_epoll_threads() and VhostUserCommon::shutdown() now happen when
dropping the WorkerThreads, and the unpark-before-join teardown now
lives in one place.
Assisted-by: Claude:Opus-4.8
Signed-off-by: Dylan Reid <dgreid@fb.com>
Signed-off-by: Rob Bradford <rbradford@meta.com>
Replace manual thread management with the use of
VhostUserCommon::spawn_worker() which is just a thin layer over
VirtioCommon::spawn_worker() but handling VhostUserCommon reset. This
removes some manual thread management and also triggers a reset upon
spawn failure.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Wrap VirtioCommon::spawn_worker() to also include vhost-user specific
backend cleanup.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Replace use of spawn_virtio_thread() helper with the new method on
VirtioCommon to handle thread management as well as spawning. As a
result this cleanly handles reset if it fails to spawn the thread.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Implement a method on VirtioCommon for spawning the worker thread. This
method also handles storing the handle to the thread and triggering a
reset on the device if thread spawning fails. The addition of this
helper replaces a repeated pattern across virtio and vhost-user devices
but also ensures correct cleanup when spawning multiple threads for a
device with multiple queues.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
Rely on the vector of threads in VirtioCommon for storing all the
handles of the spawned threads rather than storing them in the devices
or in VhostUserCommon directly.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rob Bradford <rbradford@meta.com>
process_rx writes the packet header back into the descriptor chain
it is currently processing, so the write must go through that
chain's memory snapshot. Rederefing self.mem.memory() resolves to
the same snapshot today, but couples the write on the chain to the
device's atomic handle and obscures intent. Match the pattern used
by the rest of the device by writing through desc_chain.memory().
Suggested-by: Rob Bradford <rbradford@meta.com>
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
