We cannot reliably send Request::abandon() on every kind of failure on
the sender side, as we might be in the middle of a memory transmission.
The receiver would not reliably know what to do with that. So instead,
when the receiver cannot read from the socket, we log that the migration
sender failed, which is the only likely cause of that failure.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
This improves the observability whether a migration failed because of
the sender or because of some error on the receiving side.
Using a simple log message is simpler than introducing a new error enum
to differentiate between SendError and RemoteError.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
When the receiver of a live migration encounters an error, it sends an
error response. The sender of the migration would then send an abandon
request and wait for a response. This abandon request is not necessary,
because the receiver already abandoned the migration due to the error it
encountered.
From now on this function will not send an abandon request to the
receiver anymore, thus it was renamed to "ok_or_error".
Also, this case was always broken, because after sending the error
response, the receiver just exits without waiting for the additional
abandon request.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Getting rid of the unsafe ByteValued implementation for MemoryRange,
Request and Response structures, by relying on zerocopy's safe
implementation instead.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-8
size_of is part of std::prelude as of Rust 1.80 (with size_of_val,
align_of, align_of_val), and the workspace MSRV is 1.89, so qualifying
it (mem::size_of, std::mem::size_of, core::mem::size_of) is unnecessary.
Convert every qualified size_of call-site to the bare prelude form and
drop the now-redundant `use std::mem::size_of;` imports, keeping
`use std::mem;` where it still serves non-prelude items (transmute,
swap, replace, take, zeroed, MaybeUninit, offset_of). size_of is the
only one of the four currently used in the tree.
Pure refactor, no behavioural change. Follow-up to the
clippy::absolute_paths cleanup (#7670), as discussed in #8444.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Plumb the SocketUffdMemorySource into the receiving side of live
migration. When memory_mode=postcopy is requested, the destination
brings up a dedicated fault connection, registers userfaultfd on the
restored memory regions, and serves guest pages on demand over that
connection while the VM resumes early.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-7
Introducing PageFault as the new wire command needed by both postcopy
live migration and on demand restore from the offload daemon. This new
command describes the need from the destination to fault the page
content in. This request describes the page through a MemoryRange
structure, and the response can be either 0 or the actual page size.
In case it is 0, that means the source had access to the guest memory
and was able to copy the page content directly. In case the response is
the actual page size, there is a payload associated which contains the
page content.
We can expect local live migration and offload restore to run locally
and therefore have access to the guest memory. The remote live migration
over the network is the case where we would expect the page content to
be sent over the wire.
This command is served through an additional connection happening on the
UNIX or TCP socket. The goal is to keep the same codepath between local
and remote migrations. This additional channel allows PageFault commands
to be issued asynchronously so they can be served without blocking the
main connection.
A connection role is introduced in order to identify an additional
connection related to pre-copy memory versus the newly introduced
channel for serving post-copy requests.
Signed-off-by: Sebastien Boeuf <sboeuf@meta.com>
Assisted-by: Claude:claude-opus-4-7
Import the std modules used in the crate instead of spelling the full
paths at every use site, and drop the now-unnecessary crate-level
#![expect(clippy::absolute_paths)].
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Removal of absolute paths is currently in progress. To avoid regressing
those changes add a clippy deny at the workspace level and at the crate
level override with #[expect(clippy::absolute_paths)]
See: #7670
Signed-off-by: Rob Bradford <rbradford@meta.com>
Validate that all files that are necessary for TLS encryption are in the
given folder. The knowledge which files are necessary is part of the TLS
module.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Extend ReceiveListener with a TLS-backed listener variant for migration
receivers.
Store the TCP listener together with the server TLS configuration, wrap
accepted sockets in TlsStream::new_server(), and preserver the existing
listener cloning and fd polling behavior so receive-side migration code
can treat TLS listeners like the existing TCP and UNIX cases.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Teach the migration transport to handle TLS-backed streams alongside
plain TCP and UNIX sockets.
Introduce a Tls variant in SocketStream and implement the necessary
traits.
Also updates the local-migration error path to reject any non-UNIX
transport, which now includes TLS-wrapped TCP connections.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
TLS connections have a TLS server (listens for incoming connections) and
a TLS client (initiates the connection). This commit adds the code for
the client side, which is the sender of a migration
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Validate the sender's migration protocol version when
handling the initial Start request.
Read the version from the Start command header, accept only
the supported version window n-1..=n, and reject unsupported
versions with Error. A rejected Start moves the receiver to
the aborted state.
This keeps compatibility one-way, from older protocol
versions to newer ones, and leaves later version-based
branching on the receiver side.
Log the protocol version on both sender and receiver to make
the active migration path visible.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Add protocol-side support for migration protocol versioning.
Use the existing 6-byte Start command header, which was
previously zero padding, to carry the sender's migration
protocol version without changing the wire layout.
Store the version as a little-endian u16 in the first two
bytes and ignore the remaining four bytes. A zeroed command
header continues to mean a legacy v0 sender.
This keeps the message flow unchanged for rollout:
Start is still followed by plain OK or Error (Aborted), and no new
command is needed.
On-behalf-of: SAP leander.kohler@sap.com
Signed-off-by: Leander Kohler <leander.kohler@cyberus-technology.de>
Import the std modules used in the crate instead of spelling the full
paths at every use site.
Signed-off-by: Henry Hrvoje Tonkovac <htonkovac@gmail.com>
Assisted-by: Claude:Opus-4.8
Reordering commands or adding commands in-between is breaking the
migration protocol. By using explicit numbers, we can increase the
attention required when touching this code.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Add two explicit disconnected-backend error paths before wiring them
into the call sites.
MigratableError::DeviceDisconnected is the lifecycle sentinel for
operations that were skipped because a component is already known to be
disconnected. It lets the caller log and continue without treating it as
a VMM-fatal condition.
Error::BackendDisconnected is the vhost-user-local error used when
VhostUserCommon refuses to call a backend after its disconnected flag is
set. The transport classifier treats socket close/reset/EOF and
vhost-user partial-message/disconnected cases as transport loss, while
backend NACKs, invalid protocol state, and retry-able socket errors
remain ordinary operation failures.
Signed-off-by: Dylan Reid <dgreid@fb.com>
This extends migration to also support paused VMs, preserving the
paused state on the destination.
Changes:
- Add CompletePaused protocol command that finalizes migration without
resuming the VM on the destination
- Skip the pause step during migration if the VM is already paused
- On migration failure, only restore the running state if
the VM was originally running (not paused)
Signed-off-by: Nguyen Dinh Phi <phind.uet@gmail.com>
Add migration-level context types that extend the existing memory-only
metrics with overall migration duration and downtime breakdown.
OngoingMigrationContext models the sender-side migration progress until
all inputs needed for final downtime accounting are available.
CompletedMigrationContext then stores the finalized migration metrics,
including the final memory iteration, snapshotting, snapshot transfer,
and completion phase.
This provides the data needed to log effective downtime in the VMM and
lays the groundwork for future migration statistics reporting.
# Terminology
At first glance, the use of "state" and "[VM] snapshot" may seem
confusing. As discussed in [0], we use "state" consistently in the
migration code. On the VM side, "snapshotting" is merely the mechanism
used to obtain the VM state.
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7979#discussion_r3061359899
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Expose the finalized per-iteration timing fields needed by higher-level
migration metrics and factor the iteration-overhead calculation into a
small helper.
This keeps the existing MemoryMigrationContext behavior intact while
making the timing data easier to consume from migration-level context
in the following commits.
On-behalf-of: SAP philipp.schuster@sap.com
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
For sending memory over multiple connections, we need a way to split up
the work. With these changes, we can chop a memory table into same-sized
chunks for transmit
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Wire the new `downtime_ms`, `timeout_s`, and `timeout_strategy` fields
from `VmSendMigrationData` into the precopy loop, replacing the previous
hard-coded 5-iteration cap.
Each iteration now evaluates three convergence criteria in order:
- no dirty pages remain;
- the estimated final-iteration downtime is within the configured budget
- or the overall migration timeout has elapsed.
On timeout, `TimeoutStrategy::Cancel` aborts and keeps the VM live on
the source, while `TimeoutStrategy::Force` proceeds regardless of the
downtime target. The convergence callback is updated to return a Result
to propagate the cancel error cleanly up the call stack.
With the recent changes [0], it is fairly easy to implement the new
checks and operate on actual metrics.
These changes are inspired by [1] but differ significantly in details.
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7799
[1] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7033
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
When memory_restore_mode=ondemand is specified on the restore command,
the memory manager creates a userfaultfd descriptor, registers each
guest RAM range for missing-page fault interception, and spawns a
handler thread that serves page faults from the snapshot file using
UFFDIO_COPY. This avoids reading the entire memory-ranges file into
guest RAM before restore completes.
The handler uses epoll to multiplex the userfaultfd and a stop eventfd
for clean shutdown. Concurrent faults from multiple vCPUs are handled
by treating EEXIST as a benign race and waking blocked threads with
UFFDIO_WAKE. Once all pages have been served the handler exits
automatically. If the handler thread panics the VMM is signalled to
exit since the VM cannot continue without page fault service.
MemoryZone gains a backing_page_size field so the handler resolves
fault granularity from the zone rather than the top-level config.
Errors from the UFFD setup path use a structured UffdError enum
and a new MigratableError::OnDemandRestore variant, with a From
impl to keep call sites concise.
The seccomp filter is updated to allow the userfaultfd syscall and
the four uffd ioctls (UFFDIO_API, UFFDIO_COPY, UFFDIO_REGISTER,
UFFDIO_WAKE) under the VMM thread profile.
Signed-off-by: Shayon Mukherjee <shayonj@gmail.com>
Introduce MemoryMigrationContext to track internal metrics of an ongoing
precopy memory migration.
The context aggregates information such as iteration count, transferred
bytes, durations, bandwidth, and estimated downtime. This enables
migration logic to make decisions based on runtime characteristics,
such as terminating iterations once the expected downtime is below a
target threshold.
The type is used in the next commit to implement iteration-based
migration metrics.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
The pointer created by `Vec::as_ptr` may not be used for mutation of the
underlying data [0].
This PR switches to `Vec::as_mut_ptr` and uses `cast` to avoid
mutability changes when casting.
Also improves safety reasoning, separates the unsafe call from the
call to `read_exact` to improve clarity and simplifies the vector
creation.
[0]: https://doc.rust-lang.org/alloc/vec/struct.Vec.html#method.as_ptr
On-behalf-of: SAP julian.schindel@sap.com
Signed-off-by: Julian Schindel <julian.schindel@cyberus-technology.de>
This helps to uncover expensive and needless clones in the code base.
For example, I prevented extensive clones in the snapshot path where
(nested) BTreeMap's have been cloned over and over again. Further,
the lint helps devs to much better reason about the ownership of
parameters.
All of these changes have been done manually with the necessary
caution. A few structs that are cheap to clone are now `copy` so that
this lint won't trigger for them.
I didn't enable the lint so far as it is a massive rabbit hole and
needs much more fixes. Nevertheless, it is very useful.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This better aligns with the rest of the code and makes it clearer
that these tests can run "as is" in a normal hosted environments
without the special test environment.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Adding itertools as dependency improves the iteration code in the
following significantly.
With this change, we don't need a copy of the vector. Just something
that can be coerced into an iterator. We also use the bit position
iterator to make the code somewhat clearer. The new code is much
faster, because it will not iterate over every bit, just each 1 bit in
the input.
The next commit will complete this optimization and have some concrete
numbers.
On-behalf-of: SAP julian.stecklina@sap.com
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Fix clippy warning `uninlined_format_args` reported by rustc rustc
1.89.0 (29483883e 2025-08-04).
```console
warning: variables can be used directly in the `format!` string
--> block/src/lib.rs:649:17
|
649 | info!("{} failed to create io_uring instance: {}", error_msg, e);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
= note: `#[warn(clippy::uninlined_format_args)]` on by default
help: change this to
|
649 - info!("{} failed to create io_uring instance: {}", error_msg, e);
649 + info!("{error_msg} failed to create io_uring instance: {e}");
|
```
Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
# What
This commit introduces file-based advisory locking for the files backing
up the block devices by using the fcntl() syscall with OFD locks. The
per-open-file-descriptor (OFD) locks are more robust than traditional
POSIX locks (F_SETLK) as they are not tied to process IDs and avoid
common issues in multithreaded or multi-fd scenarios [1]. Therefore,
we don't use `std::fs::File::try_lock()`, which is backed by F_SETLKW.
The locking mechanism is aware of the `readonly` property and allows
`n` readers or `1` writer (exclusive mode).
As the locks are advisory, multiple cloud-hypervisor processes can
prevent themselves from writing to the same file. However, this is not
a system-wide file-system level locking mechanism preventing to open()
a file.
The introduced new locking mechanism does not cover vhost-user devices.
# Why
To prevent misconfiguration and improve safety, it is good practice to
protect disk image files with a locking mechanism. Experience and common
best practices suggest that advisory locks are preferable over mandatory
locks due to better compatibility and fewer pitfalls (in fs space).
The introduced functionality is aligned with the approach taken by
QEMU [0], and is also recommended in [1].
# Implementation Details
We need to ensure that not only normal operation keeps working but also
state save/resume and live-migration. Especially for live migration,
it is crucial that the sender VMM releases the locks when the VM stops
so the receiver VMM can acquire them right after that.
Therefore, the locking and releasing happen directly on the block
device struct. The device manager knows all block devices and can
forward requests to these types.
Last but not least, this commit uses on explicit lock acquiring
but implicit lock releasing (FD close). It only explicitly releases
the locks where this integrates more smoothly into the existing
code.
# Testing
I tested
- normal operation
- state save/resume,
- device hot plugging,
- and live-migration
with read/shared and write/exclusive locks.
One can use the `fcntl-tool` to test if locks are actually acquired
or released [2].
# Links
[0] 825b96dbce/util/osdep.c (L266)
[1] https://apenwarr.ca/log/20101213
[2] https://crates.io/crates/fcntl-tool
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Historically the Cloud Hypervisor coding style has been to ensure that
all imports are ordered and placed in a single group. Unfortunately
cargo fmt has no support for ensuring that all imports are in a single
group so if whitespace lines were added as part of the import statements
then they would only be odered correctly in the group.
By adopting "group_imports="StdExternalCrate" we can enforce a style
where imports are placed in at most three groups for std, external
crates and the crate itself. Choosing a style enforceable by the tooling
reduces the reviewer burden.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
An example warning output is:
error: first doc comment paragraph is too long
--> virtio-devices/src/lib.rs:158:1
|
158 | / /// Convert an absolute address into an address space (GuestMemory)
159 | | /// to a host pointer and verify that the provided size define a valid
160 | | /// range within a single memory region.
161 | | /// Return None if it is out of bounds or if addr+size overlaps a single region.
| |_
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#too_long_first_doc_paragraph
= note: `-D clippy::too-long-first-doc-paragraph` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::too_long_first_doc_paragraph)]`
Signed-off-by: Bo Chen <chen.bo@intel.com>
This method will return the existing Response if the status is
successful (Status::Ok) otherwise issue a command to abandon the
migration and return the desired error.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
