This gate behaves like a barrier, but it can be opened, meaning that
threads can be released before all threads arrived at the gate. This
lets us release waiting threads in case of an error, which will be
important for the sender side of a live migration with multiple
TCP connections.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Adds the functionality to accept multiple connections on the receiver
side of a live migration. A thread listens for incoming connections and
creates a worker for each new connection.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
With this, the receiver side of a migration can wait for incoming
connections, while also being able to abort the accept when the
migration is done.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
That way we avoid having to grab a lock when receiving a chunk of memory
over the migration socket. This is a necessary prerequisite for having
multiple memory receiving threads.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
The memory manager is guarded by a mutex, thus parallel accesses to it
and its members are not possible. But we have to execute this function
in parallel when we introduce multiple TCP connections. Otherwise, the
workers who receive the data and write it into guest memory will block
on each other, and thus slow down the migration.
Also rename the function to receive_memory_ranges for better naming
consistency.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
When doing a local migration using a UNIX socket, we removed the UNIX
socket file after accepting the connection. The VMM does not own this
socket file, which makes this an unsafe operation. Thus, we stop doing
that.
On-behalf-of: SAP sebastian.eydam@sap.com
Signed-off-by: Sebastian Eydam <sebastian.eydam@cyberus-technology.de>
Refactor PciDevice::allocate_bars trait and all implementations
to take &mut SystemAllocator instead of &Arc<Mutex<SystemAllocator>>,
removing double indirection.
The caller in device_manager.rs now acquires the lock before
calling allocate_bars.
Signed-off-by: Chinmoy <daschinmoyy21@gmail.com>
Remove the old async_io::DiskFile trait implementation from
RawFileDiskAio, now that the new disk_file trait hierarchy
is fully implemented.
Clean up unused imports: DiskFile and DiskFileResult from
crate::async_io.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Previously, GSIs were eagerly allocated for all MSI-X vectors a device
advertises (i.e. the maximum the device can support). This can easily
exhaust KVM_MAX_IRQ_ROUTES (4096) with modern NVMe devices that support
up to 2048 MSI-X vectors.
Defer GSI allocation to the first time an interrupt vector is
unmasked. The EventFd is still created eagerly since external
components (e.g. VFIO) need it at device init time.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Previously, KVM_KVMCLOCK_CTRL was skipped when kvm_hyperv=on because
Windows does not use pvclock directly. However, KVM internally uses
pvclock data structures as the basis for computing the Hyper-V
Reference TSC page parameters. Not calling KVM_KVMCLOCK_CTRL means
there is no mechanism to signal time discontinuity to Windows guests
after pause/resume, contributing to multi-minute hangs.
Remove the kvm_hyperv guard so all guests receive the clock-paused
notification.
Signed-off-by: CMGS <ilskdw@gmail.com>
Reorder resume() to: set_clock, device_manager.resume,
cpu_manager.resume. This matches the inverse of pause()
which correctly saves the clock before pausing vCPUs.
Signed-off-by: CMGS <ilskdw@gmail.com>
Classify the io::Error as BlockErrorKind::Io with ErrorOp::Open.
Update vmm CreateFixedVhdDiskSync to take BlockError.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Update device_manager.rs to construct DiskBackend::Next
instead of DiskBackend::Legacy for the synchronous raw
backend.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Update device_manager.rs to construct DiskBackend::Next
instead of DiskBackend::Legacy for the io_uring raw backend.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Fix broken URL reported by Lychee. The binutils-gdb github repo no
longer exists. Use the equivalent sourceware.org link.
Signed-off-by: Anirudh Rayabharam <anrayabh@microsoft.com>
When landlock support was added, creation of file descriptors was
moved out into a function called pre_create_console_devices, with the
idea being that this could be run before Landlock rules are applied
and access to all the necessary paths are dropped.
This idea didn't take reboots into account, though. When a VM is
rebooted, pre_create_console_devices is called again, but now the
Landlock rules have been applied, so they need to allow access to all
those paths anyway.
I imagine the way this was intended to work was that file descriptors
would be preserved across reboot, but that's not currently the case,
and it's not a trivial change to make because they get dropped when
the VM is destroyed. Longer term it would be ideal if Cloud
Hypervisor's implementation was more focused on file descriptors than
paths[1], and if created VMs only took references to file descriptors,
so they were easily preserved across reboots.
Fixes: b3e5738b4 ("vmm: Introduce ApplyLandlock trait")
Closes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7547
Link: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7704 [1]
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Add BLKGETSIZE64 0x80081272 to the VMM seccomp ioctl allow list
alongside the existing BLK* ioctls. This is needed for querying
block device size without seeking.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Following the `VfioContainer` to `VfioOps` trait switch, update the
remaining field names, method names, comments, and log messages to use
`vfio_ops` and "host IOMMU address space" consistently.
No fucntional changes.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
Replace the concrete `VfioContainer` type with the `VfioOps` trait
object for device passthrough. This decouples the VFIO DMA mapping
interface from the legacy VFIO container/group implementation, allowing
it to be extended to support VFIO cdev and iommufd in the future.
Signed-off-by: Bo Chen <bchen@crusoe.ai>
The actual size of the balloon is taken directly from the guest. A
misbehaving guest can set it to an arbitrary value and cause underflow
on the next vm.info call. Use a saturation_sub instead to avoid a panic
in a debug build or a crazy number in a release build.
Signed-off-by: Dylan Reid <dgreid@dylanreid.com>
As far as I can tell, PIDTYPE_PID is a kernel-internal name, and
PR_SCHED_CORE_SCOPE_THREAD is the UAPI name. There's no PIDTYPE_PID
in the UAPI headers, and the core scheduling documentation says that
the fourth prctl argument should be a "PR_SCHED_CORE_SCOPE_-prefixed
macro constant".
Link: https://www.kernel.org/doc/html/v6.19/admin-guide/hw-vuln/core-scheduling.html#usage
Fixes: 3f800d2bb ("vmm: Add core scheduling support for vCPU threads")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Wire the new `downtime_ms`, `timeout_s`, and `timeout_strategy` fields
from `VmSendMigrationData` into the precopy loop, replacing the previous
hard-coded 5-iteration cap.
Each iteration now evaluates three convergence criteria in order:
- no dirty pages remain;
- the estimated final-iteration downtime is within the configured budget
- or the overall migration timeout has elapsed.
On timeout, `TimeoutStrategy::Cancel` aborts and keeps the VM live on
the source, while `TimeoutStrategy::Force` proceeds regardless of the
downtime target. The convergence callback is updated to return a Result
to propagate the cancel error cleanly up the call stack.
With the recent changes [0], it is fairly easy to implement the new
checks and operate on actual metrics.
These changes are inspired by [1] but differ significantly in details.
[0] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7799
[1] https://github.com/cloud-hypervisor/cloud-hypervisor/pull/7033
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Management software needs fine-grained control over live migration to
meet QoS requirements for VM guests. Add `downtime_ms`, `timeout_s`, and
`timeout_strategy` fields to `VmSendMigrationData`, exposed via API.
This commit contains the API changes only; the VMM does not yet act on
these values. This follows in the next commit.
For the JSON API, downtime and timeout are represented as plain integers
(downtime_ms and timeout_s) to make the units explicit. Using Duration
directly would require custom (de)serialization logic, so instead the
internal raw integers are exposed as Duration via getters. This
introduces minor conversion overhead but keeps the Rust API clear and
unambiguous.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
This change prepares upcoming options (following commit) that are added
to VmSendMigrationData.
VmSendMigrationData is a special case as it is currently the only
"rich configuration" type that lives outside `config.rs`, as it is
purely API-facing. Therefore, it isn't integrated into the existing
OptionParser infrastructure. We therefore introduce a `parse()` method
to use that in `ch-remote` in the following.
In `ch-remote`, we remove `--local` for `send-migration` and switch to
the new option string parsing constructor (breaking change!). This
prepares the addition of downtime and timeout options in the following
and streamlines the `ch-remote` command line interface with other
commands, such as `ch-remote add-net`.
Lastly, this commit updates the integration tests.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
Bump to the released versions that are compatible wherever possible but
for the vhost and vfio crates they are git hashes as no releases with
compatible versions have yet been made.
Signed-off-by: Rob Bradford <rbradford@meta.com>
This now required after Rust-VMM crate bumps. Also reorder some syscalls
so that they are now in alphabetical order.
Signed-off-by: Rob Bradford <rbradford@meta.com>
Change Block to hold DiskBackend instead of
Box<dyn async_io::DiskFile>. In device_manager, existing formats
(raw, vhd, vhdx) are wrapped in DiskBackend::Legacy while
QcowDiskSync uses DiskBackend::Next. The fuzz target is updated
accordingly.
The Error::DiskResize variant now carries BlockError instead of
DiskFileError, matching the BlockResult return type of
DiskBackend::resize().
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add an option that can be used when restoring to resume the VM. This is
particularly useful when restoring the VM via the direct VMM command
line, when you might not want/have an API socket configured.
Signed-off-by: Rob Bradford <rbradford@meta.com>
The virtio vhost-user device backend prefers to use externally-provided
eventfds as irqfds. This allows the frontend VM to notify the backend
VM directly, without the need for a userspace proxy process. Since the
frontend can provide irqfds at any time, the backend needs to register
and unregister irqfds dynamically.
This is tricky because the functions that access the irqfd table all
take `&self`, not `&mut self`. The obvious solution to this problem is
to wrap the table in a mutex. Most of these functions are not called on
hot paths, but `.notifier()` is called whenever Cloud Hypervisor needs
to inject an interrupt into a guest. Most devices don't need to
register irqfds at runtime, and for them, slowing down interrupt
injection would be wasteful.
Instead, require devices to opt-in to irqfd registration. The irqfd
table now comes in two forms: one that contains a mutex and one that
does not. The one containing a mutex can be mutated freely, while
attempting to mutate the one that does not will panic.
Right now, no code registeres irqfds at runtime, but this will change in
subsequent commits.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
This allows creating an InterruptSourceGroup with an externally provided
file descriptor. It also allows changing the file descriptor
afterwards.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
