Since our firmware files are still designed to be used via PVH use the
load_kernel() function to load the firmware falling back to legacy
firmware loading if necessary.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Only the ovs-dpdk live-migration tests need to run sequentially as they
use the same ovs-dpdk setup.
This is to reduce our CI time, particularly for the live-migration
and aarch64 jobs.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The documentation states to install qemu-img. This is located in the
package qemu-utils. The documentation imples the binary is wihtin the
qemu-img package, which doesn't exist.
Signed-off-by: Steven Dake <sdake@lambdal.com>
If contributors add this pre-commit hook we should have fewer issues
with build churn on our CI as the most basic checks will pass.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This enables the Windows test module. One basic test is enabled,
while all others are disabled yet for aarch64. Jenkins file is
extended with the corresponding step for aarch64.
installAzureCli() is parametrized.
It seems that transferring a 30GB image would take >= 15 minutes. An
optimization here is having a gzip'ed image to 10GB which would unpack
in 3 minutes. Expect to be quicker than transferring an uncompressed
image while on another network.
Signed-off-by: Anatol Belski <ab@php.net>
This removes the requirement for the tests (dev-dependencies) to build
with all supported toolchains including the MSRV.
See: #4318
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Adding new I/O ports for both the ACPI shutdown and the ACPI PM timer
devices so they can be triggered from both addresses. The reason for
this change is that TDX expects only certain I/O ports to be enabled
based on what QEMU exposes. We follow this to avoid new ports from being
opened exclusively for Cloud Hypervisor.
We have to keep the former I/O ports available given all firmwares
haven't been updated yet. Once we reach a point where we know both Rust
Hypervisor Firmware, OVMF, TDVF and TDSHIM have been updated with the
new port values, we'll be able to remove the former ports.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The current fuzzer defines a 'format' for the random input 'bytes' from
libfuzzer, but this 'format' failed to improve the fuzzing
efficiency. Instead, the 'format' parsing process obfuscates the fuzzer and
makes the fuzzing engine much harder to focus on the actual fuzzing
target (e.g. virtio-block queue event handling). It is actually worse than
simply using the random inputs as the virt queue content for fuzzing.
We can later introduce a different 'format' to the input 'bytes' for
better fuzzing, say focusing more on virito-block fuzzing through
ensuring the virt queue content always has a valid 'available'
descriptor chain to process.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This also ensures that the 'queue_evt' is fully processed, as we enforce
the main thread is waiting for the virtio-block thread to process the
'kill_evt' which is after the 'queue_evt' processing.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Currently the main thread returns immediately after sending a 'queue'
event which is rarely received and processed by the virtio-block
thread (unless system is in high workload). In this way, the fuzzer is
mostly doing nothing and is unable to reproduce its behavior
deterministically (from the same inputs). This patch relies on a
'level-triggered' epoll to ensure a 'queue' event is properly processed
before return from the main thread.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The old API remains usable, and will remain usable for two releases but
we should only advertise the new API.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Introduce a new top level member of VmConfig called PayloadConfig that
(currently) encompasses the kernel, commandline and initramfs for the
guest to use.
In future this can be extended for firmware use. The existing
"--kernel", "--cmdline" and "initramfs" CLI parameters now fill the
PayloadConfig.
Any config supplied which uses the now deprecated config members have
those members mapped to the new version with a warning.
See: #4445
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Whenever starting a test with the capture_output option, both stdout and
stderr pipes are resized to a greater capacity. But the problem is that
when the system call fcntl() fails, we need to know about the error
returned by the system so that we can apply the necessary changes.
This patch simply enhances the error checking and reporting related to
the invocation of fcntl() when trying to resize a pipe.
Refers to #4475
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The test test_virtio_block_topology has been recently failing due to an
error happening in losetup while trying to set the block size. Since
there's no option in losetup for retrying, we took the approach of
programming the expected behavior of creating a loop device relying
directly on the system ioctl LOOP_CONFIGURE. We apply a retry loop based
on the result returned by this ioctl, so that we don't fail on the first
try. We also added a sleep before retrying, hoping this would help the
next iteration to succeed.
Fixes#3494
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Remove the use of 'unwrap()' that assumes the guest address for request
status is always valid, which avoid virtio-block thread panic on
malformed descriptors from the guest.
Signed-off-by: Bo Chen <chen.bo@intel.com>
By checking in the validation logic we get checking for both devices
specified in the initial config but also hotplug too.
Fixes: #4453
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
There are PCI extended capabilities that can't be passed through the VM
as they would be unusable from a guest perspective. That's why we
introduce a way to patch what is returned to the guest when the PCI
configuration space is accessed. The list of patches is created from the
parsing of the extended capabilities in that case, and particularly
based on the presence of the SRIOV and Resizable BAR capabilities.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Steven Dake <sdake@lambdal.com>
The new option parsing for "[..,]" syntax is much stricter and requires
that everything is balanced in terms of brackets. Change the stripping
of the brackets to only strip the first and last one rather than
multiple:
e.g. "[[a,b], [c,d]]" becomes "[a,b],[c,d]" rather than "a,b], [c"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The parsing is now O(1) as we only consider each character in the input
once and further it removes the need for allocating memory for
reassembling the option parameter.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the BAR for the VFIO device is marked as prefetchable on the
underlying device ensure that the BAR exposed through PciConfiguration
is also marked as prefetchable.
Fixes problem where NVIDIA devices are not usable with PCI VFIO
passthrough. See related NVIDIA kernel driver bug:
https://github.com/NVIDIA/open-gpu-kernel-modules/issues/344.
Fixes: #4451
Signed-off-by: Steven Dake <sdake@lambdal.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The original value didn't include the size of the entry point structure.
The caused the last entry to be corrupted by other code.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Previously the code used the SmbiosSysInfo structure for that purpose.
Handle 0x0003, DMI type 127, 27 bytes
<TRUNCATED>
Wrong DMI structures length: 130 bytes announced, structures occupy 131 bytes.
Fix this by using the correct structure and padding.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
There is no need to manually implement Clone for some structures.
It is better to explicitly spell out repr(C) to avoid the compiler
reordering the fields.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
It's been observed on the Bionic image that udev and snapd services can
cause some delay in the VM's shutdown. Disabling them before shutting
down the VM improves the reliability of the test.
Also increasing slightly the sleep time to ensure we give the VM enough
time to shutdown before checking the list of events provided by the
event monitor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Some dependencies are not tracking the latest version in the .toml file
so update all dependencies to the latest version.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The test virtio_net_latency_us recently failed because of the ethr tool
couldn't complete the latency measurement with the following error
message:
Error dialing the latency connection: dial tcp4
:0-\u003e172.19.0.2:8888: connect: connection refused
The "connection refused" error comes from the Golang Dial function which
couldn't complete as expected because there was no server listening on
the specified port at the specified destination.
In other words, the ethr server wasn't ready when the ethr client tried
to connect to it.
The easy way to avoid this problem is simply by increasing the sleep
time between the command spawning the ethr server in the guest and the
command running the ethr client from the host.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since it's not possible to run the integration test test_vfio on Azure
at the moment (because of some nested virtualization issues), we can
temporarily run it on the baremetal CI where we already run some VFIO
tests.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Rather than using the QCOW2 image, rely on the RAW one, the same way the
host relies on this RAW image to boot the first VM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The uuid indicates the unique ID of a virtual machine.
cloud-hypervisor takes the uuid passed by libvirt
and uses it to initialize cloud-init.
Signed-off-by: lizhaoxin1 <Lxiaoyouling@163.com>
Move the live migration tests to a 'jammy' worker rather than
'jammy-small'. This type of worker has more CPUs (64 vs 16) and more RAM
(256G vs 64G), which should improve the time it takes to run each test.
With this improvement, the test shouldn't fail anymore due to timeout
being reached.
A second improvement is to reduce the amount of vCPUs created for each
VM. The point is simply to check we can migrate a VM with multiple
vCPUs, therefore using 2 instead of 6 should be enough when possible.
When testing NUMA, we can't lower the amount of vCPUs since there's a
quite complex topology that is expected there.
Also, the total amount of vCPUs is reduced from 12 to 4 (again when not
testing with NUMA).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Fuzz the HTTP API handling code with a minimum HTTP API
receiver (e.g. mocking the behavior of our "vmm" thread).
Signed-off-by: Bo Chen <chen.bo@intel.com>
The parameter "poll_queue" was useful at the time Cloud Hypervisor was
responsible for spawning vhost-user backends, as it was carrying the
information the vhost-user-block backend should have this option enabled
or not.
It's been quite some time that we walked away from this design, as we
now expect a management layer to be responsible for running vhost-user
backends.
That's the reason why we can remove "poll_queue" from the DiskConfig
structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Improve error catching on the steps creating the block device so that we
can understand if qemu-img or losetup is the faulty command leading to
an empty device path.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Both of these tests have been sporadically failing through multiple CI
runs. The reason is related to cloud-init which fails to run the
"init-local" script during the second boot of the VM. This causes the
network interface to not be available, and therefore the test can't SSH
into the VM as expected. The root cause is the filesystem and cache
corruption that happens on the cloud-init disk.
The way to prevent from this issue is to sync the guest filesystem
before we shut it down, and as a security harness, we also wait for a
few seconds for the shutdown command to complete inside the guest before
we trigger the API shutdown or delete.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that we rely on pop_descriptor_chain() rather than iter() to iterate
over a queue, there's no more borrow on the queue itself, meaning we can
invoke add_used() directly for the iteration loop. This simplifies the
processing of the queues for each virtio device, and bring some possible
performance improvement given we don't have to iterate twice over the
list of descriptors to invoke add_used().
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Using pop_descriptor_chain() is much more appropriate than iter() since
it recreates the iterator every time, avoiding the queue to be borrowed
and allowing the virtio-net implementation to match all the other ones.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The new virtio-queue version introduced some breaking changes which need
to be addressed so that Cloud Hypervisor can still work with this
version.
The most important change is about removing a handle to the guest memory
from the Queue, meaning the caller has to provide the guest memory
handle for multiple methods from the QueueT trait.
One interesting aspect is that QueueT has been widely extended to
provide every getter and setter we need to access and update the Queue
structure without having direct access to its internal fields.
This patch ports all the virtio and vhost-user devices to this new crate
definition. It also updates both vhost-user-block and vhost-user-net
backends based on the updated vhost-user-backend crate. It also updates
the fuzz directory.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There is no point in wasting resources building use Jenkins if the
change only modifies the fuzzers.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Adding a new limitation related to the TDX guest kernel as it doesn't
allow for most ACPI devices, meaning the PCI hotplug through ACPI isn't
supported unless we use 'tdx_disable_filter' boot parameter.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Handle the case `body_offset` is `None` instead of calling `unwrap()`
which leads to a panic.
Signed-off-by: Maximilian Nitsch <maximilian.nitsch@d3tn.com>
When starting the VM such that it is already on a breakpoint (via
stop_on_boot) when attached to gdb then start the vCPUs in a paused
state rather than starting the vCPUs later (upon resume).
Further, make the resumption/break of the VM more resilient by only
attempting to resume the vCPUs if were are already in a break point and
only attempting to pause/break if we were already running.
Fixes: #4354
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Remove the hardcoded addresses.
Also remove PM_TMR_BLK as spec compliant implementation will use
X_PM_TMR_BLK over this field.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The field can be set directly.
This eliminates one place where dyn Device is used outside of KVM
aarch64 code.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The code was moved from the vmm crate to the hypervisor crate. After the
move it is trivially obvious that it only works with KVM. Use concrete
types where possible.
This allows us to drop create_device from the Vm trait.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The original code uses kvm_device_attr directly outside of the
hyeprvisor crate. That leaks hypervisor details.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Whenever a virtio reset happens, the vhost-user backend should be
notified that the vring should be stopped. This is performed by calling
GET_VRING_BASE on the appropriate queue indexes.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Rather than relying on the amount of queues to enable or disable the
queue that have been activated, we rely on the actual queue indexes
provided through the tuple including the queue index, the Queue and the
EventFd. By storing the list of indexes, we simplify the code and also
make it more accurate in case some queues aren't activated.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of passing separately a list of Queues and the equivalent list
of EventFds, we consolidate these two through a tuple along with the
queue index.
The queue index can be very useful if looking for the actual index
related to the queue, no matter if other queues have been enabled or
not.
It's also convenient to have the EventFd associated with the Queue so
that we don't have to carry two lists with the same amount of items.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When preparing the activator, we must provide the correct queue index to
clone the right EventFd associated with the queue.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It's not mandatory for the virtio-fs driver to enable all virtqueues
provided by the backend since all it needs is one request queue to work
correctly. Therefore we lower the minimal amount of enabled queues to 1.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It might sometimes take a few seconds for the guest to trigger the OOM
and report it back to the host. That's why this patch adds some sleep
time between the command in the guest supposedly triggering the OOM and
the check of the balloon size from the host.
Fixes#4336
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This requires making get/set_lapic_reg part of the type.
For the moment we cannot provide a default variant for the new type,
because picking one will be wrong for the other hypervisor, so I just
drop the test cases that requires LapicState::default().
Signed-off-by: Wei Liu <liuwe@microsoft.com>
CpuId is an alias type for the flexible array structure type over
CpuIdEntry. The type itself and the type of the element in the array
portion are tied to the underlying hypervisor.
Switch to using CpuIdEntry slice or vector directly. The construction of
CpuId type is left to hypervisors.
This allows us to decouple CpuIdEntry from hypervisors more easily.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
We only need to do this for x86 since MSHV does not have aarch64 support
yet. This reduces unnecessary code churn.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
VmState was introduced to hold hypervisor specific VM state. KVM does
not need it and MSHV does not really use it yet.
Just drop the code. It can be easily revived once there is a need.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
It is however only used for KVM right now because MSHV does not need it
yet.
Nonetheless a stub MSHV constructor should be there and get/set
functions should be implemented for MSHV.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Previously, we were assuming that every time an eventfd notified us,
there was only a single event waiting for us. This meant that if,
while one API request was being processed, two more arrived, the
second one would not be processed (until the next one arrived, when it
would be processed instead of that event, and so on). To fix this,
make sure we're processing the number of API and debug requests we've
been told have arrived, rather than just one. This is easy to
demonstrate by sending lots of API events and adding some sleeps to
make sure multiple events can arrive while each is being processed.
For other uses of eventfd, like the exit event, this doesn't matter —
even if we've received multiple exit events in quick succession, we
only need to exit once. So I've only made this change where receiving
an event is non-idempotent, i.e. where it matters that we process the
event the right number of times.
Technically, reset requests are also non-idempotent — there's an
observable difference between a VM resetting once, and a VM resetting
once and then immediately resetting again. But I've left that alone
for now because two resets in immediate succession doesn't sound like
something anyone would ever want to me.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Some `arch_target = "arm"' usages on VCPU related code are not correct.
And we don't support 32-bit ARM architecture.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Function `system_registers` took mutable vector reference and modified
the vector content. Now change the definition to `get/set` style.
And rename to `get/set_sys_regs` to align with other functions.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
On AArch64, the function `core_registers` and `set_core_registers` are
the same thing of `get/set_regs` on x86_64. Now the names are aligned.
This will benefit supporting `gdb`.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
For installing packages for the custom Ubuntu image, we
need to setup DNS inside the chroot.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The VM specific signal (currently only SIGWINCH) should only be handled
when the VM is running.
The generic VMM signals (SIGINT and SIGTERM) need handling at all times.
Split the signal handling into two separate threads which have differing
lifetimes.
Tested by:
1.) Boot full VM and check resize handling (SIGWINCH) works & sending
SIGTERM leads to cleanup (tested that API socket is removed.)
2.) Start without a VM and send SIGTERM/SIGINT and observe cleanup (API
socket removed)
3.) Boot full VM, delete VM and observe 2.) holds.
4.) Boot full VM, delete VM, recreate VM and observe 1.) holds.
Fixes: #4269
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In the original code:
If the first header was invalid and the second header had the same
sequence number (zero) then the invalid first header would be returned.
The new code is more Rust idiomatic and avoids this situation.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=48858
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
And along with virtio-queue, we must also bump vhost-user-backend from
0.3.0 to 0.5.0 (since it relies on virtio-queue 0.4.0).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The vhost-user backend was always provided the maximum queue size but
this is incorrect. Instead it must be informed of the actual queue size
that has been negotiated with the virtio driver running in the guest.
This ensures proper functioning of vhost-user-block with the Rust
Hypervisor Firmware, which uses a hardcoded queue size of 16.
Partially fixes#4285
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The latest vhost-user specification describes VHOST_USER_RESET_OWNER
command as deprecated with the following explanation:
This is no longer used. Used to be sent to request disabling all
rings, but some back-ends interpreted it to also discard connection
state (this interpretation would lead to bugs). It is recommended that
back-ends either ignore this message, or use it to disable all rings.
Also, it's been observed that when using either Rust Hypervisor Firmware
or EDK2 OVMF firmware with SPDK (using the block device as the boot
disk), the virtio reset that happens when the firmware no longer needs
to access the block device caused a failure by triggering the command
VHOST_USER_RESET_OWNER.
For all these reasons, this patch simplifies the virtio reset
implementation by simply disabling the virtqueues and no longer calling
into VHOST_USER_RESET_OWNER.
Partially fixes#4285
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Enabling the TAP interface is only about the IFF_UP flag, meaning we
shouldn't be testing the presence of IFF_RUNNING, and therefore we
shouldn't set it when not present.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The Linux kernel now checks for this before marking CPUs as
hotpluggable:
commit aa06e20f1be628186f0c2dcec09ea0009eb69778
Author: Mario Limonciello <mario.limonciello@amd.com>
Date: Wed Sep 8 16:41:46 2021 -0500
x86/ACPI: Don't add CPUs that are not online capable
A number of systems are showing "hotplug capable" CPUs when they
are not really hotpluggable. This is because the MADT has extra
CPU entries to support different CPUs that may be inserted into
the socket with different numbers of cores.
Starting with ACPI 6.3 the spec has an Online Capable bit in the
MADT used to determine whether or not a CPU is hotplug capable
when the enabled bit is not set.
Link: https://uefi.org/htmlspecs/ACPI_Spec_6_4_html/05_ACPI_Software_Programming_Model/ACPI_Software_Programming_Model.html?#local-apic-flags
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This check is new in the beta version of clippy and exists to avoid
potential deadlocks by highlighting when the test in an if or for loop
is something that holds a lock. In many cases we would need to make
significant refactorings to be able to pass this check so disable in the
affected crates.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: accessing first element with `data.get(0)`
--> virtio-devices/src/transport/pci_device.rs:1055:34
|
1055 | if let Some(v) = data.get(0) {
| ^^^^^^^^^^^ help: try: `data.first()`
|
= note: `#[warn(clippy::get_first)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#get_first
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Tested:
1. SIGTERM based
2. VM shutdown/poweroff
3. Injected VM boot failure after calling Vm::setup_tty()
Fixes: #4248
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The snapshots are stored in a BTree which is ordered however as the ids
are strings lexical ordering places "11" ahead of "2". So encode the
vCPU id with zero padding so it is lexically sorted.
This fixes issues with CPU restore on aarch64.
See: #4239
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Git will refuse to generate information for the report if the repository
is owned by a different user (root in the container vs the real owner
outside).
e.g:
Error generating human readable git reference: fatal: unsafe repository ('/cloud-hypervisor' is owned by someone else)
To add an exception for this directory, call:
git config --global --add safe.directory /cloud-hypervisor
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
It is reasonable for these to fail as it the tool could be run outside
of a git repository but by not giving any error message we cannot see
issues when we expect the report to have the git details.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
By default Microsoft Hypervisor send a GP to the guest if it tries
read/write an unimplemented MSR from the hypervisor prospective. Instead
change this behavior to ignore read/write operations for unimplemented
MSRs
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
When restoring a VM, the restore codepath will take care of mapping the
MMIO regions based on the information from the snapshot, rather than
having the mapping being performed during device creation.
When the device is created, information such as which BARs contain the
MSI-X tables are missing, preventing to perform the mapping of the MMIO
regions.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the VfioCommon implementation, the VfioUserPciDevice now
implements the Migratable trait.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the VfioCommon implementation, the VfioPciDevice now implements
the Migratable trait.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Introduces the common code to handle one aspect of the migration
support. Particularly, the ability to store VMM internal states related
to such device. The internal state of the device will happen later in a
dedicated patchset that will implement the VFIO migration API.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
So that we can save and restore the whole structure through snapshot and
restore operations.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This reverts commit 0d0013c46e.
Grovvy shell script execution engine does not like backslash as the
escape character. So we need to put another backslash to escape the
backslash character. This would most likely fix the issue that we saw
with the CI.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
In order to conserve resources it is better to not run CI, whenever there
are changes only in fuzz/Cargo.toml or fuzz/Cargo.lock.
Fixes#4148
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Based on recent KVM host patches (merged in Linux 5.16), it's forbidden
to call into KVM_SET_CPUID2 after the first successful KVM_RUN returned.
That means saving CPU states during the pause sequence, and restoring
these states during the resume sequence will not work with the current
design starting with kernel version 5.16.
In order to solve this problem, let's simply move the save/restore logic
to the snapshot/restore sequences rather than the pause/resume ones.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The vCPU is created and set after all the devices on a VM's boot.
There's no reason to follow a different order on the restore codepath as
this could cause some unexpected behaviors.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Combined the `GicDevice` struct in `arch` crate and the `Gic` struct in
`devices` crate.
After moving the KVM specific code for GIC in `arch`, a very thin wapper
layer `GicDevice` was left in `arch` crate. It is easy to combine it
with the `Gic` in `devices` crate.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
In order to ensure that the virtio device thread is spawned from the vmm
thread we use an asynchronous activation mechanism for the virtio
devices. This change optimises that code so that we do not need to
iterate through all virtio devices on the platform in order to find the
one that requires activation. We solve this by creating a separate short
lived VirtioPciDeviceActivator that holds the required state for the
activation (e.g. the clones of the queues) this can then be stored onto
the device manager ready for asynchronous activation.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
As coredump function is to make a vmcore for crash tool to analyze,
in order not to introduce a big thing in integration, we just check
if ch-remote command runs no error report here.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Based on the newly added guest_debug feature, this patch adds http
endpoint support.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Co-authored-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The crash tool use a special note segment which named 'QEMU' to
analyze kaslr info and so on. If we don't add the 'QEMU' note
segment, crash tool can't find linux version to move on.
For now, the most convenient way is to add 'QEMU' note segment to
make crash tool happy.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Guest memory is needed for analysis in crash tool, so save it
for coredump.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Co-authored-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It's useful to dump the guest, which named coredump so that crash
tool can be used to analysize it when guest hung up.
Let's add GuestDebuggable trait and Coredumpxxx error to support
coredump firstly.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Co-authored-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The error message incorrectly said that the user was trying to combine
cache_size without dax whereas it is only usuable with dax.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Remove the code from the DeviceManager that prepares the DAX cache since
the functionality has now been removed.
Fixes: #3889
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
`GicDevice` trait was defined for the common part of GicV3 and ITS.
Now that the standalone GicV3 do not exist, `GicDevice` is not needed.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Why combine:
- GicV3 is not required alone
- GicV3 and GicV3Its has separate snapshot/pause code. But the code of
GicV3 was never used.
- Reduce the code complexity of GIC related traits and structs.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
From the logs it appears that booting the VM to the point at which it
can signal to the host can sometimes take longer than then 30 seconds
specified.
Fixes: #4136
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Refactoring and features in this patch:
- Add support for aarch64 build
- Add offline build configuration using vendored crates
- Add script to generate vendored sources
- Fix openssl-sys dependency
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
This reverts commit 6a099257e8.
It is now clear that pinning the toolchain for cross compilation is not
necessary since we only use it for building to aarch64:
https://github.com/rust-lang/rust/issues/95926#issue-1199547707
"This problem doesn't currently show up when cross-compiling from x86_64
to aarch64, since aarch64 doesn't use static-pie by default, but
enabling PIE with -C relocation-model=pie triggers the same bug and
makes the resulting binaries segfault."
Fixes: #3962
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the capturing of the debug port event without requiring more
verbose logging which itself has a performance impact.
Unless users are using our custom kernel or their own custom patches
they are unlikely to see the impact.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This reverts commit f160572f9d.
There has been increased flakiness around the live migration tests since
this was merged. Speculatively reverting to see if there is increased
stability.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to ensure that the virtio device thread is spawned from the vmm
thread we use an asynchronous activation mechanism for the virtio
devices. This change optimises that code so that we do not need to
iterate through all virtio devices on the platform in order to find the
one that requires activation. We solve this by creating a separate short
lived VirtioPciDeviceActivator that holds the required state for the
activation (e.g. the clones of the queues) this can then be stored onto
the device manager ready for asynchronous activation.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The current patch fixes the following error that was raised by clippy:
error: this let-binding has unit value
--> tests/integration.rs:6538:13
|
6538 | / let _ = stdin
6539 | | .write_all("type=7".as_bytes())
6540 | | .expect("failed to write stdin");
| |_________________________________________________^
|
= note: `-D clippy::let-unit-value` implied by `-D warnings`
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#let_unit_value
help: omit the `let` binding
|
6538 ~ stdin
6539 + .write_all("type=7".as_bytes())
6540 + .expect("failed to write stdin");
|
error: could not compile `cloud-hypervisor` due to previous error
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Latest cargo beta version raises warnings about unused macro rules.
Simply remove them to fix the beta build.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Cut and paste doesn't quite work. Not that cut and pasters will get
this far along, but the docs should be correct in any regards.
Signed-off-by: Steven Dake <steven.dake@gmail.com>
The qemu-img tool is used as part of the tutorial but is not installed
by default on a fresh ubuntu 22.04 server.
Signed-off-by: Steven Dake <steven.dake@gmail.com>
There is no need to include serde_derive separately,
as it can be specified as serde feature instead.
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
In this way, we can cover a broad range of events from the event monitor
while avoiding code duplication.
Fixes: #4054
Signed-off-by: Bo Chen <chen.bo@intel.com>
Rely on the newly added helper from vm-virtio crate to keep cloning the
list of Queue structures.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since the QueueState structure has been updated by not implementing
Clone anymore, we need a way to keep cloning the Queue.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This variable name is residual from when these functions acted directly
on the vCPU fd rather than the hypervisor wrapper.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Explicitly re-export types from the hypervisor specific modules. This
makes it much clearer what the common functionality that is exposed is.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
All the required functionality is already exported from the hypervisor
crate so for consistency make this module private.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
And thus only export what is necessary through a `pub use`. This is
consistent with some of the other modules and makes it easier to
understand what the external interface of the hypervisor crate is.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Move this enum from vm-device to hypervisor crate so that hypervisor
crate does not gain an extra dependency.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
By taking advantage of the fact that IrqRoutingEntry is exported by the
hypervisor crate (that is typedef'ed to the hypervisor specific version)
then the code for handling the MsiInterruptManager can be simplified.
This is particularly useful if in this future it is not a typedef but
rather a wrapper type.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This removes the requirement to leak as many datastructures from the
hypervisor crate into the vmm crate.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The trait and functionality is about operations on the VM rather than
the VMM so should be named appropriately. This clashed with with
existing struct for the concrete implementation that was renamed
appropriately.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Whenever going through the codepath of loading a RAW firmware, we always
add an extra RAM region to the guest memory through the memory manager.
But we must be careful to use the updated guest memory rather than a
previous reference that wasn't containing the new region, as this can
lead to the following error:
VmBoot(FirmwareLoad(InvalidGuestAddress(GuestAddress(4290772992))))
This is fixed by the current patch, getting the latest reference onto
the guest memory from the memory manager right after the new region has
been added.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We can return prematurely from 'map_mmio_regions()' (e.g. when a mmap call
failed for vfio or 'create_user_memory_region()' failed for vfio-user)
without updating the 'MmioRegion::user_memory_regions' with the
information of previous successful mmaps, which in turn would cause mmap
leaks particularly for the case of hotplug where the 'vmm' thread will
keep running. To fix the issue, let's keep 'MmioRegion::user_memory_regions'
updated right after successful mmap calls.
Fixes: #4068
Signed-off-by: Bo Chen <chen.bo@intel.com>
This is required when hot-removing a vfio-user device. Details code path
below:
Thread 6 "vcpu0" received signal SIGSYS, Bad system call.
[Switching to Thread 0x7f8196889700 (LWP 2358305)]
0x00007f8196dae7ab in shutdown () at ../sysdeps/unix/syscall-template.S:78
78 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
(gdb) bt
0x00007f8196dae7ab in shutdown () at ../sysdeps/unix/syscall-template.S:78
0x000056189240737d in std::sys::unix::net::Socket::shutdown ()
at library/std/src/sys/unix/net.rs:383
std::os::unix::net::stream::UnixStream::shutdown () at library/std/src/os/unix/net/stream.rs:479
0x000056189210e23d in vfio_user::Client::shutdown (self=0x7f8190014300)
at vfio_user/src/lib.rs:787
0x00005618920b9d02 in <pci::vfio_user::VfioUserPciDevice as core::ops::drop::Drop>::drop (
self=0x7f819002d7c0) at pci/src/vfio_user.rs:551
0x00005618920b8787 in core::ptr::drop_in_place<pci::vfio_user::VfioUserPciDevice> ()
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x00005618920b92e3 in core::ptr::drop_in_place<core::cell::UnsafeCell<dyn pci::device::PciDevice>>
() at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x00005618920b9362 in core::ptr::drop_in_place<std::sync::mutex::Mutex<dyn pci::device::PciDevice>> () at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x00005618920d8a3e in alloc::sync::Arc<T>::drop_slow (self=0x7f81968852b8)
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/sync.rs:1092
0x00005618920ba273 in <alloc::sync::Arc<T> as core::ops::drop::Drop>::drop (self=0x7f81968852b8)
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/sync.rs:1688
0x00005618920b76fb in core::ptr::drop_in_place<alloc::sync::Arc<std::sync::mutex::Mutex<dyn pci::device::PciDevice>>> ()
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x0000561891b5e47d in vmm::device_manager::DeviceManager::eject_device (self=0x7f8190009600,
pci_segment_id=0, device_id=3) at vmm/src/device_manager.rs:4000
0x0000561891b674bc in <vmm::device_manager::DeviceManager as vm_device::bus::BusDevice>::write (
self=0x7f8190009600, base=70368744108032, offset=8, data=&[u8](size=4) = {...})
at vmm/src/device_manager.rs:4625
0x00005618921927d5 in vm_device::bus::Bus::write (self=0x7f8190006e00, addr=70368744108040,
data=&[u8](size=4) = {...}) at vm-device/src/bus.rs:235
0x0000561891b72e10 in <vmm::vm::VmOps as hypervisor::vm::VmmOps>::mmio_write (
self=0x7f81900097b0, gpa=70368744108040, data=&[u8](size=4) = {...}) at vmm/src/vm.rs:378
0x0000561892133ae2 in <hypervisor::kvm::KvmVcpu as hypervisor::cpu::Vcpu>::run (
self=0x7f8190013c90) at hypervisor/src/kvm/mod.rs:1114
0x0000561891914e85 in vmm::cpu::Vcpu::run (self=0x7f819001b230) at vmm/src/cpu.rs:348
0x000056189189f2cb in vmm::cpu::CpuManager::start_vcpu::{{closure}}::{{closure}} ()
at vmm/src/cpu.rs:953
Signed-off-by: Bo Chen <chen.bo@intel.com>
Since both Net and vhost_user::Net implement the Migratable trait, we
can factorize the common part to simplify the code related to the net
creation.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since both Block and vhost_user::Blk implement the Migratable trait, we
can factorize the common part to simplify the code related to the disk
creation.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend the validate() function for both DiskConfig and NetConfig so that
we return an error if a vhost-user-block or vhost-user-net device is
expected to be placed behind the virtual IOMMU. Since these devices
don't support this feature, we can't allow iommu to be set to true in
these cases.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Reorganizing the code to leverage the same mechanics implemented for
vfio-user and aimed at supporting sparse memory mappings for a single
region.
Relying on the capabilities returned by the vfio-ioctls crate, we create
a list of sparse areas depending if we get SPARSE_MMAP or MSIX_MAPPABLE
capability, or a single sparse area in case we couldn't find any
capability.
The list of sparse areas is then used to create both the memory mappings
in the Cloud Hypervisor address space and the hypervisor user memory
regions.
This allowed for the simplification of the MmioRegion structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This is a cleaner approach to handling the I/O port write to 0x80.
Whilst doing this also use generate the timestamp at the start of the VM
creation. For consistency use the same timestamp for the ARM equivalent.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We don't use the VmmOps trait directly for manipulating memory in the
core of the VMM as it's really designed for the MSHV crate to handle
instruction decoding. As I plan to make this trait MSHV specific to
allow reduced locking for MMIO and PIO handling when running on KVM this
use should be removed.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Instead of always creating a single large mmap for the MMIO region of a
BAR, we create multiple mmaps for the BARs that need multiple kvm user
memory regions. In this way, we can simplify 'unmap_mmio_regions()' (by
reusing information kept from 'MmioRegion::user_memory_region').
Signed-off-by: Bo Chen <chen.bo@intel.com>
To correctly map MMIO regions to the guest, we will need to wait for valid
MMIO region information which is generated from 'PciDevice::allocate_bars()'
(as a part of 'DeviceManager::add_pci_device()').
Signed-off-by: Bo Chen <chen.bo@intel.com>
To prepare the support of region capabilities, this commit factorizes
the `VFIO_USER_DEVICE_GET_REGION_INFO` operation in its own function. In
this process, it also removes the need of the hard-coded arbitrary value
for `vfio_region_info.argsz` (the maximum size of the reply payload).
Signed-off-by: Bo Chen <chen.bo@intel.com>
Similar to what's being supported for vfio devices, vfio-user devices
may also have BARs that need multiple kvm user memory regions,
e.g. device regions with `VFIO_REGION_INFO_CAP_SPARSE_MMAP`.
Signed-off-by: Bo Chen <chen.bo@intel.com>
For devices that cannot be named by the user use the "__" prefix to
identify them as internal devices. Check that any identifiers provided
in the config do not clash with those internal names. This prevents the
user from creating a disk such as "__serial" which would then cause a
failure in unpredictable manner.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
All hotpluggable devices were properly removed from the VmConfig when a
remove-device command was issued, except for the "fs" type. Fix this
lack of support as it is causing the integration tests to fail with the
recent addition of verifying that identifiers are unique.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The device identifiers generated from the DeviceManager were not
guaranteed to be unique since they were not taking the list of
identifiers provided through the configuration.
By returning the list of unique identifiers from the configuration, and
by providing it to the DeviceManager, the generation of new identifiers
can rely both on the DeviceTree and the list of IDs from the
configuration.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This prevents a conflict since the old API socket will not have been
cleaned up (due to the use of SIGKILL.)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The socket will safely deleted on shutdown and so it is not necessary to
delete the API socket when starting the HTTP server.
Fixes: #4026
Signed-off-by: LiHui <andrewli@kubesphere.io>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Start loading the kernel as possible in the VM in a separate thread.
Whilst it is loading other work can be carried out such as initialising
the devices.
The biggest performance improvement is seen with a more complex set of
devices. If using e.g. four virtio-net devices then the time to start the
kernel improves by 20-30ms. With the simplest configuration the
improvement was of the order of 2-3ms.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the same code for generating the kernel command line to be
used on both aarch64 and x86_64 when the latter starts loading the
kernel in asynchronously.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This is not required for x86_64 and maintains a tight coupling between
kernel loading and the DeviceManager.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This reverts commit 87eed369cd.
The reason we're reverting this is that OpenAPI Specification[0] doesn't
know how to deal with unsigned types. :-/
Right now the best to do is keep it as it's, as an int64, and try to fix
OpenAPI, or even switch to swagger, as the latter knows how to properly
deal with those. However, switching to swagger is far from being an 1:1
transition and will require time to experiment, thus reverting this for
now seems the best approach.
[0]: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#data-types
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The Token Bucket fields are, on the Cloud Hypervisor side, u64.
However, we expose those as int64 in the OpenAPI YAML file.
With that in mind, let's adjust the yaml file to expose those as uint64.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This means that the automatic enabling of the virtio-iommu will also be
applied to VMs creates via the API as well as the CLI.
Fixes: #4016
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Improve the request parsing/handling code by allowing an error status to
be returned back to the guest driver before we return an error
internally.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend the Domain structure to store the information about each domain
being in bypass mode or not. Based on this new information, the address
translation of the virtio devices is performed according to the bypass
mode of each domain. And both MAP/UNMAP requests are generating errors
in case the domain has been previously set to bypass mode.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for associating more than mappings with a domain, we
factorize the list of mappings associated with a domain behind a
dedicated Domain structure. We also update the field name so that it
reads better in the code.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Exposing the VIRTIO_IOMMU_F_BYPASS_CONFIG feature to the guest, which
allows to update the bypass global knob through virtio configuration.
Based on the value of this global knob, the address translations for
endpoints that have not been added to a domain is allowed with a simple
identity mapping.
By default, we enable the bypass mode for all endpoints that are not
attached to any domain.
Fixes#3987
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the VIRTIO specification, we must be able to support multiple
endpoints per domain. This is fixed along with the introduction of some
simplification regarding how we can retrieve the external mapping
directly based on the endpoint.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
If using the ACPI based hotplug only memory can be added so if the
hotplug RAM size is the same as the boot RAM size then do not include
the memory manager DSDT entries.
Also: this change simplifies the code marginally by making the
HotplugMethod enum Copyable.
This was identified from the following perf output:
1.78% 0.00% vmm cloud-hypervisor [.] <vmm::memory_manager::MemorySlots as acpi_tables::aml::Aml>::append_aml_bytes
|
---<vmm::memory_manager::MemorySlots as acpi_tables::aml::Aml>::append_aml_bytes
<vmm::memory_manager::MemorySlot as acpi_tables::aml::Aml>::append_aml_bytes
acpi_tables::aml::Name::new
<acpi_tables::aml::Path as acpi_tables::aml::Aml>::append_aml_bytes
__libc_malloc
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
No further changes are necessary that adding a #[derive(Error)] as there
is a manual implementation of Display.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Extend VfioCommon to simplify the overall code, and also in preparation
for supporting the restore code path.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend VfioCommon structure to own the legacy interrupt manager. This
will be useful for implementing the restore code path.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend VfioCommon structure to own the MSI interrupt manager. This will
be useful for implementing the restore code path.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We need to split the parsing functions into one function dedicated to
the actual parsing and a second function for initializing the interrupt
type. This will be useful on the restore path as the parsing won't be
needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding support for snapshot and restore to the MsiConfig structure, as
it will be needed part of VFIO migration.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This carries a string that is exposed via DMI/SMBIOS and is particularly
useful for cloud-init initialisation.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This is particularly helpful as the cloud-init DMI serial number
encoding scheme uses "=" and ";" for key value pairs.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the guest has not activated the virtio-mem device then reject an
attempt to resize using it.
Fixes: #4001
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Live upgrade is currently not guaranteed during this development cycle
and we will try to enable these tests after the next release.
Signed-off-by: Bo Chen <chen.bo@intel.com>
By augmenting existing set of tests, this patch added a set of
tests for live-upgrade that covers use cases with NUMA,
vhost-user (OVS-DPDK), and local-migration.
Fixes: #3949
Signed-off-by: Bo Chen <chen.bo@intel.com>
Use a single enum member for representing errors from the internal API.
This avoids the ugly duplication of the API call name in the error
message:
e.g.
$ target/debug/ch-remote --api-socket /tmp/api resize --cpus 2
Error running command: Server responded with an error: InternalServerError: VmResize(VmResize(CpuManager(DesiredVCpuCountExceedsMax)))
Becomes:
$ target/debug/ch-remote --api-socket /tmp/api resize --cpus 2
Error running command: Server responded with an error: InternalServerError: ApiError(VmResize(CpuManager(DesiredVCpuCountExceedsMax)))
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In case a list of resources is provided to allocate_bars(), it directly
means we're restoring some existing BARs. That's why we shouldn't share
the codepath that creates BARs from scratch as we don't need to interact
with the device to retrieve the information.
Whenever resources are provided, we simply iterate over the list of
possible BAR indexes and create the BARs if the resource could be found.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of defining some very generic resources as PioAddressRange or
MmioAddressRange for each PCI BAR, let's move to the new Resource type
PciBar in order to make things clearer. This allows the code for being
more readable, but also removes the need for hard assumptions about the
MMIO and PIO ranges. PioAddressRange and MmioAddressRange types can be
used to describe everything except PCI BARs. BARs are very special as
they can be relocated and have special information we want to carry
along with them.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to make the code more consistent and easier to read, we remove
the former tuple that was used to describe a BAR, replacing it with the
existing structure PciBarConfiguration.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The code was quite unclear regarding the type of index that was being
used regarding a BAR. This is improved by differenciating register
indexes and BAR indexes more clearly.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By factorizing the algorithm untangling TDVF sections from guest RAM
into a dedicated function, we can write some unit tests to validate it
properly achieves what we expect.
Adding the "tdx" feature to the unit tests, otherwise it wouldn't get
tested.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The I/O operations knobs are prefixed `ops_` rather than `bw_`, as `bw_`
refers to the "bandwidth" knobs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
By adding a new method id() to the PciDevice trait, we allow the caller
to retrieve a unique identifier. This is used in the context of BAR
relocation to identify the device being relocated, so that we can update
the DeviceTree resources for all PCI devices (and not only
VirtioPciDevice).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Relying on the list of resources, VfioCommon is now able to allocate the
BARs at specific addresses. This will be useful for restoring VFIO and
vfio-user devices.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By returning the new PCI resources from add_pci_device(), we allow the
factorization of the code translating the BARs into resources. This
allows VIRTIO, VFIO and vfio-user to add the resources to the DeviceTree
node.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Relying on the function introduced recently to get the PCI resources and
handle the restore case, both VFIO and vfio-user device creation paths
now have access to PCI resources, which can be provided to the function
add_pci_device().
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Create a dedicated function for getting the PCI segment, b/d/f and
optional resources. This is meant for handling the potential case of a
restore.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Updating the way of restoring BAR addresses for virtio-pci by providing
a more generic approach that will be reused for other PciDevice
implementations (i.e VfioPcidevice and VfioUserPciDevice).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The calls to these functions are always preceded by a call to
InterruptSourceGroup::update(). By adding a masked boolean to that
function call it possible to remove 50% of the calls to the
KVM_SET_GSI_ROUTING ioctl as the the update will correctly handle the
masked or unmasked case.
This causes the ioctl to disappear from the perf report for a boot of
the VM.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
It suffered from the same issue like the main program after switching to
Rust 2021 edition, but the issue was not caught by the CI.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
EDK2 execution requires a flash device at address 0.
The new added device is not a fully functional flash. It doesn't
implement any spec of a flash device. Instead, a piece of memory is used
to simulate the flash simply.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
We modified a test case to workaround the RAM calculation error caused
by hidding 4MiB memory for UEFI. Now change it back to normal.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Size of `memory` node in FDT was reduced by 4MiB. Now it is returned.
In memory regions, a `Ram` region of 4MiB was created at address 0. Now
it is removed.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Rust 2021 edition has a few improvements over the 2018 edition. Migrate
the project to 2021 edition by following recommended migration steps.
Luckily, the code itself doesn't require fixing.
Bump MSRV to 1.56 as it is required by the 2021 edition. Also fix the
clap build dependency to make Cloud Hypervisor build again.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This is a refactoring commit to simplify source code.
Removed some functions that only return a layout const.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Some addresses defined in `layout.rs` were of type `GuestAddress`, and
are `u64`. Now align the types of all the `*_START` definitions to
`GuestAddress`.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Now that address translations performed by virtio-iommu can error out if
the address can't be translated, we uncovered an issue in integration
test aarch64_acpi::test_virtio_iommu.
We disable the test until we can investigate and fix the root cause.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It doesn't matter if we're trying to translate a GVA or a GPA address,
but in both cases we must error out if the address couldn't be
translated.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Remove EOLed distributions and clarify the stability guarantees and what
the experimental features are.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The reserved space is for devices.
Some devices (like TPM) require arbitrary addresses close to 4GiB.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
`RAM_64BIT_START` was set to 1 GiB, not a real 64-bit address. Now
rename it `RAM_START` to avoid confusion.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
The extra vDPA device in the test is hotplugged behind the vIOMMU, which
covers the use case of placing a vDPA device behind a virtual IOMMU.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Whenever a virtio device is placed behind a vIOMMU, we have some code in
pci_common_config.rs to translate the queue addresses (descriptor table,
available ring and used ring) from GVA to GPA, so that they can be used
correctly.
But in case of vDPA, we also need to provide the queue addresses to the
vhost backend. And since the vhost backend deals with consistent IOVAs,
all addresses being provided should be GVAs if the device is placed
being a vIOMMU. For that reason, we perform a translation of the queue
addresses back from GPA to GVA if necessary, and only to be provided to
the vhost backend.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Add a new iommu parameter to VdpaConfig in order to place the vDPA
device behind a virtual IOMMU.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In case an external mapping would have been added after the virtio-iommu
device has been activated, it would have simply be ignored because the
code wasn't using a shared object between the vmm thread and the iommu
thread. This behavior is only triggered on the hotplug codepath, and
only if the hotplugged device is placed behind the virtual IOMMU.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for the vDPA need to translate a GPA back into a GVA, we
extend the existing trait DmaRemapping and AccessPlatform to perform
such operation.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Renaming translate() to translate_gva() to clarify we want to translate
a GVA address into a GPA.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
According to latest official images based on latest TDX guest kernel, we
update the documentation to reflect the change regarding serial support.
New guest kernels have been updated to disable usage of serial ports,
meaning adding console=ttyS0 to the kernel boot parameters will have no
effect.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The list of memory resources provided through the HOB wasn't accurate
because of the broken logic. The fix provides correct ranges to the
firmware.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on latest QEMU patches from branch tdx-qemu-2022.03.29-v7.0.0-rc1
we should only report as memory resources the TempMem sections from TDVF
sections.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on latest QEMU patches from branch tdx-qemu-2022.03.29-v7.0.0-rc1
we don't need EFI_RESOURCE_ATTRIBUTE_ENCRYPTED as part of the attributes
we must enable with EFI_RESOURCE_SYSTEM_MEMORY and
EFI_RESOURCE_MEMORY_RESERVED resource types.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The introduction of a error if live resizing is not possible is a
regression compared to the original behaviour where the new size would
be stored in the config and reflected in the next boot. This behaviour
was also inconsistent with the effect of resizing with no VM booted.
Instead of generating an error allow the code to go ahead and update the
config so that the new size will be available upon the reboot.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
As reported by the periodic CI runs, it may take more time for the NVMe
device to present in the guest after being hotplugged as a VFIO user
device on `aarch64` (especially under high load). Let's increase the
timeout after device hotplug from `1s` to `10s` to increase the test
stability.
Fixes: #3495
Signed-off-by: Bo Chen <chen.bo@intel.com>
Similarly to the previous commit restricting the cpu resizing error only
to the situations where the vcpu amount has changed, let's do the same
with the memory and be consistent throughout our code base.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
188078467d made clear that resize should
only happen when dealing with a "dynamic" CpuManager. Although this is
very much correct, it causes a regression on Kata Containers (and on any
other consumer of Cloud Hypervisor) in cases where a resize would be
triggered but the vCPUs values wouldn't be changed.
There's no doubt Kata Containers could do better and do not call a
resize in such situations, and that's something that should **also** be
solved there. However, we should also work this around on Cloud
Hypervisor side as it introduces a regression with the current Kata
Containers code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Extend the Hypervisor API in order to retrieve the TDX capabilities from
the underlying hypervisor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By enabling the VIRTIO feature VIRTIO_F_IOMMU_PLATFORM for all
vhost-user devices when needed, we force the guest to use the DMA API,
making these devices compatible with TDX. By using DMA API, the guest
triggers the TDX codepath to share some of the guest memory, in
particular the virtqueues and associated buffers so that the VMM and
vhost-user backends/processes can access this memory.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
If EFI reset fails on the Linux kernel then it will fallthrough to CMOS
reset. Implement this as one of our reset solutions.
Fixes: #3912
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Compile this feature in by default as it's well supported on both
aarch64 and x86_64 and we only officially support using it (no non-acpi
binaries are available.)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
These were erroneously skipping features for the unit tests and the
"build" target for dev_cli.sh
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
These are never run by the CI and is inconsistent with the way we build
test which is specified inside the .github workflows.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This includes the removal of testing without the "acpi" feature. The
command have been reordered to reduce the amount of recompilation
required.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
It seems the vdpa_sim_block isn't behaving properly after the vhost
device is closed, as it sometimes returns EBUSY when we try to open it
again. The easiest way to deal with this issue is by simplifying the
integration test, avoid to plug the same device after it's been
unplugged.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit unifies the custom linux kernel build in x86, Arm, and
performance metrics to the same function. Therefore, when bumping
the kernel version, we can make sure we only need to make the change
in one place.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
With the addition of the amx feature, add a new build workflow to
enable the feature and a clippy quality check.
Signed-off-by: William Douglas <william.douglas@intel.com>
AMX is an x86 extension adding hardware units for matrix
operations (int and float dot products). The goal of the extension is
to provide performance enhancements for these common operations.
On Linux, AMX requires requesting the permission from the kernel prior
to use. Guests wanting to make use of the feature need to have the
request made prior to starting the vm.
This change then adds the first --cpus features option amx that when
passed will enable AMX usage for guests (needs a 5.17+ kernel) or
exits with failure.
The activation is done in the CpuManager of the VMM thread as it
allows migration and snapshot/restore to work fairly painlessly for
AMX enabled workloads.
Signed-off-by: William Douglas <william.douglas@intel.com>
Now that we rely on vhost v0.4.0, which contains the fix for
get_iova_range(), we don't need the workaround anymore, and we can
actually call into the dedicated function.
Fixes#3861
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Rely on newly released versions of the vhost and vhost-user-backend
crates from rust-vmm.
The new vhost version includes the fixes needed for vDPA.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit makes sure that the custom linux kernel is always
rebuilt when running the performance metrics tests, and therefore
changes to the kernel config file is always caught.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Explain the reason why using vDPA might be interesting and how to use it
with Cloud Hypervisor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Disable the DAX feature from the virtio-fs implementation as the feature
is still not stable. The feature is deprecated, meaning the 'dax'
parameter will be removed in about 2 releases cycles.
In the meantime, the parameter value is ignored and forced to be
disabled.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The test is sporadically failing whenever we try to hotplug the vDPA
device we've just unplugged. This is causing the kernel to complain with
EBUSY because the device hasn't been released yet. This is happening
because the CI system is under very high load, therefore taking quite
some time to the host to update the state of this device.
The easy way to fix such issue is by increasing the sleep time between
the unplug and the replug.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When running non-dynamic or with virtio-mem for hotplug the ACPI
functionality should not be included on the DSDT nor does the
MemoryManager need to be placed on the MMIO bus.
Fixes: #3883
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This is now consistent with not supplying the _CRS for the device when
CpuManager is not dynamic.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The file descriptor provided to fs_slave_map() and fs_slave_io() is
passed as a AsRawFd trait, meaning the caller owns it. For that reason,
there's no need for these functions to close the file descriptor as it
will be closed later on anyway.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
As ethr is not stable on AArch64, disable it until we find other tool.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
fio test depends on hot-add disk which needs uefi on AArch64, that
will lead to lots of performance issues. So, it's better to use
cold-plug disk.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
On Arm64, the fio binary that comes from ubuntu distribution cannot
support io_uring, which is a required engine for the block
performance tests in performance metrics. Therefore this commit
build the fio from a relatively new version, and replace the native
ubuntu fio with the newly build one.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
To emulate debug I/O port on arm64, we need trap from guest to VMM.
Generally, there are 3 ways to go:
1) execute a privilege intruction like "wfi";
2) access a system register;
3) access device MMIO region;
However, 1) and 2) often handled in kvm, thus 3) it's the choice.
There maybe lots of repeated work and enlarge the size of clh to develop
a new device, so it's better to reuse the current device. Luckily, I
find that pl011 is the eariest device initialized in kernel and, there
is reserved region in it which we can reuse for emulating debug I/O.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Rather than just printing a message return an error back through the API
if the user attempts to hotplug a device that supports being behind an
IOMMU where that device isn't placed on an IOMMU segment.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Ensure devices that are specified to be on a PCI segment that is behind
the IOMMU are IOMMU enabled if possible or error out for those devices
that do not support it.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Previously it was not possible to enable vIOMMU for a virtio device.
However with the ability to place an entire PCI segment behind the
IOMMU the IOMMU mapping needs to be setup for the virtio device if it is
behind the IOMMU.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This can already be calculated by the summing the tables reported by the
Linux kernel but this is more convenient.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Changes in commit "scripts: aarch64: Reduce source code build in CI"
for the Linux kernel will lead to the kernel cannot be rebuilt if
the config file is changed. Therefore should be reverted.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
As Linux kernel, edk2, virtiofd, SPDK are all build from source, we
need to find a way to reduce the number of source code build.
To address this issue, this commit adds a stage that touching a
".built" file to each repo of the source code after a successfully
build. By checking if the ".built" file exists, we can determine
if there is already an old build. Since the source code repo will
be removed and cloned again when we bump the repo hash, the CI will
rebuild the repo.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Whenever a vfio-user device is dropped, the communication between the
VMM and the backend should be shutdown.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Separate the destruction and cleanup of original VM and the creation of
the new one. In particular have a clear hand off point for resources
(e.g. reset EventFd) used by the new VM from the original. In the
situation where vm.shutdown() generates an error this also avoids the
Vmm reference to the Vm (self.vm) from being maintained.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding two new integration tests for vDPA, relying on both block and net
simulators from the host kernel.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the newly added Vdpa device along with the new vdpa parameter,
this patch enables the support for vDPA devices.
It's important to note this the only virtio device for which we provide
an ExternalDmaMapping instance. This will allow for the right DMA ranges
to be mapped/unmapped.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Introduce a new --vdpa parameter associated with a VdpaConfig for the
future creation of a Vdpa device.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
vDPA is a kernel framework introduced fairly recently in order to handle
devices complying with virtio specification on their datapath, while the
control path is vendor specific. For the datapath, that means the
virtqueues are handled through DMA directly between the hardware and the
guest, while the control path goes through the vDPA framework,
eventually exposed through a vhost-vdpa device.
vDPA, like VFIO, aims at achieving baremetal performance for devices
that are passed into a VM. But unlike VFIO, it provides a simpler/better
framework for achieving migration. Because the DMA accesses between the
device and the guest are going through virtio queues, migration can be
achieved way more easily, and doesn't require each device driver to
implement the migration support. In the VFIO case, each vendor is
expected to provide an implementation of the VFIO migration framework,
which makes things harder as it must be done for each and every device.
So to summarize the point is to support migration for hardware devices
through which we can achieve baremetal performances.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Update the content of the documentation to clarify how the cache option
from virtiofsd has an impact on the host page cache, while the dax
option for the filesystem in the guest has an impact on the guest page
cache.
The document is also updated to not have lines exceeding 80 characters.
Fixes#3826
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Document the reason why someone might need to use the balloon device and
detail the different parameters associated with it.
Fixes#3807
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This will significantly reduce the size of the DSDT and the effort
required to parse them if there is no requirement to support
hotplug/unplug.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the CpuManager is dynamic it devices CPUs can be
hotplugged/unplugged.
Since TDX does not support CPU hotplug this is currently the only
determinator as to whether the CpuManager is dynamic.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since the code has been adapted to support VIRTIO_F_EVENT_IDX we use
Queue::needs_notification() to determine whether to signal the guest so
it is no longer necessary to check if there are any used descriptors. If
the feature is not negotiated then Queue::needs_notification() will
return true triggering an interrupt of the guest. Theoretically this
could be a spurious interrupt of the guest if there were no used used
descriptors but this is unlikely as we only generate used descriptors
for the control queue as a result of an interrupt of the VMM by the
guest.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This indicates if anything has been added to the available queue since
it was last iterated through. If it returns true then it is necessary to
iterate through the queue again otherwise it is appropriate to break out
from the loop.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
vmm.ping/vm.info will hang for PUT method, vm.create/vmm.shutdonw hang for GET method.
Because these four APIs do not write the response body when the HTTP method does not match.
Signed-off-by: LiHui <andrewli@kubesphere.io>
In case the virtio device which requires DMA mapping is placed behind a
virtual IOMMU, we shouldn't map/unmap any region manually. Instead, we
provide the DMA handler to the virtio-iommu device so that it can
trigger the proper mappings.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
If a virtio device is associated with a DMA handler, the DMA mapping and
unmapping is performed from the device manager through the handler.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Given that some virtio device might need some DMA handling, we provide a
way to store this through the VirtioPciDevice layer, so that it can be
accessed when the PCI device is removed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for handling potential DMA mapping/unmapping operations for a
virtio device, we extend the MetaVirtioDevice with an additional field
that holds an optional DMA handler.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The tuple of information related to each virtio device is too big, and
it's better to factorize it through a dedicated structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
With the VIRTIO_F_EVENT_IDX handling now conducted inside the
virtio-queue crate it is necessary to activate the functionality on
every queue if it is negotiatated. Otherwise this leads to a failure of
the guest to signal to the host that there is something in the available
queue as the queue's internal state has not been configured correctly.
Fixes: #3829
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When mask a msi irq, we set the entry.masked to be true, so kvm
hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING
ioctl which update kvm->irq_routing. This will trigger kernel
panic on AMD platform when the gsi is the largest one in kernel
kvm->irqfds.items:
crash> bt
PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8"
#0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397
#1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d
#2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d
#3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d
#4 [ffffb1ba6707fb90] no_context at ffffffff856692c9
#5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51
#6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace
[exception RIP: svm_update_pi_irte+227]
RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086
RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001
RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8
RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200
R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001
R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm]
#8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm]
#9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm]
RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b
RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020
RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0
R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0
R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0
ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b
To solve this problem, move route.disable() before set_gsi_routes() to
remove the gsi from irqfds.items first.
This problem only exists on AMD platform, 'cause on Intel platform
kernel just return when update irte while it only prints a warning on
AMD.
Also, this patch adjusts the order of enable() and set_gsi_routes() in
unmask(), which should do no harm.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
When mask a msi irq, we set the entry.masked to be true, so kvm
hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING
ioctl which update kvm->irq_routing. This will trigger kernel
panic on AMD platform when the gsi is the largest one in kernel
kvm->irqfds.items:
crash> bt
PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8"
#0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397
#1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d
#2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d
#3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d
#4 [ffffb1ba6707fb90] no_context at ffffffff856692c9
#5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51
#6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace
[exception RIP: svm_update_pi_irte+227]
RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086
RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001
RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8
RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200
R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001
R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm]
#8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm]
#9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm]
RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b
RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020
RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0
R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0
R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0
ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b
To solve this problem, move route.disable() before set_gsi_routes() to
remove the gsi from irqfds.items first.
This problem only exists on AMD platform, 'cause on Intel platform
kernel just return when update irte while it only prints a warning on
AMD.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Reduce the number of open pull requests dependabot will create to avoid
excessive rebase and rebuilds of PRs. This avoids a situation with a 2nd
PR being rebased after a 1st is merged resulting in a wasteful rebuild.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Move to release version v0.2.0 for both vm-virtio and vhost-user-backend
crates rather than relying on their main branch, as they might be
subject to breaking changes.
Fixes#3800
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Replace the thread for handling SIGSYS with a simple signal handler.
This resolves inconsistent delivery of signals to the SIGSYS thread due
to other threads manipulating the signals.
Tested by removing key syscalls from vCPU and virtio device filters and
observing correct notice.
Fixes: #3811
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
To be consistently with the integration scripts, we now pass through the
`--test-filter` option of the test scripts to the `performance-metrics`
binary. Now the following two commands achieve the same results:
`$./scripts/dev_cli.sh tests --metrics -- --test-filter boot_time_ -- --test-filter latency`
`$./scripts/dev_cli.sh tests --metrics -- -- --test-filter boot_time_ --test-filter latency`
Note that, the test scripts support only single occurrence of the
`--test-filter` option while the `performance-metrics` binary allows
multiple occurrences for filtering based on different keywords.
Fixes: #3787
Signed-off-by: Bo Chen <chen.bo@intel.com>
Add a field for its length and fix up users.
Things work just because all hardcoded values agree with each other.
This is prone to breakage.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
In the context of virtualization, it is always convenient to be able to share a directory from the host with the guest.
In the context of virtualization, it is always convenient to be able to share a
directory from the host with the guest.
__virtio-fs__, also known as __vhost-user-fs__ is a virtual device defined by the VIRTIO specification which allows any VMM to perform filesystem sharing.
__virtio-fs__, also known as __vhost-user-fs__ is a virtual device defined by
the VIRTIO specification which allows any VMM to perform filesystem sharing.
## Pre-requisites
### The daemon
This virtual device relies on the _vhost-user_ protocol, which assumes the backend (device emulation) is handled by a dedicated process running on the host. This daemon is called __virtiofsd__ and needs to be present on the host.
This virtual device relies on the _vhost-user_ protocol, which assumes the
backend (device emulation) is handled by a dedicated process running on the
host. This daemon is called __virtiofsd__ and needs to be present on the host.
_Build virtiofsd_
```bash
@@ -25,31 +29,46 @@ mkdir /tmp/shared_dir
_Run virtiofsd_
```bash
./virtiofsd \
-d\
--log-level debug\
--socket-path=/tmp/virtiofs \
--shared-dir=/tmp/shared_dir \
--cache=never
--cache=never\
--thread-pool-size=$N
```
The `cache=never` option should be the default when using `virtiofsd` with the __cloud-hypervisor__ VMM. This prevents from using the guest page cache, which reduces the memory footprint of the guest. When running multiple virtual machines on the same host, this will let the host deal with page cache, which will increase the density of virtual machines which can be launched.
The `cache=never` option is the default when using `virtiofsd` with
Cloud Hypervisor. This prevents from using the host page cache, reducing the
overall footprint on host memory. This increases the maximum density of virtual
machines that can be launched on a single host.
The `cache=always` option will allow for the guest page cache to be used, which will increase the memory footprint of the guest. This option should be used only for specific use cases where a single VM is going to be running on a host.
The `cache=always` option will allow the host page cache to be used, which can
result in better performance for the guest's workload at the cost of increasing
the footprint on host memory.
The `thread-pool-size` option controls how many IO threads are spawned. For
very fast storage like NVMe spawning enough worker threads is critical to
getting an acceptable performance compared to native.
### Kernel support
Modern Linux kernels starting (at least v5.10) have support for virtio-fs. Use
of older kernels, with additional patches, are not supported.
Modern Linux kernels (at least v5.10) have support for virtio-fs. Use of older
kernels, with additional patches, are not supported.
## How to share directories with cloud-hypervisor
### Start the VM
Once the daemon is running, the option `--fs` from __cloud-hypervisor__ needs to be used.
Direct kernel boot is the preferred option, but we can boot from an EFI cloud image if it contains a recent enough kernel.
Once the daemon is running, the option `--fs` from Cloud Hypervisor needs
to be used.
Because _vhost-user_ expects a dedicated process (__virtiofsd__ in this case) to be able to access the guest RAM to communicate through the _virtqueues_ with the driver running in the guest, `--memory` option needs to be slightly modified. It must specify `shared=on` to share the memory pages so that an external process can access them.
Both direct kernel boot and EFI firmware can be used to boot a VM with
virtio-fs, given that the cloud image contains a recent enough kernel.
Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your system, here is the __cloud-hypervisor__ command you need to run:
Correct functioning of `--fs` requires `--memory shared=on` to facilitate
interprocess memory sharing.
Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your
system, here is the Cloud Hypervisor command you need to run:
```bash
./cloud-hypervisor \
--cpus boot=1\
@@ -60,26 +79,20 @@ Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your system
By default, DAX is enabled with a cache window of 8GiB. You can specify a custom size (let's say 4GiB for this example) for the cache by explicitly setting DAX and the cache size:
In case you don't want to use a shared window of cache to pass the shared files content, this means you will have to explicitly disable DAX with `dax=off`. Note that in this case, the `cache_size` parameter will be ignored.
The last step is to mount the shared directory inside the guest, using the `virtiofs` filesystem type.
The last step is to mount the shared directory inside the guest, using the
`virtiofs` filesystem type.
```bash
mkdir mount_dir
mount -t virtiofs -o dax myfs mount_dir/
mount -t virtiofs myfs mount_dir/
```
The `tag` needs to be consistent with what has been provided through the __cloud-hypervisor__ command line, which happens to be `myfs` in this example.
The `-o dax` option must be removed in case the shared cache region is not enabled from the VMM.
The `tag` needs to be consistent with what has been provided through the
Cloud Hypervisor command line, which happens to be `myfs` in this example.
## DAX feature
Given the DAX feature is not stable yet from a daemon standpoint, it is not
If profiling with a network device attached either the TAP device must be already created and configured or the profiling must be done as root so that the TAP device can be created.
## Userspace only profiling with LBR
The use of LBR (Last Branch Record; available since Haswell) offers lower
overhead if only userspace profiling is required. This lower overhead can allow
a higher frequency of sampling. This also removes the requirement to compile
with custom `RUSTFLAGS` however debug symbols should still be included:
e.g.
```
$ perf record --call-graph lbr --all-user --user-callchains -g target/release/cloud-hypervisor \
This document provides guidance to users, downstream maintainers and
any other consumers of the Cloud Hypervisor project, this document
describes the release process, release cadence, stability expectations and
related topics.
## Basic Terms
### Stability
For Cloud Hypervisor the following areas are subject to stability guarantees:
- [REST API](api.md#rest-api)
- [Command line options](api.md#command-line-interface)
- [Device Model](device_model.md)
- Device tree, device list, ACPI, Hyper-V enlightenments and any other
features exposed to guest
- KVM compatibility
- Rust edition compatibility
This list is incomplete but this document serves as a best effort guide to stability
across releases.
### Experimental features
Experimental features are under active development and no guarantees are made about their stability.
List of experimental features:
- TDX
- vfio-user
- vDPA
### Security
Security fixes should be included in a new point release.
For security issues an advisory will be published via the GitHub security advisory process along with the release. Watching the project on GitHub will notify you of those issues.
## Releases
### Versioning
The versioning scheme uses `MAJOR.POINT` pattern:
-`MAJOR` can introduce incompatible changes along with support for new features. Changes to the [API](api.md#rest-api),
[CLI options](api.md#command-line-interface) and [device model](device_model.md)
require a notice at least 2 releases in advance for the actual change to take
Cloud Hypervisor is under active development. A new major release is issued approximately
every 6 weeks. Point releases are issued on demand, when important bug fixes are in
the queue. A major release would receive bug fixes for the next two cycles (~12 weeks)
and then be considered EOL.
```
+ - Active release support
E - EOL
2021 2022 2023
| | | | | | | | |
18.0 | | | ++++++++E
19.0 | | | |++++++++E
20.0 | | | | ++++++++E
21.0 | | | | | ++++++++E
22.0 | | | | | +++++++++E
23.0 | | | | | | +++++++++E
```
### Major Release Stability Considerations
Snapshot/restore support is not compatible across `MAJOR` versions.
Live migration support is not compatible across `MAJOR` versions.
### LTS Release Cadence
A regular release is promoted to LTS every 12 months. An LTS release is supported for 18 months. This gives a 6 months window for users to move to the new LTS.
@@ -6,30 +6,31 @@ use super::{vnet_hdr_len, MacAddr, Tap, TapError};
usestd::net::Ipv4Addr;
usestd::path::Path;
usestd::{fs,io};
usethiserror::Error;
#[derive(Debug)]
#[derive(Error, Debug)]
pubenumError{
/// Failed to convert an hexadecimal string into an integer.
#[error("Failed to convert an hexadecimal string into an integer: {0}")]
ConvertHexStringToInt(std::num::ParseIntError),
/// Error related to the multiqueue support (no support TAP side).
#[error("Error related to the multiqueue support (no support TAP side).")]
MultiQueueNoTapSupport,
/// Error related to the multiqueue support (no support device side).
#[error("Error related to the multiqueue support (no support device side).")]
MultiQueueNoDeviceSupport,
/// Failed to read the TAP flags from sysfs.
#[error("Failed to read the TAP flags from sysfs: {0}")]
ReadSysfsTunFlags(io::Error),
/// Open tap device failed.
#[error("Open tap device failed: {0}")]
TapOpen(TapError),
/// Setting tap IP failed.
#[error("Setting tap IP failed: {0}")]
TapSetIp(TapError),
/// Setting tap netmask failed.
#[error("Setting tap netmask failed: {0}")]
TapSetNetmask(TapError),
/// Setting MAC address failed
#[error("Setting MAC address failed: {0}")]
TapSetMac(TapError),
/// Getting MAC address failed
#[error("Getting MAC address failed: {0}")]
TapGetMac(TapError),
/// Setting vnet header size failed.
#[error("Setting vnet header size failed: {0}")]
TapSetVnetHdrSize(TapError),
/// Enabling tap interface failed.
#[error("Enabling tap interface failed: {0}")]
TapEnable(TapError),
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
