error: all if blocks contain the same code at the end
--> vmm/src/memory_manager.rs:884:9
|
884 | / Ok(mm)
885 | | }
| |_________^
Signed-off-by: Bo Chen <chen.bo@intel.com>
This concept ends up being broken with multiple types on input connected
e.g. console on TTY and serial on PTY. Already the code for checking for
injecting into the serial device checks that the serial is configured.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Introduce a common solution for spawning the virtio threads which will
make it easier to add the panic handling.
During this effort I discovered that there were no seccomp filters
registered for the vhost-user-net thread nor the vhost-user-block
thread. This change also incorporates basic seccomp filters for those as
part of the refactoring.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Currently we need to test both device tree and ACPI on AArch64. As
the number of ACPI test cases is gradually increasing and expected
to increase in the future, it is better to extract all ACPI test
cases on AArch64 to a single module.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Current AArch64 power button is only for device tree using a PL061
GPIO controller device. Since AArch64 now supports ACPI, this
commit extend the power button on AArch64 to:
- Using GED for ACPI+UEFI boot.
- Using PL061 for device tree boot.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This message only occurs sporadically and so it should be included at
info!() level. Enhance the output to also include the BAR number.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
These statements are useful for understanding the cause of reset or
shutdown of the VM and are not spammy so should be included at info!()
level.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This patch adds a separate function to perform common numa checks, so
that we can reuse this function in other tests, e.g. the test for
live-migration with numa.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Despite setting up a dedicated thread for signal handling, we weren't
making sure that the signals we were listening for there were actually
dispatched to the right thread. While the signal-hook provides an
iterator API, so we can know that we're only processing the signals
coming out of the iterator on our signal handling thread, the actual
signal handling code from signal-hook, which pushes the signals onto
the iterator, can run on any thread. This can lead to seccomp
violations when the signal-hook signal handler does something that
isn't allowed on that thread by our seccomp policy.
To reproduce, resize a terminal running cloud-hypervisor continuously
for a few minutes. Eventually, the kernel will deliver a SIGWINCH to
a thread with a restrictive seccomp policy, and a seccomp violation
will trigger.
As part of this change, it's also necessary to allow rt_sigreturn(2)
on the signal handling thread, so signal handlers are actually allowed
to run on it. The fact that this didn't seem to be needed before
makes me think that signal handlers were almost _never_ actually
running on the signal handling thread.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Move the processing of the input from stdin, PTY or file from the VMM
thread to the existing virtio-console thread. The handling of the resize
of a virtio-console has not changed but the name of the struct used to
support that has been renamed to reflect its usage.
Fixes: #3060
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Downcasting of GicDevice trait might fail. Therefore we try to
downcast the trait first and only if the downcasting succeeded we
can then use the object to call methods. Otherwise, do nothing and
log the failure.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit implements the GIC (including both GICv3 and GICv3ITS)
Pausable trait. The pause of device manager will trigger a "pause"
of GIC, where we flush GIC pending tables and ITS tables to the
guest RAM.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This test exercises the local live-migration between two Cloud
Hypervisor VMs on the same host. It ensures the following behaviors:
1. The source VM is up and functional (including various virtio-devices
are working properly);
2. The 'send-migration' and 'receive-migration' command finished
successfully;
3. The source VM terminated gracefully after live migration;
4. The destination VM is functional (including various virtio-devices
are working properly) after live migration.
Note: This test does not use vsock as we can't create two identical
vsock on the same host.
Fixes: #2965
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch adds a dedicate function to include the common checks on the
virtio-devices from the 'test_snapshot_restore' test, which will also be
reused for the upcoming 'test_live_migration' test.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This prevents the boot of the guest kernel from being blocked by
blocking I/O on the serial output since the data will be buffered into
the SerialBuffer.
Fixes: #3004
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Introduce a dynamic buffer for storing output from the serial port. The
SerialBuffer implements std::io::Write and can be used in place of the
direct output for the serial device.
The internals of the buffer is a vector that grows dynamically based on
demand up to a fixed size at which point old data will be overwritten.
Currently the buffer is only flushed upon writes.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The rust-vmm crates we're pulling from git have renamed their main
branches. We need to update the branch names we're giving to Cargo,
or people who don't have these dependencies cached will get errors
like this when trying to build:
error: failed to get `vm-fdt` as a dependency of package `arch v0.1.0 (/home/src/cloud-hypervisor/arch)`
Caused by:
failed to load source for dependency `vm-fdt`
Caused by:
Unable to update https://github.com/rust-vmm/vm-fdt?branch=master#031572a6
Caused by:
object not found - no match for id (031572a6edc2f566a7278f1e17088fc5308d27ab); class=Odb (9); code=NotFound (-3)
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Remove the indirection of a dispatch table and simply use the enum as
the event data for the events.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Definition of kvm_tdx_init_vm used by INIT_VM has been updated in latest
kernel, needing an update on the Cloud Hypervisor side as well.
Update structure TdxInitVm to fit this change and avoid -EINVAL to be
returned by the kernel.
Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Moving to the latest kernel 5.14 simplifies greatly the list of patches
that we need to carry. Since virtio-iommu has been merged as part of the
5.14 release, the only remaining patches are virtio-watchdog and the EFI
reboot.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Currently the latest cloudhypervisor/dev docker container is the
multi-arch image. We can pull the arm image directly.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
To enable live-migration for vhost-user devices, we now always shutdown
the source VM when the migration is completed. This patch reflects this
change in the documentation.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This resolves an issue with hotplug -> removal -> hotplug of a vfio-user
device as the allocator was not updated with the now unused entries.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Use two separate events for the console and serial PTY and then drive
the handling of the inputs on the PTY separately. This results in the
correct behaviour when both console and serial are attached to the PTY
as they are triggered separately on the epoll so events are not lost.
Fixes: #3012
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Check the config to find out which device is attached to the tty and
then send the input from the user into that device (serial or
virtio-console.)
Fixes: #3005
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The type is to change from 32-bit to 64-bit. See
https://github.com/rust-lang/libc/issues/1848.
The change is announced via a deprecation warning. Cloud Hypervisor's
code does not need changing. Simply suppress these warnings.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The Jenkins master is now known as the controller and the agent it provides is
called "built-in".
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The fuzzer needs to take a larger input for the whole disk image to
be most useful. Since the file is small we can test by reading and
writing over the whole file.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Signed-off-by: Fazla Mehrab <akm.fazla.mehrab@intel.com>
Two tests for booting Linux cloud image from the different VHDx files:
fixed and dynamic. Another test for testing the dynamic expansion of a
generated VHDx file.
Signed-off-by: Fazla Mehrab <akm.fazla.mehrab@intel.com>
vhdx_sync.rs in block_util implements traits to represent the vhdx
crate as a supported block device in the cloud hypervisor. The vhdx
is added to the block device list in device_manager.rs at the vmm
crate so that it can automatically detect a vhdx disk and invoke the
corresponding crate.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Fazla Mehrab <akm.fazla.mehrab@intel.com>
Microsoft’s VHDx block device format specification is implemented
here as a crate. This commit includes the implementation for the
fixed and dynamic formats, where the other format is known as
differencing. The vhdx_header.rs, vhdx_bat.rs, vhdx_metadata.rs
implements parser and manipulators for the VHDx header, Block
Allocation Table, and metadata, respectively, for the VHDx file.
The vhdx_io.rs implements read and write routines for the VHDx file.
The vhdx.rs implements the Vhdx structure, which provides the wrapper
functions for standard I/O operations like read, write, and seek.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Fazla Mehrab <akm.fazla.mehrab@intel.com>
This commit adds a github action to release statically-linked arm64
binaries - ch-remote and cloud-hypervisor.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
We are relying on applying empty 'seccomp' filters to support the
'--seccomp false' option, which will be treated as an error with the
updated 'seccompiler' crate. This patch fixes this issue by explicitly
checking whether the 'seccomp' filter is empty before applying the
filter.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The current code was expecting a reply with only the header, which
wasn't reading the rest of payload that was provided. This was causing
the following replies to be completely wrong as they were shifted by the
previous payload that wasn't read.
The simple way to fix this issue is by reading the header and the
expected payload when getting a reply from a DmaUnmap request.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Change the block size to the standard 512 byte sector size to that
disk images can be used (since their partition tables will be specified
in terms of 512 byte sectors.)
Also remove the hugepages=on option from the command line as it is not
necessary.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When mapping the region into the guest ensure that all the fields are
updated correctly as the unmap code path checks that they are set.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This commit adds an AArch64-only integration test case called
`test_guest_numa_nodes_dt` so that it is possible to test the
NUMA for the FDT on AArch64 platform.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Based on `--memory-zone` and `--numa` param in the Cloud Hypervisor
cmdline, the NUMA memory configuration is described. This commit
adds such NUMA memory configuration to the FDT memory node.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
It is forbidden that the same memory zone belongs to more than one
NUMA node. This commit adds related validation to the `--numa`
parameter to prevent the user from specifying such configuration.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
For the purpose of identification, each NUMA node is associated
with a unique token known as a `numa-node-id`. For the purpose of
device tree binding, a `numa-node-id` is a 32-bit integer.
The CPU node is associated with a NUMA node by the presence of a
`numa-node-id` property which contains the node id of the device.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The optional device tree node distance-map describes the relative
distance (memory latency) between all NUMA nodes.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This is to make sure the NUMA node data structures can be accessed
both from the `vmm` crate and `arch` crate.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The AArch64 platform provides a NUMA binding for the device tree,
which means on AArch64 platform, the NUMA setup can be extended to
more than the ACPI feature.
Based on above, this commit extends the NUMA setup and data
structures to following scenarios:
- All AArch64 platform
- x86_64 platform with ACPI feature enabled
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Signed-off-by: Michael Zhao <Michael.Zhao@arm.com>
Introducing a new structure VhostUserCommon allowing to factorize a lot
of the code shared between the vhost-user devices (block, fs and net).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of panicking with an expect() function, the QcowDiskSync::new
function now propagates the error properly. This ensures the VMM will
not panic, which might be the source of weird errors if only one thread
exits while the VMM continues to run.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of running the generic block fuzzer with QCOW, it's better to
use a RAW file since it's less complex and it will focus on virtqueues.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We cannot let vhost-user devices connect to the backend when the Block,
Fs or Net object is being created during a restore/migration. The reason
is we can't have two VMs (source and destination) connected to the same
backend at the same time. That's why we must delay the connection with
the vhost-user backend until the restoration is performed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Introducing a new function to factorize a small part of the
initialization that is shared between a full reinitialization and a
restoration.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The code wasn't doing what it was expected to. The '?' was simply
returning the error to the top level function, meaning the Err() case in
the match was never hit. Moving the whole logic to a dedicated function
allows to identify when something got wrong without propagating to the
calling function, so that we can still stop the dirty logging and
unpause the VM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In case the migration succeeds, the destination VM will be correctly
running, with potential vhost-user backends attached to it. We can't let
the source VM trying to reconnect to the same backends, which is why
it's safer to shutdown the source VM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to prevent the vhost-user devices from reconnecting to the
backend after the migration has been successfully performed, we make
sure to kill the thread in charge of handling the reconnection
mechanism.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Add a way to let every Migratable object know when the migration is
complete, so they can take appropriate actions.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
During a migration, the vhost-user device talks to the backend to
retrieve the dirty pages. Once done with this, a snapshot will be taken,
meaning there's no need to communicate with the backend anymore. Closing
the communication is needed to let the destination VM being able to
connect to the same backend.
That's why we shutdown the communication with the backend in case a
migration has been started and we're asked for a snapshot.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This anticipates the need for creating a new Blk, Fs or Net object
without having performed the connection with the vhost-user backend yet.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for creating vhost-user devices in a different way when
being restored compared to a fresh start, this commit introduces a new
boolean created by the Vm depending on the use case, and passed down to
the DeviceManager. In the future, the DeviceManager will use this flag
to assess how vhost-user devices should be created.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It was incorrect to call Vec::from_raw_parts() on the address pointing
to the shared memory log region since Vec is a Rust specific structure
that doesn't directly translate into bytes. That's why we use the same
function from std::slice in order to create a proper slice out of the
memory region, which is then copied into a Vec.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that the common vhost-user code can handle logging dirty pages
through shared memory, we need to advertise it to the vhost-user
backends with the protocol feature VHOST_USER_PROTOCOL_F_LOG_SHMFD.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Correct operation of user devices (vfio-user) requires shared memory so
flag this to prevent it from failing in strange ways.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Create the vfio-user / user devices from the config. Currently hotplug
of the devices is not supported nor can they be placed behind the
(virt-)iommu.
Removal of the coldplugged device is however supported.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Taking advantage of the refactored VFIO code implement a new
VfioUserPciDevice that wraps the client for vfio-user and exposes the
BusDevice and PciDevice into the VMM.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the user to specify devices that are running in a different
userspace process and communicated with vfio-user.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Implement (most) of the client side (i.e. VMM side) of the vfio-user
protocol:
https://github.com/nutanix/libvfio-user/blob/master/docs/vfio-user.rst
Items that are not implemented (because they are optimisations or unused
due to alternative solutions:
* VFIO_USER_DMA_READ/WRITE - this is a way for the server to read guest
memory if the guest memory is not shared by fd where the client
doesn't support it. However since we do support sharing the memory by
fd this is not required.
* VFIO_USER_GET_REGION_IO_FDS - an optimisation to bypass the VMM by
having KVM talk directly to the backend using ioregionfd
* VFIO_USER_DIRTY_PAGES - for the implementation of live migration
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This doesn't really affect the build as we ship a Cargo.lock with fixed
versions in. However for clarity it makes sense to use fixed versions
throughout and let dependabot update them.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
AArch64 CPU topology can be described using either device tree or
ACPI. Therefore, the integration test should also cover the AArch64
ACPI CPU topology tests.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The optional Processor Properties Topology Table (PPTT) table is
used to describe the topological structure of processors controlled
by the OSPM, and their shared resources, such as caches. The table
can also describe additional information such as which nodes in the
processor topology constitute a physical package.
The ACPI PPTT table supports topology descriptions for ACPI guests.
Therefore, this commit adds the PPTT table for AArch64 to enable
CPU topology feature for ACPI.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The Arm CPU topology is defined within the `cpu-map` node, which is
a direct child of the cpus node and provides a container where the
actual topology nodes are listed.
This commit adds an optional cpu-map node in device tree, based on
the Cloud Hypervisor command line vCPU topology information.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
In an Arm system, the hierarchy of CPUs is defined through three
entities that are used to describe the layout of physical CPUs in
the system:
- cluster
- core
- thread
All these three entities have their own FDT node field. Therefore,
This commit adds an AArch64-specific helper to pass the config from
the Cloud Hypervisor command line to the `configure_system`, where
eventually the `create_fdt` is called.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Make sure the DeviceManager is triggered for all migration operations.
The dirty pages are merged from MemoryManager and DeviceManager before
to be sent up to the Vmm in lib.rs.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that Migratable provides the methods for starting, stopping and
retrieving the dirty pages, we move the existing code to these new
functions.
No functional change intended.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This patch connects the dots between the vm.rs code and each Migratable
device, in order to make sure Migratable methods are correctly invoked
when migration happens.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding the common vhost-user code for starting logging dirty pages when
the migration is started, and its counterpart for stopping, as well as
the code in charge of retrieving the bitmap of the dirty pages that have
been logged.
All these functions are meant to be leveraged from vhost-user devices.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding a simple field `migration_support` to VhostUserHandle in order to
store the information about the device supporting migration or not. The
value of this flag depends on the feature set negotiated with the
backend. It's considered as supporting migration if VHOST_F_LOG_ALL is
present in the virtio features and if VHOST_USER_PROTOCOL_F_LOG_SHMFD is
present in the vhost-user protocol features.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Add new methods to the Migratable trait so that each device implementing
this trait can be notified when the migration starts/stops, as well as
when a dirty bitmap must be returned.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for supporting the merge of multiple dirty pages coming
from multiple devices, this patch factorizes the creation of a
MemoryRangeTable from a bitmap, as well as providing a simple method for
merging the dirty pages regions under a single MemoryRangeTable.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Replacing QCOW mentions with other viable options. There is still
docs/custom-image.md which talks about creating QCOW, however it
might be more convenient to be touched when the actual changes
are indeed there.
Related to #1985.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
These crates are written to produce log messages using the error!
macro, but their logs didn't actually go anywhere, which made it very
difficult to debug when they're not working.
I've used env_logger here because it's the same log implementation
that the hypervisor crate uses.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
I don't think the min_values(1) was doing anything, since
takes_value(true) was already set, and after this change I still get
an error if I try to supply the argument with no value.
But the unwrap below would fail if the argument wasn't supplied at
all. By setting required(true) we get a nice message from clap
instead.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
After the refactoring to split the common VFIO code out for vfio-user
there were some inconsistencies in the error handling. Correct this so
that the error is independent of the transport (hardware vs user) VFIO
and migrate to anyhow/thiserror in the process. Some unused errors from
earlier refactoring have also been removed.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Returning a reference is not possible for the vfio-user code as it is
constructed for the function.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Rename the wrapper trait and structs since this will be used for more
than reading the PCI config.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the code to be used from a different module in the same
crate for vfio-user support.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Split data that will need to be common between VfioPciDevice and
VfioUserPciDevice into a common struct. Currently this has no methods
but they will be added soon.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
By splitting this into a trait with common code extracted then this
will allow extensive reuse of logic in the vfio-user version.
This commit also changed the order of parameters on
::write_config_dword() to place offset first to match the other
functions.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Moving the MSHV crate form Cloud Hypervisor to rust-vmm
is done. This patch update the MSHV referent to rust-vmm
Signed-off-by: Muminul Islam <muislam@microsoft.com>
backend like SPDK required to know how many virt queues to be handled
before gets VHOST_USER_SET_INFLIGHT_FD message.
fix dpdk core dump while processing vhost_user_set_inflight_fd:
#0 0x00007fffef47c347 in vhost_user_set_inflight_fd (pdev=0x7fffe2895998, msg=0x7fffe28956f0, main_fd=545) at ../lib/librte_vhost/vhost_user.c:1570
#1 0x00007fffef47e7b9 in vhost_user_msg_handler (vid=0, fd=545) at ../lib/librte_vhost/vhost_user.c:2735
#2 0x00007fffef46bac0 in vhost_user_read_cb (connfd=545, dat=0x7fffdc0008c0, remove=0x7fffe2895a64) at ../lib/librte_vhost/socket.c:309
#3 0x00007fffef45b3f6 in fdset_event_dispatch (arg=0x7fffef6dc2e0 <vhost_user+8192>) at ../lib/librte_vhost/fd_man.c:286
#4 0x00007ffff09926f3 in rte_thread_init (arg=0x15ee180) at ../lib/librte_eal/common/eal_common_thread.c:175
Signed-off-by: Arafatms <arafatms@outlook.com>
This patch adds all the seccomp rules missing for MSHV.
With this patch MSFT internal CI runs with seccomp enabled.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch adds a fallback path for sending live migration, where it
ensures the following behavior of source VM post live-migration:
1. The source VM will be paused only when the migration is completed
successfully, or otherwise it will keep running;
2. The source VM will always stop dirty pages logging.
Fixes: #2895
Signed-off-by: Bo Chen <chen.bo@intel.com>
The vhost-user-net backend needs to prepare all queues before enabling vring.
For example, DPVGW will report 'the RX queue can't find' error, if we enable
vring immediately after kicking it out.
Signed-off-by: Arafatms <arafatms@outlook.com>
This rule is needed to boot windows guest.
This bug was introduced while we tried to boot
windows guest on MSHV.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch modify the existing live migration code
to support MSHV. Adds couple of new functions to enable
and disable dirty page tracking. Add missing IOCTL
to the seccomp rules for live migration.
Adds necessary flags for MSHV.
This changes don't affect KVM functionality at all.
In order to get better performance it is good to
enable dirty page tracking when we start live migration
and disable it when the migration is done.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Right now, get_dirty_log API has two parameters,
slot and memory_size.
MSHV needs gpa to retrieve the page states. GPA is
needed as MSHV returns the state base on PFN.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Now that vhost-user supports being snapshot and restored, we extend the
existing test_ovs_dpdk to validate snapshot/restore feature works as
expected.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding the support for snapshot/restore feature for all supported
vhost-user devices.
The complexity of vhost-user-fs device makes it only partially
compatible with the feature. When using the DAX feature, there's no way
to store and remap what was previously mapped in the DAX region. And
when not using the cache region, if the filesystem is mounted, it fails
to be properly restored as this would require a special command to let
the backend know that it must remount what was already mounted before.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This patch moves all vhost-user common functions behind a new structure
VhostUserHandle. There is no functional changes intended, the only goal
being to prepare for storing information through this new structure,
limiting the amount of parameters that are needed for each function.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We didn't implement the vhost-user-net backend to support EVENT_IDX
feature, hence it is incorrect to advertise VIRTIO_RING_F_EVENT_IDX to
the VMM and the guest.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It's totally acceptable to snapshot and restore a virtio-fs device that
has no cache region, since this is a valid mode of functioning for
virtio-fs itself.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
With the new beta version, clippy complains about redundant allocation
when using Arc<Box<dyn T>>, and suggests replacing it simply with
Arc<dyn T>.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
As we are now using an global control to start/stop dirty pages log from
the `hypervisor` crate, we need to explicitly tell the hypervisor (KVM)
whether a region needs dirty page tracking when it is created.
This reverts commit f063346de3.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Following KVM interfaces, the `hypervisor` crate now provides interfaces
to start/stop the dirty pages logging on a per region basis, and asks
its users (e.g. the `vmm` crate) to iterate over the regions that needs
dirty pages log. MSHV only has a global control to start/stop dirty
pages log on all regions at once.
This patch refactors related APIs from the `hypervisor` crate to provide
a global control to start/stop dirty pages log (following MSHV's
behaviors), and keeps tracking the regions need dirty pages log for
KVM. It avoids leaking hypervisor-specific behaviors out of the
`hypervisor` crate.
Signed-off-by: Bo Chen <chen.bo@intel.com>
To support different CPUID entry semantics, we now allow to
specify the compatible condition for each feature entry. Most entries
are considered compatible when they are "bitwise subset", with few
exceptions: 1. "equal", e.g. EBX/ECX/EDX of leaf `0x4000_0000` (KVM
CPUID SIGNATURE); 2. "smaller or equal as a number", e.g. EAX of leaf
`0x7` and leaf `0x4000_0000`;
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch adds a common function "Vmm::vm_check_cpuid_compatibility()"
to be shared by both live-migration and snapshot/restore.
Signed-off-by: Bo Chen <chen.bo@intel.com>
We now send not only the 'VmConfig' at the 'Command::Config' step of
live migration, but also send the 'common CPUID'. In this way, we can
check the compatibility of CPUID features between the source and
destination VMs, and abort live migration early if needed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Add missing option of hugepage_size in argument of memory,
and add some details about lacking huge pages.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
With the support of dynamically turning on/off dirty-pages-log during
live-migration (only for guest RAM regions), we now can create guest
memory regions without dirty-pages-log by default both for guest RAM
regions and other regions backed by file/device.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch extends slightly the current live-migration code path with
the ability to dynamically start and stop logging dirty-pages, which
relies on two new methods added to the `hypervisor::vm::Vm` Trait. This
patch also contains a complete implementation of the two new methods
based on `kvm` and placeholders for `mshv` in the `hypervisor` crate.
Fixes: #2858
Signed-off-by: Bo Chen <chen.bo@intel.com>
The existing test_macvtap is factorized to be able to support both
coldplug and hotplug of a macvtap interface through virtio-net. Adding
the new test_macvtap_hotplug test allows for validating that sending a
TAP file descriptor through control message along with the add-net
command works.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
To help with readability, we rely on exec_host_command_status() from the
macvtap test, which replaces the former "bash -c ..." syntax.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Enable specifically for the add-net command the ability to send file
descriptors along with the HTTP request. This is useful to hotplug a
macvtap interface after the VMM has already been started.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Whenever a file descriptor is sent through the control message, it
requires fcntl() syscall to handle it, meaning we must allow it through
the list of syscalls authorized for the HTTP thread.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When running TDX guest, the Guest Physical Address space is limited by
a shared bit that is located on bit 47 for 4 level paging, and on bit 51
for 5 level paging (when GPAW bit is 1). In order to keep things simple,
and since a 47 bits address space is 128TiB large, we ensure to limit
the physical addressable space to 47 bits when runnning TDX.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When running a TDX guest, we need the virtio drivers to use the DMA API
to share specific memory pages with the VMM on the host. The point is to
let the VMM get access to the pages related to the buffers pointed by
the virtqueues.
The way to force the virtio drivers to use the DMA API is by exposing
the virtio devices with the feature VIRTIO_F_IOMMU_PLATFORM. This is a
feature indicating the device will require some address translation, as
it will not deal directly with physical addresses.
Cloud Hypervisor takes care of this requirement by adding a generic
parameter called "force_iommu". This parameter value is decided based on
the "tdx" feature gate, and then passed to the DeviceManager. It's up to
the DeviceManager to use this parameter on every virtio device creation,
which will imply setting the VIRTIO_F_IOMMU_PLATFORM feature.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This refactoring ensures all CPUID related operations are centralized in
`arch::x86_64` module, and exposes only two related public functions to
the vmm crate, e.g. `generate_common_cpuid` and `configure_vcpu`.
Signed-off-by: Bo Chen <chen.bo@intel.com>
In order to let a separate process open a TAP device and pass the file
descriptor through the control message mechanism, this patch adds the
support for sending a file descriptor over to the Cloud Hypervisor
process along with the add-net HTTP API command.
The implementation uses the NetConfig structure mutably to update the
list of fds with the one passed through control message. The list should
always be empty prior to this, as it makes no sense to provide a list of
fds once the Cloud Hypervisor process has already been started.
It is important to note that reboot is supported since the file
descriptor is duplicated upon receival, letting the VM only use the
duplicated one. The original file descriptor is kept open in order to
support a potential reboot.
Fixes#2525
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There are some seccomp rules needed for MSHV
in virtio-devices but not for KVM. We only want to
add those rules based on MSHV feature guard.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The micro-http crate now uses recvmsg() syscall in order to receive file
descriptors through control messages. This means the syscall must be
part of the authorized list in the seccomp filters.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The vfio-ioctls crate now contains a KVM feature gate. Make use of it in
Cloud Hypervisor.
That crate has two users. For the vmm crate is it straight-forward. For
the vm-device crate, we introduce a KVM feature gate as well so that the
vmm crate can pass on the configuration.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The help of arguments `memory` and `memory-zone` missing a comma.
Before adding, these parts are as follows:
> hugepage_size=<hugepage_size>hotplug_method=acpi|virtio-mem
After adding, these parts will be:
> hugepage_size=<hugepage_size>,hotplug_method=acpi|virtio-mem
Signed-off-by: Yukiteru Lee <wfly1998@sina.com>
The edk2 upstream has already suppoorted AArch64 Cloud Hypervisor,
and hence we can use upstream edk2 in CI and doc.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This new option allows the user to define a list of SGX EPC sections
attached to a specific NUMA node.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to uniquely identify each SGX EPC section, we introduce a
mandatory option `id` to the `--sgx-epc` parameter.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Enabling CONFIG_HYPERV in the Linux Kernel allows
guest to use hyperv clock source. This changes
improves guest performance. Without this changes
we saw slowness in the guest on MSHV.
Signed-off-by: Vineeth Pillai <viremana@linux.microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The guest can see that SGX supports provisioning as it is exposed
through the CPUID. This patch enables the proper backing of this
feature by having the host open the provisioning device and enable
this capability through the hypervisor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We need a dedicated function to enable the SGX attribute capability
through the Hypervisor abstraction.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The cloud hypervisor tells the VM and the backend to support the PACKED_RING feature,
but it actually processes various variables according to the split ring logic, such
as last_avail_index. Eventually it will cause the following error (SPDK as an example):
vhost.c: 516:vhost_vq_packed_ring_enqueue: *ERROR*: descriptor has been used before
vhost_blk.c: 596:process_blk_task: *ERROR*: ====== Task 0x200113784640 req_idx 0 failed ======
vhost.c: 629:vhost_vring_desc_payload_to_iov: *ERROR*: gpa_to_vva((nil)) == NULL
Signed-off-by: Arafatms <arafatms@outlook.com>
This patch fixes a few things to support TDVF correctly.
The HOB memory resources must contain EFI_RESOURCE_ATTRIBUTE_ENCRYPTED
attribute.
Any section with a base address within the already allocated guest RAM
must not be allocated.
The list of TD_HOB memory resources should contain both TempMem and
TdHob sections as well.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
If GICR_CTLR is restored before GICR_PROPBASER and GICR_PENDBASER,
the restoring of the latter registers will fail, as the LPI enable
bit is already set in GICR_CTLR. Therefore, in this commit, the
order of restoring GICR registers is changed.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
This commit implements the GicV3Its Snapshottable trait, including:
- GicV3Its state: GIC registers and ITS registers
- Save/restore logic of GicV3Its state
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit implements two helper functions `gicv3_its_attr_access`
and `gicv3_its_tables_access` to access ITS device attributes and
ITS tables.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
In current code, the ITS device fd of GICv3 will be lost after the
creation of GIC. This commit adds a new `its_device` field for the
`GicV3Its` struct, which will be useful to save the ITS device fd.
This fd will be used in restoring the ITS device.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The BAR calculation code was incorrect for calculating I/O BARs but also
has misleading comments (mixing bits and bytes, first and least
significant, etc).
This change adjusts the algorithm to more closely match the version
described in the PCI specification and takes advantage of Rust's binary
literals for ease of reading. Although this is slightly longer by
calculating the 64-bit and 32-bit paths separately I think this is
easier to read.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Previously the same function was used to both create and remove regions.
This worked on KVM because it uses size 0 to indicate removal.
MSHV has two calls -- one for creation and one for removal. It also
requires having the size field available because it is not slot based.
Split set_user_memory_region to {create/remove}_user_memory_region. For
KVM they still use set_user_memory_region underneath, but for MSHV they
map to different functions.
This fixes user memory region removal on MSHV.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The to-be-introduced MSHV rules don't need to contain KVM rules and vice
versa.
Put KVM constants into to a module. This avoids the warnings about
dead code in the future.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The TX rate limiter didn't correctly handle the `-EAGAIN` situation
where it should replenish the rate-limiter tokens. This patch simplifies
the TX rate-limiter code path by following the similar trade-off made in
the TX rate limiter, where we always let the `last` descriptor
go-through (even if it was over the rate limit. We simply stop
processing the oncoming `descriptors` if any.
Fixes: #2817
Signed-off-by: Bo Chen <chen.bo@intel.com>
This warning isn't present on on the read case and we now have better
handling of the -EAGAIN situation including retries.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If writing to the TAP returns EAGAIN then listen for the TAP to be
writable. When the TAP becomes writable attempt to process the TX queue
again.
Fixes: #2807
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The output from getty ("login:") does not always appear. This can be
observed interactively when booting the VM. (Mashing return will bring
it up.) Instead of checking for that string to ensure the VM has booted
instead check for a message from systemd to say it has started the SSH
daemon.
Fixes: #2799
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: use of `expect` followed by a function call
--> test_infra/src/lib.rs:598:10
|
598 | .expect(&format!("Expected '{}' to run", command))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `unwrap_or_else(|_| panic!("Expected '{}' to run", command))`
|
= note: `#[warn(clippy::expect_fun_call)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#expect_fun_call
warning: use of `expect` followed by a function call
--> test_infra/src/lib.rs:605:10
|
605 | .expect(&format!("Expected '{}' to run", command))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `unwrap_or_else(|_| panic!("Expected '{}' to run", command))`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#expect_fun_call
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This commit introduces the `ProcessorGiccAffinity` struct for the
AArch64 platform. This struct will be created and included into
the SRAT table to enable AArch64 NUMA setup.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This vcpu API is necessary for MSHV related debugging.
These two registers controls the vcpu_run in the
/dev/mshv driver code.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
It ensures all handlers for `ApiRequest` in `control_loop` are
consistent and minimum and should read better.
No functional changes.
Signed-off-by: Bo Chen <chen.bo@intel.com>
It simplifies a bit the `Vmm::control_loop` and reads better to be
consistent with other `ApiRequest` handlers. Also, it removes the
repetitive `ApiError::VmAlreadyCreated` and makes `ApiError::VmCreate`
useful.
No functional changes.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Simplified the build instruction on arm64.
And the binary built with default option can be used for both UEFI and
direct-kernel test.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
AArch64 tests were divided into 2 steps:
- Build and test with 'acpi' feature
- Build and test without 'acpi'
This can be optimized. We need only to build and test once with default
features ('acpi' is enabled).
On AArch64, ACPI only works with UEFI. If UEFI is not available, guest
kernel fall back to use FDT. Most AArch64 test cases boot from direct
kernel, the guest will keep using FDT even if ACPI is enabled. So
nothing is broken.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
We have been building Cloud Hypervisor with command like:
`cargo build --no-default-features --features ...`.
After implementing ACPI, we donot have to use specify all features
explicitly. Default build command `cargo build` can work.
This commit fixed some build warnings with default build option and
changed github workflow correspondingly.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
UEFI need to be loaded to a flash area at the beginning of guest memory
address space. To simulate the flash, we take a piece of RAM and hide
it to the guest. As this is a temporary solution, the hiden RAM for UEFI
should be as little as possible. The size was 64 MiB, that's too much,
4 MiB is enough.
The down side of such simulation is that there is a gap (4 MiB) between
the memory size in VMM's view and that in guest's view. This is to be
fixed by implementing a flash device in future.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
This dependency bump needed some manual handling since the API changed
quite a lot regarding some RawFd being changed into either File or
AsRawFd traits.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
error: avoid using `collect()` when not needed
--> vmm/src/vm.rs:630:86
|
630 | let node_id_list: Vec<u32> = configs.iter().map(|cfg| cfg.guest_numa_id).collect();
| ^^^^^^^
...
664 | if !node_id_list.contains(&dest) {
| ---------------------------- the iterator could be used here instead
|
= note: `-D clippy::needless-collect` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_collect
Signed-off-by: Bo Chen <chen.bo@intel.com>
Issue from beta verion of clippy:
Error: --> vm-virtio/src/queue.rs:700:59
|
700 | if let Some(used_event) = self.get_used_event(&mem) {
| ^^^^ help: change this to: `mem`
|
= note: `-D clippy::needless-borrow` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
Signed-off-by: Bo Chen <chen.bo@intel.com>
Issue from beta verion of clippy:
error: field is never read: `type`
--> vmm/src/cpu.rs:235:5
|
235 | pub r#type: u8,
| ^^^^^^^^^^^^^^
|
= note: `-D dead-code` implied by `-D warnings`
Signed-off-by: Bo Chen <chen.bo@intel.com>
Initialize MTRR defType register the same way the KVM code does - WB caching by default.
Tested with latest mshv code.
Without this patch, these lines are present in guest serial log:
[ 0.000032] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[ 0.000036] CPU MTRRs all blank - virtualized system.
This indicates the guest is detecting the set MTRR.
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The Linux kernel expects that any PCI devices that advertise I/O bars
have use an address that is within the range advertised by the bus
itself. Unfortunately we were not advertising any I/O ports associated
with the PCI bus in the ACPI tables.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This patch addresses this issue https://github.com/rust-lang/rust-bindgen/pull/2064.
While we access field of packed struct the compiler can generate the
correct code to create a temporary variable to access the packed struct
field. Access withing {} ensures that.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The virtio_balloon test is a bit flaky since we can't really know how
much the balloon is gonna be deflated when the guest is under memory
pressure. That's why it's safer to simply check that the balloon is not
the initial size anymore.
One small detail, but we don't need to check for the balloon size to be
higher than 0 since the returned value is a u64.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we ran into issues while using the Azure credentials plugin for
Jenkins, let's rely directly on the Azure CLI to download the Windows
guest image along with the modified OVMF firmware.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
To debug the FDT (Flattened Device Tree), we usually need to modify
source code to save the generted DTB data to disk, and use 'dtc' command
to decode the binary file into a text file to analyze.
It would be ideal if the FDT content can be seen in log.
This commit makes it real by:
- Introducing 'fdt' crate for parsing FDT.
- Printing the content of the FDT in tree view.
The parsing and printing only happen when Debug level logging enabled.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Additionally, he disk creation routine is extended to support NTFS and
variable image size.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
In order to allow a hotplugged vCPU to be assigned to the correct NUMA
node in the guest, the DSDT table must expose the _PXM method for each
vCPU. This method defines the proximity domain to which each vCPU should
be attached to.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The _PXM method always return 0, which is wrong since the SRAT might
tell differently. The point of the _PXM method is to be evaluated by the
guest OS when some new memory slot is being plugged, but this will never
happen for Cloud Hypervisor since using NUMA nodes along with memory
hotplug only works for virtio-mem.
Memory hotplug through ACPI will only happen when there's only one NUMA
node exposed to the guest, which means the _PXM method won't be needed
at all.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Make sure the unique PCI bus is tied to the default NUMA node 0, and
update the documentation to let the users know about this special case.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Fixed wrong MPIDR value setting for VCPUs in FDT.
The wrong setting made only 16 VCPUs can be enabled at most, all other
VCPUs were showing off-line.
The issue was introduced when we were migrating FDT-generating code to
vmm-fdt crate.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Now that vhost crate allows the caller to set the header flags, we can
set NEED_REPLY whenever the REPLY_ACK protocol feature is supported from
both ends.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The new test_virtio_balloon() is to verify if the 'deflate_on_oom'
parameter works. Its testing result is as follows:
1. Start a 4G guest with 2G balloon, check memory once starts up.
total_mem is 4294967296 bytes
actual_mem is 2147483648 bytes
orig_balloon is 2147483648 bytes
total used free shared buff/cache available
Mem: 3.8Gi 2.1Gi 1.6Gi 0.0Ki 140Mi 1.6Gi
Swap: 0B 0B 0B
2. Run a command in guest to eat up 25G memory, and check again.
total_mem is 4294967296 bytes
actual_mem is 3121610752 bytes
deflated_balloon is 1173356544 bytes
total used free shared buff/cache available
Mem: 3.8Gi 1.2Gi 2.6Gi 0.0Ki 49Mi 2.5Gi
Swap: 0B 0B 0B
From above, we can notice the balloon memory indeed deflates from
2147483648 bytes to 1173356544 bytes once an oom is going to be
triggered.
Signed-off-by: Fei Li <lifei.shirley@bytedance.com>
For vhost-user devices, memory should be shared between CLH and
vhost-user backend. However, madvise DONTNEED doesn't working in
this case. So, let's use fallocate PUNCH_HOLE to discard those
memory regions instead.
Signed-off-by: Li Hangjing <lihangjing@bytedance.com>
Sometimes we need balloon deflate automatically to give memory
back to guest, especially for some low priority guest processes
under memory pressure. Enable deflate_on_oom to support this.
Usage: --balloon "size=0,deflate_on_oom=on" \
Signed-off-by: Fei Li <lifei.shirley@bytedance.com>
Since using the VIRTIO configuration to expose the virtual IOMMU
topology has been deprecated, the virtio-iommu implementation must be
updated.
In order to follow the latest patchset that is about to be merged in the
upstream Linux kernel, it must rely on ACPI, and in particular the newly
introduced VIOT table to expose the information about the list of PCI
devices attached to the virtual IOMMU.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since the reconnection thread took on the responsibility to handle
backend initiated requests as well, the variable naming should reflect
this by avoiding the "reconnect" prefix.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
A new method has been introduced for WindowsGuest, that would poll on
the SSH connection and try to execute a test command. When successfull,
the polling stops and the guest is considered finished boot. The
downside of this method is that the network setup is needed always even
if the test doesn't use network, however it complies with the behavior
of other tests. I also observe a bit more stability on the local test
run, however it still appears to be a resource issue so some sporadic
fails are possible on a slower machine.
The hardcoded timeouts for guest boot and DHCP setup have been removed.
The dnsmasq invocation uses `--bind-dynamic` so then the daemon can be
started while the target interface could be down yet.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Co-authored-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Vhost user INFLIGHT_SHMFD protocol feature supports inflight I/O
tracking, this commit implement the vhost-user device (master) support
of the feature. Till this commit, specific vhost-user devices (blk, fs,
or net) have not enable this feature.
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
Add the support for reconnecting the backend request handler after a
disconnection/crash happened.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since the slave request handler is common to all vhost-user devices, the
same way the reconnection is, it makes sense to handle the requests from
the backend through the same thread.
The reconnection thread now handles both a reconnection as well as any
request coming from the backend.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit enables socket reconnection for vhost-user-fs backends. Note
that, till this commit:
- The re-establish of the slave communication channel is no supported. So
the socket reconnection does not support virtiofsd with DAX enabled.
- Inflight I/O tracking and restoring is not supported. Therefore, only
virtio-fs daemons that are not processing inflight requests can work
normally after reconnection.
- To make the restarted virtiofsd work normally after reconnection, the
internal status of virtiofsd should also be recovered. This is not the
work of cloud-hypervisor. If the virtio-fs daemon does not support
saving or restoring its internal status, then a re-mount in guest after
socket reconnection should be performed.
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
With the ability of getting host IPA size in `hypervisor` crate,
we can query the host IPA size through ioctl instead of hardcoding
a maximum IPA size. Therefore this commit removes the hardcoded
maximum host IPA size.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Not all AArch64 platforms support IPAs up to 40 bits. Since the
kvm-ioctl crate now supports `get_host_ipa_limit` for AArch64,
when creating the VM, it is better to get the IPA size from the
host and use that to create the VM.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit adds a helper `get_host_ipa_limit` to the AArch64
`KvmHypervisor` struct. This helper can be used to get the
`Host_IPA_Limit`, which is the maximum possible value for
IPA_Bits on the host and is dependent on the CPU capability
and the kernel configuration.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
test_aarch64_pe_boot was added at the very beginning of AArch64 support.
Now it can be combined to test_vmlinux_boot.
Renamed test_vmlinux_boot to test_direct_kernel_boot for generality.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Enabled test case test_vmlinux_boot_noacpi on AArch64 and renamed it
test_direct_kernel_boot_noacpi for generality.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Some new integration tests will require the "stress" binary to be
present in the guest in order to run correctly.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit added support for building the edk2 binary dedicated
for the Cloud Hypervisor (CLOUDHV_EFI.fd).
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
EDK2 requires the beginning of PCIe high space above 4G address.
In CLH the space follows the RAM. If the RAM space is small, the PCIe
high space could fall bellow 4G.
Here we put it above 512G in FDT to workaround the EDK2 check only when
ACPI is enabled, because EDK2 collects PCIe information from FDT.
The address written in ACPI is not impacted.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Implemented an architecture specific function for loading UEFI binary.
Changed the logic of loading kernel image:
1. First try to load the image as kernel in PE format;
2. If failed, try again to load it as formatless UEFI binary.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
_DSM (Device Specific Method) is a control method that enables devices
to provide device specific control functions. Linux kernel will evaluate
this device then initialize preserve_config in acpi pci initialization.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Re-enable virtiofsd testing now that issues with capstone repository
have been resolved.
This reverts commit a14c70019a.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This new function allows for better readability of the code by
factorizing a few of lines of code into a single one.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It's important to verify the actual exit code returned from the new
function exec_host_command_status() to ensure the command ran
successfully.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This new function allows for better readability of the code by
factorizing a bunch of lines of code into a single one.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There's no need for ssh_command_ok() anymore since ssh_command() now
returns an error in case the executed command returned with an exit code
different than 0.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit enables socket reconnection for vhost-user-blk backends.
Note that, till this commit, inflight I/O trakcing and restoring is not
supported. Therefore, only vhost-user-blk backend that are not processing
inflight requests can work normally after reconnection.
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
We should try to read the last avail index from the vring memory aera. This
is necessary when handling vhost-user socket reconnection.
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
Make sure the reconnection functionaliity is well tested from the OVS
DPDK integration test.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to simplify and speed up the OVS DPDK test, we switch from
using 'iperf3' to 'netcat' to validate the connectivity is functional.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since OVS DPDK deprecated the use for the server mode, let's make sure
the integration test uses the client mode instead. This means the OVS
backend is the client and the VMM acts as the server, therefore the
reason why we use "vhost_mode=server".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Re-enable the OVS DPDK integration test by assigning both VMs to the
NUMA node 0. This ensures the processes are being run on NUMA node 0,
preventing OVS DPDK from abnormal functioning.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
As discussed in the working PR in the upstream vm-memory crate repo,
some special functions (e.g. return raw pointers to the wrapped guest
memory) require manual dirty page tracking from their users (e.g.the
VMM). One of the special functions is `VolatileSlice::as_ptr(), which is
used in our code base for supporting async block I/O. This patch
manually mark dirty for guest pages touched while reading from block
devices.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Live migration currently handles guest memory writes from the guest
through the KVM dirty page tracking and sends those dirty pages to the
destination. This patch augments the live migration support with dirty
page tracking of writes from the VMM to the guest memory(e.g. virtio
devices).
Fixes: #2458
Signed-off-by: Bo Chen <chen.bo@intel.com>
Function "GuestMemory::with_regions(_mut)" were mainly temporary methods
to access the regions in `GuestMemory` as the lack of iterator-based
access, and hence they are deprecated in the upstream vm-memory crate [1].
[1] https://github.com/rust-vmm/vm-memory/issues/133
Signed-off-by: Bo Chen <chen.bo@intel.com>
As the first step to complete live-migration with tracking dirty-pages
written by the VMM, this commit patches the dependent vm-memory crate to
the upstream version with the dirty-page-tracking capability. Most
changes are due to the updated `GuestMemoryMmap`, `GuestRegionMmap`, and
`MmapRegion` structs which are taking an additional generic type
parameter to specify what 'bitmap backend' is used.
The above changes should be transparent to the rest of the code base,
e.g. all unit/integration tests should pass without additional changes.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Add a helper to VirtioCommon which returns duplicates of the EventFds
for kill and pause event.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The control queue was missing rt_sigprocmask syscall, which was causing
a crash when the VM was shutdown.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The CI is failing due the git server that the submodules required for
this fork of QEMU need to build from is unavailable.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
(cherry picked from commit 2aec0a92a5)
When in client mode, the VMM will retry connecting the backend for a
minute before giving up.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The reconnection code is moved to the vhost-user module as it is a
common place to be shared across all vhost-user devices.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit implements the reconnection feature for vhost-user-net in
case the connection with the backend is shutdown.
The guest has no knowledge about what happens when a disconnection
occurs, which simplifies the code to handle the reconnection. The
reconnection happens with the backend, and the VMM goes through the
vhost-user setup once again.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
After adding "get_interrupt_controller()" function in DeviceManager,
"enable_interrupt_controller()" became redundant, because the latter
one is the a simple wrapper on the interrupt controller.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
The function used to calculate "gicr-typer" value has nothing with
DeviceManager. Now it is moved to AArch64 specific files.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
In order to avoid regression regarding OVS-DPDK support, a new
integration test is added. This test consists of running two VMs, both
attached to a distinct OVS port, where both ports are connected to an
OVS bridge. Once the VM are running, the test validates the connection
between the two VMs works correctly.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding a new function ssh_command_ok() as a wrapper around the existing
ssh_command() function. The goal being to identify if the command
returned without any error.
This new function allow for a bit of factorization through the codebase.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The Dockerfile is extended with some new instructions to install the
OVS-DPDK packages. This will be useful for writing new integration tests
and make sure OVS-DPDK is working properly.
This also updates the Rust toolchain from 1.51.0 to 1.52.1.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We recently realized the control queue is not meant to be handled by the
backend, that's why we move it out of vhost-user-net implementation.
Revert "vhost_user_net: Handle control queue"
This reverts commit af88ff1c49.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We thought we could move the control queue to the backend as it was
making some good sense. Unfortunately, doing so was a wrong design
decision as it broke the compatibility with OVS-DPDK backend.
This is why this commit moves the control queue back to the VMM side,
meaning an additional thread is being run for handling the communication
with the guest.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
A lot of the VIRTIO reserved features should be supported or not by the
vhost-user backend. That means on the VMM side, these features should be
available, so that they don't get lost during the negotiation.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
On FDT, VMM can allocate IRQ from 0 for devices.
But on ACPI, the lowest range below 32 has to be avoided.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
This commit enables the PSCI (Power State Coordination Interface)
for the AArch64 platform, which allows the VMM to manage the power
status of the guest. Also, multiple vCPUs can be brought up using
PSCI.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit implements the IO Remapping Table (IORT) for AArch64.
The IORT is one of the required ACPI table for AArch64, since
it describes the GICv3ITS node.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit implements an AArch64-required ACPI table: Serial
Port Console Redirection Table (SPCR). The table provides
information about the configuration and use of the serial port
or non-legacy UART interface.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit implements an AArch64-specific ACPI table: Generic
Timer Description Table (GTDT). The GTDT provides OSPM with
information about a system’s Generic Timers configuration.
The Generic Timer (GT) is a standard timer interface implemented
on ARM processor-based systems.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Added the final PCI bus number in MCFG table. This field is mandatory on
AArch64. On X86 it is optional.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Simplified definition block of CPU's on AArch64. It is not complete yet.
Guest boots. But more is to do in future:
- Fix the error in ACPI definition blocks (seen in boot messages)
- Implement CPU hot-plug controller
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
The VIRTIO features should not be set before they are acked from the
guest. This code was only present to overcome a vhost crate limitation
that was expecting the VIRTIO features to be set before we could fetch
and set the protocol features.
The vhost crate has been recently fixed by removing the limitation,
therefore there's no need for this workaround in the Cloud Hypervisor
codebase anymore.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Everything that was shared in the net_util.rs file has been now moved to
the net_util crate. The only remaining bit was only used by the
virtio-net implementation, that is why this commit moves this code to
virtio-net, and since there's nothing left in net_util.rs, it can be
removed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since the net_util crate contains the common code needed for processing
the control queue, let's use it and remove the duplicate from inside the
virtio-devices crate.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Moving helpers to the net_util crate since we don't want virtio-net
common code to be split between two places. The net_util crate should be
the only place to host virtio-net common code.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Factorize the virtio features and vhost-user protocol features
negotiation through a common function that blk, fs and net
implementations can directly rely on.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Make sure the virtio features are set upon device activation. At the
time the device is activated, we know the guest acknowledged the
features, which mean it's safe to set them back to the backend.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend the current list of available virtio features in order to make it
work correctly with the VMM side.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend the current list of available virtio features in order to make it
work correctly with the VMM side.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In migration, vm object is created by new_from_migration with
NULL kvm clock. so vm.set_clock will not be called during vm resume.
If the guest using kvm-clock, the ticks will be stopped after migration.
As clock was already saved to snapshot, add a method to restore it before
vm resume in migration. after that, guest's kvm-clock works well.
Signed-off-by: Ren Lei <ren.lei4@zte.com.cn>
Connecting a restored KVM clock vm will take long time, as clock
is NOT restored immediately after vm resume from snapshot.
this is because 9ce6c3b incorrectly remove vm_snapshot.clock, and
always pass None to new_from_memory_manager, which will result to
kvm_set_clock() never be called during restore from snapshot.
Fixes: 9ce6c3b
Signed-off-by: Ren Lei <ren.lei4@zte.com.cn>
The vhost crate does not support the need_reply flag yet, meaning we
can't be sure the backend is properly setup before the guest goes on.
One can run in a race condition where the VMM enables the vring, but
never gets any acknowledgement, meaning it assumes everything went well
and finalize the virtio device activation. Once the device is seen as
ready by the guest, it keeps going by sending some messages through the
virtqueues. Problem is, if it took some time for the backend to enable
the queue, one of the backend thread might receive a kick from the guest
while the corresponding queue is not enabled. This leads to the loss of
the event as it is discarded because the queue is not enabled.
Until vhost crate allows for requests with ACK, the way to mitigate this
issue is by ignoring an event coming up on a queue that has not been
enabled.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The virtio features are negotiated and set at the time the device is
created, hence there's no need to set the features again while going
through the vhost-user setup that is performed upon queue activation.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that the control queue is correctly handled by the backend, there's
no need to handle it as well from the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The virtio-net control queue must be handled by the backend, the same
way all other queues already are.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This code is ported from the net_util.rs in virtio-devices. The point
being to move it to the net_util crate so that it can later be reused
from vhost-user-net backend.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Yet another small refactoring step for WindowsGuest
after f56471566b.
For this particular case - there's currently neither overloading nor
default argument support in Rust (except a macro or other tricky stuff),
so keep the timeout and other options default for now.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Mark it as unreachable for now in the default implementation as this is
currently only used on tdx code path which is KVM only.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Before this change, the FDT was loaded at the end of RAM. The address of
FDT was not fixed.
While UEFI (edk2 now) requires fixed address to find FDT and RSDP.
Now the FDT is moved to the beginning of RAM, which is a fixed address.
RSDP is wrote to 2 MiB after FDT, also a fixed address.
Kernel comes 2 MiB after RSDP.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
These messages are predominantly during the boot process but will also
occur during events such as hotplug.
These cover all the significant steps of the boot and can be helpful for
diagnosing performance and functionality issues during the boot.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This will make `mkfs.ntfs` available in the next image update, so then
it can be used in the tests.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Separate the population of the memory and the HOB from the TDX
initialisation of the memory so that the latter can happen after the CPU
is initialised.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Some refactoring is performed in order to always expect the irqfd to be
provided by VirtioInterrupt trait. In case no irqfd is available, we
simply fail initializing the vhost-user device. This allows for further
simplification since we can assume the interrupt will always be
triggered directly by the vhost-user backend without proxying through
the VMM. This allows for complete removal of the dedicated thread for
both block and net.
vhost-user-fs is a bit more complex as it requires the slave request
protocol feature in order to support DAX. That's why we still need the
VMM to interfere and therefore run a dedicated thread for it.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now all crates use edition = "2018" then the majority of the "extern
crate" statements can be removed. Only those for importing macros need
to remain.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This will allow progressive porting of parts of the VMM to using
versioned state. A new trait VersionMapped is introduced with a default
implementation that returns an empty VersionMap to ease implementation.
If a struct is updated the the trait will need to be fully implemented
with VersionMapped::version_map() returning a new VersionMap handling
the change.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Bump the sleep time before checking the guest RAM size from 10 to 30
seconds. This will help the VFIO baremetal CI passing more consistently.
Fixes#2606
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
error: all if blocks contain the same code at the start
--> virtio-devices/src/mem.rs:508:9
|
508 | / if plug {
509 | | let handlers =
self.dma_mapping_handlers.lock().unwrap();
|
|_____________________________________________________________________^
|
= note: `-D clippy::branches-sharing-code` implied by `-D warnings`
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#branches_sharing_code
help: consider moving the start statements out like this
|
508 | let handlers = self.dma_mapping_handlers.lock().unwrap();
509 | if plug {
|
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: usage of `contains_key` followed by `insert` on a `BTreeMap`
--> virtio-devices/src/iommu.rs:439:17
|
439 | / if !mappings.contains_key(&domain) {
440 | | mappings.insert(domain, BTreeMap::new());
441 | | }
| |_________________^ help: try this:
`mappings.entry(domain).or_insert_with(|| BTreeMap::new());`
|
= note: `-D clippy::map-entry` implied by `-D warnings`
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#map_entry
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Issue from beta version of clippy:
--> virtio-devices/src/vsock/csm/txbuf.rs:69:34
|
69 | Box::new(unsafe {mem::MaybeUninit::<[u8;
Self::SIZE]>::uninit().assume_init()}));
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[deny(clippy::uninit_assumed_init)]` on by default
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#uninit_assumed_init
Fix backported Firecracker a8c9dffad439557081f3435a7819bf89b87870e7.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: reference to packed field is unaligned
--> virtio-devices/src/vhost_user/fs.rs:85:21
|
85 | fs.flags[i].bits() as i32,
| ^^^^^^^^^^^
|
= note: `-D unaligned-references` implied by `-D warnings`
= warning: this was previously accepted by the compiler but is being
phased out; it will become a hard error in a future release!
= note: for more information, see issue #82523
<https://github.com/rust-lang/rust/issues/82523>
= note: fields of packed structs are not properly aligned, and
creating a misaligned reference is undefined behavior (even if that
reference is never dereferenced)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Regernating the bindings required some minor changes to accomodate
changes around the accessing of unions.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since SGX and VFIO tests don't need to be run on pull requests, we
should not have to wait for the corresponding node (bionic-sgx or
bionic-vfio) to become available in order to skip them.
Fixes#2607
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Re-enable virtiofsd testing now that issues with capstone repository
have been resolved.
This reverts commit 2aec0a92a5.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Remove unnecessary code for these structs. Moving this also allows the
removal of the arch_gen crate.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Relying on dnsmasq running on the host, the Windows guest are now
getting allocated with the expected IP addresses. This allows for
multiple VMs, therefore multiple tests to run in parallel.
The end goal is to reduce the time spent running Windows integration
tests.
Fixes#1891
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This new wrapping structure allows for a better factorization of the
Windows specific code. This makes each test simpler and easier to read.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By using a Box around the DiskConfig trait, it becomes Sized. For that
reason, we can pass the DiskConfig to the Guest so that it can own it.
This allows for further simplification as the Guest does not need to be
bound to a specific lifetime, which makes things easier.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Updated the obsoleted instructions and scripts, based on Open vSwitch
v2.13.1, DPDK stable v19.11.3, and Cloud Hypervisor v15.0 (on Ubuntu
20.04.1).
Signed-off-by: Bo Chen <chen.bo@intel.com>
Refactor the existing vhost-user-net integration tests in order to
extend it with an extra test for vhost-user client mode.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Relying on latest changes from vhost_user_backend crate, this patch
enables the client mode support for vhost-user-net backend.
In this mode, the backend behaves as a client instead of the default
mode where it behaves as the server.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Introduce some new code to support running a vhost-user backend as a
client instead of the default server mode.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The point being to identify clearly that we're running the backend as a
server. This anticipates the addition of a new function for running the
backend as a client.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding the support for an OVS vhost-user backend to connect as the
vhost-user client. This means we introduce with this patch a new
option to our `--net` parameter. This option is called 'server' in order
to ask the VMM to run as the server for the vhost-user socket.
Fixes#1745
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In some situations (booting with OVMF) fewer queues will be enabled
therefore we should iterate over the number of enabled queues (as passed
into VirtioDevice::activate()) rather than the number of create tap
devices.
Fixes: #2578
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Create a temporary copy of the config, add the new device and validate
that. This needs to be done separately to adding it to the config to
avoid race conditions that might be result in config changes being
overwritten.
Fixes: #2564
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
To handle that devices are stored in an Option<Vec<T>> and reduce
duplicated code use generic function to add the devices to the the
struct.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Relying on guest Ubuntu image 21.04, including a 5.11 kernel, this patch
adds some additional tests to the VFIO baremetal integration tests. It
adds a test for ACPI memory hotplug, another one for virtio-mem memory
hotplug, and finally a test for hotplugging the NVIDIA card.
The existing test already taking care of the reboot has been renamed.
The script running "cargo test" has been modified to run only one thread
at a time, so that each test run sequentially. This is mandatory since
the card can't be shared across multiple VMs.
Fixes#2404
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to support most recent Ubuntu distributions, we must update
the way of detecting a reboot through the journal since there is no
more "-- Reboot --" logs.
Using the `--list-boots` option is the preferred way for getting the
boot count information from journalctl command. We simply need to add 1
to the count in order to get the reboot count.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Moving to latest Ubuntu version as the guest image is needed to move to
more recent guest kernel (5.11). With more recent kernels, we'll be able
to add hotplug and virtio-mem tests to the VFIO baremetal CI.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The CI is failing due the git server that the submodules required for
this fork of QEMU need to build from is unavailable.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The latest kvm-sgx code has renamed sgx_virt_epc device node
to sgx_vepc. Update cloud-hypervisor code and documentation to
follow this.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The Rust manifiest requires a three part version number however we
continue with our plan to use two digits so the last part will not be
used.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
test_reboot became flaky after the refactoring of the VM reboot code.
This is because we removed the ability to specify a custom timeout.
This patch fixes the issue by allowing a custom timeout of 120s to be
set.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Because the http thread no longer needs to create the api socket,
remove the socket, bind and listen syscalls from the seccomp filter.
Signed-off-by: William Douglas <william.douglas@intel.com>
Instead of using the http server's method to have it create the
fd (causing the http thread to need to support the socket, bind and
listen syscalls). Create the socket fd in the vmm thread and use the
http server's new method supporting passing in this fd for the api
socket.
Signed-off-by: William Douglas <william.douglas@intel.com>
This allows the guest to reprogram the offload settings and mitigates
issues where the Linux kernel tries to reprogram the queues even when
the feature is not advertised.
Fixes: #2528
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Rather than erroring out and stalling the queue instead report an error
message if the command is invalid and return an error to the guest via
the status field.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Cleanup the control queue handling in preparation for supporting
alternative commands.
Note that this change does not make the MQ handling spec compliant.
According to the specification MQ should only be enabled once the number
of queue pairs the guest would like to use has been specified. The only
improvement towards the specication in this change is correct error
handling if the guest specifies an inappropriate number of queues (out
of range.)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Factorize NVIDIA GPU checks into its own function so that it can be
reused.
Factorize linux guest reboot into its own function to reduce the amount
of code.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
With recent kernels, it is now expected to use "bind" control from the
sysfs in order to bind a specific device to the vfio-pci driver.
The use of "new_id" is still required but it is only needed once per
device type. Let's say you want to bind two virtio-net devices, you will
need to add the device_id:vendor_id pair through "new_id", while you
will be required to use "bind" for each device.
Fixes#2545
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Use the PVH vmlinux for all tests (with the exception of the specific
bzImage test.)
See: #2231
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The new kernel 5.12 requires the devices to be manually bound to
vfio-pci while adding a new_id is only needed once per
device_id:vendor_id pair.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The 5.12 kernel contains a fix for issues observed with nested snapshot
& restore.
Fixes: #2535
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
To avoid race issues where the api-socket may not be created by the
time a cloud-hypervisor caller is ready to look for it, enable the
caller to pass the api-socket fd directly.
Avoid breaking current callers by allowing the --api-socket path to be
passed as it is now in addition to through the path argument.
Signed-off-by: William Douglas <william.r.douglas@gmail.com>
Setting the tap offload should only be done based on the features that
are acked by the guest. Therefore it is incorrect to set these upon
opening the tap.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In the reference vhost_user_net backend don't advertise the offload
features. For correctness we should only advertise them if we can set
them when they are acked by the guest.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
It appears that mshv is not yet there to succeed with these tests. It is
suggested to ignore them and enable later one by one as the
functionality gets fixed.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
In order to support using Versionize for state structures it is necessary
to use simpler, primitive, data types in the state definitions used for
snapshot restore.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to support using Versionize for state structures it is necessary
to use simpler, primitive, data types in the state definitions used for
snapshot restore.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to support using Versionize for state structures it is necessary
to use simpler, primitive, data types in the state definitions used for
snapshot restore.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the tap file descriptor is not writable then try again later. Update
the RX side to match the test on std::io::ErrorKind::WouldBlock
Fixes: #2517
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Duplicate the fd that is specified in the config so that be used again
after a reboot. When rebooting we destroy all VM state and restore from
the config.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Windows guests read this field upon PCI device ejection. Let's make sure
we don't return an error as this is valid. We simply return an empty u32
since the ejection is done right away upon write access, which means
there's no pending ejection that might be reported to the guest.
Here is the error that was shown during PCI device removal:
ERROR:vmm/src/device_manager.rs:3960 -- Accessing unknown location at
base 0x7ffffee000, offset 0x8
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The logic wasn't quite right, as it wasn't detecting BAR reprogramming
when the upper part of the address was identical. For instance, a BAR
moved from 0x7fc0000000 to 0x7fd0000000 wasn't detected properly.
The logic has been updated and cleaned up to fix this issue, which was
observed when running Windows guests. This fixes the network hotplug
support as well.
Fixes#1797Fixes#1798
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to avoid possible issues with Bionic Beaver (18.04) as it's
getting old, we move the CI container to Focal Fossa (20.04).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of tracking on a block level of 64 pages, we are now collecting
dirty pages one by one. It improves the efficiency of dirty memory
tracking while live migration.
Signed-off-by: Bo Chen <chen.bo@intel.com>
With current serde_derive it is possible to #[derive(Serialize)] on
packed structures if they implement Copy. This allows the removal of the
manual equivalent code.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
By disabling this KVM feature, we prevent the guest from using APF
(Asynchronous Page Fault) mechanism. The kernel has recently switched to
using interrupts to notify about a page being ready, but for some
reasons, this is causing unexpected behavior with Cloud Hypervisor, as
it will make the vcpu thread spin at 100%.
While investigating the issue, it's better to disable the KVM feature to
prevent 100% CPU usage in some cases.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The original code had a generic type E. It was later replaced by a
concrete type. The code should have been simplified when the replacement
happened.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
For now, memory layout on arm64 is sparse and is conflict with uefi.
Here, we do some rearrangement to let it compact and compatible with
uefi support.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
We now reply on the value from '/sys/kernel/mm/ksm/shared_pages' to
validate our "--memory mergeable=on|off" option. For `mergeable=on`,
we are expecting to see more 'shared_pages' reported by the kernel when
we start more VMs with this option. For `mergeable=off`, we are
expecting the 'shared_pages' value to be always 0, as we are assuming
the rest of the system (in our CI) is not using mergeable memory.
Fixes: #2138
Signed-off-by: Bo Chen <chen.bo@intel.com>
The MCRS method returns a 64-bit memory range descriptor. The
calculation is supposed to be done as follows:
max = min + len - 1
However, every operand is represented not as a QWORD but as combination
of two DWORDs for high and low part. Till now, the calculation was done
this way, please see also inline comments:
max.lo = min.lo + len.lo //this may overflow, need to carry over to high
max.hi = min.hi + len.hi
max.hi = max.hi - 1 // subtraction needs to happen on the low part
This calculation has been corrected the following way:
max.lo = min.lo + len.lo
max.hi = min.hi + len.hi + (max.lo < min.lo) // check for overflow
max.lo = max.lo - 1 // subtract from low part
The relevant part from the generated ASL for the MCRS method:
```
Method (MCRS, 1, Serialized)
{
Acquire (MLCK, 0xFFFF)
\_SB.MHPC.MSEL = Arg0
Name (MR64, ResourceTemplate ()
{
QWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, Cacheable, ReadWrite,
0x0000000000000000, // Granularity
0x0000000000000000, // Range Minimum
0xFFFFFFFFFFFFFFFE, // Range Maximum
0x0000000000000000, // Translation Offset
0xFFFFFFFFFFFFFFFF, // Length
,, _Y00, AddressRangeMemory, TypeStatic)
})
CreateQWordField (MR64, \_SB.MHPC.MCRS._Y00._MIN, MINL) // _MIN: Minimum Base Address
CreateDWordField (MR64, 0x12, MINH)
CreateQWordField (MR64, \_SB.MHPC.MCRS._Y00._MAX, MAXL) // _MAX: Maximum Base Address
CreateDWordField (MR64, 0x1A, MAXH)
CreateQWordField (MR64, \_SB.MHPC.MCRS._Y00._LEN, LENL) // _LEN: Length
CreateDWordField (MR64, 0x2A, LENH)
MINL = \_SB.MHPC.MHBL
MINH = \_SB.MHPC.MHBH
LENL = \_SB.MHPC.MHLL
LENH = \_SB.MHPC.MHLH
MAXL = (MINL + LENL) /* \_SB_.MHPC.MCRS.LENL */
MAXH = (MINH + LENH) /* \_SB_.MHPC.MCRS.LENH */
If ((MAXL < MINL))
{
MAXH += One /* \_SB_.MHPC.MCRS.MAXH */
}
MAXL -= One
Release (MLCK)
Return (MR64) /* \_SB_.MHPC.MCRS.MR64 */
}
```
Fixes#1800.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Simplify snapshot & restore code by using generics to specify helper
functions that take / make a Serialize / Deserialize struct
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since using bzImage is now deprecated, let's update the SGX integration
test to rely on vmlinux instead.
Fixes#2476
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This allows the return of errors which will be printed using the
existing code and removes panic()s
Fixes: #2342
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the return of errors which will be printed using the
existing code and removes panic()s
Fixes: #2342
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Both changes aim to document the absence of the CPU hot-remove
functionality on Windows.
Closes#2457.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Update the Ubuntu Focal image used as the guest image. It's based on the
latest Focal image released on April 1st 2021, and customized to include
all the utilities we need. As usual, snapd and pollinate services have
been removed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This patch moves out the actual processing on the TX queue from the
`handle_tx_event()` function into a separate function,
e.g. `process_tx()`. This allows us to resume the TX queue processing
without reading from the TX queue EventFd, which is needed for rate
limiting support.
No functional change.
Signed-off-by: Bo Chen <chen.bo@intel.com>
To support I/O throttling on virt-net devices, we need to use the
'rate_limiter' module from the 'net_utils' crate. Given the
'virtio-devices' crate has dependency on the 'net_utils', we will need
to move the 'rate_limiter' module out of the 'virtio-devices' crate to
avoid circular dependency issue. Considering the 'rate_limiter' is not
virtio specific and could be reused for non virtio devices, we move it
to its own crate.
Signed-off-by: Bo Chen <chen.bo@intel.com>
DescriptorChain::is_valid() wrongly used .checked_offset() to attempt to
validate that the descriptor's data is in valid memory. This works in
all cases except where the guest has placed the data at the very end of
the guest memory as the offset + offset will be outside the range (as
the combined offset will be the next byte and as such out of the guest
memory). Instead use the function .check_range() takes an offset and a
length to validate
This fixes issues see with error messages featuring the
DescriptorChainTooShort error.
Fixes: #2424
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Fixes the current codebase so that every cargo clippy can be run with
the beta toolchain without any error.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There are two parts:
- Unconditionally zero the output area. The length of the incoming
vector has been seen from 1 to 4 bytes, even though just the first
byte might need to be handled. But also, this ensures any possibly
unhandled offset will return zeroed result to the caller. The former
implementation used an I/O port which seems to behave differently from
MMIO and wouldn't require explicit output zeroing.
- An access with zero offset still takes place and needs to be handled.
Fixes#2437.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
On nightly, clippy expects the structs to be constructed with fields in
the same order they're declared in. Seems sensible enough, so let's do
that.
Signed-off-by: Gaelan Steele <gbs@canishe.com>
The following is from the Hyper-V specification v6.0b.
Cpuid leaf 0x40000003 EDX:
Bit 3: Support for physical CPU dynamic partitioning events is
available.
When Windows determines to be running under a hypervisor, it will
require this cpuid bit to be set to support dynamic CPU operations.
Cpuid leaf 0x40000004 EAX:
Bit 5: Recommend using relaxed timing for this partition. If
used, the VM should disable any watchdog timeouts that
rely on the timely delivery of external interrupts.
This bit has been figured out as required after seeing guest BSOD
when CPU hotplug bit is enabled. Race conditions seem to arise after a
hotplug operation, when a system watchdog has expired.
Closes#1799.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
error: an implementation of `From` is preferred since it gives you `Into<_>` for free where the reverse isn't true
Error: --> devices/src/legacy/rtc_pl031.rs:73:1
|
73 | impl Into<libc::clockid_t> for ClockType {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `-D clippy::from-over-into` implied by `-D warnings`
= help: consider to implement `From` instead
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#from_over_into
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: redundant slicing of the whole range
Error: --> devices/src/legacy/gpio_pl061.rs:298:37
|
298 | let value = read_le_u32(&data[..]);
| ^^^^^^^^^ help: use the original slice instead: `data`
|
= note: `-D clippy::redundant-slicing` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_slicing
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: name `GPIOInterruptDisabled` contains a capitalized acronym
Error: --> devices/src/legacy/gpio_pl061.rs:46:5
|
46 | GPIOInterruptDisabled,
| ^^^^^^^^^^^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `GpioInterruptDisabled`
|
= note: `-D clippy::upper-case-acronyms` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: an implementation of `From` is preferred since it gives you `Into<_>` for free where the reverse isn't true
--> hypervisor/src/vm.rs:41:1
|
41 | impl Into<u64> for DataMatch {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(clippy::from_over_into)]` on by default
= help: consider to implement `From` instead
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#from_over_into
warning: 1 warning emitted
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: name `AcpiPMTimerDevice` contains a capitalized acronym
--> devices/src/acpi.rs:175:12
|
175 | pub struct AcpiPMTimerDevice {
| ^^^^^^^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `AcpiPmTimerDevice`
|
= note: `#[warn(clippy::upper_case_acronyms)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: name `IORegion` contains a capitalized acronym
--> pci/src/configuration.rs:320:5
|
320 | IORegion = 0x01,
| ^^^^^^^^ help: consider making the acronym lowercase, except the initial letter (notice the capitalization): `IoRegion`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: name `ConvertFromUTF8` contains a capitalized acronym
--> virtio-devices/src/vsock/unix/mod.rs:32:5
|
32 | ConvertFromUTF8(std::str::Utf8Error),
| ^^^^^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `ConvertFromUtf8`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: name `InvalidCPUCount` contains a capitalized acronym
--> src/bin/ch-remote.rs:24:5
|
24 | InvalidCPUCount(std::num::ParseIntError),
| ^^^^^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `InvalidCpuCount`
|
= note: `#[warn(clippy::upper_case_acronyms)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: name `TranslateGVA` contains a capitalized acronym
--> hypervisor/src/arch/emulator/mod.rs:51:5
|
51 | TranslateGVA(#[source] anyhow::Error),
| ^^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `TranslateGva`
|
= note: `#[warn(clippy::upper_case_acronyms)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: redundant slicing of the whole range
--> vhost_user_block/src/lib.rs:396:31
|
396 | right.copy_from_slice(&data[..]);
| ^^^^^^^^^ help: use the original slice instead: `data`
|
= note: `-D clippy::redundant-slicing` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_slicing
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: name `RSDPPastRamEnd` contains a capitalized acronym
--> arch/src/lib.rs:59:5
|
59 | RSDPPastRamEnd,
| ^^^^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `RsdpPastRamEnd`
|
= note: `-D clippy::upper-case-acronyms` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: writing `&PathBuf` instead of `&Path` involves a new object where a slice will do.
--> block_util/src/lib.rs:68:31
|
68 | fn build_device_id(disk_path: &PathBuf) -> result::Result<String, Error> {
| ^^^^^^^^ help: change this to: `&Path`
|
= note: `-D clippy::ptr-arg` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
error: writing `&PathBuf` instead of `&Path` involves a new object where a slice will do.
--> block_util/src/lib.rs:83:39
|
83 | pub fn build_disk_image_id(disk_path: &PathBuf) -> Vec<u8> {
| ^^^^^^^^ help: change this to: `&Path`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
error: writing `&PathBuf` instead of `&Path` involves a new object where a slice will do.
--> block_util/src/lib.rs:68:31
|
68 | fn build_device_id(disk_path: &PathBuf) -> result::Result<String, Error> {
| ^^^^^^^^ help: change this to: `&Path`
|
= note: `-D clippy::ptr-arg` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
error: writing `&PathBuf` instead of `&Path` involves a new object where a slice will do.
--> block_util/src/lib.rs:83:39
|
83 | pub fn build_disk_image_id(disk_path: &PathBuf) -> Vec<u8> {
| ^^^^^^^^ help: change this to: `&Path`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: redundant slicing of the whole range
--> net_util/src/mac.rs:60:35
|
60 | bytes[..].copy_from_slice(&src[..]);
| ^^^^^^^^ help: use the original slice instead: `src`
|
= note: `-D clippy::redundant-slicing` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_slicing
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: name `TYPE_UNKNOWN` contains a capitalized acronym
--> vm-virtio/src/lib.rs:48:5
|
48 | TYPE_UNKNOWN = 0xFF,
| ^^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `Type_Unknown`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
error: name `GetDeviceID` contains a capitalized acronym
--> block_util/src/lib.rs:138:5
|
138 | GetDeviceID,
| ^^^^^^^^^^^ help: consider making the acronym lowercase, except the initial letter: `GetDeviceId`
|
= note: `-D clippy::upper-case-acronyms` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#upper_case_acronyms
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
With CONFIG_PVH in stable kernels for some time we should deprecate the
use of alternative boot methods since this will lead to a much simpler
boot flow and CI process.
See: #2231
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
There are multiple reports of DescriptorChainTooShort errors and so add
some extra debugging to aid the debugging of this issue.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Drop the generic type E and use IrqRoutngEntry directly. This allows
dropping a bunch of trait bounds from code.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Their make_entry functions look the same now. Extract the logic to a
common function.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This returns trues if this descriptor has another descriptor linked to
it. Not whether this descriptor chain has another one following it.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The 0.3.0 release (re)supports booting stock Ubuntu images so users
should be pointed towards that.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
There is no point in queueing an empty descriptor in the list of iovecs.
Let's simply ignore such case and avoid some unnecessary processing.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The hotplug documentation was missing the explanation on how to use
memory hotplug with virtio-mem.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Document that seccomp is on and how to disable it along with a pointer
on how to identify missing syscalls during development.
Fixes#993
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It must be specified as excluded from the workspace as it must not be
built on non-test targets due to issues with the ssh2 dependency and the
musl toolchain.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
refactor vec_with_array_field to common hypervisor code
so that mshv can also make use of it.
Signed-off-by: Vineeth Pillai <viremana@linux.microsoft.com>
Bumps iced-x86 from 1.10.3 to 1.11.0.
Manual update of the code was needed since memory_displacement() was
deprecated and replaced with either memory_displacement32() or
memory_displacement64().
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There's no need to have the code creating the passthrough_device being
duplicated since we can factorize it in a function used in both cases
(both cold plugged and hot plugged devices VFIO devices).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend and use the existing DeviceTree to retrieve useful information
related to PCI devices. This removes the duplication with pci_devices
field which was internal to the DeviceManager.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Make the code a bit clearer by changing the naming of the structure
holding the list of IRQs reserved for PCI devices. It is also modified
into an array of 32 entries since we know this is the amount of PCI
slots that is supported.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We define a new enum in order to classify PCI device under virtio or
VFIO. This is a cleaner approach than using the Any trait, and
downcasting it to find the object back.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Introduces a tuple holding both information needed by pci_id_list and
pci_devices.
Changes pci_devices to be a BTreeMap of this new tuple.
Now that pci_devices holds the information needed from pci_id_list,
pci_id_list is no longer needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for further factorization, the pci_id_list is now a
hashmap of PCI b/d/f leading to each device name.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of relying on a PCI specific device list, we use the DeviceTree
as a reference to determine if a device name is already in use or not.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Remove the startup message containing incomplete VM configuration
details. It's a bit unusual for a tool such as Cloud Hypervisor to print
those kind of details which are a direct representation of the command
line.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This includes:
* OS disk image management
* Cloud init creation
* SSH to guest access
* Waiting for guest to boot
This will be useful in other projects that want to do similar things in
their integration tests.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Only if we have a valid API server path then create the API server. For
now this has no functional change there is a default API server path in
the clap handling but rather prepares to do so optionally.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In case of the virtio frontend driver doesn't need interupts for
certain queue event, it may explicitly write VIRTIO_MSI_NO_VECTOR
to the virtio common configuration, or it may doesn't configure
the event type vector at all.
This patch initializes both MSI-X configuration vector and queue vector
with VIRTIO_MSI_NO_VECTOR, so that the backend drivers won't trigger
unexpected interrupts to the guest.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Relying on a NVIDIA Tesla T4 card present in the SGX machine, this patch
enables baremetal VFIO testing, validated by running several NVIDIA
tools in the guest. The guest image has been prepared to include all the
software needed to run these tests.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit implements ARM PrimeCell General Purpose Input/Output
(GPIO) PL061 device specification.
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit switches the default serial device from 16550 to the
Arm dedicated UART controller PL011. The `ttyAMA0` can be enabled.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Enabled all "ttyS0" related test cases:
- test_serial_off
- test_serial_tty
- test_serial_file
Enabled mandatory guest kernel driver for "ns16550a" on AArch64.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
On AArch64, interrupt controller (GIC) is emulated by KVM. VMM need to
set IRQ routing for devices, including legacy ones.
Before this commit, IRQ routing was only set for MSI. Legacy routing
entries of type KVM_IRQ_ROUTING_IRQCHIP were missing. That is way legacy
devices (like serial device ttyS0) does not work.
The setting of X86 IRQ routing entries are not impacted.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Extend the existing url_to_path() to take the URL string and then use
that to simplify the snapshot/restore code paths.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Now that virtio devices can be updated with add_memory_region(), there's
no need to keep update_memory() around.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Relies on the preliminary work allowing virtio devices to be updated
with a single memory at a time instead of updating the entire memory at
once.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Assuming vhost-user devices support CONFIGURE_MEM_SLOTS protocol
feature, we introduce a new method to the VirtioDevice trait in order to
update one single memory at a time.
In case CONFIGURE_MEM_SLOTS is not supported by the backend (feature not
acked), we fallback onto the current way of updating the memory
mappings, that is with SET_MEM_TABLE.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to support GET_MAX_MEM_SLOTS, ADD_MEM_REG and REM_MEM_REG, the
protocol feature CONFIGURE_MEM_SLOTS must be enabled.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to support GET_MAX_MEM_SLOTS, ADD_MEM_REG and REM_MEM_REG, the
protocol feature CONFIGURE_MEM_SLOTS must be enabled.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The latest vhost version adds the support for the new commands
get_max_mem_slots(), add_mem_region() and remove_mem_region(), all
related to the new vhost-user protocol feature CONFIGURE_MEM_SLOTS.
The vhost_user_backend crate is updated accordingly in order to support
these new commands, mostly related to the capability of updating the
guest memory mappings with a finer control than set_mem_table() command.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
On x86_64 architecture, multiple syscalls were missing when shutting
down the vhost-user-net device along with the VM. This was causing the
usual crash related to seccomp filters.
This commit adds these missing syscalls to fix the issue.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There is no need to get the vring base when resetting the vhost-user
device. This was mostly ignored, but in some cases, it was causing some
actual errors.
A reset must simply be a combination of disabling the vrings along with
the reset of the owner.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Originally, VhostUserSetup is only used by vhost-user-fs. While
vhost-user-blk and vhost-user-net have their own error messages,
we rename VhostUserSetup to VhostUserFsSetup.
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
The MMIO structure contains the length rather than the maximum address
so it is necessary to subtract the starting address from the end address
to calculate the length.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In the absence of a way of integration testing this testing that it
compiles is reasonable compromise.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Load the sections backed from the file into their required addresses in
memory and populate the HOB with details of the memory. Using the HOB
address initialize the TDX state in the vCPUs and finalize the TDX
configuration.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add API to the hypervisor interface and implement for KVM to allow the
special TDX KVM ioctls on the VM and vCPU FDs.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When booting with TDX no kernel is supplied as the TDFV is responsible
for loading the OS. The requirement to have the kernel is still
currently enforced at the validation entry point; this change merely
changes function prototypes and stored state to use Option<> to support.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add support extracting the sections out for a TDVF file which can be
then used to load the TDVF and TD HOB data into their appropriate
locations.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add the skeleton of the "tdx" feature with a module ready inside the
arch crate to store implementation details.
TEST=cargo build --features="tdx"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
These need to be updated together as the kvm-ioctls depends upon a
strictly newer version of kvm-bindings which requires a rebase in the CH
fork.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When a vm is created with a pty device, on reboot the pty fd (sub
only) will only be associated with the vmm through the epoll event
loop. The fd being polled will have been closed due to the vm itself
dropping the pty files (and potentially reopening the fd index to a
different item making things quite confusing) and new pty fds will be
opened but not polled on for input.
This change creates a structure to encapsulate the information about
the pty fd (main File, sub File and the path to the sub File). On
reboot, a copy of the console and serial pty structs is then passed
down to the new Vm instance which will be used instead of creating a
new pty device.
This resolves the underlying issue from #2316.
Signed-off-by: William Douglas <william.r.douglas@gmail.com>
Now that virtio-mem devices can update VFIO mappings through dedicated
handlers, let's provide them from the DeviceManager.
Important to note these handlers should either be provided to virtio-mem
devices or to the unique virtio-iommu device. This must be mutually
exclusive.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Create two functions for registering/unregistering DMA mapping handlers,
each handler being associated with a VFIO device.
Whenever the plugged_size is modified (which means triggered by the
virtio-mem driver in the guest), the virtio-mem backend is responsible
for updating the DMA mappings related to every VFIO device through the
handler previously provided.
It's important to update the map when the handler is either registered
or unregistered as well, as we don't want to miss some plugged memory
that would have been added before the VFIO device is added to the VM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of letting the VfioPciDevice take the decision on how/when to
perform the DMA mapping/unmapping, we move this to the DeviceManager
instead.
The point is to let the DeviceManager choose which guest memory regions
should be mapped or not. In particular, we don't want the virtio-mem
region to be mapped/unmapped as it will be virtio-mem device
responsibility to do so.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When memory is resized through ACPI, a new region is added to the guest
memory. This region must also be added to the corresponding memory zone
in order to keep everything in sync.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Replace "--monitor-fd" with "--event-monitor" which can either take
"fd=<int>" or "path=<path>" which can point to e.g. a named pipe and
allow more flexibility.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since the SGX server is down for maintenance, all builds are waiting on
the node agent to answer, causing all PRs to be blocked.
Let's disable temporarily the SGX CI until the server is back up.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
So far we've only had the need to emulate one instruction. There is no
need to use a HashMap when a simple tuple for the initial mapping will
do.
We can bring back the HashMap once more sophisticated use cases surface.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This avoids code complexity down the line when we get around
implementing Windows support.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Moving to the latest version of the rust-vmm/vhost crate, before it gets
published on crates.io.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In particular update for the vmm-sys-util upgrade and all the other
dependent packages. This requires an updated forked version of
kvm-bindings (due to updated vfio-ioctls) but allowed the removal of our
forked version of kvm-ioctls.
The changes to the API from kvm-ioctls and vmm-sys-util required some
other minor changes to the code.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The vhost crate from rust-vmm is ready, which is why we do the switch
from the Cloud Hypervisor fork to the upstream crate.
At the same time, we rename the crate from vhost_rs to vhost.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit moves both pci and vmm code from the internal vfio-ioctls
crate to the upstream one from the rust-vmm project.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that ExternalDmaMapping is defined in vm-device, let's use it from
there.
This commit also defines the function get_host_address_range() to move
away from the vfio-ioctls dependency.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By moving the trait and its VFIO implementation out of vfio-ioctls, we
give anticipate for the move to the vfio-ioctls from rust-vmm.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This has been lagging behind on an older Rust version and we have
enough coverage from our Jenkins CI agent.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This is only use in the integration test and was erroneously included in
the main binary dependencies.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The interrupt tests were not being run as they were erroneously under a
feature guard that does not exist in arch.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This removes the dependency on "tempdir" which in turn depends on the
large rand dependency chain.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The help information displayed for our `--disk` option is incorrect and
incomplete, e.g. missing the `direct` and `poll_queue` field.
Signed-off-by: Bo Chen <chen.bo@intel.com>
According to Intel's mnemonic (which is used by iced-x86) the first
argument is destination while the second is source.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The main idea behind this commit is to remove all the complexity
associated with TX/RX handling for virtio-net. By using writev() and
readv() syscalls, we could get rid of intermediate buffers for both
queues.
The complexity regarding the TAP registration has been simplified as
well. The RX queue is only processed when some data are ready to be
read from TAP. The event related to the RX queue getting more
descriptors only serves the purpose to register the TAP file if it's not
already.
With all these simplifications, the code is more readable but more
performant as well. We can see an improvement of 10% for a single
queue device.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Currently script does not exit early if the image/firmware not present
in the host. We should not progress further if the images are not pre
downloaded.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This crate exposes the abililty for the VMM to set a file that events
should be written to. The event!() macro provides an interface to report
those events allowing the specification of an event source, an event
type and optional extra data. This will be written to the provided file
descriptor as JSON data.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We don't have an easy way to figure out if a GPA points to normal memory
or device memory, but the guest's normal memory regions shouldn't
overlap with device regions. We can simply try to do a normal memory
read / write, and proceed to do device memory read / write if that
fails.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This function can then be used by the TDX code to allocate the memory at
specific locations required for the TDVF to run from.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
OVMF would use string IO on those ports. String IO has not been
implemented, so that leads to panics.
Skip them explicitly in MSHV. Leave a long-ish comment in code to
explain the situation. We should properly implement string IO once it
becomes feasible / necessary.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Update for clippy in Rust 1.50.0:
error: Unnecessary nested match
--> vmm/src/vm.rs:419:17
|
419 | / if let vm_device::BusError::MissingAddressRange = e {
420 | | warn!("Guest MMIO write to unregistered address 0x{:x}", gpa);
421 | | }
| |_________________^
|
= note: `-D clippy::collapsible-match` implied by `-D warnings`
help: The outer pattern can be modified to include the inner pattern.
--> vmm/src/vm.rs:418:17
|
418 | Err(e) => {
| ^ Replace this binding
419 | if let vm_device::BusError::MissingAddressRange = e {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ with this pattern
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#collapsible_match
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the function can never return an error this is now a clippy failure:
error: this function's return value is unnecessarily wrapped by `Result`
--> virtio-devices/src/watchdog.rs:215:5
|
215 | / fn set_state(&mut self, state: &WatchdogState) -> io::Result<()> {
216 | | self.common.avail_features = state.avail_features;
217 | | self.common.acked_features = state.acked_features;
218 | | // When restoring enable the watchdog if it was previously enabled. We reset the timer
... |
223 | | Ok(())
224 | | }
| |_____^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_wraps
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This addresses a newly added clippy issue:
error: methods called `as_*` usually take self by reference or self by mutable reference; consider choosing a less ambiguous name
--> vm-migration/src/protocol.rs:57:34
|
57 | fn as_mut_bytes<T: Sized>(p: &mut T) -> &mut [u8] {
| ^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#wrong_self_convention
error: aborting due to 2 previous errors
error: could not compile `vm-migration`
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Depending on the host OS the code for looking up the time for the CMOS
make require extra syscalls to be permitted for the vCPU thread.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In case the VFIO device does not support MSI or MSI-X, the capabilities
should not be parsed, avoiding the exposure of unsupported capabilities.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Make sure to propagate the error coming from VfioDevice when trying to
enable INTx, MSI or MSI-X interrutps.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since INTx is now supported for VFIO PCI devices, there is no more
limitations regarding VFIO, which means the documentation must be
updated accordingly.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
With all the preliminary work done in the previous commits, we can
update the VFIO implementation to support INTx along with MSI and MSI-X.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Here we are adding the PCI routing table, commonly called _PRT, to the
ACPI DSDT. For simplification reasons, we chose not to implement PCI
links as this involves dynamic decision from the guest OS, which result
in lots of complexity both from an AML perspective and from a device
manager perspective.
That's why the _PRT creates a static list of 32 entries, each assigned
with the IRQ number previously reserved by the device manager.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to support INTx for PCI devices, each PCI device must be
assigned an IRQ. This is preliminary work to reserve 8 IRQs which will
be shared across the 32 PCI devices.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for accessing the legacy interrupt manager from the
function creating a VFIO PCI device, we store it as part of the
DeviceManager, to make it available for all methods.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The DeviceManager already has a hold onto the MSI interrupt manager,
therefore there's no need to pass it through every function. Instead,
let's simplify the code by using the attribute from DeviceManager's
instance.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Both GIC and IOAPIC must implement a new method notifier() in order to
provide the caller with an EventFd corresponding to the IRQ it refers
to.
This is needed in anticipation for supporting INTx with VFIO PCI
devices.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for supporting the notifier function for the legacy
interrupt source group, we need this function to return an EventFd
instead of a reference to this same EventFd.
The reason is we can't return a reference when there's an Arc<Mutex<>>
involved in the call chain.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Swap the last two parameters of guest_mem_{read,write} to be consistent
with other read / write functions.
Use more descriptive parameter names.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This reflects that it generates CPUID state used across all vCPUs.
Further ensure that errors from this function get correctly propagated.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Move the code for populating the CPUID with KVM HyperV emulation details from
the per-vCPU CPUID handling code to the shared CPUID handling code.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Move the code for populating the CPUID with details of the CPU
identification from the per-vCPU CPUID handling code to the shared CPUID
handling code.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Move the code for populating the CPUID with details of the maximum
address space from the per-vCPU CPUID handling code to the shared CPUID
handling code.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add the ability for cloud-hypervisor to create, manage and monitor a
pty for serial and/or console I/O from a user. The reasoning for
having cloud-hypervisor create the ptys is so that clients, libvirt
for example, could exit and later re-open the pty without causing I/O
issues. If the clients were responsible for creating the pty, when
they exit the main pty fd would close and cause cloud-hypervisor to
get I/O errors on writes.
Ideally the main and subordinate pty fds would be kept in the main
vmm's Vm structure. However, because the device manager owns parsing
the configuration for the serial and console devices, the information
is instead stored in new fields under the DeviceManager structure
directly.
From there hooking up the main fd is intended to look as close to
handling stdin and stdout on the tty as possible (there is some future
work ahead for perhaps moving support for the pty into the
vmm_sys_utils crate).
The main fd is used for reading user input and writing to output of
the Vm device. The subordinate fd is used to setup raw mode and it is
kept open in order to avoid I/O errors when clients open and close the
pty device.
The ability to handle multiple inputs as part of this change is
intentional. The current code allows serial and console ptys to be
created and both be used as input. There was an implementation gap
though with the queue_input_bytes needing to be modified so the pty
handlers for serial and console could access the methods on the serial
and console structures directly. Without this change only a single
input source could be processed as the console would switch based on
its input type (this is still valid for tty and isn't otherwise
modified).
Signed-off-by: William Douglas <william.r.douglas@gmail.com>
This thread is virtio-net specific, so it is not handled in the common
virtio device code.
The non-vhost implementation resumes the thread itself. Do the same
thing for vhost-user-net.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Use the newly added hugepages_size option if provided by the user to
pick a huge page size when creating the memfd region. If none is
specified use the system default.
Sadly different huge pages cannot be tested by an integration test as
creating a pool of the non-default size cannot be done at runtime
(requires kernel to be booted with certain parameters.)
TETS=Manually tested with a kernel booted with both 1GiB and 2MiB huge
pages (hugepagesz=1G hugepages=1 hugepagesz=2M hugepages=512)
Fixes: #2230
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Including a warning that the user is respsonsible for ensuring that they
have sufficient pages of the specified size.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the user to use an alternative huge page size otherwise the
default size will be used.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This commit introduces a new information to the VirtioMemZone structure
in order to know if the memory zone is backed by hugepages.
Based on this new information, the virtio-mem device is now able to
determine if madvise(MADV_DONTNEED) should be performed or not. The
madvise documentation specifies that MADV_DONTNEED advice will fail if
the memory range has been allocated with some hugepages.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Hui Zhu <teawater@antfin.com>
This commit performs some refactoring to make all functions a method
from a specific object, and in particular methods for MemEpollHandler.
The point is to simplify the code to make it more readable.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adjust the code to comply better with the virtio-mem specification by
adding some validation for the virtio-mem configuration, but also by
updating the virtio-mem configuration itself.
Nowhere in the virtio-mem specification is stated the usable region size
must be adjusted everytime the plugged size changes. For simplification
reason, and without going against the specification, the usable region
size is now kept static, setting its value to the size of the whole
region.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By introducing a ResizeSender object, we avoid having a Resize clone
with a different content than the original Resize object.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
MOV R/RM is a special case of MOVZX, so we generalize the mov_r_rm macro
to make it support both instructions.
Fixes: #2227
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The vhd module is the implementation of the VHD specification, which is
why it is important to unit test it.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Let's create a fixed VHD disk file from the existing RAW file thanks to
qemu-img, and create a new integration test to validate that
Cloud-Hypervisor can boot VHD disk image.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Relying on the simplified version of the synchronous support for RAW
disk files, the new fixed_vhd_sync module in the block_util crate
introduces the synchronous support for fixed VHD disk files.
With this patch, the fixed VHD support is complete as it is implemented
in both synchronous and asynchronous versions.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Using directly preadv and pwritev, we can simply use a RawFd instead of
a file, and we don't need to use the more complex implementation from
the qcow crate.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit adds the asynchronous support for fixed VHD disk files.
It introduces FixedVhd as a new ImageType, moving the image type
detection to the block_util crate (instead of qcow crate).
It creates a new vhd module in the block_util crate in order to handle
VHD footer, following the VHD specification.
It creates a new fixed_vhd_async module in the block_util crate to
implement the asynchronous version of fixed VHD disk file. It relies on
io_uring.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
_EJx built in should not return.
dsdt.dsl 813: Return (CEJ0 (0x00))
Warning 3104 - ^ Reserved method should not return a value (_EJ0)
dsdt.dsl 813: Return (CEJ0 (0x00))
Error 6080 - ^ Called method returns no value
Fixes: #2216
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The mutex timeout should be 0xffff rather than 0xfff to disable the
timeout feature.
dsdt.dsl 745: Acquire (\_SB.PRES.CPLK, 0x0FFF)
Warning 3130 - ^ Result is not used, possible operator timeout will be missed
dsdt.dsl 767: Acquire (\_SB.PRES.CPLK, 0x0FFF)
Warning 3130 - ^ Result is not used, possible operator timeout will be missed
dsdt.dsl 775: Acquire (\_SB.PRES.CPLK, 0x0FFF)
Warning 3130 - ^ Result is not used, possible operator timeout will be missed
Fixes: #2216
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
By using `net_util::open_tap` to create the TAP interface, the created
interface will be deleted when the returned variable (`net_utils::Tap`)
is dropped.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch enables multi-queue support for creating virtio-net devices by
accepting multiple TAP fds, e.g. '--net fds=3:7'.
Fixes: #2164
Signed-off-by: Bo Chen <chen.bo@intel.com>
This helper can open a TAP device and configure the interface on it. If
the device needs to be opened multiple times for MQ then it also handles
that correctly.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Because of the behavior of the NVIDIA proprietary driver, we can't
expect NVIDIA cards with only MSI support to be functioning correctly
after they've been passed through with Cloud-Hypervisor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Building with 1.51 nightly produces the following warning:
warning: unnecessary trailing semicolon
--> vmm/src/device_manager.rs:396:6
|
396 | };
| ^ help: remove this semicolon
|
= note: `#[warn(redundant_semicolons)]` on by default
warning: 1 warning emitted
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The unit tests ask the Linux kernel to generate a TAP device name on
demand by passing in a format string. I suspect, but haven't been able
to confirm that there might be a rare race that triggers when creating
lots of devices in a short period of time. This is appearing in our unit
test as the occassional flake of the test_tap_read() which although it
has successfully created the device it fails to set the IP address on it
when looking it back up by it's name.
Since this is the most frequent cause of failures on our CI use a lock
to ensure that multiple TAP devices are not created simultaneously.
Fixes: #2135
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This skeleton commit brings in the support for compiling aarch64 with
the "acpi" feature ready to the ACPI enabling. It builds on the work to
move the ACPI hotplug devices from I/O ports to MMIO and conditionalises
any code that is x86_64 only (i.e. because it uses an I/O port.)
Filling in the aarch64 specific details in tables such as the MADT it
out of the scope.
See: #2178
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
e.g.
scripts/dev_cli.sh tests --integration -- --test-filter test_watchdog
This used to be supported by passing "$@" but was broken when multiple
hypervisor support was added.
Fixes: #2182
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since QCOW and RAW synchronous implementation are very close, it makes
sense to introduce some common functions that can be shared between
these two.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It might be useful debugging information for the user to know what kind
of disk file implementation is in use.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that BlockIoUring is the only implementation of virtio-block,
handling both synchronous and asynchronous backends based on the
AsyncIo trait, we can rename it to Block.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that both synchronous and asynchronous backends rely on the
asynchronous version of virtio-block (namely BlockIoUring), we can
get rid of the synchronous version (namely Block).
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the synchronous QCOW file implementation present in the qcow
crate, we created a new qcow_sync module in block_util that ports this
synchronous implementation to the AsyncIo trait.
The point is to reuse virtio-blk asynchronous implementation for both
synchronous and asynchronous backends.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the synchronous RAW file implementation present in the qcow
crate, we created a new raw_sync module in block_util that ports this
synchronous implementation to the AsyncIo trait.
The point is to reuse virtio-blk asynchronous implementation for both
synchronous and asynchronous backends.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the new DiskFile and AsyncIo traits, the implementation of
asynchronous block support does not have to be tied to io_uring anymore.
Instead, the only thing the virtio-blk implementation knows is that it
is using an asynchronous implementation of the underlying disk file.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Both DiskFile and AsyncIo traits are introduced to allow all kind of
files (RAW, QCOW, VHD) to be able to handle asynchronous access to the
underlying file.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Currently the GED control is in a fixed I/O port address but instead use
an MMIO address that has been chosen by the allocator.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This patch refines the sccomp filter list for the vCPU thread, as we are
no longer spawning virtio-device threads from the vCPU thread.
Fixes: #2170
Signed-off-by: Bo Chen <chen.bo@intel.com>
Even though the driver can provide fewer queues than those advertised
for some device types their is a minimum number that is required for
operation.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
It is permissable for the driver to program fewer queues than offered by
the device. Filter the queues so that only the ready ones are included
and check that they have valid addresses configured.
Fixes: #2136
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Don't assume that the number of queues provided match the number of
queues offered. The virtio spec allows the driver to program fewer
queues.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We should use full memory barrier to ensure both guest and us
can see the correct avail_idx and avail_event_idx. Something
like this pattern:
VM: CLH:
update vring.avail->idx update avail_event_idx
mb() mb()
read avail_event_idx read vring.avail->idx
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
Rather than having to give and return the ioeventfd used for a device
clone them each time. This will make it simpler when we start handling
the driver enabling fewer queues than advertised by the device.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We have killed the device thread by writing to the exit EventFd but we
should wait for them to quit to ensure consistency.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This will lead to the triggering of an ACPI button inside the guest in
order to cleanly shutdown the guest.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Use the ACPI GED device to trigger a notitifcation of type
POWER_BUTTON_CHANGED which will ultimately lead to the guest being
notified.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If a GED event of type 0x8 (for power button) is received notify the
guest OS that this button device has been activated.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Renamed this bitfield as it will also be used for non-hotplug purposes
such as synthesising a power button.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to make the thread naming more useful derive their name from
the device id (which can be supplied by the user) and a device specific
suffix that has details of the individual queue (or queue pair.)
e.g.
rob@artemis:~$ pstree -p -c -l -t `pidof cloud-hypervisor`
cloud-hyperviso(27501)─┬─{_console}(27525)
├─{_disk0_q0}(27529)
├─{_disk0_q1}(27532)
├─{_net1_ctrl}(27533)
├─{_net1_qp0}(27534)
├─{_net1_qp1}(27535)
├─{_rng}(27526)
├─{http-server}(27504)
├─{seccomp_signal_}(27502)
├─{signal_handler}(27523)
├─{vcpu0}(27520)
├─{vcpu1}(27522)
└─{vmm}(27503)
Fixes: #2077
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The Windows image is quite large (about 20GiB), hence it takes some time
to copy it for every test in order to avoid potential corruption.
One way to mitigate that without compromising on safety between each
test is by using device mapper. By creating a read-only base, we ensure
the image won't be modified by any of the tests, and by creating one
snapshot for each test, we avoid copying the entire image each time.
A dedicated Copy On Write disk image is created to handle any change
that might be performed on the base image, letting the tests behave as
expected.
Fixes#2155
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By relying on the Guest object, Windows dedicated tests copy the Windows
guest image before booting from it. The point being to avoid corruption
between multiple tests. This is already how the rest of the integration
tests work, Windows tests were the only ones missing this feature.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The tight coupling of these packages caused issues for dependabot trying
to update them separately so instead combine them together into one
update.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This image does not have the pollinate service which can sometimes fail
and prevent SSH from starting as it marks itself as a prerequisite. This
service will never fully succeed as it tries to make a network
connection which will fail inside our test VMs.
Fixes: #2113
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the removal of most of the custom patches in particular all
the virtio-fs ones have been integrated.
Fixes: #2070
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The ch-5.10.6 branch is rebased on top of Linux stable and includes the
latest virtio-iommu support patches.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Using --net=host is not necessary for any of the integration tests, so
let's use the default network option called "bridge".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Older libc (like RHEL7) uses open() rather than openat(). This was
demonstrated through a failure to open /etc/localtime as used by
gmtime() libc call trigged from the vCPU thread (CMOS device.)
Fixes: #2111
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Because we're back on transient builder, let's download the image
everytime.
This reverts commit b5653d5278.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Some sporadic failures were due to an early connection to the VM while
it was not fully ready. Increasing sleep times fixes these issues.
Fixes#2104
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Sometimes when running under the CI tests fail due to a barrier not
being released and the guest blocks on an MMIO write. Add further
debugging to try and identify the issue.
See: #2118
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Given we already check the connected IP address matches the expected
guest IP address, the check on the "booted" message is not needed.
Fixes: #2117
Signed-off-by: Bo Chen <chen.bo@intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
On aarch64, the openat() syscall was missing from the seccomp filters
list, preventing the test_watchdog from running properly.
Fixes#2103
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Temporarily disable fast failing to try and make progress on CI
stabilisation to allow more test coverage.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This test is very flaky and regularly causing CI failures. Until we can
identify the root cause we should disable this test.
See: #2103
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Simplify our image handling by not copying both QCOW2 and raw images for
every test. Allow the test to choose QCOW2 or raw by specifying the
image name manually. A follow on patch will add explicity QCOW2 tests.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When an SSH command fails we want to be able to see, via a panic() why
and where it failed. Replace use of .unwrap_or_default() from SSH
command calls to ensure that we can see the location of the panic.
Also enhance the existing SSH output code to show the error if there is
one.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Based on the LKML feedback, the devices under /dev/sgx/* are
not justified. SGX RFC v40 moves the SGX device nodes to /dev/sgx_*
and this is reflected in kvm-sgx (next branch) too.
Update cloud-hypervisor code and documentation to follow this.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
This reverts commit 7f1f335950.
This breaks the compilation on Fedora/RedHat machines by requiring
dpkg-deb in the build.rs
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the removal of most of the custom patches in particular all
the virtio-fs ones have been integrated.
Partially fixes: #2070
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the vCPU thread calls log!() the time difference between the call
time and the boot up time is reported. On most environments and
architectures this covered by a vDSO call rather than a syscall. However
on some platforms this turns into a syscall.
Fixes: #2080
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: field assignment outside of initializer for an instance created with Default::default()
Error: --> hypervisor/src/kvm/mod.rs:1239:9
|
1239 | state.mp_state = self.get_mp_state()?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `-D clippy::field-reassign-with-default` implied by `-D warnings`
note: consider initializing the variable with `kvm::aarch64::VcpuKvmState { mp_state: self.get_mp_state()?, ..Default::default() }` and removing relevant reassignments
--> hypervisor/src/kvm/mod.rs:1237:9
|
1237 | let mut state = CpuState::default();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#field_reassign_with_default
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: field assignment outside of initializer for an instance created with Default::default()
--> virtio-devices/src/mem.rs:496:9
|
496 | resp.resp_type = resp_type;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: consider initializing the variable with `mem::VirtioMemResp { resp_type: resp_type, ..Default::default() }` and removing relevant reassignments
--> virtio-devices/src/mem.rs:495:9
|
495 | let mut resp = VirtioMemResp::default();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#field_reassign_with_default
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We must explicitly mark these values as u8 as the function that consumes
them takes a T and needs to use the specific width.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: field assignment outside of initializer for an instance created with Default::default()
--> hypervisor/src/kvm/mod.rs:318:9
|
318 | cap.cap = KVM_CAP_SPLIT_IRQCHIP;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `-D clippy::field-reassign-with-default` implied by `-D warnings`
note: consider initializing the variable with `kvm_bindings::kvm_enable_cap { cap: KVM_CAP_SPLIT_IRQCHIP, ..Default::default() }` and removing relevant reassignments
--> hypervisor/src/kvm/mod.rs:317:9
|
317 | let mut cap: kvm_enable_cap = Default::default();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#field_reassign_with_default
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: this returns a `Result<_, ()>
--> net_util/src/mac.rs:68:5
|
68 | pub fn from_bytes(src: &[u8]) -> Result<MacAddr, ()> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `-D clippy::result-unit-err` implied by `-D warnings`
= help: use a custom Error type instead
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#result_unit_err
Replace with std::io::Error like other locations in the same file.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
With Rust 1.49 using attributes on a function parameter is not allowed.
The recommended workaround is to put it in a new block.
error[E0658]: attributes on expressions are experimental
--> vmm/src/memory_manager.rs:698:17
|
698 | #[cfg(target_arch = "x86_64")]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: see issue #15701 <https://github.com/rust-lang/rust/issues/15701> for more information
error: removing an expression is not supported in this position
--> vmm/src/memory_manager.rs:698:17
|
698 | #[cfg(target_arch = "x86_64")]
|
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The boot time for direct kernel boot based tests is significantly
quicker than booting via the firmware and stock kernel as it triggers a
reboot during the boot process due to the initrd handling.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When doing a direct kernel boot only have console=ttyS0 in the command
line if we are explicitly testing the serial output. The default
behaviour is `--serial null` so this output will not be visible but will
trigger a KVM exit for every byte which is very costly when running
under nested virtualization.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
On the CI we are seeing issues with the activation barriers not being released:
cloud-hypervisor: 12.452434193s: INFO:vmm/src/vm.rs:413 -- Waiting for barrier
cloud-hypervisor: 12.452499794s: INFO:virtio-devices/src/block.rs:382 -- Changing cache mode to writeback
cloud-hypervisor: 12.452605195s: INFO:vmm/src/vm.rs:413 -- Waiting for barrier
cloud-hypervisor: 12.452684596s: INFO:virtio-devices/src/transport/pci_device.rs:671 -- Waiting for barrier
cloud-hypervisor: 12.452708196s: INFO:virtio-devices/src/transport/pci_device.rs:673 -- Barrier released
cloud-hypervisor: 12.452717596s: INFO:vmm/src/vm.rs:415 -- Barrier released
Add some debugging to try and identify the vause of this issue.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Now that we can support supplying an fd to the TAP device we can support
the MACVTAP and MACVLAN use cases.
Fixes: #872
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add an "fd=" parameter to allow specifying a TAP fd to use. Currently
only one fd for one queue pair is supported.
Fixes: #2052
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add support for creating virtio-net device from existing TAP fd.
Currently only a single fd and thus no-more than 2 queues (one pair) is
suppored.
Fixes: #2052
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Ultimately this will allow the creation of a virtio-net device that is
backed by a file descriptor.
This function ensures that the TAP device is correctly setup with
offloading and non-blocking.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
And fix related warnings: op_kind and op_register are being deprecated
as they might panic.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Starting the virtio device threads from the VMM thread has slowed down
the start of the VM when running on a highly contested system like the
CI.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When a device is ready to be activated signal to the VMM thread via an
EventFd that there is a device to be activated. When the VMM receives a
notification on the EventFd that there is a device to be activated
notify the device manager to attempt to activate any devices that have
not been activated.
As a side effect the VMM thread will create the virtio device threads.
Fixes: #1863
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We need to be able to return the barrier from the code that prepares to
activate the virtio device. This triggered by a write to the
configuration fields stored in the PCI BAR. Since bars can be accessed
by both memory mapping and through PCI config I/O several prototypes
must be changed.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This can be uses to indicate to the caller that it should wait on the
barrier before returning as there is some asynchronous activity
triggered by the write which requires the KVM exit to block until it's
completed.
This is useful for having vCPU thread wait for the VMM thread to proceed
to activate the virtio devices.
See #1863
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Currently these two macros(msr, msr_data) reside both on kvm and mshv
module. Definition is same for both module. Moving them to arch/x86
module eliminates redundancy and makes more sense.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
On the CI we are seeing that sometimes the epoll is receiving these
errors which do not indicate a failure but that we should retry.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since we can't test mshv and kvm at the same time, --all-features no
longer work.
We factorize all, non-hypervisor related features into a common set and
mix that with either mshv and kvm.
Co-Developed-by: Samuel Ortiz <sameo@linux.intel.com>
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Co-Developed-by: Wei Liu <liuwe@microsoft.com>
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
We don't have IrqFd and IOEventFd support in the kernel for now.
So an emulation layer is needed. In the future, we will be adding this
support in the kernel.
Co-Developed-by: Wei Liu <liuwe@microsoft.com>
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
vmmops trait object is needed to get access some
of the upper level vmm functionalities i.e guest
memory access, IO read write etc.
Co-Developed-by: Praveen Paladugu <prapal@microsoft.com>
Signed-off-by: Praveen Paladugu <prapal@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Adding hv_state (hyperv state) to Vm and Vcpu struct for mshv.
This state is needed to keep some kernel data(for now hypercall page)
in the vmm.
Co-Developed-by: Praveen Paladugu <prapal@microsoft.com>
Signed-off-by: Praveen Paladugu <prapal@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This is the initial folder structure of the mshv module inside
the hypervisor crate. The aim of this module is to support Microsoft
Hyper-V as a supported Hypervisor.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
There are some code base and function which are purely KVM specific for
now and we don't have those supports in mshv at the moment but we have plan
for the future. We are doing a feature guard with KVM. For example, KVM has
mp_state, cpu clock support, which we don't have for mshv. In order to build
those code we are making the code base for KVM specific compilation.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The customized hashmap macro can't be lifted to common MockVMM code.
MockVMM only needs a collection to iterate over to get initial register
states. A vector is just as good as a hashmap.
Switch to use a vector to store initial register states. This allows us
to drop the hashmap macro everywhere.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Unfortunately Rust stable does not yet have inline ASM support the flag
calculation will have to be implemented in software.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
nightly / beta are not required actions but if they fail then all other
builds will be cancelled and so prevent the PR from being cleared as
buildable.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Unfortunately it seems patch entries are ignored when obtaining
dependencies from another workspace.
Remove the problematic kvm-ioctls and kvm-bindings patch entries and use
the forked repository directly.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When using an PIO write to 0x80 which is a special case handle that and
then return without going through the resolve.
This removes an extra warning that is reported.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
There is no need to have a pair of curly brackets for structures without
any member.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The mapping between code and its handler is static. We can drop the
HashMap in favour of a static match expression.
This has two benefits:
1. No more memory allocation and deallocation for the HashMap.
2. Shorter look-up time.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When a total ordering between multiple atomic variables is not required
then use Ordering::Acquire with atomic loads and Ordering::Release with
atomic stores.
This will improve performance as this does not require a memory fence
on x86_64 which Ordering::SeqCst will use.
Add a comment to the code in the vCPU handling code where it operates on
multiple atomics to explain why Ordering::SeqCst is required.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The DeviceNode cannot be fully represented as it embeds a Rust style
enum (i.e. with data) which is instead represented by a simple
associative array.
Fixes: #1167
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When the x86 instruction decoder tells us about some missing bytes from
the instruction stream, we call into the platform fetch method and
emulate one last instruction.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
In preparation for the instruction fetching step, we modify the decoding
loop so that we can check what the last decoding error is.
We also switch to explictly using decode_out() which removes a 32 bytes
copy compared to decode().
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
In order to validate emulated memory accesses, we need to be able to get
all the segments descriptor attributes.
This is done by abstracting the SegmentRegister attributes through a
trait that each hypervisor will have to implement.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We need to be able to build the CPU mode from its state in order to
start implementing mode related checks in the x86 emulator.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
If we rely on timeouts at the top level we can get builds being aborted
simply because they took too long to be scheduled rather than because
the actual integration tests took too long.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The MockVMM platform will be used by other instructions emulation
implementations, but also by the emulator framework.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The observation here is PlatformEmulator can be seen as the context for
emulation to take place. It should be rather easy to construct a context
that satisfies the lifetime constraints for instruction emulation.
The thread doing the emulation will have full ownership over the
context, so this removes the need to wrap PlatformEmulator in Arc and
Mutex, as well as the need for the context to be either Clone or Copy.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The emulator gets a CPU state from a CpuStateManager instance, emulates
the passed instructions stream and returns the modified CPU state.
The emulator is a skeleton for now since it comes with an empty
instruction mnemonic map.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
And an InstructionMap helper structure to map x86 mnemonic codes
to instruction handlers.
Any instruction emulation implementation should then boil down with
implementing InstructionHandler for any supported mnemonic.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Minimal will be defined by the amount of emulated instructions.
Carrying all GPRs, all CRs, segment registers and table registers should
cover quite a few instructions.
Co-developed-by: Wei Liu <liuwe@microsoft.com>
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
For efficiently emulating x86 instructions, we need to build and pass a
CPU state copy/reference to instruction emulation handlers. Those handlers
will typically modify the CPU state and let the caller commit those
changes back through the PlatformEmulator trait set_cpu_state method.
Hypervisors typically have internal CPU state structures, that maps back
to the correspinding kernel APIs. By implementing the CpuState trait,
instruction emulators will be able to directly work on CPU state
instances that are directly consumable by the underlying hypervisor and
its kernel APIs.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
In order to emulate instructions, we need a way to get access to some of
the guest resources. The PlatformEmulator interface provides guest
memory and CPU state access to emulator implementations.
Typically, an hypervisor will implement PlatformEmulator for architecture
specific instruction emulators to build their framework on top of.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
We will need the GDT API for the hypervisor's x86 instruction
emulator implementation, it's better if the arch crate depends on the
hypervisor one rather than the other way around.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The configuration is stored separately to the Vm in the VMM. The failure
to store the config was preventing the VM from shutting down correctly
as Vmm::vm_delete() checks for the presence of the config.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Rather return the None to the caller to handle instead. This removes the
source of a potential panic.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The live migration support added use of this ioctl but it wasn't
included in the permitted list.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This interface is used by the vCPU thread to delegate responsibility for
handling MMIO/PIO operations and to support different approaches than a
VM exit.
During profiling I found that we were spending 13.75% of the boot CPU
uage acquiring access to the object holding the VmmOps via
ArcSwap::load_full()
13.75% 6.02% vcpu0 cloud-hypervisor [.] arc_swap::ArcSwapAny<T,S>::load_full
|
---arc_swap::ArcSwapAny<T,S>::load_full
|
--13.43%--<hypervisor::kvm::KvmVcpu as hypervisor::cpu::Vcpu>::run
std::sys_common::backtrace::__rust_begin_short_backtrace
core::ops::function::FnOnce::call_once{{vtable-shim}}
std::sys::unix::thread::Thread::new::thread_start
However since the object implementing VmmOps does not need to be mutable
and it is only used from the vCPU side we can change the ownership to
being a simple Arc<> that is passed in when calling create_vcpu().
This completely removes the above CPU usage from subsequent profiles.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
As we switched to focal for this test we no longer get any output during
the boot unless serial is used over virtio-console.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
There have been a lot of flakes around tests such as
test_virtio_fs_hotplug_dax_on_w_vhost_user_fs_daemon() or
test_virtio_fs_hotplug_dax_on() which all try and hotplug memory.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add config parameter to --disk called "_disable_io_uring" (the
underscore prefix indicating it is not for public consumpion.) Use this
option to disable io_uring if it would otherwise be used.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
With the removal of vhost-user self-spawning support we should migrate
the tests to use the binaries so that we can remove the functionality
from the cloud-hypervisor binary itself.
See: #1925
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Now the VM is paused/resumed by the migration process itself.
0. The guest configuration is sent to the destination
1. Dirty page log tracking is started by start_memory_dirty_log()
2. All guest memory is sent to the destination
3. Up to 5 attempts are made to send the dirty guest memory to the
destination...
4. ...before the VM is paused
5. One last set of dirty pages is sent to the destination
6. The guest is snapshotted and sent to the destination
7. When the migration is completed the destination unpauses the received
VM.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows code running in the VMM to access the VM's MemoryManager's
functionality for managing the dirty log including resetting it but also
generating a table.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Prior to sending the memory the full state is not needed only the
configuration. This is sufficient to create the appropriate structures
in the guest and have the memory allocations ready for filling.
Update the protocol documentation to add a separate config step and move
the state to after the memory is transferred. As the VM is created in a
separate step to restoring it the requires a slightly different
constructor as well as saving the VM object for the subsequent commands.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to do this we must extend the MemoryManager API to add the
ability to specify the tracking of the dirty pages when creating the
userspace mappings and also keep track of the userspace mappings that
have been created for RAM regions.
Currently the dirty pages are collected into ranges based on a block
level of 64 pages. The algorithm could be tweaked to create smaller
ranges but for now if any page in the block of 64 is dirty the whole
block is added to the range.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
While the addressable space size reduction of 4k in necessary due to
the Linux bug, the 64k alignment of the addressable space size is
required by Windows. This patch satisfies both.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Cloning the ArcSwapOption (like the ArcSwap) does not act like a
.clone() on an Arc, instead an entirely new ArcSwap is created with the
same contents. To correctly share the ArcSwap needs to be placed inside
an Arc.
See: 2433d5719b (diff-6c6d94533c44c19bd1416ef17bad1a878e63dca6e98d59181228fbe8f967c62bR6)
Due to this being wrongly used ::clone() was removed from
ArcSwap/ArcSwapOption in 1.0.0.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This is tested by:
Source VMM:
target/debug/cloud-hypervisor --kernel ~/src/linux/vmlinux \
--pmem file=~/workloads/focal.raw --cpus boot=1 \
--memory size=2048M \
--cmdline"root=/dev/pmem0p1 console=ttyS0" --serial tty --console off \
--api-socket=/tmp/api1 -v
Destination VMM:
target/debug/cloud-hypervisor --api-socket=/tmp/api2 -v
And the following commands:
target/debug/ch-remote --api-socket=/tmp/api1 pause
target/debug/ch-remote --api-socket=/tmp/api2 receive-migration unix:/tmp/foo &
target/debug/ch-remote --api-socket=/tmp/api1 send-migration unix:/tmp/foo
target/debug/ch-remote --api-socket=/tmp/api2 resume
The VM is then responsive on the destination VMM.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the code to be reused when creating the VM from a snapshot
when doing VM migration.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add API entry points with stub implementation for sending and receiving
a VM from one VMM to another.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Due to a known limitation in OpenAPITools/openapi-generator tool,
it's impossible to send go zero types, like false and 0 to
cloud-hypervisor because `omitempty` is added if a field is not
required.
Set cache_size, dax, num_queues and queue_size as required to remove
`omitempty` from the json tag.
fixes#1961
Signed-off-by: Julio Montes <julio.montes@intel.com>
This also removes the need to lookup up the "exe" symlink for finding
the VMM executable path.
Fixes: #1925
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The integration script failed to obtain the new guest kernel commit when
the code folder had been existing on CI machine.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
The logic to handle AArch64 system event was: SHUTDOWN and RESET were
all treated as RESET.
Now we handle them differently:
- RESET event will trigger Vmm::vm_reboot(),
- SHUTDOWN event will trigger Vmm::vm_shutdown().
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Now Vcpu::run() returns a boolean value to VcpuManager, indicating
whether the VM is going to reboot (false) or just continue (true).
Moving the handling of hypervisor VCPU run result from Vcpu to
VcpuManager gives us the flexibility to handle more scenarios like
shutting down on AArch64.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Set the test case test_snapshot_restore X86 only, instead of excluding
it from test command line.
The command line option was added because we used to support migration
with Virtio-MMIO, but not Virtio-PCI.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Rather than filling the guest memory from a file at the point of the the
guest memory region being created instead fill from the file later. This
simplifies the region creation code but also adds flexibility for
sourcing the guest memory from a source other than an on disk file.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-30 12:31:47 +01:00
263 changed files with 36216 additions and 27187 deletions
- [1. What is Cloud Hypervisor?](#1-what-is-cloud-hypervisor)
- [Objectives](#objectives)
- [High Level](#high-level)
- [Architectures](#architectures)
- [Guest OS](#guest-os)
- [2. Getting Started](#2-getting-started)
- [Install prerequisites](#install-prerequisites)
- [Clone and build](#clone-and-build)
- [Containerized builds and tests](#containerized-builds-and-tests)
- [Run](#run)
@@ -23,10 +22,12 @@
- [5. Community](#5-community)
- [Contribute](#contribute)
- [Join us](#join-us)
- [Security issues](#security-issues)
# 1. What is Cloud Hypervisor?
Cloud Hypervisor is an open source Virtual Machine Monitor (VMM) that runs on top of [KVM](https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt).
Cloud Hypervisor is an open source Virtual Machine Monitor (VMM) that runs on top of [KVM](https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt) and the MSHV hypervisors .
The project focuses on exclusively running modern, cloud workloads, on top of a limited set of hardware architectures and platforms.
Cloud workloads refers to those that are usually run by customers inside a cloud provider. For our purposes this means modern operating systems with most I/O handled by paravirtualised devices (i.e. virtio), no requirement for legacy devices, and 64-bit CPUs.
@@ -36,7 +37,7 @@ Cloud Hypervisor is implemented in [Rust](https://www.rust-lang.org/) and is bas
### High Level
-KVM based
-Runs on KVM or MSHV
- Minimal emulation
- Low latency
- Low memory footprint
@@ -53,7 +54,7 @@ Cloud Hypervisor supports the `x86-64` and `AArch64` architectures. There are so
### Guest OS
Cloud Hypervisor supports `64-bit Linux`with support for _modern_ 64-bit Windows guests currently under development.
Cloud Hypervisor supports `64-bit Linux`and Windows 10/Windows Server 2019.
# 2. Getting Started
@@ -148,7 +149,7 @@ We need to get the latest `rust-hypervisor-firmware` release and also a working
@@ -171,7 +172,7 @@ Multiple arguments can be given to the `--disk` parameter.
#### Building your kernel
Cloud Hypervisor also supports direct kernel boot into a `vmlinux` ELF kernel or `bzImage`. In order to support virtio-fs and virtio-iommu we have our own development branch. You are of course able to use your own kernel but these instructions will continue with the version that we develop and test against.
Cloud Hypervisor also supports direct kernel boot into a `vmlinux` ELF kernel. In order to support virtio-iommu we have our own development branch. You are of course able to use your own kernel but these instructions will continue with the version that we develop and test against.
Direct kernel boot to userspace should work with a rootfs from most distributions.
## Hot Plug
Cloud Hypervisor supports hotplug of CPUs, passthrough devices (VFIO), `virtio-{net,block,pmem,fs,vsock}` and memory resizing. This [document](https://github.com/cloud-hypervisor/cloud-hypervisor/blob/master/docs/hotplug.md) details how to add devices to
Cloud Hypervisor supports hotplug of CPUs, passthrough devices (VFIO), `virtio-{net,block,pmem,fs,vsock}` and memory resizing. This [document](docs/hotplug.md) details how to add devices to
a running VM.
## Device Model
Details of the device model can be found in this [documentation](https://github.com/cloud-hypervisor/cloud-hypervisor/blob/master/docs/device_model.md).
Details of the device model can be found in this [documentation](docs/device_model.md).
## TODO
@@ -317,3 +335,7 @@ etc, are all equal and welcome means of contribution. See the [CONTRIBUTING](CON
Get an [invite to our Slack channel](https://join.slack.com/t/cloud-hypervisor/shared_invite/enQtNjY3MTE3MDkwNDQ4LWQ1MTA1ZDVmODkwMWQ1MTRhYzk4ZGNlN2UwNTI3ZmFlODU0OTcwOWZjMTkwZDExYWE3YjFmNzgzY2FmNDAyMjI)
and [join us on Slack](https://cloud-hypervisor.slack.com/).
## Security issues
Please use the GitHub security advisories feature for reporting issues: https://github.com/cloud-hypervisor/cloud-hypervisor/security/advisories/new
# How to build and run Cloud-hypervisor on AArch64
# How to build and test Cloud Hypervisor on AArch64
Cloud-hypervisor is partially enabled on AArch64 architecture.
Although all features are not ready yet, you can begin to test Cloud-hypervisor on a AArch64 host by following this guide.
This document introduces how to build and test Cloud Hypervisor on AArch64 servers. Currently Cloud Hypervisor cannot be tested on Raspberry PI. Because on AArch64, Cloud Hypervisor requires GICv3-ITS device for PCIe MSI interrupt handling. But GICv3-ITS has not been equipped on any Raspberry PI product so far.
Now Cloud Hypervisor supports 2 ways of booting on AArch64: UEFI booting and direct-kernel booting. The document covers both of the ways.
All the steps are based on Ubuntu. We use the Ubuntu cloud image for guest VM disk.
## Getting started
We create a folder to build and run Cloud Hypervisor at `$HOME/cloud-hypervisor`
```shell
$ exportCLOUDH=$HOME/cloud-hypervisor
$ mkdir $CLOUDH
```
## Prerequisites
On AArch64 machines, Cloud-hypervisor depends on an external library `libfdt-dev` for generating Flattened Device Tree (FDT).
You need to install some prerequisite packages to build and test Cloud Hypervisor.
The long-term plan is to replace `libfdt-dev` with some pure-Rust component to get rid of such dependency.
### Tools
```bash
sudo apt-get update
sudo apt-get install libfdt-dev
# Install rust tool chain
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install the tools used for building guest kernel, EDK2 and converting guest disk
$ qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-arm64.img focal-server-cloudimg-arm64.raw
$ popd
```
## Containerized build
## UEFI booting
If you want to build and test Cloud Hypervisor without having to install all the required dependencies, you can also turn to the development script: dev_cli.sh.
This part introduces how to build EDK2 firmware and boot Cloud Hypervisor with it.
@@ -35,45 +35,42 @@ The `cache=none` option should be the default when using `virtiofsd` with the __
The `cache=always` option will allow for the guest page cache to be used, which will increase the memory footprint of the guest. This option should be used only for specific use cases where a single VM is going to be running on a host.
### The kernel
### Kernel support
In order to leverage __virtio-fs__ support from within the guest, and because the code has not been merged in upstream Linux kernel yet, it is required to build a custom kernel embedding the patches.
The following branch `virtio-fs-virtio-iommu` on the repository https://github.com/cloud-hypervisor/linux.git includes all the needed patches to support __virtio-fs__.
Make sure to build a kernel out of this branch that can be then used to boot the VM.
Modern Linux kernels starting (at least v5.10) have support for virtio-fs. Use
of older kernels, with additional patches, are not supported.
## How to share directories with cloud-hypervisor
### Start the VM
Once the daemon is running, the option `--fs` from __cloud-hypervisor__ needs to be used.
Direct kernel boot option is preferred since we need to provide the custom kernel including the __virtio-fs__ patches. We could boot from `hypervisor-fw` if we had previously edited the image to replace the kernel binary.
Direct kernel boot is the preferred option, but we can boot from an EFI cloud image if it contains a recent enough kernel.
Because _vhost-user_ expects a dedicated process (__virtiofsd__ in this case) to be able to access the guest RAM to communicate through the _virtqueues_ with the driver running in the guest, `--memory` option needs to be slightly modified. It needs to specify a backing file for the memory so that an external process can access it.
Because _vhost-user_ expects a dedicated process (__virtiofsd__ in this case) to be able to access the guest RAM to communicate through the _virtqueues_ with the driver running in the guest, `--memory` option needs to be slightly modified. It must specify `shared=on` to share the memory pages so that an external process can access them.
Assuming you have `focal-server-cloudimg-amd64.raw` and `custom-vmlinux.bin` on your system, here is the __cloud-hypervisor__ command you need to run:
Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your system, here is the __cloud-hypervisor__ command you need to run:
By default, DAX is enabled with a cache window of 8GiB. You can specify a custom size (let's say 4GiB for this example) for the cache by explicitly setting DAX and the cache size:
In case you don't want to use a shared window of cache to pass the shared files content, this means you will have to explicitly disable DAX with `dax=off`. Note that in this case, the `cache_size` parameter will be ignored.
The extra vCPU threads will be created and advertised to the running kernel. The kernel does not bring up the CPUs immediately and instead the user must "online" them from inside the VM:
@@ -56,19 +56,23 @@ After a reboot the added CPUs will remain.
Removing CPUs works similarly by reducing the number in the "desired_vcpus" field of the reisze API. The CPUs will be automatically offlined inside the guest so there is no need to run any commands inside the guest:
As per adding CPUs to the guest, after a reboot the VM will be running with the reduced number of vCPUs.
## Memory Hot Plug
### ACPI method
Extra memory can be added from a running Cloud Hypervisor instance. This is controlled by two mechanisms:
1. Allocating some of the guest physical address space for hotplug memory.
2. Making a HTTP API request to the VMM to ask for a new amount of RAM to be assigned to the VM. In the case of expanding the memory for the VM the new memory will be hotplugged into the running VM, if reducing the size of the memory then change will take effect after the next reboot.
To use memory hotplug start the VM specifying some size RAM in the "hotplug_size" parameter to the memory configuration. Not all the memory specified in this parameter will be available to hotplug as there are spacing and alignment requirements so it is recommended to make it larger than the hotplug RAM needed.
To use memory hotplug start the VM specifying some size RAM in the `hotplug_size` parameter to the memory configuration. Not all the memory specified in this parameter will be available to hotplug as there are spacing and alignment requirements so it is recommended to make it larger than the hotplug RAM needed.
Because the ACPI method is the default, there is no need to add the extra option `hotplug_method=acpi`.
The new memory is now available to use inside the VM:
@@ -111,3 +115,134 @@ Due to guest OS limitations is is necessary to ensure that amount of memory adde
The same API can also be used to reduce the desired RAM for a VM but the change will not be applied until the VM is rebooted.
Memory and CPU resizing can be combined together into the same HTTP API request.
### virtio-mem method
Extra memory can be added and removed from a running Cloud Hypervisor instance. This is controlled by two mechanisms:
1. Allocating some of the guest physical address space for hotplug memory.
2. Making a HTTP API request to the VMM to ask for a new amount of RAM to be assigned to the VM.
To use memory hotplug start the VM specifying some size RAM in the `hotplug_size` parameter along with `hotplug_method=virtio-mem` to the memory configuration.
The new memory is now available to use inside the VM:
```shell
free -h
total used free shared buff/cache available
Mem: 3.0Gi 71Mi 2.8Gi 0.0Ki 47Mi 2.8Gi
Swap: 32Mi 0B 32Mi
```
The same API can also be used to reduce the desired RAM for a VM. It is important to note that reducing RAM size might only partially work, as the guest might be using some of it.
## PCI Device Hot Plug
Extra PCI devices can be added and removed from a running Cloud Hypervisor instance. This is controlled by making a HTTP API request to the VMM to ask for the additional device to be added, or for the existing device to be removed.
To use PCI device hotplug start the VM with the HTTP server.
00:01.0 Unassigned class [ffff]: Red Hat, Inc. Virtio console (rev 01)
00:02.0 Mass storage controller: Red Hat, Inc. Virtio block device (rev 01)
00:03.0 Unassigned class [ffff]: Red Hat, Inc. Virtio RNG (rev 01)
```
After a reboot the added PCI device will remain.
### Remove PCI device
Removing a PCI device works the same way for all kind of PCI devices. The unique identifier related to the device must be provided. This identifier can be provided by the user when adding the new device, or by default Cloud Hypervisor will assign one.
Cloud Hypervisor supports using a MACVTAP device which is derived from a MACVLAN. Full details of configuring MACVLAN or MACVTAP is out of scope of this document. However the example below indicates how to bridge the guest directly onto the network the host is on. Due to the lack of hairpin mode it not usually possible to reach the guest directly from the host.
```bash
# The MAC address must be attached to the macvtap and be used inside the guest
mac="c2:67:4f:53:29:cb"
# Host network adapter to bridge the guest onto
host_net="eno1"
# Create the macvtap0 as a new virtual MAC associated with the host network
sudo ip link add link "$host_net" name macvtap0 type macvtap
sudo ip link set macvtap0 address "$mac" up
sudo ip link show macvtap0
# A new character device is created for this interface
tapindex=$(< /sys/class/net/macvtap0/ifindex)
tapdevice="/dev/tap$tapindex"
# Ensure that we can access this device
sudo chown "$UID.$UID""$tapdevice"
# Use --net fd=3 to point to fd 3 which the shell has opened to point to the /dev/tapN device
target/debug/cloud-hypervisor \
--kernel ~/src/linux/vmlinux \
--disk path=~/workloads/focal.raw \
--cpus boot=1 --memory size=512M \
--cmdline "root=/dev/vda1 console=hvc0"\
--net fd=3,mac=$mac 3<>$"$tapdevice"
```
As the guest is now connected to the same L2 network as the host you can obtain an IP address based on your host network (potentially including via DHCP)
Specifies if the memory must be `mmap(2)` with `MAP_HUGETLB` and `MAP_HUGE_2MB`
flags. This performs a memory mapping relying on 2MiB pages instead of the
default 4kiB pages.
Specifies if the memory must be created and `mmap(2)` with `MAP_HUGETLB` and size
flags. This performs a memory mapping relying on the specified huge page size. If no huge page size is supplied the system's default huge page size is used.
By using hugepages, one can improve the overall performance of the VM, assuming
the guest will allocate hugepages as well. Another interesting use case is VFIO
as it speeds up the VM's boot time since the amount of IOMMU mappings are
reduced.
The user is responsible for ensuring there are sufficient huge pages of the specified size for the VMM to use.
Failure to do so may result in strange VMM behaviour, e.g. error with `ReadKernelImage` is common.
If there is a strange error with `hugepages` enabled, just disable it or check whether there are enough huge pages.
By default this option is turned off.
_Example_
```
--memory size=1G,hugepages=on
--memory size=1G,hugepages=on,hugepage_size=2M
```
### `hotplug_method`
@@ -341,15 +345,16 @@ it allows for specifying the distance between each NUMA node.
```rust
structNumaConfig{
id: u32,
guest_numa_id: u32,
cpus: Option<Vec<u8>>,
distances: Option<Vec<NumaDistance>>,
memory_zones: Option<Vec<String>>,
sgx_epc_sections: Option<Vec<String>>,
}
```
```
--numa <numa> Settings related to a given NUMA node "id=<node_id>,cpus=<cpus_id>,distances=<list_of_distances_to_destination_nodes>,memory_zones=<list_of_memory_zones>"
--numa <numa> Settings related to a given NUMA node "guest_numa_id=<node_id>,cpus=<cpus_id>,distances=<list_of_distances_to_destination_nodes>,memory_zones=<list_of_memory_zones>,sgx_epc_sections=<list_of_sgx_epc_sections>"
cloud-hypervisor can emulate one or more virtual network interfaces, represented at the hypervisor host by [tap devices](https://www.kernel.org/doc/Documentation/networking/tuntap.txt). This guide briefly describes, in a manual and distribution neutral way, how to setup and use networking with cloud-hypervisor.
## Multiple queue support for net devices ##
## Multiple queue support for net devices
While multiple vcpus defined for guest, to gain the benefit of vcpu scalable to improve performance, it suggests to define multiple queue pairs for net devices, one Tx/Rx queue pair per one vcpu, that means the number of queue pairs at least is equal to the vcpu count. In that case, after virtnet driver set cpu affinity for virtqueues in guest kernel, vcpus could handle interrupt from different virtqueue pairs in parallel.
@@ -23,7 +23,7 @@ Use one `--net` command-line argument from cloud-hypervisor to specify the emula
```bash
./cloud-hypervisor \
--cpus 4\
--cpus boot=4\
--memory "size=512M"\
--disk path=focal-server-cloudimg-amd64.raw \
--kernel my-vmlinux.bin \
@@ -34,14 +34,14 @@ Use one `--net` command-line argument from cloud-hypervisor to specify the emula
The `--net` argument takes 1 or more space-separated strings of key value pairs containing the following 4 keys or fields:
num_queues is the total number of tx and rx queues, the default value is 2, and it could be increased by multiples of 2. Additionally, num_queues is suggested to be as 2 times of vcpu count. The default value for queue_size is 256.
@@ -104,6 +104,7 @@ bridge name bridge id STP enabled interfaces
ich-dpl 8000.067afc1b9a67 no ich1
ich-int 8000.725412ffce6f no ich0
```
This completes the layer 2 wiring: The cloud-hypervisor is now connected to the hypervisor host via the 2 linux bridges.
## IP (Layer 3) provisioning
@@ -116,6 +117,7 @@ On the hypervisor host add the network gateway IP address of each network to the
root@host:~# ip addr add 192.168.4.1/24 dev ich-int
root@host:~# ip addr add 10.0.1.1/24 dev ich-dpl
```
The routing table of the hypervisor host should now also have corresponding routing entries:
Within the virtual machine set the vNIC's to up state and provision the corresponding IP addresses on the 2 vNIC's. The steps outlined below use the ip command as an example. Alternative distribution specific procedures can also apply.
Within the virtual machine set the vNIC's to up state and provision the corresponding IP addresses on the 2 vNIC's. The steps outlined below use the ip command as an example. Alternative distribution specific procedures can also apply.
```bash
root@guest:~# ip link set up enp0s2
@@ -165,7 +168,7 @@ root@192.168.4.2's password:
Linux cloud-hypervisor 5.2.0 #2 SMP Thu Jul 11 08:08:16 CEST 2019 x86_64
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
permitted by applicable law.
Last login: Fri Jul 12 13:27:56 2019 from 192.168.4.1
@@ -189,7 +192,7 @@ nameserver 192.168.178.1
make sure that the default gateway of the hypervisor host (in this example host 192.168.178.1 which is an adsl router) has an entry in the routing table for the 192.168.4.0/24 network otherwise IP connectivity will not work.
`perf` can be used to profile the `cloud-hypervisor` binary but it is necessary to make some modifications to the build in order to produce a binary that gives useful results.
## Building a suitable binary
Modify the `Cargo.toml` file to add `debug = 1` to the `[profile.release]` block. It should look like this:
```
[profile.release]
lto = true
debug = 1
```
This adds the symbol information to the release binary but does not otherwise affect the performance.
The binary must also be built with frame pointers included so that the call graph can be captured by the profiler.
$ perf record -g target/release/cloud-hypervisor \
--kernel ~/src/linux/vmlinux \
--pmem file=~/workloads/focal.raw \
--cpus boot=1 --memory size=1G \
--cmdline "root=/dev/pmem0p1 console=ttyS0" \
--serial tty --console off \
--api-socket=/tmp/api1
```
For analysing the samples:
```
$ perf report -g
```
If profiling with a network device attached either the TAP device must be already created and configured or the profiling must be done as root so that the TAP device can be created.
@@ -35,8 +35,42 @@ To make Cloud Hypervisor use UEFI boot, pass the `OVMF.fd` file path as an argum
The same firmware can be used with Cloud Hypervisor or with QEMU. This is particularly useful if using QEMU for the preparation phase.
## Building UEFI Firmware with Compatibility Support Module (CSM)
CSM is a module that allows to boot legacy operating systems using the OVMF firmware. OVMF can embed a CSM build of SeaBIOS. To build the SeaBIOS with CSM support, add `CONFIG_CSM=y` to `.config` before the build. The outcome `out/Csm16.bin` is to be moved into `OvmfPkg/Csm/Csm16/Csm16.bin` before OVMF is built. Then, the OVMF build will have to be passed the `-D CSM_ENABLE` option in order to generate a legacy aware UEFI firmware. At the current stage, all the necessary patches are included in the Cloud Hypervisor specific [SeaBIOS branch](https://github.com/cloud-hypervisor/seabios/tree/ch). Taking into account the previous instructions, the modified command sequence to compile an OVMF binary with CSM support is the following one:
Please note, that the CSM support has currently only been tested with Linux guests. There are no plans to provide legacy support for other OSes (e.g. Windows).
VFIO-user is an *experimental* protocol for allowing devices to be implemented in another process and communicate over a socket; ie.e VFIO-user is to VFIO as virtio is to vhost-user.
The protocol is documented here: https://github.com/nutanix/libvfio-user/blob/master/docs/vfio-user.rst
The Cloud Hypervisor support for such devices is *experimental*. Not all Cloud Hypervisor functionality is supported in particular: virtio-mem and iommu are not supported.
## Usage
The `--user-device socket=<path>` parameter is used to create a vfio-user device when creating the VM specifying the socket to connect to. The device can also be hotplugged with `ch-remote add-user-device socket=<path>`.
## Example (GPIO device)
There is a simple GPIO device included in the libvfio-user repository: https://github.com/nutanix/libvfio-user#gpio
The test runs with multiple queue (MQ) support enabled, using 2 pairs of TX/RX queues defined for both OVS and the virtual machine. Here are the details on how the test can be run.
The test runs with multiple queue (MQ) support enabled, using 2 pairs of
TX/RX queues defined for both OVS and the virtual machine. Here are the
detailed instructions.
_Setup OVS_
`ovs_test.sh` is created to setup and start OVS. OVS will provide the `dpdkvhostuser` backend running in server mode.
Here is an example how to configure a basic OpenVswitch using DPDK:
This allows for SSH login from a remote machine, for example through the `administrator` user: `ssh administrator@192.168.249.2`. For a more detailed OpenSSH guide, please follow the MSDN article from the [links](#links) section.
## Hotplug capability
CPU hotplug is supported. The VM operating system needs to support hotplug and be appropriately licensed. SKU limitations like constraints on the number of cores are to be taken into consideration. Note, that Windows doesn't support CPU hot-remove. When `ch-remote` is invoked to reduce the number of CPUs, the result will be visible after the OS reboot within the same hypervisor instance.
RAM hotplug is supported. Note, that while the `pnpmem.sys` driver in use supports RAM hot-remove, the RAM being unplugged has to be not in use and have no reserved pages. In most cases it means, hot-remove won't work. Same as with the CPU hot-remove, when `ch-remote` is invoked to reduce the RAM size, the result will be visible after the OS reboot.
Network device hotplug and hot-remove are supported.
Disk hotplug and hot-remove are supported. After the device has been hotplugged, it will need to be onlined from within the guest. Among other tools, powershell applets `Get-Disk` and `Set-Disk` can be used for the disk configuration and activation.
## Debugging
The Windows guest debugging process relies heavily on QEMU and [socat](http://www.dest-unreach.org/socat/). The procedure requires two Windows VMs:
@@ -282,6 +292,17 @@ Note, this VM has the networking enabled. It is needed, because symbols and sour
Also, notice the `-serial` parameter - that's what does the magic on exposing the serial port to the guest while connecting the debugger VM with a client VM through the network. SAC/EMS needs to be disabled in the debugger VM, as otherwise the COM device might be blocked.
Hereafter, WinDbg can be started using a command below:
```cmd
set_NT_DEBUG_PORT=com1
set_NT_DEBUG_BAUD_RATE=115200
windbg -v -d -k
```
Once started, WinDbg will wait for an incoming connection which is going to be initialized by the debuggee VM started in the next section.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
