This is required when hot-removing a vfio-user device. Details code path
below:
Thread 6 "vcpu0" received signal SIGSYS, Bad system call.
[Switching to Thread 0x7f8196889700 (LWP 2358305)]
0x00007f8196dae7ab in shutdown () at ../sysdeps/unix/syscall-template.S:78
78 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
(gdb) bt
0x00007f8196dae7ab in shutdown () at ../sysdeps/unix/syscall-template.S:78
0x000056189240737d in std::sys::unix::net::Socket::shutdown ()
at library/std/src/sys/unix/net.rs:383
std::os::unix::net::stream::UnixStream::shutdown () at library/std/src/os/unix/net/stream.rs:479
0x000056189210e23d in vfio_user::Client::shutdown (self=0x7f8190014300)
at vfio_user/src/lib.rs:787
0x00005618920b9d02 in <pci::vfio_user::VfioUserPciDevice as core::ops::drop::Drop>::drop (
self=0x7f819002d7c0) at pci/src/vfio_user.rs:551
0x00005618920b8787 in core::ptr::drop_in_place<pci::vfio_user::VfioUserPciDevice> ()
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x00005618920b92e3 in core::ptr::drop_in_place<core::cell::UnsafeCell<dyn pci::device::PciDevice>>
() at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x00005618920b9362 in core::ptr::drop_in_place<std::sync::mutex::Mutex<dyn pci::device::PciDevice>> () at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x00005618920d8a3e in alloc::sync::Arc<T>::drop_slow (self=0x7f81968852b8)
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/sync.rs:1092
0x00005618920ba273 in <alloc::sync::Arc<T> as core::ops::drop::Drop>::drop (self=0x7f81968852b8)
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/sync.rs:1688
0x00005618920b76fb in core::ptr::drop_in_place<alloc::sync::Arc<std::sync::mutex::Mutex<dyn pci::device::PciDevice>>> ()
at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ptr/mod.rs:188
0x0000561891b5e47d in vmm::device_manager::DeviceManager::eject_device (self=0x7f8190009600,
pci_segment_id=0, device_id=3) at vmm/src/device_manager.rs:4000
0x0000561891b674bc in <vmm::device_manager::DeviceManager as vm_device::bus::BusDevice>::write (
self=0x7f8190009600, base=70368744108032, offset=8, data=&[u8](size=4) = {...})
at vmm/src/device_manager.rs:4625
0x00005618921927d5 in vm_device::bus::Bus::write (self=0x7f8190006e00, addr=70368744108040,
data=&[u8](size=4) = {...}) at vm-device/src/bus.rs:235
0x0000561891b72e10 in <vmm::vm::VmOps as hypervisor::vm::VmmOps>::mmio_write (
self=0x7f81900097b0, gpa=70368744108040, data=&[u8](size=4) = {...}) at vmm/src/vm.rs:378
0x0000561892133ae2 in <hypervisor::kvm::KvmVcpu as hypervisor::cpu::Vcpu>::run (
self=0x7f8190013c90) at hypervisor/src/kvm/mod.rs:1114
0x0000561891914e85 in vmm::cpu::Vcpu::run (self=0x7f819001b230) at vmm/src/cpu.rs:348
0x000056189189f2cb in vmm::cpu::CpuManager::start_vcpu::{{closure}}::{{closure}} ()
at vmm/src/cpu.rs:953
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 42c19e14c5)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
All hotpluggable devices were properly removed from the VmConfig when a
remove-device command was issued, except for the "fs" type. Fix this
lack of support as it is causing the integration tests to fail with the
recent addition of verifying that identifiers are unique.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
(cherry picked from commit a5a2e591c9)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This prevents a conflict since the old API socket will not have been
cleaned up (due to the use of SIGKILL.)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
(cherry picked from commit 4fed2d4ed7)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The I/O operations knobs are prefixed `ops_` rather than `bw_`, as `bw_`
refers to the "bandwidth" knobs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit a87d1bbaa1)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The calls to these functions are always preceded by a call to
InterruptSourceGroup::update(). By adding a masked boolean to that
function call it possible to remove 50% of the calls to the
KVM_SET_GSI_ROUTING ioctl as the the update will correctly handle the
masked or unmasked case.
This causes the ioctl to disappear from the perf report for a boot of
the VM.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
It suffered from the same issue like the main program after switching to
Rust 2021 edition, but the issue was not caught by the CI.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
EDK2 execution requires a flash device at address 0.
The new added device is not a fully functional flash. It doesn't
implement any spec of a flash device. Instead, a piece of memory is used
to simulate the flash simply.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
We modified a test case to workaround the RAM calculation error caused
by hidding 4MiB memory for UEFI. Now change it back to normal.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Size of `memory` node in FDT was reduced by 4MiB. Now it is returned.
In memory regions, a `Ram` region of 4MiB was created at address 0. Now
it is removed.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Rust 2021 edition has a few improvements over the 2018 edition. Migrate
the project to 2021 edition by following recommended migration steps.
Luckily, the code itself doesn't require fixing.
Bump MSRV to 1.56 as it is required by the 2021 edition. Also fix the
clap build dependency to make Cloud Hypervisor build again.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This is a refactoring commit to simplify source code.
Removed some functions that only return a layout const.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Some addresses defined in `layout.rs` were of type `GuestAddress`, and
are `u64`. Now align the types of all the `*_START` definitions to
`GuestAddress`.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Now that address translations performed by virtio-iommu can error out if
the address can't be translated, we uncovered an issue in integration
test aarch64_acpi::test_virtio_iommu.
We disable the test until we can investigate and fix the root cause.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It doesn't matter if we're trying to translate a GVA or a GPA address,
but in both cases we must error out if the address couldn't be
translated.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Remove EOLed distributions and clarify the stability guarantees and what
the experimental features are.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The reserved space is for devices.
Some devices (like TPM) require arbitrary addresses close to 4GiB.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
`RAM_64BIT_START` was set to 1 GiB, not a real 64-bit address. Now
rename it `RAM_START` to avoid confusion.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
The extra vDPA device in the test is hotplugged behind the vIOMMU, which
covers the use case of placing a vDPA device behind a virtual IOMMU.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Whenever a virtio device is placed behind a vIOMMU, we have some code in
pci_common_config.rs to translate the queue addresses (descriptor table,
available ring and used ring) from GVA to GPA, so that they can be used
correctly.
But in case of vDPA, we also need to provide the queue addresses to the
vhost backend. And since the vhost backend deals with consistent IOVAs,
all addresses being provided should be GVAs if the device is placed
being a vIOMMU. For that reason, we perform a translation of the queue
addresses back from GPA to GVA if necessary, and only to be provided to
the vhost backend.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Add a new iommu parameter to VdpaConfig in order to place the vDPA
device behind a virtual IOMMU.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In case an external mapping would have been added after the virtio-iommu
device has been activated, it would have simply be ignored because the
code wasn't using a shared object between the vmm thread and the iommu
thread. This behavior is only triggered on the hotplug codepath, and
only if the hotplugged device is placed behind the virtual IOMMU.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for the vDPA need to translate a GPA back into a GVA, we
extend the existing trait DmaRemapping and AccessPlatform to perform
such operation.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Renaming translate() to translate_gva() to clarify we want to translate
a GVA address into a GPA.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
According to latest official images based on latest TDX guest kernel, we
update the documentation to reflect the change regarding serial support.
New guest kernels have been updated to disable usage of serial ports,
meaning adding console=ttyS0 to the kernel boot parameters will have no
effect.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The list of memory resources provided through the HOB wasn't accurate
because of the broken logic. The fix provides correct ranges to the
firmware.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on latest QEMU patches from branch tdx-qemu-2022.03.29-v7.0.0-rc1
we should only report as memory resources the TempMem sections from TDVF
sections.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on latest QEMU patches from branch tdx-qemu-2022.03.29-v7.0.0-rc1
we don't need EFI_RESOURCE_ATTRIBUTE_ENCRYPTED as part of the attributes
we must enable with EFI_RESOURCE_SYSTEM_MEMORY and
EFI_RESOURCE_MEMORY_RESERVED resource types.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The introduction of a error if live resizing is not possible is a
regression compared to the original behaviour where the new size would
be stored in the config and reflected in the next boot. This behaviour
was also inconsistent with the effect of resizing with no VM booted.
Instead of generating an error allow the code to go ahead and update the
config so that the new size will be available upon the reboot.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
As reported by the periodic CI runs, it may take more time for the NVMe
device to present in the guest after being hotplugged as a VFIO user
device on `aarch64` (especially under high load). Let's increase the
timeout after device hotplug from `1s` to `10s` to increase the test
stability.
Fixes: #3495
Signed-off-by: Bo Chen <chen.bo@intel.com>
Similarly to the previous commit restricting the cpu resizing error only
to the situations where the vcpu amount has changed, let's do the same
with the memory and be consistent throughout our code base.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
188078467d made clear that resize should
only happen when dealing with a "dynamic" CpuManager. Although this is
very much correct, it causes a regression on Kata Containers (and on any
other consumer of Cloud Hypervisor) in cases where a resize would be
triggered but the vCPUs values wouldn't be changed.
There's no doubt Kata Containers could do better and do not call a
resize in such situations, and that's something that should **also** be
solved there. However, we should also work this around on Cloud
Hypervisor side as it introduces a regression with the current Kata
Containers code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Extend the Hypervisor API in order to retrieve the TDX capabilities from
the underlying hypervisor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By enabling the VIRTIO feature VIRTIO_F_IOMMU_PLATFORM for all
vhost-user devices when needed, we force the guest to use the DMA API,
making these devices compatible with TDX. By using DMA API, the guest
triggers the TDX codepath to share some of the guest memory, in
particular the virtqueues and associated buffers so that the VMM and
vhost-user backends/processes can access this memory.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
If EFI reset fails on the Linux kernel then it will fallthrough to CMOS
reset. Implement this as one of our reset solutions.
Fixes: #3912
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Compile this feature in by default as it's well supported on both
aarch64 and x86_64 and we only officially support using it (no non-acpi
binaries are available.)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
These were erroneously skipping features for the unit tests and the
"build" target for dev_cli.sh
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
These are never run by the CI and is inconsistent with the way we build
test which is specified inside the .github workflows.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This includes the removal of testing without the "acpi" feature. The
command have been reordered to reduce the amount of recompilation
required.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
It seems the vdpa_sim_block isn't behaving properly after the vhost
device is closed, as it sometimes returns EBUSY when we try to open it
again. The easiest way to deal with this issue is by simplifying the
integration test, avoid to plug the same device after it's been
unplugged.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit unifies the custom linux kernel build in x86, Arm, and
performance metrics to the same function. Therefore, when bumping
the kernel version, we can make sure we only need to make the change
in one place.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
With the addition of the amx feature, add a new build workflow to
enable the feature and a clippy quality check.
Signed-off-by: William Douglas <william.douglas@intel.com>
AMX is an x86 extension adding hardware units for matrix
operations (int and float dot products). The goal of the extension is
to provide performance enhancements for these common operations.
On Linux, AMX requires requesting the permission from the kernel prior
to use. Guests wanting to make use of the feature need to have the
request made prior to starting the vm.
This change then adds the first --cpus features option amx that when
passed will enable AMX usage for guests (needs a 5.17+ kernel) or
exits with failure.
The activation is done in the CpuManager of the VMM thread as it
allows migration and snapshot/restore to work fairly painlessly for
AMX enabled workloads.
Signed-off-by: William Douglas <william.douglas@intel.com>
Now that we rely on vhost v0.4.0, which contains the fix for
get_iova_range(), we don't need the workaround anymore, and we can
actually call into the dedicated function.
Fixes#3861
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Rely on newly released versions of the vhost and vhost-user-backend
crates from rust-vmm.
The new vhost version includes the fixes needed for vDPA.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This commit makes sure that the custom linux kernel is always
rebuilt when running the performance metrics tests, and therefore
changes to the kernel config file is always caught.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Explain the reason why using vDPA might be interesting and how to use it
with Cloud Hypervisor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Disable the DAX feature from the virtio-fs implementation as the feature
is still not stable. The feature is deprecated, meaning the 'dax'
parameter will be removed in about 2 releases cycles.
In the meantime, the parameter value is ignored and forced to be
disabled.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The test is sporadically failing whenever we try to hotplug the vDPA
device we've just unplugged. This is causing the kernel to complain with
EBUSY because the device hasn't been released yet. This is happening
because the CI system is under very high load, therefore taking quite
some time to the host to update the state of this device.
The easy way to fix such issue is by increasing the sleep time between
the unplug and the replug.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When running non-dynamic or with virtio-mem for hotplug the ACPI
functionality should not be included on the DSDT nor does the
MemoryManager need to be placed on the MMIO bus.
Fixes: #3883
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This is now consistent with not supplying the _CRS for the device when
CpuManager is not dynamic.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The file descriptor provided to fs_slave_map() and fs_slave_io() is
passed as a AsRawFd trait, meaning the caller owns it. For that reason,
there's no need for these functions to close the file descriptor as it
will be closed later on anyway.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
As ethr is not stable on AArch64, disable it until we find other tool.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
fio test depends on hot-add disk which needs uefi on AArch64, that
will lead to lots of performance issues. So, it's better to use
cold-plug disk.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
On Arm64, the fio binary that comes from ubuntu distribution cannot
support io_uring, which is a required engine for the block
performance tests in performance metrics. Therefore this commit
build the fio from a relatively new version, and replace the native
ubuntu fio with the newly build one.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
To emulate debug I/O port on arm64, we need trap from guest to VMM.
Generally, there are 3 ways to go:
1) execute a privilege intruction like "wfi";
2) access a system register;
3) access device MMIO region;
However, 1) and 2) often handled in kvm, thus 3) it's the choice.
There maybe lots of repeated work and enlarge the size of clh to develop
a new device, so it's better to reuse the current device. Luckily, I
find that pl011 is the eariest device initialized in kernel and, there
is reserved region in it which we can reuse for emulating debug I/O.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Rather than just printing a message return an error back through the API
if the user attempts to hotplug a device that supports being behind an
IOMMU where that device isn't placed on an IOMMU segment.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Ensure devices that are specified to be on a PCI segment that is behind
the IOMMU are IOMMU enabled if possible or error out for those devices
that do not support it.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Previously it was not possible to enable vIOMMU for a virtio device.
However with the ability to place an entire PCI segment behind the
IOMMU the IOMMU mapping needs to be setup for the virtio device if it is
behind the IOMMU.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This can already be calculated by the summing the tables reported by the
Linux kernel but this is more convenient.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Changes in commit "scripts: aarch64: Reduce source code build in CI"
for the Linux kernel will lead to the kernel cannot be rebuilt if
the config file is changed. Therefore should be reverted.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
As Linux kernel, edk2, virtiofd, SPDK are all build from source, we
need to find a way to reduce the number of source code build.
To address this issue, this commit adds a stage that touching a
".built" file to each repo of the source code after a successfully
build. By checking if the ".built" file exists, we can determine
if there is already an old build. Since the source code repo will
be removed and cloned again when we bump the repo hash, the CI will
rebuild the repo.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Whenever a vfio-user device is dropped, the communication between the
VMM and the backend should be shutdown.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Separate the destruction and cleanup of original VM and the creation of
the new one. In particular have a clear hand off point for resources
(e.g. reset EventFd) used by the new VM from the original. In the
situation where vm.shutdown() generates an error this also avoids the
Vmm reference to the Vm (self.vm) from being maintained.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding two new integration tests for vDPA, relying on both block and net
simulators from the host kernel.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the newly added Vdpa device along with the new vdpa parameter,
this patch enables the support for vDPA devices.
It's important to note this the only virtio device for which we provide
an ExternalDmaMapping instance. This will allow for the right DMA ranges
to be mapped/unmapped.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Introduce a new --vdpa parameter associated with a VdpaConfig for the
future creation of a Vdpa device.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
vDPA is a kernel framework introduced fairly recently in order to handle
devices complying with virtio specification on their datapath, while the
control path is vendor specific. For the datapath, that means the
virtqueues are handled through DMA directly between the hardware and the
guest, while the control path goes through the vDPA framework,
eventually exposed through a vhost-vdpa device.
vDPA, like VFIO, aims at achieving baremetal performance for devices
that are passed into a VM. But unlike VFIO, it provides a simpler/better
framework for achieving migration. Because the DMA accesses between the
device and the guest are going through virtio queues, migration can be
achieved way more easily, and doesn't require each device driver to
implement the migration support. In the VFIO case, each vendor is
expected to provide an implementation of the VFIO migration framework,
which makes things harder as it must be done for each and every device.
So to summarize the point is to support migration for hardware devices
through which we can achieve baremetal performances.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Update the content of the documentation to clarify how the cache option
from virtiofsd has an impact on the host page cache, while the dax
option for the filesystem in the guest has an impact on the guest page
cache.
The document is also updated to not have lines exceeding 80 characters.
Fixes#3826
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Document the reason why someone might need to use the balloon device and
detail the different parameters associated with it.
Fixes#3807
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This will significantly reduce the size of the DSDT and the effort
required to parse them if there is no requirement to support
hotplug/unplug.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the CpuManager is dynamic it devices CPUs can be
hotplugged/unplugged.
Since TDX does not support CPU hotplug this is currently the only
determinator as to whether the CpuManager is dynamic.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since the code has been adapted to support VIRTIO_F_EVENT_IDX we use
Queue::needs_notification() to determine whether to signal the guest so
it is no longer necessary to check if there are any used descriptors. If
the feature is not negotiated then Queue::needs_notification() will
return true triggering an interrupt of the guest. Theoretically this
could be a spurious interrupt of the guest if there were no used used
descriptors but this is unlikely as we only generate used descriptors
for the control queue as a result of an interrupt of the VMM by the
guest.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This indicates if anything has been added to the available queue since
it was last iterated through. If it returns true then it is necessary to
iterate through the queue again otherwise it is appropriate to break out
from the loop.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
vmm.ping/vm.info will hang for PUT method, vm.create/vmm.shutdonw hang for GET method.
Because these four APIs do not write the response body when the HTTP method does not match.
Signed-off-by: LiHui <andrewli@kubesphere.io>
In case the virtio device which requires DMA mapping is placed behind a
virtual IOMMU, we shouldn't map/unmap any region manually. Instead, we
provide the DMA handler to the virtio-iommu device so that it can
trigger the proper mappings.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
If a virtio device is associated with a DMA handler, the DMA mapping and
unmapping is performed from the device manager through the handler.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Given that some virtio device might need some DMA handling, we provide a
way to store this through the VirtioPciDevice layer, so that it can be
accessed when the PCI device is removed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In anticipation for handling potential DMA mapping/unmapping operations for a
virtio device, we extend the MetaVirtioDevice with an additional field
that holds an optional DMA handler.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The tuple of information related to each virtio device is too big, and
it's better to factorize it through a dedicated structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
With the VIRTIO_F_EVENT_IDX handling now conducted inside the
virtio-queue crate it is necessary to activate the functionality on
every queue if it is negotiatated. Otherwise this leads to a failure of
the guest to signal to the host that there is something in the available
queue as the queue's internal state has not been configured correctly.
Fixes: #3829
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When mask a msi irq, we set the entry.masked to be true, so kvm
hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING
ioctl which update kvm->irq_routing. This will trigger kernel
panic on AMD platform when the gsi is the largest one in kernel
kvm->irqfds.items:
crash> bt
PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8"
#0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397
#1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d
#2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d
#3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d
#4 [ffffb1ba6707fb90] no_context at ffffffff856692c9
#5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51
#6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace
[exception RIP: svm_update_pi_irte+227]
RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086
RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001
RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8
RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200
R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001
R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm]
#8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm]
#9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm]
RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b
RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020
RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0
R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0
R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0
ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b
To solve this problem, move route.disable() before set_gsi_routes() to
remove the gsi from irqfds.items first.
This problem only exists on AMD platform, 'cause on Intel platform
kernel just return when update irte while it only prints a warning on
AMD.
Also, this patch adjusts the order of enable() and set_gsi_routes() in
unmask(), which should do no harm.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
When mask a msi irq, we set the entry.masked to be true, so kvm
hypervisor will not pass the gsi to kernel through KVM_SET_GSI_ROUTING
ioctl which update kvm->irq_routing. This will trigger kernel
panic on AMD platform when the gsi is the largest one in kernel
kvm->irqfds.items:
crash> bt
PID: 22218 TASK: ffff951a6ad74980 CPU: 73 COMMAND: "vcpu8"
#0 [ffffb1ba6707fa40] machine_kexec at ffffffff8565b397
#1 [ffffb1ba6707fa90] __crash_kexec at ffffffff85788a6d
#2 [ffffb1ba6707fb58] crash_kexec at ffffffff8578995d
#3 [ffffb1ba6707fb70] oops_end at ffffffff85623c0d
#4 [ffffb1ba6707fb90] no_context at ffffffff856692c9
#5 [ffffb1ba6707fbf8] exc_page_fault at ffffffff85f95b51
#6 [ffffb1ba6707fc50] asm_exc_page_fault at ffffffff86000ace
[exception RIP: svm_update_pi_irte+227]
RIP: ffffffffc0761b53 RSP: ffffb1ba6707fd08 RFLAGS: 00010086
RAX: ffffb1ba6707fd78 RBX: ffffb1ba66d91000 RCX: 0000000000000001
RDX: 00003c803f63f1c0 RSI: 000000000000019a RDI: ffffb1ba66db2ab8
RBP: 000000000000019a R8: 0000000000000040 R9: ffff94ca41b82200
R10: ffffffffffffffcf R11: 0000000000000001 R12: 0000000000000001
R13: 0000000000000001 R14: ffffffffffffffcf R15: 000000000000005f
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffffb1ba6707fdb8] kvm_irq_routing_update at ffffffffc09f19a1 [kvm]
#8 [ffffb1ba6707fde0] kvm_set_irq_routing at ffffffffc09f2133 [kvm]
#9 [ffffb1ba6707fe18] kvm_vm_ioctl at ffffffffc09ef544 [kvm]
RIP: 00007f143c36488b RSP: 00007f143a4e04b8 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 00007f05780041d0 RCX: 00007f143c36488b
RDX: 00007f05780041d0 RSI: 000000004008ae6a RDI: 0000000000000020
RBP: 00000000000004e8 R8: 0000000000000008 R9: 00007f05780041e0
R10: 00007f0578004560 R11: 0000000000000246 R12: 00000000000004e0
R13: 000000000000001a R14: 00007f1424001c60 R15: 00007f0578003bc0
ORIG_RAX: 0000000000000010 CS: 0033 SS: 002b
To solve this problem, move route.disable() before set_gsi_routes() to
remove the gsi from irqfds.items first.
This problem only exists on AMD platform, 'cause on Intel platform
kernel just return when update irte while it only prints a warning on
AMD.
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Reduce the number of open pull requests dependabot will create to avoid
excessive rebase and rebuilds of PRs. This avoids a situation with a 2nd
PR being rebased after a 1st is merged resulting in a wasteful rebuild.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Move to release version v0.2.0 for both vm-virtio and vhost-user-backend
crates rather than relying on their main branch, as they might be
subject to breaking changes.
Fixes#3800
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Replace the thread for handling SIGSYS with a simple signal handler.
This resolves inconsistent delivery of signals to the SIGSYS thread due
to other threads manipulating the signals.
Tested by removing key syscalls from vCPU and virtio device filters and
observing correct notice.
Fixes: #3811
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
To be consistently with the integration scripts, we now pass through the
`--test-filter` option of the test scripts to the `performance-metrics`
binary. Now the following two commands achieve the same results:
`$./scripts/dev_cli.sh tests --metrics -- --test-filter boot_time_ -- --test-filter latency`
`$./scripts/dev_cli.sh tests --metrics -- -- --test-filter boot_time_ --test-filter latency`
Note that, the test scripts support only single occurrence of the
`--test-filter` option while the `performance-metrics` binary allows
multiple occurrences for filtering based on different keywords.
Fixes: #3787
Signed-off-by: Bo Chen <chen.bo@intel.com>
Add a field for its length and fix up users.
Things work just because all hardcoded values agree with each other.
This is prone to breakage.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Needed since:
commit 0ab4097606
Author: Bo Chen <chen.bo@intel.com>
Date: Thu Feb 10 18:29:53 2022 -0800
test_infra: Enable cross-build for "musl" and "aarch64" targets
With enabling the `vendered-openssl` feature, we can now cross-build the
`test_infra` crate for "musl" and "aarch64" targets. In this way, we
can remove the `test_infra` crate from the "exclude" list, so that this
crate can be checked and processed by "cargo clippy" and "cargo fmt".
More details can be found: https://docs.rs/openssl/latest/openssl/#vendored
As 'musl-gcc' is required, this commit also installs the `musl-tools`
package for our "build" github action on the musl target [1].
[1] https://github.com/actions-rs/toolchain/issues/102
Signed-off-by: Bo Chen <chen.bo@intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Some host kernels are IPv6-only, and they cannot make an AF_INET socket.
The tun ioctls such as SIOCGIFHWADDR work with an AF_UNIX socket, which
all host kernels should have.
Signed-off-by: Barret Rhoden <brho@google.com>
Adding the missing TARGET_CC environment variable to get the build to
complete correctly.
Fixes#3776
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
After writing to an address, Windows 11 on ARM64 unconditionally reads
it back. It is harmless. Drop the error message to avoid spamming.
Fixes: #3732
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This looks like it was copy and pasted from the network test which
required 2 vCPUs per queue pair but has since been resolved.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Using the default queue sizes provides more realistic data about what
our users are testing. Extra metrics can be added later that also
modify the queue size but overloading the existing metrics is confusing.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Don't treat it as the number of pairs. Instead calculate the number of
pairs later when necessary.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Also be clear that the block tests are in bytes per second (Bps) vs
network that is in bits per second (bps).
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The former is ambiguous as it could be considered the ordinal (and is
used elsewhere in the codebase for that).
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Many of the tests already amortize their results over a longer time
period/sample so it is not necessary to run 30 iterations.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Let's officially have a way to pass the features used to build
cloud-hypervisor to the dev_cli.sh script.
This doesn't invalidate the previous commit, as we still don't what the
features_build variable to be quoted, otherwise we face the following
issue:
```
error: Found argument '--features tdx' which wasn't expected, or isn't valid in this context
Did you mean --features?
USAGE:
cargo build --all --features <FEATURES>...
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2805e7b1dc quoted enclose variables to
prevent globbing or incorrect splitting. However, by doing with with
$features_build it broke the capability to call the script as:
```
$ ./scripts/dev_cli.sh build --release --libc musl -- --features tdx
```
Before 2805e7b1dc it simply worked, after,
the result is:
```
docker run --user 1000:1000 --workdir /cloud-hypervisor --rm --volume /dev/kvm --volume /home/ffidenci/go/src/github.com/cloud-hypervisor/cloud-hypervisor:/cloud-hypervisor --env RUSTFLAGS= cloudhypervisor/dev:20220223-0 cargo build --all '' --target-dir /cloud-hypervisor/build/cargo_target --features tdx --release --target x86_64-unknown-linux-musl
error: Found argument '' which wasn't expected, or isn't valid in this context
USAGE:
cargo build --all
For more information try --help
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For now only generate the boot time related tests as the full metrics
test suite needs some more time to bed in.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The dev container interface script (e.g. 'dev_cli.sh') now supports the
following arguments syntax for running tests:
`tests [--unit|--cargo|--all] [--libc musl|gnu] [-- [<test scripts args>] [-- [<test binary args>]]] `
In this way, we can pass custom arguments to the test binary (either
"cargo test" or "performance-metrics") with our dev container script.
For example:
`$ ./dev_cli.sh tests --metrics -- -- --report-file /tmp/metrics.json --test-filter latency`
`$ ./dev_cli.sh tests --integration -- --test-filter "test_serial" -- --nocapture --test-threads=1`
Fixes: #3739
Signed-off-by: Bo Chen <chen.bo@intel.com>
We don't need to install docker inside the container. Add missing
depenencies that docker was pulling in.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The same way we mask the writes coming from the guest to the message
control register related to MSI-X capability, let's do the same for MSI.
The point is to prevent the guest from writing to read-only bits.
The correct writable bits for MSI are only bits 0, 4, 5 and 6 of 2nd
16-bit word.
Those are:
* MSI Enable: 0
* Multiple Message Enable: 6-4
See "Table 7-39 Message Control Register for MSI" from
"NCB-PCI_Express_Base_5.0r1.0-2019-05-22.pdf".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Starting from Rust 1.56 Cargo supports specifying the minimum supported
rust version (MSRV) via "rust-version". If the compiler version is not
satisfied, Cargo prints an error and exits early.
MSRV is useful information to packagers. Using this field also saves us
from adding another file to the tree.
The version is currently set to 1.54, which is tested to build Cloud
Hypervisor successfully. Although anyone who uses 1.54 will see a
warning because "rust-version" is only introduced in 1.56. The warning
can be safely ignored.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This commit adds event fds and the event handler to send/receive
requests and responses from the GDB thread. It also adds `--gdb` flag to
enable GDB stub feature.
Signed-off-by: Akira Moroo <retrage01@gmail.com>
This commit adds `stop_on_boot` to `Vm` so that the VM stops before
starting on boot requested. This change is required to keep the target
VM stopped before a debugger attached as the user expected.
Signed-off-by: Akira Moroo <retrage01@gmail.com>
This commit adds `Vm::debug_request` to handle `GdbRequestPayload`,
which will be sent from the GDB thread.
Signed-off-by: Akira Moroo <retrage01@gmail.com>
This commit adds initial gdb.rs implementation for `Debuggable` trait to
describe a debuggable component. Some part of the trait bound
implementations is based on the crosvm GDB stub code [1].
[1] https://github.com/google/crosvm/blob/main/src/gdb.rs
Signed-off-by: Akira Moroo <retrage01@gmail.com>
This commit adds `KVM_SET_GUEST_DEBUG` and `KVM_TRANSLATE` ioctls to
seccomp filter to enable guest debugging without `--seccomp=false`.
Signed-off-by: Akira Moroo <retrage01@gmail.com>
This commit adds `VmState::BreakPoint` to handle hardware breakpoint.
The VM will enter this state when a breakpoint hits or a debugger
interrupts the execution.
Signed-off-by: Akira Moroo <retrage01@gmail.com>
This commit adds `VmExit::Debug` for x86/KVM. When the guest hits a
hardware breakpoint, `VcpuExit::Debug` vm exit occurs. This vm exit
will be handled with code implemented in the following commits.
Signed-off-by: Akira Moroo <retrage01@gmail.com>
This commit adds `translate_gva` for x86/KVM. The same name function is
already implemented for MSHV, but the implementation differs as
KVM_TRANSLATE does not take the flag argument and does not return status
code. This change requires the newer version of kvm-ioctls [1].
[1]
97ff779b6e
Signed-off-by: Akira Moroo <retrage01@gmail.com>
`translate_gva` returns mshv-specific type `hv_translate_gva_result`.
This return type is not a problem since this function is implemented
only for mshv, but we need to remove the type as the same function will
be implemented for KVM in PR #3575. This commit replaces the
mshv-specific type with `u32`.
Signed-off-by: Akira Moroo <retrage01@gmail.com>
error: writing `&mut Vec` instead of `&mut [_]` involves a new object
where a slice will do
--> virtio-devices/src/transport/pci_common_config.rs:93:17
|
93 | queues: &mut
Vec<Queue<GuestMemoryAtomic<GuestMemoryMmap>>>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: change this to: `&mut [Queue<GuestMemoryAtomic<GuestMemoryMmap>>]`
|
= note: `-D clippy::ptr-arg` implied by `-D warnings`
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
Signed-off-by: Akira Moroo <retrage01@gmail.com>
I incorrectly used the MSI message control register values for the mask
not the the MSI-X control registers.
The correct writable fields for MSI-X are only bits 14 and 15 of 2nd
16-bit word.
Those are:
* Function Mask: 14
* MSI-X Enable: 15
See "Table 7-47 Message Control Register for MSI-X" from
"NCB-PCI_Express_Base_5.0r1.0-2019-05-22.pdf"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
42b5d4a2f7 has changed how the PciBdf
field of a DeviceNode is represented (from an int32 to its own struct).
To avoid marshelling / demarshelling issues for the projects relying on
the openapi auto generated code, let's propagate the change, updating
the yaml file accordingly.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`Dies per package` setting of VCPU topology doesnot apply on AArch64.
Now we only accept `1` value. This way we can make the `dies` field
transparent, avoid it from impacting the topology setting.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
The logic of determining VCPU index in creating `cpu-map` node of FDT
can be optimized.
The code is invoked when VCPU topology is specified.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
In Flattened Device Tree (FDT) on AArch64, the VCPU topology is
represented by `cpu-map` node. The source code of creating the node
can be simplified.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
If `--local` is provided or if the version is not available then build
the container before use. This allows combining updates to the
Dockerfile with a full CI run.
Drop the "--dev" parameter as we only support one container type for
simplicity.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
For vhost-user devices, we don't want to loose the vhost-user protocol
feature through the negotiation between guest and device. Since we know
VIRTIO has no knowledge of the vhost-user protocol feature, there is no
way it would ever be acknowledged by the guest. For that reason, we
create each vhost-user device with the set of acked features containing
the vhost-user protocol feature is this one was part of the available
list.
Having the set of acked features containing this bit allows for solving
a bug that was happening through the migration process since the
vhost-user protocol feature wasn't explicitely enabled.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In this way, we can cover local-migration with dpdk in our regular CI,
to prevent similar regressions reported and fixed by #3657.
Fixes: #3659
Signed-off-by: Bo Chen <chen.bo@intel.com>
Based on the helpers from the hypervisor crate, the VMM can identify
what type of hypercall has been issued through the KVM_EXIT_TDX reason.
For now, we only log warnings and set the status to INVALID_OPERAND
since these hypercalls aren't supported. The proper handling will be
implemented later.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Relying on the recent additions to the kvm-ioctls crate, this commit
implements the support for providing the exit reason details to the
caller, which allows the identification of the type of hypercall that
was issued. It also introduces a way for the consumer to set the status
code that must be sent back to the guest.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since the object returned from CpuManager.create_vcpu() is never used,
we can avoid the cloning of this object.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Rely on latest version of rust-vmm/kvm-ioctls to be able to invoke the
new method get_kvm_run() that returns a mutable reference of the kvm_run
structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
A new fork of the kvm-bindings crate has been submitted to the
ch-v0.5.0-tdx branch. It contains updated bindings for x86 to support
TDX.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This will allow easier transitioning between versions of the container.
A later PR will update the CI to use this new tag.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This reverts commit 98bbfa9738.
Some tests are continuing to fail even after reverting
d27316dab6. This is the only other
relevant change.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This reverts commit d27316dab6.
Jenkins testing showed up issues that didn't appear during manual
testing even after using "dev_cli.sh build-container" before running
integration tests.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Remove unused packages or more specific dependencies. In particular use
the packaged virtiofsd to avoid compiling it.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
To run unit tests correctly on musl target, We don't need to provide
specific "CFLAGS" or "TARGET_CC", as long as we use the correct build
target `*-linux-musl` with the "cargo test" command.
Signed-off-by: Bo Chen <chen.bo@intel.com>
It provides a CLI optoin to list and filter tests, e.g.:
`$ performance-metrics --test-filter boot_time --test-filter net_latency`
Signed-off-by: Bo Chen <chen.bo@intel.com>
Performance tests are expected to be executed sequentially, so we can
start VM guests with the same IP while putting them on a different
private network. The default constructor "Guest::new()" does not work
well, as we can easily create more than 256 VMs from repeating various
performance tests dozens times in a single run.
Signed-off-by: Bo Chen <chen.bo@intel.com>
In this way, we can avoid create/delete large temporary files during
every iteration of block I/O performance tests. Also, we can reuse the
'init/clean_tests()' interface in the future for other setup/cleanup.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This commit adds unit tests for parsing iperf3/ethr/boot_time/fio, e.g.:
`$ cargo test --package performance_metrics `
Signed-off-by: Bo Chen <chen.bo@intel.com>
This commit creates a new binary target for generating performance metrics,
and also adds a set of performance tests for network throughput/latency,
block I/O performance, and boot time.
Signed-off-by: Bo Chen <chen.bo@intel.com>
error: unused variable: `fw_path`
--> test_infra/src/lib.rs:837:9
|
837 | fw_path: &str,
| ^^^^^^^ help: if this is intentional, prefix it with an underscore: `_fw_path`
|
= note: `-D unused-variables` implied by `-D warnings`
Signed-off-by: Bo Chen <chen.bo@intel.com>
With enabling the `vendered-openssl` feature, we can now cross-build the
`test_infra` crate for "musl" and "aarch64" targets. In this way, we
can remove the `test_infra` crate from the "exclude" list, so that this
crate can be checked and processed by "cargo clippy" and "cargo fmt".
More details can be found: https://docs.rs/openssl/latest/openssl/#vendored
As 'musl-gcc' is required, this commit also installs the `musl-tools`
package for our "build" github action on the musl target [1].
[1] https://github.com/actions-rs/toolchain/issues/102
Signed-off-by: Bo Chen <chen.bo@intel.com>
keep it as default to print out commandline when spawning guest in favor
of integration tests for debugging purpose, while allowing to disable
it when needed (e.g. for performance tests).
Signed-off-by: Bo Chen <chen.bo@intel.com>
The PCI spec specifies that only the following bits are writable:
16: MSI Enable
20,21,22: Multiple Message Enable
26: Extended Message Data Enable
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
By having the DeviceNode storing a PciBdf, we simplify the internal code
as well as allow for custom Serialize/Deserialize implementation for the
PciBdf structure. These custom implementations let us display the PCI
s/b/d/f in a human readable format.
Fixes#3711
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
As we've added support for cold adding devices to a VM that was created
but not already started, we should propagate the `204` response
generated on those cases to the yaml file, so openapi-generator can
produce the correct client code on the go side, to handle both `200` and
`204` successful results.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of erroring out when trying to change the configuration of the
VM somewhere between the VM was created but not yet booted, let's allow
users to change that without any issue, as long as the VM has already
been created.
Fixes: #3639
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add very basic unit for the vm_add_$device() functions, so we can
easily expand those when changing its behaviour in the coming commits.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of doing the validation of the configuration change as part of
the vm, let's do this in the uper layer, in the Vmm.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move add_to_config to config.rs so it can be used from both inside
and outside of the vm.rs file.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The GitHub security reporting link only works for those who are
repository owners.
Fixes: #3701
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
TDx support is already present on the project for quite some time, but
the TDx configuration was not yet exposed to the ones using CH via the
OpenAPI auto generated code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Since the devices behind the IOMMU cannot be changed at runtime we offer
the ability to place all devices on user chosen segments behind the
IOMMU. This allows the hotplugging of devices behind the IOMMU provided
that they are assigned to a segment that is located behind the iommu.
Fixes: #911
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Introducing a new integration test relying on the virtio-balloon ability
to free host pages that have been reported as freed by the guest.
This test checks that after consuming a lot of RAM in the guest, the VMM
process is able to releases the pages reported by the guest. Simply done
by checking the RSS associated with the VMM's process follows the memory
trend in the guest.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Implement the VIRTIO_BALLOON_F_REPORTING feature, indicating to the
guest it can report set of free pages. A new virtqueue dedicated for
receiving the information about the free pages is created. The VMM
releases the memory by punching holes with fallocate() if the guest
memory is backed by a file, and madvise() the host about the ranges of
memory that shouldn't be needed anymore.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding a new parameter free_page_reporting=on|off to the balloon device
so that we can enable the corresponding feature from virtio-balloon.
Running a VM with a balloon device where this feature is enabled allows
the guest to report pages that are free from guest's perspective. This
information is used by the VMM to release the corresponding pages on the
host.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Improving the existing code for better readability and in anticipation
for adding an additional virtqueue for the free page reporting feature.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
No need to run the Jenkins CI on pull request for which exclusively
markdown files have been modified.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since AZURE_CONNECTION_STRING is only useful for the Windows build,
let's remove it from other builds where it's not invoked.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding a bit of documentation and links about the TDShim and how to run
it with Cloud Hypervisor.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This should not occur as ioeventfd is used for notification. Such an
error message would have made the discovery of the underlying cause of
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to allow for human readable output for the VM configuration, we
pull it out of the snapshot, which becomes effectively the list of
states from the VM. The configuration is stored through a dedicated file
in JSON format (not including any binary output).
Having the ability to read and modify the VM configuration manually
between the snapshot and restore phases makes debugging easier, as well
as empowers users for extending the use cases relying on the
snapshot/restore feature.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
As per this kernel documentation:
For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR, KVM_EXIT_XEN,
KVM_EXIT_EPR, KVM_EXIT_X86_RDMSR and KVM_EXIT_X86_WRMSR the corresponding
operations are complete (and guest state is consistent) only after userspace
has re-entered the kernel with KVM_RUN. The kernel side will first finish
incomplete operations and then check for pending signals.
The pending state of the operation is not preserved in state which is
visible to userspace, thus userspace should ensure that the operation is
completed before performing a live migration. Userspace can re-enter the
guest with an unmasked signal pending or with the immediate_exit field set
to complete pending operations without allowing any further instructions
to be executed.
Since we capture the state as part of the pause and override it as part
of the resume we must ensure the state is consistent otherwise we will
lose the results of the MMIO or PIO operation that caused the exit from
which we paused.
Fixes: #3658
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Remove the bits about this feature being experimental, as well as
limitations around virtio-mem and vhost-user devices.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The patch mechanism in Cargo.toml doesn't support features to be
specified, and was leading to the following warning:
warning: patch for `kvm-bindings` uses the features mechanism.
default-features and features will not take effect because the patch
dependency does not support this mechanism
Removing the "features" parameter from the patch fixes this warning
without breaking the build.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In this way, we allows to reuse the struct `Guest` with kernel paths and
kernel commands (e.g. hardcoded constants) that are tests-specific.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Instead of using hardcoded firmware paths inside the `Guest` struct
constructor, this commit removes `fw_path` related code paths from the
`Guest` struct and asks each test constructs its firmware path
explicitly. This allows better flexibility for the `Guest` struct so
that it can be reused for the performance tests we are adding soon.
Signed-off-by: Bo Chen <chen.bo@intel.com>
If a payload is found in the TDVF section, and after it's been copied to
the guest memory, make sure to create the corresponding TdPayload
structure and insert it through the HOB.
Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding a new method to the TdHob structure so that we can easily insert
a HOB_PAYLOAD_INFO_TABLE into the HOB.
Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In case of TDX, if a kernel and/or a command line are provided by the
user, they can't be treated the same way as for the non-TDX case. That
is why this patch ensures the function load_kernel() is only invoked for
the non-TDX case.
For the TDX case, whenever TDVF contains a Payload and/or PayloadParam
sections, the file provided through --kernel and the parameters provided
through --cmdline are copied at the locations specified by each TDVF
section.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the recent updates of the TDVF specification introducing new
types of TDVF sections, let's extend the enum in our code.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The TDVF specification has been updated with the ability to provide a
specific payload, which means we will be able to achieve direct kernel
boot.
For that reason, let's not prevent the user from using --kernel
parameter when running with TDX.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Make sure Cloud Hypervisor relies on upstream and actively maintained
vfio-ioctls crate from the rust-vmm/vfio repository instead of the
deprecated version coming from rust-vmm/vfio-ioctls repository.
Fixes#3673
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that we introduced a separate method to indicate when the migration
is started, both start_dirty_log() and stop_dirty_log() don't have to
carry an implicit meaning as they can focus entirely on the dirty log
being started or stopped.
For that reason, we can now safely move stop_dirty_log() to the code
section performing non-local migration. It makes only sense to stop
logging dirty pages if this has been started before.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to clearly decouple when the migration is started compared to
when the dirty logging is started, we introduce a new method to the
Migratable trait. This clarifies the semantics as we don't end up using
start_dirty_log() for identifying when the migration has been started.
And similarly, we rely on the already existing complete_migration()
method to know when the migration has been ended.
A bug was reported when running a local migration with a vhost-user-net
device in server mode. The reason was because the migration_started
variable was never set to "true", since the start_dirty_log() function
was never invoked.
Signed-off-by: lizhaoxin1 <Lxiaoyouling@163.com>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since both vhost-user-block and vhost-user-net backend now rely on the
upstream vhost-user-backend crate from rust-vmm, we can simply remove
the local version of the crate from our repository.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of relying on the local version of vhost-user-backend, this
patch allows the block backend implementation to rely on the upstream
version of the crate from rust-vmm.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since virtio-iommu is now part of the upstream Linux kernel, the main
reason for having our own Linux kernel fork is to carry the patches for
virtio-watchdog support.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we moved to the CloudHvX64 target from EDK2, we can't use the same
OVMF firmware for both Cloud Hypervisor and QEMU. The documentation
needed some updates.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of relying on the local version of vhost-user-backend, this
patch allows the block backend implementation to rely on the upstream
version of the crate from rust-vmm.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The Linux Foundation have recommended the use of the this license for
documentation so this change included the license in the documentation
directory.
The file "networking.md" was remove as the copyright holder did not give
their consent.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
While cloud-hypervisor does support receiving the file descriptors of a
tuntap device, advertising the fds structure via the openAPI can lead to
misinterpretations of what can and what should be done.
An unadvertised consumer will think that they could rather just set the
file descriptors there directly, or even pass them as a byte array.
However, the proper way to go in those cases would be actually sending
those via send_msg(), together with the request.
As hacking the openAPI auto-generated code to properly do this is not
*that* trivial, and as doing so during a `create VM` request is not
supported, we better not advertising those.
Please, for more details, also check:
https://github.com/cloud-hypervisor/cloud-hypervisor/pull/3607#issuecomment-1020935523
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that all the preliminary work has been merged to make Cloud
Hypervisor work with the upstream crate virtio-queue from
rust-vmm/vm-virtio repository, we can move the whole codebase and remove
the local copy of the virtio-queue crate.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that multiple file descriptors can be provided along with add-net,
that means we can hotplug a multiqueue macvtap interface to the VM.
The common macvtap test is updated, meaning that both coldplug and
hotplug codepath now use multiqueue.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on the latest code from the micro-http crate, this patch adds the
support for multiple file descriptors to be sent along with the add-net
request. This means we can now hotplug multiqueue network interface to
the VM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This new trait simplifies the address translation of a GuestAddress by
having GuestAddress implementing it.
The three crates virtio-devices, block_util and net_util have been
updated accordingly to rely on this new trait, helping with code
readability and limiting the amount of duplicated code.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Descriptor addresses are now translated from the virtio devices directly
and the definition of the AccessPlatform trait has been moved to
vm-virtio crate. For these reasons, the virtio-queue crate can be
simplified, which makes it very close to the upstream version.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Moving the whole codebase to rely on the AccessPlatform definition from
vm-virtio so that we can fully remove it from virtio-queue crate.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Moving away from the virtio-queue mechanism for descriptor address
translation. Instead, we enable the new mechanism added to every
VirtioDevice implementation, by setting the AccessPlatform trait if one
can be found.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we're trying to move away from the translation happening in the
virtio-queue crate, the device itself is performing the address
translation when needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we're trying to move away from the translation happening in the
virtio-queue crate, the device itself is performing the address
translation when needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we're trying to move away from the translation happening in the
virtio-queue crate, the device itself is performing the address
translation when needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we're trying to move away from the translation happening in the
virtio-queue crate, the device itself is performing the address
translation when needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we're trying to move away from the translation happening in the
virtio-queue crate, the device itself is performing the address
translation when needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Since we're trying to move away from the translation happening in the
virtio-queue crate, the device itself is performing the address
translation when needed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Add a new method set_access_platform() to the VirtioDevice trait in
order to allow an AccessPlatform trait to be setup on any virtio device.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that the virtio-devices crate can take care of the queue addresses
when placed behind a vIOMMU, we can remove the corresponding code.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Upon the enablement of the queue by the guest, we perform a translation
of the descriptor table, the available ring and used ring addresses
prior to enabling the device itself. This only applies to the case where
the device is placed behind a vIOMMU, which is the reason why the
translation is needed. Indeed, the addresses allocated by the guest are
IOVAs which must be translated into GPAs before we can access the queue.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of relying on the virtio-queue crate to store the information
about the MSI-X vectors for each queue, we handle this directly from the
PCI transport layer.
This is the first step in getting closer to the upstream version of
virtio-queue so that we can eventually move fully to the upstream
version.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When freeing memory sometimes glibc will attempt to read
"/proc/sys/vm/overcommit_memory" to find out how it should release the
blocks. This happens sporadically with Cloud Hypervisor but has been
seen in use. It is not necessary to add the read() syscall to the list
as it is already included in the virtio devices common set. Similarly
the vCPU and vmm threads already have both these in the allowed list.
Fixes: #3609
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add integration tests for "pmu=on". It depends on checking if there
is "arm-pmu" item in "/proc/interrupts". As PMU info has not been added
to ACPI, the tests are only for dt.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
If a PMU is enabled in a VM, we also need to initialize the PMU
when the VM is restored. Otherwise, vCPUs cannot be started after
the VM is restored.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
When enable PMU on arm64, ioctl with group KVM_HAS_DEVICE_ATTR will be
blocked by seccomp, add it to authorized list.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Like devicefd, vcpufd also has ability to set/has attribute through kvm
ioctl. These traits are used when enable PMU on arm64, so add it here.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
As it might take more time for the VM to boot (especially under high
load) when using the firmware, let's increase the timeout waiting for
the VM to be reachable.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Relying on helpers for creating the ACPI tables and to add each table to
the HOB, this patch connects the dot to provide the set of ACPI tables
to the TD firmware.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Adding a new method to the TdHob structure so that we can easily insert
a ACPI_TABLE_HOB into the HOB.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The way to create ACPI tables for TDX is different as each table must be
passed through the HOB. This means the XSDT table is not required since
the firmware will take care of creating it. Same for RSDP, this is
firmware responsibility to provide it to the guest.
That's why this patch creates a TDX dedicated function, returning a list
of Sdt objects, which will let the calling code copy the content of each
table through the HOB.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In case of TDX, we don't want to create the ACPI tables the same way we
do for all the other use cases. That's because the ACPI tables don't
need to be written to guest memory at a specific address, instead they
are passed directly through the HOB.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
It's been decided the ACPI tables will be passed to the firmware in a
different way, rather than using TD_VMM_DATA. Since TD_VMM_DATA was
introduced for this purpose, there's no reason to keep it in our
codebase.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Both OVMF and RHF firmwares triggered an error when O_DIRECT was used
because they didn't align the buffers to the block sector size.
In order to prevent regressions, we're adding a new test validating the
VM can properly boot when the OS disk is opened with O_DIRECT and booted
from the rust-hypervisor-fw.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Whenever the backing file of our virtio-block device is opened with
O_DIRECT, there's a requirement about the buffer address and size to be
aligned to the sector size.
We know virtio-block requests are sector aligned in terms of size, but
we must still check if the buffer address is. In case it's not, we
create an intermediate buffer that will be passed through the system
call. In case of a write operation, the content of the non-aligned
buffer must be copied beforehand, and in case of a read operation, the
content of the aligned buffer must be copied to the non-aligned one
after the operation has been completed.
Fixes#3587
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This table is listed as required in the ARM Base Boot Requirements
document. The particular need arises to make the serial debugging of
Windows guest functional.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
If the guest hasn't initialised a PV clock then the KVM_KVMCLOCK_CTRL
ioctl will return -EINVAL. Therefore if running in the firmware or an OS
that doesn't use the PV clock then we should ignore that error
Tested by migrating a VM that has not yet booted into the Linux kernel
(just in firmware) by specifying no disk image:
e.g. target/debug/cloud-hypervisor --kernel ~/workloads/hypervisor-fw --api-socket /tmp/api --serial tty --console off
Fixes: #3586
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
From 15358ef79d: Device Mapper Multipath
config can avoid systemd errors related to Device Mapper multipath while
guest booting.
From 46672c384c: CONFIG_NVME_MULTIPATH is
needed to fix the observed guest hanging issue cased by systemd crash
while booting.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The SPDK-NVMe is needed for the integration test for vfio_user.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
AArch64 does not use IOBAR, and current code of panics the whole VMM if
we need to allocate the IOBAR.
This commit checks if IOBAR is enabled before the arch conditional code
of IOBAR allocation and if the IOBAR is not enabled, we can just skip
the IOBAR allocation and do nothing.
Fixes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/3479
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Ensure all pending virtio activations (as triggered by MMIO write on the
vCPU threads leading to a barrier wait) are completed before pausing the
vCPUs as otherwise there will a deadlock with the VMM waiting for the
vCPU to acknowledge it's pause and the vCPU waiting for the VMM to
activate the device and release the barrier.
Fixes: #3585
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The code can be written in a better form and the clippy warning
suppression can be dropped.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
These are the leftovers from the commit 8155be2:
arch: aarch64: vm_memory is not required when configuring vcpu
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Update documentation and CI to rely on the new CLOUDHV.fd firmware built
from the newly introduced target CloudHvX64.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
By pinning the OVMF version, we will be able to update the EDK2 fork
with a new version without potentially breaking our Cloud Hypervisor CI.
Once the new version is ready on the EDK2 fork, we'll be able to update
Cloud Hypervisor codebase, replacing the fixed version with the latest,
as well as replacing OVMF.fd with CLOUDHV.fd. This is because we'll
start building from the new target CloudHvX64.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When in local migration mode send the FDs for the guest memory over the
socket along with the slot that the FD is associated with. This removes
the requirement for copying the guest RAM and gives significantly faster
live migration performance (of the order of 3s to 60ms).
Fixes: #3566
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Create the VM using the FDs (wrapped in Files) that have been received
during the migration process.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If this FD (wrapped in a File) is supplied when the RAM region is being
created use that over creating a new one.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This function is used to open an FD (wrapped in a File) that points to
guest memory from memfd_create() or backed on the filesystem.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Currently cloud-hypervisor supports hotplug of CPUs, passthrough devices, virtio devices and memory resizing,
but in hotplug.md it still mentiones: "Currently Cloud Hypervisor only supports hot plugging of CPU devices."
We need to remove the incorrect information from hotplug.md to reflect the true situation.
Fixes: #3504
Signed-off-by: Liang Zhou <zhoul110@chinatelecom.cn>
Drop the unused parameter throughout the code base.
Also take the chance to drop a needless clone.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
error: unneeded late initalization
Error: --> devices/src/legacy/rtc_pl031.rs:294:9
|
294 | let v;
| ^^^^^^
|
= note: `-D clippy::needless-late-init` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `v` here
|
297 | let v = if (AMBA_ID_LOW..AMBA_ID_HIGH).contains(&offset) {
| +++++++
help: remove the assignments from the branches
|
299 ~ u32::from(PL031_ID[index])
300 | } else {
301 ~ match offset {
302 | RTCDR => self.get_time(),
303 | RTCMR => {
304 | // Even though we are not implementing RTC alarm we return the last value
...
help: add a semicolon after the `if` expression
|
317 | };
| +
error: unneeded late initalization
Error: --> devices/src/legacy/uart_pl011.rs:297:9
|
297 | let v;
| ^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `v` here
|
299 | let v = if (AMBA_ID_LOW..AMBA_ID_HIGH).contains(&(offset >> 2)) {
| +++++++
help: remove the assignments from the branches
|
301 ~ u32::from(PL011_ID[index])
302 | } else {
303 ~ match offset >> 2 {
304 | UARTDR => {
305 | let c: u32;
306 | let r: u32;
...
help: add a semicolon after the `if` expression
|
340 | };
| +
error: unneeded late initalization
Error: --> devices/src/legacy/uart_pl011.rs:305:21
|
305 | let c: u32;
| ^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `c` here
|
309 | let c: u32 = self.read_fifo.pop_front().unwrap_or_default().into();
| ~~~~~~~~~~
error: unneeded late initalization
Error: --> devices/src/legacy/uart_pl011.rs:306:21
|
306 | let r: u32;
| ^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `r` here
|
320 | let r: u32 = c;
| ~~~~~~~~~~
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: this boolean expression can be simplified
--> tests/integration.rs:3755:33
|
3755 | assert!(!(empty > 5), "No login on pty");
| ^^^^^^^^^^^^ help: try: `empty <= 5`
|
= note: `-D clippy::nonminimal-bool` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonminimal_bool
error: unneeded late initalization
--> tests/integration.rs:7619:13
|
7619 | let mut success;
| ^^^^^^^^^^^^^^^^
|
= note: `-D clippy::needless-late-init` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `success` here
|
7621 | let mut success = if let Some(status) = send_migration
| +++++++++++++++++
help: remove the assignments from the branches
|
7625 ~ status.success()
7626 | } else {
7627 ~ false
|
help: add a semicolon after the `if` expression
|
7628 | };
| +
error: unneeded late initalization
--> tests/integration.rs:7838:13
|
7838 | let mut success;
| ^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `success` here
|
7840 | let mut success = if let Some(status) = send_migration
| +++++++++++++++++
help: remove the assignments from the branches
|
7844 ~ status.success()
7845 | } else {
7846 ~ false
|
help: add a semicolon after the `if` expression
|
7847 | };
| +
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
error: unneeded late initalization
Error: --> arch/src/aarch64/gic/gicv3_its.rs:127:9
|
127 | let attr: u64;
| ^^^^^^^^^^^^^^
|
= note: `-D clippy::needless-late-init` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `attr` here
|
128 | let attr: u64 = if save {
| +++++++++++++++
help: remove the assignments from the branches
|
129 ~ u64::from(kvm_bindings::KVM_DEV_ARM_ITS_SAVE_TABLES)
130 | } else {
131 ~ u64::from(kvm_bindings::KVM_DEV_ARM_ITS_RESTORE_TABLES)
|
help: add a semicolon after the `if` expression
|
132 | };
| +
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We've been currently using "fd" as the field name, but it should be
called "fds" since 6664e5a6e7 introduced
the name change on the structure field.
Fixes: #3560
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Passing no boot related parameters (e.g. no --kernel) is used for e.g.
receiving a live migration or an API based boot.
marvin:~/src/cloud-hypervisor (2022-01-11-live-migration-with-fds *)$ target/debug/cloud-hypervisor --api-socket /tmp/api2
thread 'main' panicked at '`tdx` is not a name of an argument or a group.
Make sure you're using the name of the argument itself and not the name of short or long flags.', /home/rob/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-3.0.6/src/parse/matches/arg_matches.rs:598:14
stack backtrace:
0: rust_begin_unwind
at /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/std/src/panicking.rs:498:5
1: core::panicking::panic_fmt
at /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/panicking.rs:107:14
2: clap::parse::matches::arg_matches::ArgMatches::get_arg
at /home/rob/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-3.0.6/src/parse/matches/arg_matches.rs:1052:17
3: clap::parse::matches::arg_matches::ArgMatches::is_present
at /home/rob/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-3.0.6/src/parse/matches/arg_matches.rs:598:9
4: cloud_hypervisor::start_vmm
at ./src/main.rs:530:46
5: cloud_hypervisor::main
at ./src/main.rs:566:27
6: core::ops::function::FnOnce::call_once
at /rustc/7d6f948173ccb18822bab13d548c65632db5f0aa/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
`tap` has its default value set to `None`, but in the openapi yaml file
we've been setting it to `""`.
When using this code on the Kata Containers side we'd be hit by a non
expected behaviour of cloud-hypervisor, as even when using a different
method to initialise the `tuntap` device the code would be treated as if
using `--net tap` (which is a valid use-case).
Related: #3554
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When enabling the `mshv` feature, we skip quite some tests and
hence have known dead-code. This annotation silences dead-code
related warnings for our quality workflow to pass.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Given integration tests don't have special annotations, all our
integration test scripts do not need to carry special flags when running
`cargo test`. This commit also removed the "test::" prefix for
pattern-matching different set of integration tests.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Given integration tests are placed in a dedicate directory, they don't
need annotations (e.g. `#[cfg(integration_test)]` and `#[cfg(test)]`) or
defining `test mod` to exclude themselves from the common compilation
process.
Signed-off-by: Bo Chen <chen.bo@intel.com>
warning: unneeded late initalization
--> src/main.rs:134:5
|
134 | let mut app: App;
| ^^^^^^^^^^^^^^^^^
|
= note: `#[warn(clippy::needless_late_init)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `app` here
|
138 | let mut app: App = App::new("cloud-hypervisor")
| ~~~~~~~~~~~~~~~~
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: unneeded late initalization
--> vmm/src/acpi.rs:525:5
|
525 | let mut prev_tbl_len: u64;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(clippy::needless_late_init)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `prev_tbl_len` here
|
552 | let mut prev_tbl_len: u64 = madt.len() as u64;
| ~~~~~~~~~~~~~~~~~~~~~~~~~
warning: unneeded late initalization
--> vmm/src/acpi.rs:526:5
|
526 | let mut prev_tbl_off: GuestAddress;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `prev_tbl_off` here
|
553 | let mut prev_tbl_off: GuestAddress = madt_offset;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
warning: unneeded late initalization
--> virtio-queue/src/chain.rs:381:13
|
381 | let desc: Descriptor;
| ^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(clippy::needless_late_init)]` on by default
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_late_init
help: declare `desc` here
|
382 | let desc: Descriptor = if i < 3 {
| ++++++++++++++++++++++
help: remove the assignments from the branches
|
383 ~ Descriptor::new(0x1000 * i as u64, 0x1000, VIRTQ_DESC_F_NEXT, i + 1)
384 | } else {
385 ~ Descriptor::new(0x1000 * i as u64, 0x1000, 0, 0)
|
help: add a semicolon after the `if` expression
|
386 | };
| +
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Current command to run cloud-hypervisor binary has
wrong --rng argument in the README file.
This commit fixes all the references of --rng argument
by removing the --rng option to use default.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This crate contains up to date definition of the Queue, AvailIter,
DescriptorChain and Descriptor structures forked from the upstream
crate rust-vmm/vm-virtio 27b18af01ee2d9564626e084a758a2b496d2c618.
The following patches have been applied on top of this base in order to
make it work correctly with Cloud Hypervisor requirements:
- Add MSI vector field to the Queue
In order to help with MSI/MSI-X support, it is convenient to store the
value of the interrupt vector inside the Queue directly.
- Handle address translations
For devices with access to data in memory being translated, we add to
the Queue the ability to translate the address stored in the
descriptor.
It is very helpful as it performs the translation right after the
untranslated address is read from memory, avoiding any errors from
happening from the consumer's crate perspective. It also allows the
consumer to reduce greatly the amount of duplicated code for applying
the translation in many different places.
- Add helpers for Queue structure
They are meant to help crate's consumers getting/setting information
about the Queue.
These patches can be found on the 'ch' branch from the Cloud Hypervisor
fork: https://github.com/cloud-hypervisor/vm-virtio.git
This patch takes care of updating the Cloud Hypervisor code in
virtio-devices and vm-virtio to build correctly with the latest version
of virtio-queue.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This crate was used in the integration tests to allow the tests to
continue and clean up after a failure. This isn't necessary in the unit
tests and adds a large build dependency chain including an unmaintained
crate.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This updated tree now includes a fix for virtio-net regression that was
present in 5.14:
commit 732b74d647048668f0f8dc0c848f0746c69e2e2f
Author: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Date: Sat Oct 9 05:17:53 2021 -0400
virtio-net: fix for skb_over_panic inside big mode
commit 126285651b7f ("Merge ra.kernel.org:/pub/scm/linux/kernel/git/netdev/net")
accidentally reverted the effect of
commit 1a8024239da ("virtio-net: fix for skb_over_panic inside big mode")
on drivers/net/virtio_net.c
As a result, users of crosvm (which is using large packet mode)
are experiencing crashes with 5.14-rc1 and above that do not
occur with 5.13.
Crash trace:
[ 61.346677] skbuff: skb_over_panic: text:ffffffff881ae2c7 len:3762 put:3762 head:ffff8a5ec8c22000 data:ffff8a5ec8c22010 tail:0xec2 end:0xec0 dev:<NULL>
[ 61.369192] kernel BUG at net/core/skbuff.c:111!
[ 61.372840] invalid opcode: 0000 [#1] SMP PTI
[ 61.374892] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 5.14.0-rc1 linux-v5.14-rc1-for-mesa-ci.tar.bz2 #1
[ 61.376450] Hardware name: ChromiumOS crosvm, BIOS 0
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When forwarding an epoll event from the unix muxer to the
targeted connection event handler, the eventset the connection
registered is forwarded instead of the actual epoll
operation (IN/OUT).
For example, if the connection was registered for EPOLLIN,
and receives an EPOLLOUT, the connection will actually handle
an EPOLLOUT.
This is the root cause of previous bug, which caused the
introduction of some workarounds (i.e: handling ewouldblock
when reading after receiving EPOLLIN, which should never happen).
When matching the connection, we retrieve and use the evset of
the connection instead of the one passed as a parameter.
The compiler does not complain for an unused variable because
it was first logged in a debug! statement.
This is an unfortunate naming mistake that caused a lot of problems.
Fixes#3497
Signed-off-by: Eduard Kyvenko <eduard.kyvenko@gmail.com>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
If the underlying kernel is old PTY resize is disabled and this is
represented by the use of None in the provided Option<File> type. In the
virtio-console PTY path don't blindly unwrap() the value that will be
preserved across a reboot.
Fixes: #3496
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The test test_virtio_block_topology is flaky on aarch64, let's disable
it until we find the right way to fix it.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There is a corner case of using the Cloud Hypervisor on AArch64:
If the VM is started on a device where RAM is limited, and if the user
allocates nearly as much memory for the guest as is still free on the
host, we need to enable the swap memory.
This commit documented this corner case with explanation.
Also, this commit corrects the hardware requirement of the GIC interrupt
controller for running the Cloud Hypervisor on AArch64, accroding to
[1].
Fixes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/3419
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
[1] https://www.kernel.org/doc/html/latest/virt/kvm/devices/arm-vgic-its.html
It's important to maintain the ability to run in a non-TDX environment
a Cloud Hypervisor binary with the 'tdx' feature enabled.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This test relies on using losetup with a block size to create a block
device from a file that has a specific block size for the topology
detection code to pick up and passthrough to the guest.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If the disk is backed by a block device on the host a non-default
topology will be available and that topology can be advertised by virtio
block.
Fixes: #3262
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
For simplicity this trait implements a default version that has a
topology with 512 byte (i.e. sector) recommended sizes.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Setting up the SIGWINCH handler requires at least Linux 5.7. However
this functionality is not required for basic PTY operation.
Fixes: #3456
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This reverts commit 58d25b3ccc.
This change introduced a regression when running iperf with the guest
running as the server:
marvin:~/src/cloud-hypervisor ((58d25b3c...))$ iperf -c 192.168.249.2
------------------------------------------------------------
Client connecting to 192.168.249.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 1] local 192.168.249.1 port 47078 connected with 192.168.249.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 1] 0.00-10.40 sec 14.0 MBytes 11.3 Mbits/sec
marvin:~/src/cloud-hypervisor ((58d25b3c...))$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte (default)
------------------------------------------------------------
[ 1] local 192.168.249.1 port 5001 connected with 192.168.249.2 port 42866
[ ID] Interval Transfer Bandwidth
[ 1] 0.00-10.01 sec 51.2 GBytes 44.0 Gbits/sec
Fixes: #3450
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The sendmsg() syscall is limited in the number of fds it can handle.
This number matches that used by the vfio-user library and is
conservative (since we've seen it work with 64 fds.)
Fixes: #3401
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This time we use the Rust Hypervisor Firmware for test_vfio_user() in
order to fix the systemd issues we've seen so far.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Advertise the PCI MMIO config spaces here so that the MMIO config space
is correctly recognised.
Tested by: --platform num_pci_segments=1 or 16 hotplug NVMe vfio-user device
works correctly with hypervisor-fw & OVMF and direct kernel boot.
Fixes: #3432
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When restoring replace the internal value of the device tree rather than
replacing the Arc<Mutex<DeviceTree>> itself. This is fixes an issue
where the AddressManager has a copy of the the original
Arc<Mutex<DeviceTree>> from when the DeviceManager was created. The
original restore path only replaced the DeviceManager's version of the
Arc<Mutex<DeviceTree>>. Instead replace the contents of the
Arc<Mutex<DeviceTree>> so all users see the updated version.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The observation is that the code in question was used to bridge
synchronized and asynchronized code.
We can group the functions for that purpose under an adaptor trait. To
limit the scope of locking, the users of the trait are required to
implement a method to return a MutexGuard for the underlying file.
This then allows us to use concrete types (QcowFile and Vhdx) in code,
which is easier to read than a bunch of traits.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Saving the KVM clock and restoring it is key for correct behaviour of
the VM when doing snapshot/restore or live migration. The clock is
restored to the KVM state as part of the Vm::resume() method prior to
that it must be extracted from the state object and stored for later use
by this method. This change simplifies the extraction and storage part
so that it is done in the same way for both snapshot/restore and live
migration.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This commit enhances the integration test for multiple PCI segments
by:
(1) Enables the `test_virtio_fs_multi_segment` on AArch64.
(2) Adds a new integration test case for both x86_64 and AArch64 using
the direct kernel boot to test virtio-disk multiple PCI segments.
The test case does:
- Start a VM using direct kernel boot with 16 PCI segments and assign
the last PCI segment with a virtio-disk device.
- Check if the number of PCI host bridges equals to 16 after VM boots.
- Mount the virtio-disk device on the last PCI segment to the rootfs
and write/read data to the virtio-disk device.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Each PCI device under a root complex is uniquely identified by its
Requester ID (AKA RID). A Requester ID is a triplet of a Bus number,
Device number, and Function number.
MSIs may be distinguished in part through the use of sideband data
accompanying writes. In the case of PCI devices, this sideband data
may be derived from the Requester ID. A mechanism is required to
associate a device with both the MSI controllers it can address,
and the sideband data that will be associated with its writes to
those controllers.
This commit adds the `msi-map` property for PCI nodes, therefore
creating MSI mapping for each PCI device.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
This commit rewrites the `create_pci_node` in the FDT creator to
create multiple PCI nodes based on the vector of `PciSpaceInfo`,
and each PCI node in FDT reflects a PCI segment.
- The PCI MMIO config space, 32 bits PCI device space and 64 bits
PCI device space is re-calculated based on the `PciSpaceInfo` for
each PCI segment.
- A new FDT property `linux,pci-domain` is added.
- The virtio-iommu node is only created for the first PCI segment.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The constant `PCI_MMIO_CONFIG_SIZE` defined in `vmm/pci_segment.rs`
describes the MMIO configuation size for each PCI segment. However,
this name conflicts with the `PCI_MMCONFIG_SIZE` defined in `layout.rs`
in the `arch` crate, which describes the memory size of the PCI MMIO
configuration region.
Therefore, this commit renames the `PCI_MMIO_CONFIG_SIZE` to
`PCI_MMIO_CONFIG_SIZE_PER_SEGMENT` and moves this constant from `vmm`
crate to `arch` crate.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Currently, a tuple containing PCI space start address and PCI space
size is used to pass the PCI space information to the FDT creator.
In order to support the multiple PCI segment for FDT, more information
such as the PCI segment ID should be passed to the FDT creator. If we
still use a tuple to store these information, the code flexibility and
readablity will be harmed.
To address this issue, this commit replaces the tuple containing the
PCI space information to a structure `PciSpaceInfo` and uses a vector
of `PciSpaceInfo` to store PCI space information for each segment, so
that multiple PCI segment information can be passed to the FDT together.
Note that the scope of this commit will only contain the refactor of
original code, the actual multiple PCI segments support will be in
following series, and for now `--platform num_pci_segments` should only
be 1.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Previously mutex (semaphore) and file were separated. The code needed to
create artificial scopes to use mutex to protect file.
Rewrite the code to be idiomatic. The file itself is turned into a trait
object and placed inside the mutex. This requires providing a new
ReadWriteSeekFile trait to unify all helper functions.
The rewrite further simplified vhdx_sync code. The original code
contained two mutex'es for no apparent reason.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Extending the test_simple_launch() integration test to validate Cloud
Hypervisor boots correctly with both rust-hypervisor-fw and OVMF on
x86_64 platforms.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Bumping the OVMF binary version along with UEFI documentation to
reflect the latest set of patches on top of tianocore/edk2 'master'
branch, which can be found on the Cloud Hypervisor fork on 'ch' branch.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to avoid the identity map region to conflict with a possible
firmware being placed in the last 4MiB of the 4GiB range, we must set
the address to a chosen location. And it makes the most sense to have
this region placed right after the TSS region.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extending the Vm trait with set_identity_map_address() in order to
expose this ioctl to the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Commit ac25172176 bumps the rust
version of virtiofsd named `virtiofsd-rs`, which causes a warning
```
warning: use of deprecated parameter '--socket':
Please use the '--socket-path' option instead.
```
This commit updates the cmdline parameter accordingly.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Place the 3 page TSS at an explicit location in the 32-bit address space
to avoid conflicting with the loaded raw firmware.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
sh 'scripts/dev_cli.sh tests --metrics -- -- --report-file /root/workloads/metrics.json'
}
}
stage ('Upload metrics report') {
steps {
sh 'curl -X PUT https://cloud-hypervisor-metrics.azurewebsites.net/api/publishmetrics -H "x-functions-key: $METRICS_PUBLISH_KEY" -T ~/workloads/metrics.json'
}
}
}
}
}
}
}
@@ -278,3 +383,22 @@ def cancelPreviousBuilds() {
}
}
}
def installAzureCli() {
sh "sudo apt install -y ca-certificates curl apt-transport-https lsb-release gnupg"
sh "curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null"
sh "echo \"deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ focal main\" | sudo tee /etc/apt/sources.list.d/azure-cli.list"
- [Containerized builds and tests](#containerized-builds-and-tests)
@@ -26,12 +27,19 @@
# 1. What is Cloud Hypervisor?
Cloud Hypervisor is an open source Virtual Machine Monitor (VMM) that runs on top of [KVM](https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt) and the MSHV hypervisors .
Cloud Hypervisor is an open source Virtual Machine Monitor (VMM) that runs on
top of [KVM](https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt)
hypervisor and Microsoft Hypervisor (MSHV).
The project focuses on exclusively running modern, cloud workloads, on top of a limited set of hardware architectures and platforms.
Cloud workloads refers to those that are usually run by customers inside a cloud provider. For our purposes this means modern operating systems with most I/O handled by paravirtualised devices (i.e. virtio), no requirement for legacy devices, and 64-bit CPUs.
The project focuses on exclusively running modern, cloud workloads, on top of
a limited set of hardware architectures and platforms. Cloud workloads refers
to those that are usually run by customers inside a cloud provider. For our
purposes this means modern operating systems with most I/O handled by
paravirtualised devices (i.e. virtio), no requirement for legacy devices, and
64-bit CPUs.
Cloud Hypervisor is implemented in [Rust](https://www.rust-lang.org/) and is based on the [rust-vmm](https://github.com/rust-vmm) crates.
Cloud Hypervisor is implemented in [Rust](https://www.rust-lang.org/) and is
based on the [rust-vmm](https://github.com/rust-vmm) crates.
## Objectives
@@ -50,7 +58,9 @@ Cloud Hypervisor is implemented in [Rust](https://www.rust-lang.org/) and is bas
### Architectures
Cloud Hypervisor supports the `x86-64` and `AArch64` architectures. There are some small differences in functionality between the two architectures (see [#1125](https://github.com/cloud-hypervisor/cloud-hypervisor/issues/1125)).
Cloud Hypervisor supports the `x86-64` and `AArch64` architectures. There are
some small differences in functionality between the two architectures
(see [#1125](https://github.com/cloud-hypervisor/cloud-hypervisor/issues/1125)).
### Guest OS
@@ -73,9 +83,9 @@ $ mkdir $CLOUDH
## Install prerequisites
You need to install some prerequisite packages in order to build and test Cloud Hypervisor.
Here, all the steps are based on Ubuntu, for other Linux distributions please replace the
package manager and package name.
You need to install some prerequisite packages in order to build and test Cloud
Hypervisor. Here, all the steps are based on Ubuntu, for other Linux
distributions please replace the package manager and package name.
@@ -178,7 +190,10 @@ Multiple arguments can be given to the `--disk` parameter.
#### Building your kernel
Cloud Hypervisor also supports direct kernel boot into a `vmlinux` ELF kernel. In order to support virtio-iommu we have our own development branch. You are of course able to use your own kernel but these instructions will continue with the version that we develop and test against.
Cloud Hypervisor also supports direct kernel boot into a `vmlinux` ELF kernel.
In order to support virtio-watchdog we have our own development branch. You are
of course able to use your own kernel but these instructions will continue with
Direct kernel boot to userspace should work with a rootfs from most distributions.
Direct kernel boot to userspace should work with a rootfs from most
distributions.
## Hot Plug
Cloud Hypervisor supports hotplug of CPUs, passthrough devices (VFIO),`virtio-{net,block,pmem,fs,vsock}` and memory resizing. This [document](docs/hotplug.md) details how to add devices to
a running VM.
Cloud Hypervisor supports hotplug of CPUs, passthrough devices (VFIO),
`virtio-{net,block,pmem,fs,vsock}` and memory resizing. This
[document](docs/hotplug.md) details how to add devices to a running VM.
## Device Model
Details of the device model can be found in this [documentation](docs/device_model.md).
Details of the device model can be found in this
[documentation](docs/device_model.md).
## TODO
We are not tracking the Cloud Hypervisor TODO list from a specific git tracked file but through
In the context of virtualization, it is always convenient to be able to share a directory from the host with the guest.
In the context of virtualization, it is always convenient to be able to share a
directory from the host with the guest.
__virtio-fs__, also known as __vhost-user-fs__ is a virtual device defined by the VIRTIO specification which allows any VMM to perform filesystem sharing.
__virtio-fs__, also known as __vhost-user-fs__ is a virtual device defined by
the VIRTIO specification which allows any VMM to perform filesystem sharing.
## Pre-requisites
### The daemon
This virtual device relies on the _vhost-user_ protocol, which assumes the backend (device emulation) is handled by a dedicated process running on the host. This daemon is called __virtiofsd__ and needs to be present on the host.
This virtual device relies on the _vhost-user_ protocol, which assumes the
backend (device emulation) is handled by a dedicated process running on the
host. This daemon is called __virtiofsd__ and needs to be present on the host.
The `cache=none` option should be the default when using `virtiofsd` with the __cloud-hypervisor__ VMM. This prevents from using the guest page cache, which reduces the memory footprint of the guest. When running multiple virtual machines on the same host, this will let the host deal with page cache, which will increase the density of virtual machines which can be launched.
The `cache=never` option is the default when using `virtiofsd` with
Cloud Hypervisor. This prevents from using the host page cache, reducing the
overall footprint on host memory. This increases the maximum density of virtual
machines that can be launched on a single host.
The `cache=always` option will allow for the guest page cache to be used, which will increase the memory footprint of the guest. This option should be used only for specific use cases where a single VM is going to be running on a host.
The `cache=always` option will allow the host page cache to be used, which can
result in better performance for the guest's workload at the cost of increasing
the footprint on host memory.
### Kernel support
Modern Linux kernels starting (at least v5.10) have support for virtio-fs. Use
of older kernels, with additional patches, are not supported.
Modern Linux kernels (at least v5.10) have support for virtio-fs. Use of older
kernels, with additional patches, are not supported.
## How to share directories with cloud-hypervisor
### Start the VM
Once the daemon is running, the option `--fs` from __cloud-hypervisor__ needs to be used.
Direct kernel boot is the preferred option, but we can boot from an EFI cloud image if it contains a recent enough kernel.
Once the daemon is running, the option `--fs` from Cloud Hypervisor needs
to be used.
Because _vhost-user_ expects a dedicated process (__virtiofsd__ in this case) to be able to access the guest RAM to communicate through the _virtqueues_ with the driver running in the guest, `--memory` option needs to be slightly modified. It must specify `shared=on` to share the memory pages so that an external process can access them.
Both direct kernel boot and EFI firmware can be used to boot a VM with
virtio-fs, given that the cloud image contains a recent enough kernel.
Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your system, here is the __cloud-hypervisor__ command you need to run:
Correct functioning of `--fs` requires `--memory shared=on` to facilitate
interprocess memory sharing.
Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your
system, here is the Cloud Hypervisor command you need to run:
```bash
./cloud-hypervisor \
--cpus boot=1\
@@ -60,26 +74,20 @@ Assuming you have `focal-server-cloudimg-amd64.raw` and `vmlinux` on your system
By default, DAX is enabled with a cache window of 8GiB. You can specify a custom size (let's say 4GiB for this example) for the cache by explicitly setting DAX and the cache size:
In case you don't want to use a shared window of cache to pass the shared files content, this means you will have to explicitly disable DAX with `dax=off`. Note that in this case, the `cache_size` parameter will be ignored.
The last step is to mount the shared directory inside the guest, using the `virtiofs` filesystem type.
The last step is to mount the shared directory inside the guest, using the
`virtiofs` filesystem type.
```bash
mkdir mount_dir
mount -t virtiofs -o dax myfs mount_dir/
mount -t virtiofs myfs mount_dir/
```
The `tag` needs to be consistent with what has been provided through the __cloud-hypervisor__ command line, which happens to be `myfs` in this example.
The `-o dax` option must be removed in case the shared cache region is not enabled from the VMM.
The `tag` needs to be consistent with what has been provided through the
Cloud Hypervisor command line, which happens to be `myfs` in this example.
## DAX feature
Given the DAX feature is not stable yet from a daemon standpoint, it is not
cloud-hypervisor can emulate one or more virtual network interfaces, represented at the hypervisor host by [tap devices](https://www.kernel.org/doc/Documentation/networking/tuntap.txt). This guide briefly describes, in a manual and distribution neutral way, how to setup and use networking with cloud-hypervisor.
## Multiple queue support for net devices
While multiple vcpus defined for guest, to gain the benefit of vcpu scalable to improve performance, it suggests to define multiple queue pairs for net devices, one Tx/Rx queue pair per one vcpu, that means the number of queue pairs at least is equal to the vcpu count. In that case, after virtnet driver set cpu affinity for virtqueues in guest kernel, vcpus could handle interrupt from different virtqueue pairs in parallel.
It will gain better performance for guest that has multiple queues defined for net devices while it has multiple net sessions running in userspace.
To enable multiple queue support in cloud-hypervisor, multiple queue pairs will be defined, while multiple tap fds will be opened for the same tap device, it will also have multiple threads started, each thread will monitor and handle the events from each virtqueue pairs and the associated tap fd.
Note:
- Currently, it does not support to use ethtool to change the combined queue numbers in guest.
- Multiple queue is enabled for vhost-user-net backend in cloud-hypervisor, however, multiple thread is not added to handle mq, thus, the performance for vhost-user-net backend is not supposed to be improved. The multiple thread will be added for backend later.
- Performance test for vhost-user-net will be covered once vhost-user-net backend has multiple thread supported.
- Performance test for virtio-net is done by comparing 2 queue pairs with 1 queue pairs, that to run 2 iperf3 sessions in the same test environments, throughput is improved about 37%.
## Start cloud-hypervisor with net devices
Use one `--net` command-line argument from cloud-hypervisor to specify the emulation of one or more virtual NIC's. The example below instructs cloud-hypervisor to emulate for instance 2 virtual NIC's:
num_queues is the total number of tx and rx queues, the default value is 2, and it could be increased by multiples of 2. Additionally, num_queues is suggested to be as 2 times of vcpu count. The default value for queue_size is 256.
If the tap device is pre-created on host before guest boot up. To use multiple queue support for net device in guest, the tap device should be opened like this from host.
```bash
[root@localhost ~]# ip tuntap add name ich0 mode tap multi_queue
```
And the `--net` device should specify support for multiple queues. `num_queues` must be a multiple of 2 starting at least from 4 since multiple queues really means multiple queue pairs. We need at least 2 pairs for this configuration to be correct:
Different networking models can be used to provide external connectivity. In this example we will
use 2 linux bridges emulating 2 different networks. The integration bridge (ich-int) in this example will also be used
for external connectivity.
Create the bridges and connect the cloud-hypervisor tap devices to the bridges:
```bash
root@host:~# brctl addbr ich-int
root@host:~# brctl addbr ich-dpl
root@host:~# ip link set up ich-int
root@host:~# ip link set up ich-dpl
root@host:~# brctl addif ich-int ich0
root@host:~# brctl addif ich-dpl ich1
root@host:~# brctl show
bridge name bridge id STP enabled interfaces
ich-dpl 8000.067afc1b9a67 no ich1
ich-int 8000.725412ffce6f no ich0
```
This completes the layer 2 wiring: The cloud-hypervisor is now connected to the hypervisor host via the 2 linux bridges.
## IP (Layer 3) provisioning
### Hypervisor host
On the hypervisor host add the network gateway IP address of each network to the 2 linux bridges:
```bash
root@host:~# ip addr add 192.168.4.1/24 dev ich-int
root@host:~# ip addr add 10.0.1.1/24 dev ich-dpl
```
The routing table of the hypervisor host should now also have corresponding routing entries:
```bash
root@host:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.178.1 0.0.0.0 UG 60000 wlan1
10.0.1.0 0.0.0.0 255.255.255.0 U 000 ich-dpl
192.168.4.0 0.0.0.0 255.255.255.0 U 000 ich-int
192.168.178.0 0.0.0.0 255.255.255.0 U 60000 wlan1
```
### Virtual Machine
Within the virtual machine set the vNIC's to up state and provision the corresponding IP addresses on the 2 vNIC's. The steps outlined below use the ip command as an example. Alternative distribution specific procedures can also apply.
```bash
root@guest:~# ip link set up enp0s2
root@guest:~# ip link set up enp0s3
root@guest:~# ip addr add 192.168.4.2/24 dev enp0s2
root@guest:~# ip addr add 10.0.1.2/24 dev enp0s3
```
IP connectivity between the virtual machine and the hypervisor-host can be verified by sending
ICMP requests to the hypervisor-host for the gateway IP address from within the virtual machine:
```bash
root@guest:~# ping 192.168.4.1
PING 192.168.4.1 (192.168.4.1) 56(84) bytes of data.
64 bytes from 192.168.4.1: icmp_seq=1ttl=64time=0.456 ms
64 bytes from 192.168.4.1: icmp_seq=2ttl=64time=0.226 ms
root@guest:~# ping 10.0.1.1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1ttl=64time=0.449 ms
64 bytes from 10.0.1.1: icmp_seq=2ttl=64time=0.393 ms
```
The connection can now be used for instance to log into the virtual machine with
ssh under the precondition that the machine has an ssh daemon provisioned:
```bash
root@host:~# ssh root@192.168.4.2
The authenticity of host '192.168.4.2 (192.168.4.2)' can't be established.
ECDSA key fingerprint is SHA256:qNAUmTtDMW9pNuZARkpLQhfw+Yc1tqUDBrQp7aZGSjw.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.4.2' (ECDSA) to the list of known hosts.
root@192.168.4.2's password:
Linux cloud-hypervisor 5.2.0 #2 SMP Thu Jul 11 08:08:16 CEST 2019 x86_64
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Jul 12 13:27:56 2019 from 192.168.4.1
root@guest:~#
```
## Internet connectivity
To enable internet connectivity a default gw and a nameserver has to be set within
the virtual machine:
```bash
root@guest:~# ip route add default via 192.168.4.1
root@guest:~# cat /etc/resolv.conf
options timeout:2
domain vallis.nl
search vallis.nl
nameserver 192.168.178.1
```
make sure that the default gateway of the hypervisor host (in this example host 192.168.178.1 which is an adsl router) has an entry in the routing table for the 192.168.4.0/24 network otherwise IP connectivity will not work.
After the successful build, the resulting firmware binaries are available under `Build/OvmfCh/DEBUG_GCC5/FV` underneath the edk2 checkout.
After the successful build, the resulting firmware binaries are available under `Build/CloudHvX64/DEBUG_GCC5/FV` underneath the edk2 checkout.
## Using OVMF Binaries
Any UEFI capable image can be booted using the Cloud Hypervisor specific firmware. Windows guests under Cloud Hypervisor only support UEFI boot, therefore OVMF is mandatory there.
To make Cloud Hypervisor use UEFI boot, pass the `OVMF.fd` file path as an argument to the `--kernel` option. The firmware file will be opened in read only mode.
The same firmware can be used with Cloud Hypervisor or with QEMU. This is particularly useful if using QEMU for the preparation phase.
## Building UEFI Firmware with Compatibility Support Module (CSM)
CSM is a module that allows to boot legacy operating systems using the OVMF firmware. OVMF can embed a CSM build of SeaBIOS. To build the SeaBIOS with CSM support, add `CONFIG_CSM=y` to `.config` before the build. The outcome `out/Csm16.bin` is to be moved into `OvmfPkg/Csm/Csm16/Csm16.bin` before OVMF is built. Then, the OVMF build will have to be passed the `-D CSM_ENABLE` option in order to generate a legacy aware UEFI firmware. At the current stage, all the necessary patches are included in the Cloud Hypervisor specific [SeaBIOS branch](https://github.com/cloud-hypervisor/seabios/tree/ch). Taking into account the previous instructions, the modified command sequence to compile an OVMF binary with CSM support is the following one:
Please note, that the CSM support has currently only been tested with Linux guests. There are no plans to provide legacy support for other OSes (e.g. Windows).
To make Cloud Hypervisor use UEFI boot, pass the `CLOUDHV.fd` file path as an argument to the `--kernel` option. The firmware file will be opened in read only mode.
- Suitable firmware for Cloud Hypervisor (`CLOUDHV.fd`) and for QEMU (`OVMF.fd`)
- With the suggested image size of 30G, there should be enough free disk space to hold the installation ISO and any other necessary files
This step currently requires QEMU to install Windows onto the guest. QEMU is only used at the preparation stage, the resulting image is then fully functional with Cloud Hypervisor.
@@ -67,7 +67,7 @@ The basic command to boot a Windows image. The configuration section should be c
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
