In case of SEV-SNP guest devices use sw-iotlb to gain access guest
memory for DMA. For that F_IOMMU/F_ACCESS_PLATFORM must be exposed in
the feature set of virtio devices.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
According to PCIe specification, a 64-bit MMIO BAR should be
naturally aligned. In addition to being more compliant with
the specification, natural aligned BARs are mapped with
the largest possible page size by the host iommu driver, which
should speed up boot time and reduce IOTLB thrashing for virtual
machines with VFIO devices.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
The 'generate_ram_ranges' function currently hardcodes the assumption
that there are only 2 E820 RAM entries. This is not flexible enough to
handle vendor specific memory holes. Returning a Vec is also more
convenient for users of this function.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
Since the ACPI tables are generated inside the IGVM file in case of
SEV-SNP guest. So, we don't need to generate it inside the cloud
hypervisor.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
... enabled VMs. IOEvents are not supported in case of SEV-SNP VMs. All
the IO events are delievered via GHCB protocol.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This will help in identify whether a VM supports sev-snp and based on
that disable/enable certain features.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Implement workflow to to run static analysis and linting of all shell
scripts by using shfmt and shellcheck.
Fixes: #5396
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
We occasionally saw cloud-hypervisor crashed due to seccomp violations. The
coredumps showed the HTTP API thread crashing after it attempted to call
sched_yield(). The call came from rust stdlib's mpmc module, which calls
sched_yield() if several attempts to busy-wait for a condition to fulfil fall
short.
Since the system call is harmless and it comes from the stdlib, I opted to allow
all threads to call it.
Signed-off-by: Peteris Rudzusiks <rye@stripe.com>
Currently the only way to set the affinity for virtio block threads is
to boot the VM, search for the tid of each of the virtio block threads,
then set the affinity manually. This commit adds an option to pin virtio
block queues to specific host cpus (similar to pinning vcpus to host
cpus). A queue_affinity option has been added to the disk flag in
the cli to specify a mapping of queue indices to host cpus.
Signed-off-by: acarp <acarp@crusoeenergy.com>
Traditional way to configure vcpu don't work for sev-snp guests. All the
vCPU configuration for SEV-SNP guest is provided via VMSA.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This also fixes the following clippy warning on nightly build from cargo
fuzz:
warning: struct `RegionEntry` is never constructed
--> /home/chenb/project/cloud-hypervisor/cloud-hypervisor/block/src/vhdx/vhdx_header.rs:357:8
|
357 | struct RegionEntry {
| ^^^^^^^^^^^
|
= note: `RegionEntry` has derived impls for the traits `Debug` and `Clone`, but these are intentionally ignored during dead code analysis
= note: `#[warn(dead_code)]` on by default
Signed-off-by: Bo Chen <chen.bo@intel.com>
For all VFIO devices, map all non-emulated MMIO regions to
the vfio container to allow PCIe P2P between all VFIO devices
on the virtual machine. This is required for a wide variety of
advanced GPU workloads such as GPUDirect P2P (DMA between two
GPUs), GPUDirect RDMA (DMA between a GPU and an IB device).
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
Currently kernel and firmware are checked as a payload.
IGVM should be checked as well. Otherwise, it hangs indefinitely.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Unaccepted GPA is usually thrown by Microsoft hypervisor in case of
mismatch between GPA and GVA mappings. This is a fatal message from the
hypervisor perspective so we would need to error out from the vcpu run
loop. Along with add some debug message to identify the broken mapping
between GVA and GPA.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
On guests with large amounts of memory, using the `prefault` option can
lead to a very long boot time. This commit implements the strategy
taken by QEMU to prefault memory in parallel using multiple threads,
decreasing the time to allocate memory for large guests by
an order of magnitude or more.
For example, this commit reduces the time to allocate memory for a
guest configured with 704 GiB of memory on 1 NUMA node using 1 GiB
hugepages from 81.44134669s to just 6.865287881s.
Signed-off-by: Sean Banko <sbanko@crusoeenergy.com>
The `ByteValued` trait implementations for the data structures from the
'linux_loader' crate are no longer needed, and hence their wrappers can
be removed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Fix beta clippy issue:
error: file opened with `create`, but `truncate` behavior not defined
--> src/main.rs:624:26
|
624 | .create(true)
| ^^^^^^^^^^^^- help: add: `.truncate(true)`
|
= help: if you intend to overwrite an existing file entirely, call `.truncate(true)`
= help: if you instead know that you may want to keep some parts of the old file, call `.truncate(false)`
= help: alternatively, use `.append(true)` to append to the file instead of overwriting it
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#suspicious_open_options
= note: `-D clippy::suspicious-open-options` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::suspicious_open_options)]`
Signed-off-by: Bo Chen <chen.bo@intel.com>
The embedded error in the enum will be read on debug output of the
error.
Fixes beta clippy issue:
warning: field `0` is never read
--> vhost_user_block/src/lib.rs:64:23
|
64 | CreateKillEventFd(io::Error),
| ----------------- ^^^^^^^^^
| |
| field in this variant
|
= note: `#[warn(dead_code)]` on by default
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
64 | CreateKillEventFd(()),
| ~~
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The embedded error in the enum will be read on debug output of the
error.
Fixes beta clippy issue:
warning: field `0` is never read
--> performance-metrics/src/performance_tests.rs:25:11
|
25 | Infra(InfraError),
| ----- ^^^^^^^^^^
| |
| field in this variant
|
= note: `#[warn(dead_code)]` on by default
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
25 | Infra(()),
| ~~
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: field `0` is never read
--> block/src/qcow/vec_cache.rs:139:21
|
139 | struct NumCache(pub u64);
| -------- ^^^^^^^
| |
| field in this struct
|
= note: `#[warn(dead_code)]` on by default
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
139 | struct NumCache(());
| ~~
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
These structs directly embed another struct and then implement
ByteValued on that struct to implement ByteValued for the inner struct.
As such the inner struct is never directly accessed so to avoid the dead
code analysis mark this as allowed.
Beta clippy fix:
warning: field `0` is never read
--> arch/src/x86_64/mod.rs:129:32
|
129 | struct MemmapTableEntryWrapper(hvm_memmap_table_entry);
| ----------------------- ^^^^^^^^^^^^^^^^^^^^^^
| |
| field in this struct
|
= note: `MemmapTableEntryWrapper` has a derived impl for the trait `Clone`, but this is intentionally ignored during dead code analysis
= note: `#[warn(dead_code)]` on by default
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
129 | struct MemmapTableEntryWrapper(());
|
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Beta clippy fix
warning: useless use of `vec!`
--> tests/integration.rs:5845:23
|
5845 | let kernels = vec![direct_kernel_boot_path()];
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: you can use an array directly: `[direct_kernel_boot_path()]`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_vec
= note: `#[warn(clippy::useless_vec)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Beta clippy fix:
warning: initializer for `thread_local` value can be made `const`
--> vmm/src/sigwinch_listener.rs:27:40
|
27 | static TX: RefCell<Option<File>> = RefCell::new(None);
| ^^^^^^^^^^^^^^^^^^ help: replace with: `const { RefCell::new(None) }`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#thread_local_initializer_can_be_made_const
= note: `#[warn(clippy::thread_local_initializer_can_be_made_const)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Beta clippy fix:
warning: this call to `as_ref.map(...)` does nothing
--> vmm/src/device_manager.rs:1234:9
|
1234 | self.console_resize_pipe.as_ref().map(Arc::clone)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `self.console_resize_pipe.clone()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_asref
= note: `#[warn(clippy::useless_asref)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
According to crates.io the crc-any crate is actively maintained which
avoids issues with the crc32c crate and the nightly compiler.
Fixes: #6168
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
When we import a page, we have a page with
some data or empty, empty does not mean there is no data,
it rather means it's full of zeros. We can skip writing the
data as guest memory of the page is already zeroed.
A page could be partially filled and the rest of the content is zero.
Our IGVM generation tool only fills data here if there is some data
without zeros. Rest of them are padded. We only write data
without padding and compare whether we complete writing
the buffer content. Still it's a full page and update the variable
with length of the full page.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
When the HTT flag CPUID.1.EDX[HTT] is 0, it indicates that there is
only a single logical processor in the package. When HTT is 1, it
indicates that CPUID.1.EBX[23:16] contains the number of logical
processors in the package.
When this information is not included in CPUID leaf 0x1, some cpu
topology enumeration software such as hwloc are known to crash.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
This commit adds the debug-console (or debugcon) device to CHV. It is a
very simple device on I/O port 0xe9 supported by QEMU and BOCHS. It is
meant for printing information as easy as possible, without any
necessary configuration from the guest at all.
It is primarily interesting to OS/kernel and firmware developers as they
can produce output as soon as the guest starts without any configuration
of a serial device or similar. Furthermore, a kernel hacker might use
this device for information of type B whereas information of type A are
printed to the serial device.
This device is not used by default by Linux, Windows, or any other
"real" OS, but only by toy kernels and during firmware development.
In the CLI, it can be configured similar to --console or --serial with
the --debug-console parameter.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Set the SEV control register so we know where to
start running. This register configures the
SEV feature control state on a virtual processor.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This register configures the SEV feature control
state on a virtual processor.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Consistent with the other data structures and constants used in TDX
support code import the necessary structures from the kernel for
accessing the vmcall structure.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
These Default implementations either don't produce valid configs, are
no longer used outside of tests, or both.
For the tests, we can define our own local "default" values that make
the most sense for the tests, without worrying about what's
a (somewhat) sensible "global" default value.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Bumping anyhow crate from 1.0.75 to 1.0.79 will cause seccomp
failures through integration tests. Newly added backtrace support
relies on readlink and many other syscalls.
Issue noticed with test_api_http_pause_resume test, where second time
of VM PAUSE or VM RESUME prints error and causes panic.
Noticed that panic message in a thread which is not allowed to write
output triggered the issue.
So implementing Display trait for HttpError and ApiError enums to avoid
adding many syscalls to seccomp filter section.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
On hosts with >256 cpus, setting the cpu affinity to a host cpu index
>255 will return an error because type of `host_cpu` is `u8`.
This commit changes the type of `host_cpu` to `usize` to remove this
limitation.
Signed-off-by: Sean Banko <sbanko@crusoeenergy.com>
Move the duplicated logic to download OVMF into a helper function.
Explicitly specify the OVMF_FW_TAG instead of downloading the latest
so that new OVMF versions can be easily tested.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
Testing instructions are very old and type=dpdkvhostuser is
deprecated from DPDK.
logs from /var/log/openvswitch/ovs-vswitchd.log
|netdev_dpdk|WARN|dpdkvhostuser ports are considered deprecated;
please migrate to dpdkvhostuserclient ports
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Uses of the old ApiRequest enum conflated two different concerns:
identifying an API request endpoint, and storing data for an API
request. This led to ApiRequest values being passed around with junk
data just to communicate a request type, which forced all API request
body types to implement Default, which in some cases doesn't make any
sense — what's the "default" path for a vhost-user socket? The
nonsensical Default values have led to tests relying on being able to
use nonsensical data, which is an impediment to adding better
validation for these types.
Rather than having API request types be represented by an enum, which
has to carry associated body data everywhere it's used, it makes more
sense to represent API request types as trait objects. These can have
an associated type for the type of the request body, and this makes it
possible to pass API request types and data around as siblings in a
type-safe way without forcing them into a single value even where it
doesn't make sense. Trait objects also give us dynamic dispatch,
which lets us get rid of several large match blocks.
To keep it possible to fuzz the HTTP API, all the Vmm methods called
by the HTTP API are pulled out into a trait, so the fuzzer can provide
its own stub implementation of the VMM.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
With the change in 7bc764d to expose the SVM bit for nested
virtualisation test_vfio can be re-enabled on the AMD workers.
Fixes: #5895
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
When using amd topology, the svm feature flag on cpuid leaf
0x8000_0001.ecx is overwritten. We update the amd cpu topology
logic to use the flag values that originated in
KVM_GET_SUPPORTED_CPUID ioctl and override as necessary.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
warning: this URL is not a hyperlink
--> arch/src/aarch64/layout.rs:114:58
|
114 | ...in https://www.kernel.org/doc/Documentation/arm64/booting.txt.
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://www.kernel.org/doc/Documentation/arm64/booting.txt.>`
|
= note: bare URLs are not automatically turned into clickable links
= note: `#[warn(rustdoc::bare_urls)]` on by default
I also noticed that it looks like this comment was supposed to be
applied to FDT_MAX_SIZE, not FDT_START, so I moved it.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The VIRTIO specification[1] says:
> The upper 32 bits of the CID are reserved and zeroed.
We should therefore not allow the user to supply a VSOCK CID with
those bits set. To accomplish this, limit the public API of the
virtio-vsock device to only accept 32-bit CIDs, while still using
64-bit CIDs internally since that's how virtio-vsock works.
[1]: https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-4400004
Signed-off-by: Alyssa Ross <hi@alyssa.is>
I accidentally ran a VM with CID 2 (VMADDR_CID_HOST), and very strange
and difficult to debug behavior ensued. I don't think a virtio-vsock
device should be allowed to have any of the special CIDs
(VMADDR_CID_ANY, VMADDR_CID_HYPERVISOR, VMADDR_CID_LOCAL, VMADDR_CID_HOST).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The socket is nonblocking, so it's not guaranteed that it will be
possible to read the whole connect command in a single iteration of
the event loop. To reproduce:
(echo -n 'CONNECT '; sleep 1; echo 1234; cat) | socat STDIO UNIX-CONNECT:vsock.sock
This would produce the error:
cloud-hypervisor: 5.509209s: <_vsock4> INFO:virtio-devices/src/vsock/unix/muxer.rs:446 -- vsock: error adding local-init connection: UnixRead(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" })
To fix this, if we only get a partial command, we need to save it for
future iterations of the event loop, and only proceed once we've read
a complete command.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Complete the isolated import, telling the
Microsoft hypervisor that import is done so that
MSHV can issue SNP_LAUNCH_FINISH command.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Use right and exact size 32 bytes for host data field
for completing the isolated import. This way OOB
can be avoided during a function call.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Import all the isolated pages after parsing is
done on the iGVM file. Hypervisor adds those
pages for PSP measurement(part of the hashing).
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Current instructions are incorrect and there is now a new profile called
dev-opt to build the debug version of TD-SHIM.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Under the fuzzer this code appears dead:
error: field `0` is never read
--> /home/rob/src/cloud-hypervisor/arch/src/x86_64/mod.rs:128:32
|
128 | struct MemmapTableEntryWrapper(hvm_memmap_table_entry);
| ----------------------- ^^^^^^^^^^^^^^^^^^^^^^
| |
| field in this struct
|
= note: `MemmapTableEntryWrapper` has a derived impl for the trait `Clone`, but this is intentionally ignored during dead code analysis
= note: `-D dead-code` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(dead_code)]`
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
128 | struct MemmapTableEntryWrapper(());
| ~~
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Add a 'rate_limit_groups' field to VmConfig that defines a set of
named RateLimiterGroups.
When the 'rate_limit_group' field of DiskConfig is defined, all
virtio-blk queues will be rate-limited by a shared RateLimiterGroup.
The lifecycle of all RateLimiterGroups is tied to the Vm.
A RateLimiterGroup may exist even if no Disks are configured to use
the RateLimiterGroup. Disks may be hot-added or hot-removed from the
RateLimiterGroup.
When the 'rate_limiter' field of DiskConfig is defined, we construct
an anonymous RateLimiterGroup whose lifecycle is tied to the Disk.
This is primarily done for api backwards compatability. Importantly,
the behavior is not the same! This implementation rate_limits the
aggregate bandwidth / iops of an individual disk rather than the
bandwidth / iops of an individual queue of a disk.
When neither the 'rate_limit_group' or the 'rate_limiter' fields of
DiskConfig is defined, the Disk is not rate-limited.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
Add a 'rate_limiter/group' module that defines the RateLimiterGroup
and a RateLimiterGroupHandle types.
The RateLimiterGroupHandle can be used in place of a RateLimiter to
limit the aggregate bandwidth and/or ops of multiple virtio-blk or
virtio-net queues.
Each RateLimiterGroup has an associated worker thread that broadcasts
an event to each RateLimiterGroupHandle when a RateLimiter is unblocked.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
CI reports errors:
error: writing `&Vec` instead of `&[_]` involves a new object where a slice will do
--> arch/src/x86_64/mod.rs:1351:19
|
1351 | epc_sections: &Vec<SgxEpcSection>,
| ^^^^^^^^^^^^^^^^^^^ help: change this to: `&[SgxEpcSection]`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
= note: `-D clippy::ptr-arg` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::ptr_arg)]`
Signed-off-by: Yi Wang <foxywang@tencent.com>
CI reports clippy errors:
error: argument to `Path::join` starts with a path separator
--> tests/integration.rs:4076:58
|
4076 | let serial_socket = guest.tmp_dir.as_path().join("/tmp/serial.socket");
| ^^^^^^^^^^^^^^^^^^^^
|
= note: joining a path starting with separator will replace the path instead
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#join_absolute_paths
Signed-off-by: Yi Wang <foxywang@tencent.com>
CI reports clippy errors:
error: in a `match` scrutinee, avoid complex blocks or closures with blocks; instead, move the block or closure higher and bind it with a `let`
--> test_infra/src/lib.rs:93:51
|
93 | match (|| -> Result<(), WaitForBootError> {
| ___________________________________________________^
94 | | let listener =
95 | | TcpListener::bind(listen_addr.as_str()).map_err(WaitForBootError::Listen)?;
96 | | listener
... |
145 | | }
146 | | })() {
| |_________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#blocks_in_conditions
= note: `-D clippy::blocks-in-conditions` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::blocks_in_conditions)]`
Signed-off-by: Yi Wang <foxywang@tencent.com>
This PR addresses a bug in which the cpu topology of a guest
with non power-of-two number of cores is incorrect. For example,
in some contexts, a virtual machine with 2-sockets and 12-cores
will incorrectly believe that 16 cores are on socket 1 and 8
cores are on socket 2. In other cases, common topology enumeration
software such as hwloc will crash.
The root of the problem was the way that cloud-hypervisor generates
apic_id. On x86_64, the (x2) apic_id embeds information about cpu
topology. The cpuid instruction is primarily used to discover the
number of sockets, dies, cores, threads, etc. Using this information,
the (x2) apic_id is masked to determine which {core, die, socket} the
cpu is on. When the cpu topology is not a power of two
(e.g. a 12-core machine), this requires non-contiguous (x2) apic_id.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
The following tests have been temporarily disabled:
1. Live upgrade/migration test with ovs-dpdk (#5532);
2. Disk hotplug tests on windows guests (#6037);
This patch has been tested with PR #6048.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Tested-by: Bo Chen <chen.bo@intel.com>
For SEV-SNP guests we need to provide the extended memory. It follows a
very simple layout and very similar to other x86 guests.
First segment: [HIGH_RAM_START - MEM_32BIT_RESERVED_START]
PCI hole: [MEM_32BIT_RESERVED_START - RAM_64BIT_START]
Second segment: [RAM_64BIT_START - RAM_END]
Fixes#5993
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The 'test_vfio_user' is prone to fail when the system is under high
workloads with errors:
```
Error while connecting to /var/tmp/spdk.sock
Is SPDK application running?
Error details: Invalid or non-existing address: '/var/tmp/spdk.sock'
```
This is because SPDK is not fully functional before we request to
create a nvme device using the vfio_user protocol. This patch stabilize
this test with allowing retires to execute host commands.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch defines a new function 'generate_ram_ranges', to generate
usable physical memory ranges for the guest based on the existing guest
memory managed by VMM. This function is also made public, so that it can
be reused, say by the IGVM loader in the future [1].
No functional change.
See: #6020
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch adds missing new lines after functions,
fixes few typos in the comments, adds few missing
comments to SNP related functions.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
There is no requirement to call copy_from_slice, since all the member
variables are identical and we can directly assign them value.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There is a mix of v2, v3 and v4 in the codebase. Let's move to v4
everywhere because v2 seems to be using a deprecated version of nodejs.
This is throwing warnings when the Github action CI is running.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently there are some inconsistencies in Cargo.toml which is causing
the following warnings during the build process:
Error parsing Cargo.toml manifest, fallback to caching entire file:
Invalid TOML document: expected key-value, found comma
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
vmm: Add igvm module and loader module
Add a separate module named igvm to the vmm crate
with definitions to parse and load igvm to the guest memory.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add necessary API to retrieve cpuid leaf on MSHV.
This API is used to update cpuid information
during the parsing of the igvm file.
Microsoft hypervisor does not provide common
CpuID like KVM. That's why we need to call this API
during the IGVM parsing.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch adds igvm to the Vm config and params as well as
the command line argument to pass igvm file to load into
guest memory. The file must maintain the IGVM format.
The CLI option is featured guarded by igvm feature gate.
The IGVM(Independent Guest Virtual Machine) file format
is designed to encapsulate all information required to
launch a virtual machine on any given virtualization stack,
with support for different isolation technologies such as
AMD SEV-SNP and Intel TDX.
At a conceptual level, this file format is a set of commands created
by the tool that generated the file, used by the loader to construct
the initial guest state. The file format also contains measurement
information that the underlying platform will use to confirm that
the file was loaded correctly and signed by the appropriate authorities.
The IGVM file is generated by the tool:
https://github.com/microsoft/igvm-tooling
The IGVM file is parsed by the following crates:
https://github.com/microsoft/igvm
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Random failures of downloading `cloud-hypervisor-static-aarch64` with
`wget` were seen. The commit applies a workaround to retry the download
for a few times.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
This commit changes existing behavior of named TAP interfaces.
When booting a VM with configuration for a named TAP interface,
cloud-hypervisor will create the interface and apply a given
IP configuration to that interface. If the named interface
already exists on the system, the configuration is NOT overwritten.
Setting the ip and netmask fields in a tap interface configuration
for a named tap interface now works by handing this configuration
to the virtio_devices::Net object when it is created with a name.
This commit also touches net_util to make sure that the ip configuration
of existing TAP interfaces is not modified with ip or netmask handed to
open_tap.
Signed-off-by: Markus Sütter <markus.suetter@secunet.com>
The wget has been causing frequent CI failure for the past few days,
while it can't be reproduced manually. Let's show the wget details in
our CI pipeline to understand better future errors.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Running on host where vdpa_sim_blk module is not correctly loaded
test_vdpa_block passes.
"test common_parallel::test_vdpa_block ... ok"
This commit fixes the vdpa_sim_blk test to fail in that case.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Valid title format:
component1[, component2, componentN]: submodule: summary
Title should have at least one component
Components are separated by comma+space: ", "
Components are validated to be in valid_components
Components list is ended by a colon
Submodules are not validated
See: #5846
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
We can skip the CI worker build if the changes are only from gitlint, or
the hidden files in the root folder, such as .github, .gitignore,
.gitlint, .rustfmt.toml, and .typos.toml.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This is particularly useful for commits with long component names. Of
course, it is better to have concise subject that is less than 50 char.
Signed-off-by: Bo Chen <chen.bo@intel.com>
As part SMP bringup for a SEV-SNP guest, BSP sets up the VMSA page for
each AP threads and informs hypervisor about the same using a VMGEXIT.
Thus, extend the current GHCB interface to handle this scenario.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
SEV-SNP guest can request AMD's secure co-processor i.e., PSP to
generate an runtime attesation report. During this process guest needs
to inform PSP about the request and response GPAs where that report
would be generated by the PSP. This is handled via a VMGEXIT request.
Thus, extend the current GHCB handling to add support for it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
We found that it's slow to load JSON when reading snap files. As
described in [1], using from_slice instead of from_reader can fix
this.
Also, fix the error type being returned.
1. https://github.com/serde-rs/json/issues/160
Signed-off-by: Yi Wang <foxywang@tencent.com>
Since the 'write()' to the event file was moved to its own thread
(see #5633), we have no reliable way to read the latest contents of
the event file from our integration tests, since we can't ensure the
'read()' from our test always happen after 'write()' is completed from
Cloud Hypervisor. This is also why we started to see random failures on
snapshot_restore tests (particularly when the system workload is high).
This patch adds a 1s sleep before reading the event file to mitigate the
random failures.
Signed-off-by: Bo Chen <chen.bo@intel.com>
error: use of a fallible conversion when an infallible one could be used
Error: --> virtio-devices/src/vhost_user/vu_common_ctrl.rs:206:51
|
206 | let actual_size: usize = queue.size().try_into().unwrap();
| ^^^^^^^^^^^^^^^^^^^ help: use: `into()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_fallible_conversions
= note: `-D clippy::unnecessary-fallible-conversions` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::unnecessary_fallible_conversions)]`
error: could not compile `virtio-devices` (lib) due to previous error
Error: warning: build failed, waiting for other jobs to finish...
error: could not compile `virtio-devices` (lib test) due to previous error
Error: The process '/home/runner/.cargo/bin/cargo' failed with exit code 101
Signed-off-by: Bo Chen <chen.bo@intel.com>
When the script is invoked via dev_cli.sh, it always gets AUTH_DOWNLOAD_TOKEN
from the environment. The original test always returns true.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Instead of relying on "wc" and "grep", this patch provides helper
functions for checking line counts and searching/counting keywords.
To understand assertion failures better, it also generate logs for the
L1/L2 VM commands when checks fail.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Add a test that supports configuring serial and console as TTY mode
at the same time. With this configuration, the VM can set up a legacy
serial device as an early printk console device, and then change to a
virito console device after the virito console device is initialized.
In this case, we can capture the logs printed by legacy serial on early
boot, and later by the virtio console.
Signed-off-by: Yong He <alexyonghe@tencent.com>
Cloud Hypovrisor supports legacy serial device and virito console device
for VMs. Using legacy serial device, CH can capture full VM console logs,
but its implementation is based on KVM PIO emulation and has poor
performance. Using the virtio console device, the VM console logs will
be sent to CH through the virtio ring, the performance is better, but CH
will only capture the VM console logs after the virtio console device is
initialized, the VM early startup logs will be discarded.
This patch provides a way to enable both the legacy serial device and the
virtio console device as a TTY mode by setting the leagcy serial port as
the VM's early printk device and setting the virtio console as the VM's
main console device.
Then CH can capture early boot logs from the legacy serial device and
capture later logs from the virito console device with better performance.
Signed-off-by: Yong He <alexyonghe@tencent.com>
The script "run_integration_tests_vfio.sh" requires to be ran on a
specific machine with specific hardware (e.g. the "vfio" bera-metal
worker equipped with a Nvidia Tesla T4 card). So the provisioning of the
running machine is out of the scope of this script, including the custom
guest image with Nvidia drivers installed, and properly configured
Nvidia Tesla T4 card.
See: #4405, #5509
Signed-off-by: Bo Chen <chen.bo@intel.com>
With #4324 being resolved, the nested VFIO test (e.g. "test_vfio") is
now a part of the general Azure VM-based workers. No need to run it on
the bare-metal worker.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Now, default values for vcpu topology are 0s, that is not correct and may
lead to bug. Fix it by setting default value to 1s. Also add check in
case one or more of these values are zero.
Fixes: #5892
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
virtiofsd must be built with the default gnu toolchain so clear the
environment variables that may poisoned by the alternative C library
support.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The seccompiler v0.4.0 started to use `seccomp` syscall instead of the
`prctl` syscall. Also, threads for virtio-deivces should not need any of
these syscalls anyway.
Signed-off-by: Bo Chen <chen.bo@intel.com>
MMIO emulation is also performed via VMGEXIT in case of SEV-SNP guest.
Emulation is done in a very similar way like a regular guest. Just need
to make sure that guest memory is access via read/write GPA hypercall
instead of directly accessing it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Legacy port emulation requires reading RAX register from GHCB page for
SEV-SNP guest. This is the major difference between a regular guest and
SEV-SNP enabled guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Currently MSHV does not support fetching extended guest report and thus
return an appropriate error stating the NAE event is not valid.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
As part of this handling there are 4 different operations:
1. Getting the hypervisor preffered doorbell page GPA.
2. Informing hypervisor about the doorbell page chosen by the guest
3. Querying the GPA of the doorbell page
4. Clearing the GPA of the doorbell page from hypervisor
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
As part of handling this request, hypervisor is expected to three
things:
1. Maximum GHCB protocol version supported.
2. Minimum GHCB protocol version supported.
3. SEV-page table encryption bit number.
If the guest cannot support the protocol range supplied by the
hypervisor, it should terminate
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
SEV-SNP guest allocates a GHCB page and in order to update hypervisor
about the same, there is a vmgexit which allows registering GHCB page
with the hypervisor.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
A VMGEXIT exit occurs for any of the listed NAE events in the GHCB
specification [1] (e.g. CPUID, RDMSR/WRMSR, MMIO, port IO, etc.). Some
of these events are handled by hypervisor while other are handled by
VMM. Currently, we are adding support for one such request i.e.,
report supported SEV-SNP features by hypervisor.
[1] GHCB protocol specification:
https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/specifications/56421.pdf
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This is the function that needs to be called by the VMM
to inform the MSHV that isolation is complete and inform
PSP about this completion.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add hypervisor VM specific API to import the isolated
pages. Hypervisor adds those pages for PSP measurement.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch fixes following warnings:
error: boolean to int conversion using if
--> vmm/src/vm.rs:866:42
|
| .create_vm_with_type(if sev_snp_enabled.into() {
| __________________________________________^
| | 1 // SEV_SNP_ENABLED
| | } else {
| | 0 // SEV_SNP_DISABLED
| | })
| |_____________________^ help: replace with from: `u64::from(sev_snp_enabled.into())`
|
= note: `-D clippy::bool-to-int-with-if` implied by `-D warnings`
= note: `sev_snp_enabled.into() as u64` or `sev_snp_enabled.into().into()` can also be valid options
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#bool_to_int_with_if
error: useless conversion to the same type: `bool`
--> vmm/src/vm.rs:866:45
|
| .create_vm_with_type(if sev_snp_enabled.into() {
| ^^^^^^^^^^^^^^^^^^^^^^ help: consider removing `.into()`: `sev_snp_enabled`
|
= note: `-D clippy::useless-conversion` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
error: could not compile `vmm` due to 2 previous errors
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Partially revert 111225a2a5
and add the new dbus and pvpanic arguments.
As we are switching back to clap observe the following changes.
A few examples:
1. `-v -v -v` needs to be written as`-vvv`
2. `--disk D1 --disk D2` and others need to be written as `--disk D1 D2`.
3. `--option value` needs to be written as `--option=value.`
Change integration tests to adapt to the breaking changes.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
There is no need to set them in the test scripts while the main script
already has them.
The consolidates how things are done.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
There is no need to reconstruct it from within the scripts since the
main script already constructed it once.
Drop the previously useless setting of BUILD_TARGET from various
scripts. The value was always overwritten at a later point.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Sev-Snp on MSHV uses igvm crate to parse the igvm file.
igvm crate needs minimum rust version 1.65 to build.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This struct contains all configuration fields that controls the way how
we generate CPUID for the guest on x86_64. This allows cleaner extension
when adding new configuration fields.
Signed-off-by: Bo Chen <chen.bo@intel.com>
For a SEV-SNP enabled partition on MSHV, some of the VMGEXITS are
offloaded for Hypervisor to handle while the rest are handled by VMM.
By setting this additional partition property hypervisor is informed
about the VMGEXITs it needs to take care off, rest all would be handled
by the CloudHypervisor.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The lock to `vm_config` is held for accessing `cpus.kvm_hyperv` passing
as a reference to `arch::generate_common_cpuid()`, so acquiring the same
lock again while calling to the same function is a deadlock.
Fixes: 3793ffe888
Reported-by: Yi Wang <foxywang@tencent.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
As part of this initialization for a SEV-SNP VM on MSHV, it is required
that we transition the guest state to secure state using partition
hypercall. This implies all the created VPs will transition to secure
state and could access the guest encrypted memory.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
When performing integration testing on the Ampere One server, an error occurred when compiling spdk and not recognizing the CPU ID.
The latest spdk already contains fixes.
Signed-off-by: dom.song <dom.song@amperecomputing.com>
It's a requirement that a SEV-SNP enabled guest on MSHV must have
isolation policy set before launching the guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Cloud-Hypervisor takes a path for Unix socket, where it will listen
on. Users can connect to the other end of the socket and access serial
port on the guest.
"--serial socket=/path/to/socket" is the cmdline option to pass to
cloud-hypervisor.
Users can use socat like below to access guest's serial port once the
guest starts to boot:
socat -,crnl UNIX-CONNECT:/path/to/socket
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The 'derive' feature of `zerocopy` crate now is optional and requires to
be enabled explicitly [1]. Also, a version bump on `acpi_tables` is
needed to reply on a single version of `zerocopy` to avoid compilation
errors.
[1] https://github.com/google/zerocopy/pull/176
Signed-off-by: Bo Chen <chen.bo@intel.com>
Include the TSC frequency as part of the KVM state so that it will be
restored at the destination.
This ensures migration works correctly between hosts that have a
different TSC frequency if the guest is running with TSC as the source
of timekeeping.
Fixes: #5786
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The bare metal machines are now not reachable from our Jenkins
controller, and causing CI pipeline timeout for the past few days.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The cumulative average formula [1] requires to use signed integers
for proper calculations, while calculated result (e.g. cumulative
average) is always positive. This patch reflects the above requirements
in our code.
[1] https://en.wikipedia.org/wiki/Moving_average#Cumulative_averageFixes: #5745
Signed-off-by: Bo Chen <chen.bo@intel.com>
Fix make error:
$ make -C BaseTools -j `nproc`
/bin/sh: line 1: python: command not found
make[1]: *** [GNUmakefile:11: test] Error 127
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Bump these crates need to be bumped together to avoid build issues from
shared structures.
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
EntryPoint had an optional entry_addr, but there is no usage of this
struct that makes it necessary that the address is optional.
Remove the Option to avoid being able to express things that are not
useful.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
This fixes all typos found by the typos utility with respect to the config file.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
There is a "LATENCY_SCALE" being used for calculating cumulative average
latency, so it should also be used for the latency of the first op.
See: #5712
Signed-off-by: Bo Chen <chen.bo@intel.com>
Update to the latest vm-memory and all the crates that also depend upon
it.
Fix some deprecation warnings.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
We need to make sure that SEV-SNP and TDX are not enabled at the same
time. As these two features belong to mutually exclusive hardware
vendors. So, we should make sure that these two features are not enabled
at the same. Thus, a compile time check for it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Creating some brief documentation for SEV-SNP, summarizing the links on
where to find more information about SEV-SNP, as well as how to build
and run Cloud Hypervisor on it.
This document is a work in progress and will be updated in future PRs
as we add support for it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This feature flag gates the development for SEV-SNP enabled guest.
Also add a helper function to identify if SNP should be enabled for the
guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The worker is running focal not bionic - this fixes build issues on that
worker machine due to the bionic version not being installable.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is necessary for O_DIRECT based use of raw block devices which may
require access at a larger block size than that of a sector (512 bytes.)
Fixes: #5722
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
It doens't make sense to limit this to the async_io module and import
across modules so place in default create module.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Assume rax is 0xfee003e0 and the displacement is negative 0x60. The effective
address is then 0xfee00380. This is perfectly valid.
Example instruction:
c7 40 a0 00 10 00 00 movl $0x1000,-0x60(%rax)
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Logically until we have handled the first operation the latency is
infinite; this logic was applied to the minimum latency originally but
this patch extends that logic to the maximum and average latency.
To prevent the initial average latency being skewed by the inclusion of
infinity the average value is initally seeded with the first measured
latency.
Fixes: #5704
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
In the documentation of function check_latest_events_exact use same events
argument name as in the implementation
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Since kernel v6.3 the vsock packet is not split over two descriptors
and is instead included in a single one.
This change is based on the discovery and fix identified by Stefano
Garzarella for the vm-virtio vsock implementation and adapted for our
very different codebase.
Fixes: #5691
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This commit builds on top of the `Monitor::subscribe` function and
makes it possible to broadcast events published from `event-monitor`
over D-Bus.
The broadcasting functionality is enabled if the D-Bus API is enabled
and users who wish to also enable the file based `event-monitor` can do
so with the CLI arg `--event-monitor`.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
There is no need for this struct to be public and since it is used in
this module the #[allow(dead_code)] invocation can be removed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> vmm/src/device_manager.rs:1908:35
|
1908 | fn set_raw_mode(&mut self, f: &mut dyn AsRawFd) -> vmm_sys_util::errno::Result<()> {
| ^^^^^^^^^^^^^^^^ help: consider changing to: `&dyn AsRawFd`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
= note: `#[warn(clippy::needless_pass_by_ref_mut)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> src/bin/ch-remote.rs:397:52
|
397 | fn dbus_api_do_command(toplevel: &TopLevel, proxy: &mut DBusApi1ProxyBlocking<'_>) -> ApiResult {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider changing to: `&DBusApi1ProxyBlocking<'_>`
|
= note: this is cfg-gated and may require further changes
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
= note: `#[warn(clippy::needless_pass_by_ref_mut)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> virtio-devices/src/transport/pci_common_config.rs:100:17
|
100 | queues: &mut [Queue],
| ^^^^^^^^^^^^ help: consider changing to: `&[Queue]`
|
= warning: changing this function will impact semver compatibility
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
= note: `#[warn(clippy::needless_pass_by_ref_mut)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> hypervisor/src/arch/x86/emulator/instructions/mod.rs:22:15
|
22 | platform: &mut dyn PlatformEmulator<CpuState = T>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider changing to: `&dyn PlatformEmulator<CpuState = T>`
|
= note: this is cfg-gated and may require further changes
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> net_util/src/queue_pair.rs:167:14
|
167 | tap: &mut Tap,
| ^^^^^^^^ help: consider changing to: `&Tap`
|
= warning: changing this function will impact semver compatibility
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> block/src/async_io.rs:68:28
|
68 | fn query_block_size(f: &mut File, block_size_type: BlockSize) -> std::io::Result<u64> {
| ^^^^^^^^^ help: consider changing to: `&File`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: boolean expression will never evaluate to 'true'
--> block/src/vhdx/vhdx_metadata.rs:136:20
|
136 | if disk_spec.block_size < BLOCK_SIZE_MIN && disk_spec.block_size > BLOCK_SIZE_MAX {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: since `BLOCK_SIZE_MIN` < `BLOCK_SIZE_MAX`, the expression evaluates to false for any value of `disk_spec.block_size`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#impossible_comparisons
= note: `#[warn(clippy::impossible_comparisons)]` on by default
&& was used when || should have been used
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Similar to balloon inflation, memory allocation is also constrained to
align with the page size. Therefore, memory is allocated in units of the
host page size, one page at a time, until all host pages that the memory
range requested by the guest are managed. If the requested size is
smaller than the page size, the entire page will still be allocated
because smaller allocations are not possible due to the page size
limitation.
Fixes: cloud-hypervisor#5369
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Currently, virtio-balloon can't work well with page size other than 4k.
The virtio-balloon always works in units of 4kiB (BALLOON_PAGE_SIZE), but
we can only actually discard memory in units of the host page size.
We get some idea from [1] to solve this issue.
What has been done in this commit:
For balloon inflation:
A bitmap is employed to track the memory range to be released in 4k
granularity. Once it accumulates to one host page size, the corresponding
page is released, and the bitmap is cleared to handle the next record.
This process continues until all the memory range is managed. Memory will
only be released when a consecutive set of balloon request entries from
the same host page reaches the full host page size. If a balloon request
entry from a different host page is encountered, the bitmap and the base
host page address will be reset. Consequently, memory is released in
units of the page size, ensuring efficient memory management. That's say
if memory range length to be released smaller than page size or if the
guest scatters requests each of whose size is smaller than page size
across different host pages no memory will be released.
[1] https://patchwork.kernel.org/project/qemu-devel/patch/20190214043916.22128-6-david@gibson.dropbear.id.au/
Fixes: cloud-hypervisor#5369
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Add pending removed vcpu check according to VcpuState.removing, which
can avoid cloud hypervisor hangup during continual vcpu resize.
Fix#5419
Signed-off-by: Yi Wang <foxywang@tencent.com>
These had to be previously disabled because a released binary has to be
used in course of the test scenario. The necessary functionality is now
supported with the release 34.0, thus these tests should now pass.
Fixes: #5660
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
We fixed the L2 and L3 level cache sharing issues and confirmed that the
L2 level cache is independent, while the L3 level cache is shared per-socket.
See:#5505
Signed-off-by: zhongbingnan <zhongbingnan@bytedance.com>
This patch modifies `event_monitor` to ensure that concurrent access to
`event_log` from multiple threads is safe. Previously, the `event_log`
function would acquire a reference to the event log file and write
to it without doing any synchronization, which made it prone to
data races. This issue likely went under the radar because the
relevant `SAFETY` comment on the unsafe block was incomplete.
The new implementation spawns a dedicated thread named `event-monitor`
solely for writing to the file. It uses the MPMC channel exposed by
`flume` to pass messages to the `event-monitor` thread. Since
`flume::Sender<T>` implements `Sync`, it is safe for multiple threads
to share it and send messages to the `event-monitor` thread.
This is not possible with `std::sync::mpsc::Sender<T>` since it's
`!Sync`, meaning it is not safe for it to be shared between different
threads.
The `event_monitor::set_monitor` function now only initializes
the required global state and returns an instance of the
`Monitor` struct. This decouples the actual logging logic from the
`event_monitor` crate. The `event-monitor` thread is then spawned by
the `vmm` crate.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
These tests require a binary from a previous release to perform a
migration. That binary doesn't yet have the shutdown fixes and thus
these tests are supposed to fail. The tests are to be re-enabled after
the next release.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The fixup_msix_region() function added in
a718716831
made the assumption that MSI-X was always available. This is the case
with many VFIO devices and all our virtio devices but created regression
with MSI devices.
Simply return the existing region size if MSI-X is not supported by the
device.
Fixes: #5649
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
With the addition of the spinning waiting for the exit event to be
received in the CMOS device a regression was introduced into the CMOS
fuzzer. Since there is nothing to receive the event in the fuzzer and
there is nothing to update the bit the that the device is looping on;
introducing an infinite loop.
Use an Option<> type so that when running the device in the fuzzer no
Arc<AtomicBool> is provided effectively disabling the spinning logic.
Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61165
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
As the container image can be used on both Intel and AMD,
ensure the SPDK binaries are compatible. This implies -march=skylake
passed to the underlaying toolchain.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Now feature "mshv" can be built together with "kvm". There is no need to
use "--no-default-features" any more.
Fixes: #5647
Signed-off-by: Bo Chen <chen.bo@intel.com>
The reset system is asynchronous with an I/O event (PIO or MMIO) for
ACPI/i8042/CMOS triggering a write to the reset_evt event handler. The
VMM thread will pick up this event on the VMM main loop and then trigger
a shutdown in the CpuManager. However since there is some delay between
the CPU threads being marked to be killed (through the
CpuManager::cpus_kill_signalled bool) it is possible for the guest vCPU
that triggered the exit to be re-entered when the vCPU KVM_RUN is called
after the I/O exit is completed.
This is undesirable and in particular the Linux kernel will attempt to
jump to real mode after a CMOS based exit - this is unsupported in
nested KVM on AMD on Azure and will trigger an error in KVM_RUN.
Solve this problem by spinning in the device that has triggered the
reset until the vcpus_kill_signalled boolean has been updated
indicating that the VMM thread has received the event and called
CpuManager::shutdown(). In particular if this bool is set then the vCPU
threads will not re-enter the guest.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Currently if container registry is inaccessible the image will be built
locally and that takes time. This patch adds support to use mirror
registry. To use a different registry CTR_IMAGE environment variable
must be set. For example:
CTR_IMAGE="registry/cloud-hypervisor" scripts/dev_cli.sh
or
export CTR_IMAGE="registry/cloud-hypervisor"
scripts/dev_cli.sh
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Split interrupt source group restore into two steps, first restore
the irqfd for each interrupt source entry, and second restore the
GSI routing of the entire interrupt source group.
This patch will reduce restore latency of interrupt source group,
and in a 200-concurrent restore test, the patch reduced the
average IOAPIC restore time from 15ms to 1ms.
Signed-off-by: Yong He <alexyonghe@tencent.com>
The VFIO tests based on the NVMe emulation framework require cause a
high number of file descriptors to be opened on the system. On systems
with a low limit on opened files, tests like test_vfio_user can possibly
fail with "too many open files" exeption. This issue is fixed by raising
the corresponding limit.
A tricky detail here is, that the limit has to be changed in the test
container and not on the host. As the the container is executed in the
privileged mode, the setting in it will override the host value.
Related: #5426
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add integration test of coredump with no need pause.
As file of coredump has been tested in test_coredump(), so this
patch only test vm state after coredump.
Signed-off-by: Yi Wang <foxywang@tencent.com>
If the VMM is not already paused then pause the VM prior to executing
the coredump and then resume it after. If the VM is already paused then
the original state is maintained.
Signed-off-by: Yi Wang <foxywang@tencent.com>
As the commit 2f15d027c05f (KVM: x86: Properly handle APF vs disabled
LAPIC situation) has been merged into kernel 5.13, so update the minimum
performance host kernel version to that.
Signed-off-by: Yi Wang <foxywang@tencent.com>
The commit b92fe648e9 (vmm: cpu: Disable KVM_FEATURE_ASYNC_PF_INT in
CPUID) disabled APF (Asynchronous Page Fault) mechanism to address
problem that makes vcpu thread spin 100%. As the actual issue is in
KVM, which has been merged in commit 2f15d027c05f (KVM: x86: Properly
handle APF vs disabled LAPIC situation) since 2021, so it's okay to
re-enable APF now.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Add MSHV_CREATE_DEVICE, MSHV_SET_DEVICE_ATTR ioctls to filters. These
ioctls are required to passthrough PCI devices on mshv.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The pause of vcpu is async now, which makes the vm pause is not
synchronised. As virtio device calls paused_sync wait() to make
sure device_manager pause synchronously, if we make vcpu pause
synchronously, the vm pause can be synchronously then. After
vm.pause() returns the vm is really paused now.
This patch adds a AtomicBool variable to mark vcpu paused state,
to make sure the pause of CpuManager is synchronised.
Signed-off-by: Yi Wang <foxywang@tencent.com>
This test case creates a new qcow2 file using the image of ubuntu as
its backing file, and boot a virtual machine with this image file.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This preserves any data that the backing file had on a cluster when
doing a write to a subset of that cluster. These writes cause a
performance penalty on creating new clusters if a backing file is
present.
This commit is based on crosvm implementation:
5ad3bc3459
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Reads to qcow files with backing files will fall through to the backing
file if there is no allocated cluster. As of this change, a write will
still trash the cluster and hide any data already present.
This commit is based on crosvm implementation:
d8144a56e2
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit allows opening qcow with a backing file, which supports any
type implementing `BlockBackend`.
This commit is based on crosvm implementation:
9ca6039b03
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit introduces the trait `BlockBackend` with generic ops
including read, write and seek, which can be used for common I/O
interfaces for the block types without using `DiskFile` and `AsyncIo`.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit merges crates `qcow`, `vhdx` and `block_util` into the
crate `block`, which can allow `qcow` to use functions from `block_util`
without introducing a circular crate dependency.
This commit is based on crosvm implementation:
f2eecc4152
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit introduces a generic type `FixedVhd` for its sync and async
disk I/O types, and the disk I/O types will use it directly instead of
maintaining a file themselves.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Using this over the LocalApic supports APIC IDs (and hence number of
vCPUs) above 256 (size increases from u8 to u32.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
If the VM has been configured but not yet booted, all we need to do to
support removing a device is to remove it from the config, so it will
never be created.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This fixes the valid VM config unit tests, which would otherwise fail
to deserialize their expected JSON config due to the missing "gdb" field.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
error: private item shadows public glob re-export
Error: --> hypervisor/src/mshv/mod.rs:42:27
|
42 | CpuIdEntry, FpuState, LapicState, MsrEntry, SpecialRegisters, StandardRegisters,
| ^^^^^^^^^^
|
note: the name `LapicState` in the type namespace is supposed to be publicly re-exported here
--> hypervisor/src/mshv/mod.rs:16:9
|
16 | pub use mshv_bindings::*;
| ^^^^^^^^^^^^^^^^
note: but the private item here shadows it
--> hypervisor/src/mshv/mod.rs:42:27
|
42 | CpuIdEntry, FpuState, LapicState, MsrEntry, SpecialRegisters, StandardRegisters,
| ^^^^^^^^^^
= note: `-D hidden-glob-reexports` implied by `-D warnings`
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
warning: usage of `Arc<T>` where `T` is not `Send` or `Sync`
--> virtio-devices/src/vsock/device.rs:376:22
|
376 | backend: Arc::new(RwLock::new(backend)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: consider using `Rc<T>` instead or wrapping `T` in a std::sync type like `Mutex<T>`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#arc_with_non_send_sync
= note: `#[warn(clippy::arc_with_non_send_sync)]` on by default
The vsock backend may be shared between threads, so the type `B` in
`Vsock` should be `VsockBackend` and `Sync`.
Considering that `api_receiver` and `gdb_receiver` are only used in vmm
threads, the `Arc` can be replaced by `Rc`.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
warning: useless use of `vec!`
--> test_infra/src/lib.rs:111:30
|
111 | let mut events = vec![epoll::Event::new(epoll::Events::empty(), 0); 1];
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: you can use an array directly: `[epoll::Event::new(epoll::Events::empty(), 0); 1]`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_vec
= note: `#[warn(clippy::useless_vec)]` on by default
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
warning: casting raw pointers to the same type and constness is unnecessary (`*const protocol::MemoryRange` -> `*const protocol::MemoryRange`)
--> vm-migration/src/protocol.rs:280:17
|
280 | self.data.as_ptr() as *const MemoryRange as *const u8,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `self.data.as_ptr()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
= note: `#[warn(clippy::unnecessary_cast)]` on by default
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This gives users the chance to reduce the number of dependencies
included, which is generally good practice and also reduces code size.
Furthermore, `io_uring` specifically is a strong contender for something
one may wish to disable due to the syscall API's many security issues[1]
[1]: https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html
Signed-off-by: Manish Goregaokar <manishsmail@gmail.com>
test_cpu_affinity needs the number of host CPUs. Since it is possible
for the host to have more than 255 CPUs; increase the size of the
datatype used for parsing the string to accomodate this.
Signed-off-by: dom.song <dom.song@amperecomputing.com>
Add integration test for pvpanic, by two methods:
- the vendor id and device id of pci device in guest
- triggering a guest panic and check event-monitor.
Also, to support pvpanic-pci driver, add pvpanic config
in resources.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Implemetion of BusDevice and PciDevice trait for pvpanic device.
Pvpanic device can be implemented as an ISA device or as a
PCI device, we choose PCI device for now.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Introduce emulation of pvpanic device to allow cloud hypervisor to get
the notify from guest's pvpanic driver when guest kernel crash.
Signed-off-by: Yi Wang <foxywang@tencent.com>
This reverts commit efb579b224.
This PR was mistakenly merged due to the confusion around nightly
builds. The vm-memory update needs to be done in both the root workspace
and fuzz workspace together.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
On x86-64, when the underlying hypervisor platform is KVM, no
instruction emulator is necessary. KVM handles instruction boundaries
internally.
This change allows to skip the iced-x86 dependency on KVM, improving
build times, prunes the dependency graph and reduces network traffic
during the initial build.
For Hyper-V, the emulator is still necessary on x86-64, so nothing
changes there.
Signed-off-by: Christian Blichmann <cblichmann@google.com>
Otherwise, CONFIG_DEBUG_INFO will be enabled automatically bloating the
size of the kernel image.
Now the size of kernel image is reduced from 635M to 60M on x86_64.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Remove clippy:all and update it with correct clippy lint,
so that it gives proper information.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Between musl and glibc there is a difference in the signature of the
ioctl libc function. Use an anonymous cast to force the type coversion.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
According the std docs implementing From<..> is preferred since it
gives you Into<..> for free where the reverse isn’t true.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Add a Default implementation that delegates to new, so that clippy
can be removed.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Fix these warnings:
SC2086 info: Double quote to prevent globbing and word splitting.
DL3047 info: Avoid use of wget without progress bar. Use `wget --progress=dot:giga <url>`. Or consider using `-q` or `-nv` (shorthands for `--quiet` or `--no-verbose`).
SC2006 style: Use $(...) notation instead of legacy backticks `...`.
Ignore these warning cause they break the build or they do not apply:
DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
DL3003 warning: Use WORKDIR to switch to a directory
SC2016 info: Expressions don't expand in single quotes, use double quotes for that.
SC2046 warning: Quote this to prevent word splitting.
DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check
SC2155 warning: Declare and assign separately to avoid masking return values.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
hadolint error:
DL3015 info: Avoid additional packages by specifying `--no-install-recommends`
Add required packages ca-certificates unzip iproute2 dbus
Without these packages build or integration tests fail if
--no-install-recommends is specified. Previously these packages were
installed as part of recommended dependency packages.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Setting environment variable
DEBIAN_FRONTEND=noninteractive
Makes unnecessary setting it in all apt-get install commands
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Some parts of Dockerfile are indented with spaces others with tabs.
Unify indentation to use spaces.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Test was failing due to regression caused by commit
d5558aea2a
Failing command:
sudo /mnt/ch-remote --api-socket /tmp/ch_api.sock resize --memory=1073741824"
Fixes#5190
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
This does the same thing as df2a7c17 ("vmm: Ignore and warn TAP FDs
sent via the HTTP request body"), but for the vm.create endpoint,
which also previously would accept file descriptors in the body, and
try to use whatever fd occupied that number as a TAP device.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Port of df2a7c17 ("vmm: Ignore and warn TAP FDs sent via the HTTP
request body"), but for the vm.create endpoint, which would previously
accept file descriptors in the body, and try to use whatever fd
occupied that number as a TAP device.
Since I had to move the wrapping of the net config in an Arc until
after it was modified, I made the same change to all other endpoints,
so the style stays consistent.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Using the data from sysfs forward the host host cache layout to the
guest using the FDT tables.
TEST=The host cache layout (from sysfs) can be seen in inside the guest
using lscpu.
Signed-off-by: zhongbingnan <zhongbingnan@bytedance.com>
Currently, vfio device fails to initialize as the msix-cap region in BAR
is mapped as RW region.
To resolve the initialization issue, this commit avoids mapping the
msix-cap region in the BAR. However, this solution introduces another
problem where aligning the msix table offset in the BAR to the page
size may cause overlap with the MMIO RW region, leading to reduced
performance. By enlarging the entire region in the BAR and relocating
the msix table to achieve page size alignment, this problem can be
overcomed effectively.
Fixes: #5292
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
To avoid code duplication extract page related functions to their
own module and add utility functions for manipulating addresses
related to page sizes
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
This patch clarifies the assumptions we have regarding the guest address
space layout while creating memory mapping in E820 on x86_64 and fdt on
aarch64. It also explicitly checks on these assumptions and report
errors if these assumptions do not hold.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This commit renames `ram_region_sub_size` to `ram_region_available_size`
and make its value align down to the default page size or hugepage
size of the current memory zone, which can prevent the memory zone from
being split into misaligned parts. And if the available size of ram
region is zero, this region will be marked as consumed even it has
unused space.
Note that there is two methods to use hugepages.
1. Specify `hugepages` for `memory` or `memory-zone`, if the
`hugepage_size` is not specified, the value can be got by `statfs`
for `/dev/hugepages`.
2. Specify a `file` in hugetlbfs for `memory-zone`, the hugepage size
can also be got by `statfs` for the file.
The value for alignment will be the hugepage size if this memory zone
is using hugepages, otherwise the value will be default page size of
system.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The previous `arch_memory_regions` function will provide some memory
regions with the specified memory size and fill all the previous
regions before using the next one, but sometimes there may be no need
to fill up the previous one, e.g., the previous one should be aligned
with hugepage size.
This commit make `arch_memory_regions` function not take any
parameters and return the max available regions, the memory manager
can use them on demand.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The original codes did not consider that the previous memory region
might not be full and always set it to the maximum size.
This commit fixes this problem by creating memory mappings based on
the actual memory details in both E820 on x86_64 and fdt on aarch64.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The device manager will set tty or pty to raw mode, all the `\n` will
be LF without CR, which makes the output difficult to read.
This commit solves it by using `write` with `\r\n` instead of
`writeln`, which can print CR and LF explicitly.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Remove "enum_variant_names" clippy. Enumeration variant names should
specify their variant, not repeat the enumeration name.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Download firmware and focal image to fix issue of running vfio tests
with empty worlkoads directory:
$ scripts/dev_cli.sh tests --integration-vfio
+ cp /root/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw /root/workloads/vfio
cp: cannot stat '/root/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw': No such file or directory
+ cp /root/workloads/hypervisor-fw /root/workloads/vfio
cp: cannot stat '/root/workloads/hypervisor-fw': No such file or directory
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Program the APIC ID (CPUID leaf 0x1 EBX) with the CPU id. This resolves
an issue where the EDKII firmware expects the APIC ID to vary per-CPU.
Fixes: #5475
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Script `create-cloud-init.sh" uses the same cloud init data as the the CI
but this means it is somewhat overloaded with unhelpful network configuration
entries and scripts that are only needed for the CI.
For local testing data kept in test_data/cloud-init/ubuntu/local folder.
This contains minimum configuration data for user to test it locally.
Script assigns default IP address using "network-config" details
with --net "tap=" option. The default network interface will be "ens4".
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Script `create-cloud-init.sh" uses the same cloud init data as the the CI
but this means it is somewhat overloaded with unhelpful network configuration
entries and scripts that are only needed for the CI.
CI related data moved to test_data/cloud-init/ubuntu/ci folder.
Fixes: #4773
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
SerialBuffer uses VecDeque::extend, which calls realloc, which a
maximum buffer size of 1 MiB. Starting at allocation sizes of
128 KiB, musl's mallocng allocator will use mremap for the allocation.
Since this was not permitted by the seccomp rules, heavy write load
could crash cloud-hypervisor with a seccomp failure. (Encountered
using virtio-console, but I don't see any reason it wouldn't happen
for the legacy serial device too.)
Signed-off-by: Alyssa Ross <hi@alyssa.is>
In this way, our integration tests exercise the same set of build
features (e.g. "kvm,mshv") being used for releases.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Bump to the latest rust-vmm crates, including vm-memory, vfio,
vfio-bindings, vfio-user, virtio-bindings, virtio-queue, linux-loader,
vhost, and vhost-user-backend,
Signed-off-by: Bo Chen <chen.bo@intel.com>
Implemented a `TargetApi` enum to make the process of implementing
tests for the D-Bus and HTTP API more convenient.
Refactored `test_api_{create_boot, shutdown, pause_resume, delete}` tests
with the `TargetApi` enum to also implement tests for the D-Bus API.
Added a new test named `test_api_dbus_and_http_interleaved` that uses
both the HTTP and D-Bus API at the same time.
Modified integration test scripts to enable the `dbus_api` feature when
compiling and start a dbus-session when integration tests are run.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
Introduces three new CLI options, `dbus-service-name`,
`dbus-object-path` and `dbus-system-bus` to configure the DBus API.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit applies the previously created seccomp filter
to the `DbusApi` thread.
Also encloses the main loop of the `DBusApi` thread using
`std::panic::catch_unwind` and `AssertUnwindSafe` in order to mirror
the behavior of the HTTP API.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit adds support for graceful shutdown of the DBusApi thread
using `futures::channel::oneshot` channels. By using oneshot channels,
we ensure that the thread has enough time to send a response to the
`VmmShutdown` method call before it is terminated. Without this step,
the thread may be terminated before it can send a response, resulting
in an error message on the client side stating that the message
recipient disconnected from the message bus without providing a reply.
Also changes the default values for DBus service name, object path
and interface name.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit introduces three new dependencies: `zbus`, `futures`
and `blocking`. `blocking` is used to call the Internal API in zbus'
async context which is driven by `futures::executor`. They are all
behind the `dbus_api` feature flag.
The D-Bus API implementation is behind the same `dbus_api` feature
flag as well.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
Rust documentation comments follows 80 char per line. But some of
these help options doesn't have space in between. Longlines of
text is starting from beginning of the next line instead of
proper format.
Before fix:
Options:
--cpus boot=<boot_vcpus>,max=<max_vcpus>,topology=<threads_per_core>:<cores_per_die>:<dies_per_package>:<packages>,kvm_hyperv=on|off,max_phys_bits=<maximum_number_of_physical_bits>,affinity=<list_of_vcpus_with_their_associated_cpuset>,features=<list_of_features_to_enable>
After fix:
Options:
--cpus boot=<boot_vcpus>, max=<max_vcpus>,
topology=<threads_per_core>:<cores_per_die>:<dies_per_package>:<packages>,
kvm_hyperv=on|off,
max_phys_bits=<maximum_number_of_physical_bits>,
affinity=<list_of_vcpus_with_their_associated_cpuset>,
features=<list_of_features_to_enable>
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Commit 21d40d7 ("main: reset tty if starting the VM fails") changed
start_vmm() to join the vmm thread if an error happens after the vmm
thread is started. The implementation put all the error-prone code that
is run after the vmm is started in a closure, to be able to always join
the vmm thread, regardless of any error happening. However, it missed
propagating the error that might happen inside the closure back to the
main function, after joining the vmm thread.
For some cmd line options, the above issue inhibits proper error
reporting when starting a VM with invalid commands, as many parameters
are parsed after the vmm is started, thus if such parsing fails, no
error will be reported back to the user.
See: #5435
Fixes: 21d40d7 ("main: reset tty if starting the VM fails")
Signed-off-by: Rafael Mendonca <rafaelmendsr@gmail.com>
The refactoring on deferring address space allocation (#5169) broke TDX,
as TDX initialization needs to access guest memory for encryption and
measurement of guest pages.
Signed-off-by: Bo Chen <chen.bo@intel.com>
In current implementation, memory region used in vfio is assumed to
align to 4k which may cause error when the PAGE_SIZE is not 4k, like on
Arm, it can be 16k and 64k.
Remove this assumption and align memory resource used by vfio to
PAGE_SIZE then vfio can run on host with 64k PAGE_SIZE.
Fixes: #5292
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The current implementation of breadth first traversal for device tree
uses a temporary vector, therefore causes unnecessary memory copy.
Remove it and do it within vector nodes.
Signed-off-by: Hao Xu <howeyxu@tencent.com>
Unlike KVM, there's no internal handling for topoolgy under MSHV. Thus,
if no topology has been passed during the CH launch, use the boot CPUs
count to construct the topology struct.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Passing the CPUID leafs with the topology is integrated into the common
mechanism of setting and patching CPUID in Cloud Hypervisor. All the
CPUID values will be passed to the hypervisor through the register
intercept call.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Originally the AML only accepted one hex number for PCI segment
numbering. Change it to accept two numbers. That makes it possible to
add up to 256 PCI segments.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When I refactored this to centralise resetting the tty into
DeviceManager::drop, I tested that the tty was reset if an error
happened on the vmm thread, but not on the main thread. It turns out
that if an error happened on the main thread, the process would just
exit, so drop handlers on other threads wouldn't get run.
To fix this, I've changed start_vmm() to write to the VMM's exit
eventfd and then join the thread if an error happens after the vmm
thread is started.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
In particular the Std::write() API requires that the value implements
AsBytes and copies the slice representation into the table data. This
avoids unaligned writes which can cause a panic with the updated
toolchain.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
For structures that are used in SDT ACPI tables it is necessary for them
to implement this trait for the newly safe Std::write() API.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is used on older kernels where close_range() is not available.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Fixes: 505f4dfa ("vmm: close all unused fds in sigwinch listener")
On KVM this is provided by an ioctl, on MSHV this is constant. Although
there is a HV_MAXIMUM_PROCESSORS constant the MSHV ioctl API is limited
to u8.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Don't import via glob to avoid (unused) objects colliding in the
namespace. This fixes a beta clippy issue.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This fixes the following tests that have been consistently failing on
the CI:
[2023-04-22T07:00:53.760Z] failures:
[2023-04-22T07:00:53.760Z] common_parallel::test_focal_hypervisor_fw
[2023-04-22T07:00:53.760Z] common_parallel::test_focal_ovmf
I'm not sure of the origin of this check but it obviously dependent on
the underlying platform as the guest OS has not changed. Since it
depends on the host environment it doesn't make sense to assert for it.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
In this way, we can cover the scenario where a VM with hotplugged net
device using FDs can work properly with reboot.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The custom 'clone' duplicates 'preserved_fds' so that the validation
logic can be safely carried out on the clone of the VmConfig.
The custom 'drop' ensures 'preserved_fds' are safely closed when the
holding VmConfig instance is destroyed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Preserved FDs are the ones that share the same life-time as its holding
VmConfig instance, such as FDs for creating TAP devices.
Preserved FDs will stay open as long as the holding VmConfig instance is
valid, and will be closed when the holding VmConfig instance is destroyed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
If the VM is shut down, either it's going to be started again, in
which case we still want to be in raw mode, or the process is about to
exit, in which case canon mode will be set at the end of main.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Having PMU in guests isn't critical, and not all hardware supports
it (e.g. Apple Silicon).
CpuManager::init_pmu already has a fallback for if PMU is not
supported by the VCPU, but we weren't getting that far, because we
would always try to initialise the VCPU with KVM_ARM_VCPU_PMU_V3, and
then bail when it returned with EINVAL.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Cloud Hypervisor's vhost-user implementation will reconnect if it gets
disconnected from the backend. That means connections happen inside
the vhost-user seccomp sandbox, so all syscalls used in reconnecting
have to be allowed in that sandbox.
clock_nanosleep is used by Glibc, and nanosleep is used by musl.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Recently generated mshv-bindings has most of the registers
renamed. This patch renames some of the MSHV registers.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Currently we are overwriting the RUST_BACKTRACE if set
explicitly by user while running performance metrics tests
using dev_cli.sh
This change will allow user to set the RUST_BACKTRACE
while running performance metrics tests with dev_cli.sh
which invokes run_metrics.sh to run the performance binary.
We will set RUST_BACKTRACE to 1 if not set explicitly.
Signed-off-by: smit-gardhariya <gardhariya.smit@gmail.com>
Previously, we were only using it for PTYs, because for PTYs there's
no alternative. But since we have to have it for PTYs anyway, if we
also use it for TTYs, we can eliminate all of the code that handled
SIGWINCH for TTYs.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Now that the SIGWINCH listener has fallbacks for older kernels, we
don't expect it to routinely fail, so if there's an error setting it
up, we want to know about it.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This will allow the SIGWINCH listener to run on kernels older than
5.5, although on those kernels it will have to make 64 syscalls to
reset all the signal handlers.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The PTY main file descriptor had to be introduced as a parameter to
start_sigwinch_listener, so that it could be closed in the child.
Really the SIGWINCH listener process should not have any file
descriptors open, except for the ones it needs to function, so let's
make it more robust by having it close all other file descriptors.
For recent kernels, we can do this very conveniently with
close_range(2), but for older kernels, we have to fall back to closing
open file descriptors one at a time.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
It seems like these examples were always intended to be doctests,
since there are lines marked with "#" so that they are excluded from
the generated documentation, but they were not recognised as doc tests
because they were not formatted correctly.
The code needed some adjustments so that it would actually compile and
run as doctests.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
When doctests are built, the crate is built with itself as a
dependency via --extern. This causes a compiler error if using a
module with the name same as the crate, because it's ambiguous whether
it's referring to the module, or the extern version of the crate, so
it's necessary to disambiguate when using the hypervisor module here.
Fixes running cargo test --doc --workspace.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Doc comments are Markdown, and can include HTML tags. Anything in
angle brackets will therefore be inserted as an HTML tag into
rustdoc's output. If that's not intentional, the left angle bracket
needs to be escaped.
I haven't fixed the doc comments in src/main.rs, because argh doesn't
understand the escaping, so the backslashes would show up in the
--help output. I've opened https://github.com/google/argh/issues/159
about that.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
These need to be //! comments, because they apply to the module as a
whole, not to whatever directly follows the comment. Using ///
comments here resulted in documentation being attached to the wrong
thing, or not rendered at all.
I've also checked the Markdown formatting of these comments as
rendered by rustdoc, and fixed it where appropriate.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This doesn't need to be rendered in the HTML API documentation, and
wouldn't be formatted correctly if it was.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
It was not possible to build just hypervisor with Cargo's -p flag,
because it was not properly specifying the features it requires from
vfio-ioctls.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
balloon_free_page_reporting test case should not
work as expected. The reason is that MSHV pins
all the pages during the memory map for the guest.
Those pages can not be altered without unpinning the pages.
MSHV does not support modifying the pages during the guest
life cycle. This test case can be enabled once we add
VA backed VM support.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Unfortunately the build had a transient error and so needed to be
restarted which generated a build with a different date tag.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This commit will add timeout parameter to performance metrics
testcases as runtime argument. Testcases are facing the timeout
sometimes when we run with MSHV hypervisor. This change will
provide mechanism to pass the timeout as per the need while
running the testcases. If nothing is passed, default timeout or
timeout set as per the testcase will be followed.
Signed-off-by: smit-gardhariya <gardhariya.smit@gmail.com>
Tests of rate-limiter and vfio are running on bare-metal, so we need to
ensure the custom kernel is updated and rebuilt for them.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The tests require a fixed amount of huge page memory to run, rather than
encode that as fixed number of huge pages it should be calculated from
the size of memory required and the huge page size
Signed-off-by: Dom <peng6662001@163.com>
Significant API changes have occured, most significantly is the switch
to an approach which does not require vm-memory and can run no_std.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This latest stable Rust toolchain can help reduce the final binary size
around ~300k when comparing with building with Rust 1.62. Specifically,
the binary size now is 3.3M (previously 3.6M) with default feature, and
4.0M (previously 4.2M) with "kvm+mshv" feature (e.g. our release action
build) .
Signed-off-by: Bo Chen <chen.bo@intel.com>
Now cloud hypervisor will start signal thread to catch
SIGWINCH signal, cloud hypervisor then will resize the
guest console via vconsole.
This patch skip starting signal thread when there is no
need to resize guest console, such as console is not
configured.
Signed-off-by: Yong He <alexyonghe@tencent.com>
The PR #2333 added I/O rate limiter on block device, with some options
in `DiskConfig`. And the PR #2401 added rate limiter on virtio-net
device with same options, but it still throws `Error::ParseDisk`.
This commit fixes it with correct values.
Fixes: #2401
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Unused import 'xml' is redefined at:
> xml = ET.ElementTree(element=svg)
Hence, remove unused xml import.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Any error from the backend or from the protocol handling code will now
result in an error reply being sent. This is cleanly achieved by
splitting the command handling out into its own method and using the
Rust Result<> based error handling to trigger the generation of the
error reply.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the implementation of PCI devices in a different process
using the vfio-user protocol.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Replace the use of an enum with a bitfield representation which means
that is now possible to logical OR flags together.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Once error occur, vcpu thread may exit, this should
be critical event for the whole VM, we should fire
exit event and set vcpu state.
If we don't set vcpu state, the shutdown process
will hang at signal_thread, which is waiting the
vcpu state to change.
Signed-off-by: Yong He <alexyonghe@tencent.com>
This test (which relies on nesting) is failing on the VFIO worker. The tests that use the
dedicated hardware pass fine.
See: #5190
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We need to provide valid FDs while creating 'NetConfig' instances even
for unit tests. Closing invalid FDs would cause random unit test
failures.
Also, two identical 'NetConfig' instances are not allowed any more,
because it would lead to close the same FD twice. This is consistent
with the fact that a clone of a "NetConfig" instance is no
longer *equal* to the instance itself.
Fixes: #5203
Signed-off-by: Bo Chen <chen.bo@intel.com>
These are owned by the config (and are duplicated before being used to
create the `Tap` for the virtio-net device.)
By implementing Drop on NetConfig we have issues with moving out of
members that don't implement the Copy trait. This requires a small
adjustment to the unit tests that use the Default::default() function.
Fixes: #5197
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The custom version duplicates any FDs that have been provided so that
the validation logic used on hotplug, which takes a clone of the config,
can be safely carried out.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This code is indentical to what is in this repository. When a release
gets made we can then switch to that.
Fixes: #5122
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If swtpm becomes unresponsive, guest gets blocked at "recvmsg" on tpm's
data FD. This change adds a timeout to the data fd socket. If swtpm
becomes unresponsive guest waits for "timeout" (secs) and continues to
run after returning an I/O error to tpm commands.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The updated image is configured in a same way as the previously used
2019, it has same
- Credentials
- Services configured, like SAC, SSH, RDP
- Size
All the Windows updates are applied so the state is current to the date.
Also, the latest stable version 0.1.229 of the VirtIO Windows drivers
is installed.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
We can ideally defer the address space allocation till we start the
vCPUs for the very first time. Because the VM will not access the memory
until the CPUs start running. Thus there is no need to allocate the
address space eagerly and wait till the time we are going to start the
vCPUs for the first time.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This hypervisor leaf includes details of the TSC frequency if that is
available from KVM. This can be used to efficiently calculate time
passed when there is an invariant TSC.
TEST=Run `cpuid` in the guest and observe the frequency populated.
Fixes: #5178
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Updates include:
- Add references to 'TDX Tools'
- Expand instructions on buidling and using TDShim
- Add version information of guest/host kernel, TDVF, TDShim being tested
Signed-off-by: Bo Chen <chen.bo@intel.com>
This is required for booting Linux:
From: https://lore.kernel.org/all/20221028141220.29217-3-kirill.shutemov@linux.intel.com/
"""
Virtualization Exceptions (#VE) are delivered to TDX guests due to
specific guest actions such as using specific instructions or accessing
a specific MSR.
Notable reason for #VE is access to specific guest physical addresses.
It requires special security considerations as it is not fully in
control of the guest kernel. VMM can remove a page from EPT page table
and trigger #VE on access.
The primary use-case for #VE on a memory access is MMIO: VMM removes
page from EPT to trigger exception in the guest which allows guest to
emulate MMIO with hypercalls.
MMIO only happens on shared memory. All conventional kernel memory is
private. This includes everything from kernel stacks to kernel text.
Handling exceptions on arbitrary accesses to kernel memory is
essentially impossible as handling #VE may require access to memory
that also triggers the exception.
TDX module provides mechanism to disable #VE delivery on access to
private memory. If SEPT_VE_DISABLE TD attribute is set, private EPT
violation will not be reflected to the guest as #VE, but will trigger
exit to VMM.
Make sure the attribute is set by VMM. Panic otherwise.
There's small window during the boot before the check where kernel has
early #VE handler. But the handler is only for port I/O and panic as
soon as it sees any other #VE reason.
SEPT_VE_DISABLE makes SEPT violation unrecoverable and terminating the
TD is the only option.
Kernel has no legitimate use-cases for #VE on private memory. It is
either a guest kernel bug (like access of unaccepted memory) or
malicious/buggy VMM that removes guest page that is still in use.
In both cases terminating TD is the right thing to do.
"""
With this change Cloud Hypervisor can boot the current Linux guest
kernel.
Reported-By: Jiaqi Gao <jiaqi.gao@intel.com
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Do the following:
1. Use from_be_bytes to drop mutable slices.
2. Check for the exact buffer size throughout.
3. Simplify ptm_to_request where possible.
4. Make error messages style consistent.
Fix a typo in code comment while at it.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
There is no guarantee that the write can send the whole buffer at once.
In those rare occasions, we should return a sensible error.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The largest possible PTM response is only 16 bytes. Size the output
buffer correctly.
In the socket read function, rely on the caller to provide a
sufficiently large buffer. That eliminates another large stack variable.
In total this saves almost 8KB stack space.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Change "thead" to "thread".
Also make sure the two messages are distinguishable by adding "vmm" and
"vm" prefix.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The number of aligned operations can not be larger than the number of
descriptors. Initializing the capacity to 1 is good enough per the
observation that most of time there is only one data descriptor in a
given request.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Before Linux v6.0, AArch64 didn't support "socket" in "cpu-map"
(CPU topology) of FDT.
We found that clusters can be used in the same way of sockets. That is
the way we implemented the socket settings in Cloud Hypervisor. But in
fact it was a bug.
Linux commit 26a2b7 fixed the mistake. So the cluster nodes can no
longer act as sockets. And in a following commit dea8c0, sockets were
supported.
This patch fixed the way to configure sockets. In each socket, a default
cluster was added to contain all the cores, because cluster layer is
mandatory in CPU topology on AArch64.
This fix will break the socket settings on the guests where the kernel
version is lower than v6.0. In that case, if socket number is set to
more than 1, the kernel will treat that as FDT mistake and all the CPUs
will be put in single cluster of single socket.
The patch only impacts the case of using FDT, not ACPI.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Fix lowercase label to avoid "mkfs.fat: Warning: lowercase labels
might not work properly on some systems".
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
This patch adds a global execution timeout to the Jenkinsfile to avoid
infinite pending Jenkins pipelines, such as when certain worker nodes
are not available. The global execution timeout is now set to 4 hours
which is derived from total timeout of our longest stage (e.g. the
`Worker build`).
Fixes: #5148
Signed-off-by: Bo Chen <chen.bo@intel.com>
As a first time user of cloud-hypervisor and Rust environment
you get build errors with out this.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Right now integration test fails during the test run if
/dev/mshv or /dev/kvm does not exist. We should not
progress and exit early if not present.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Make the code more idiomatic by wrapping the actual size configured in
the returning Result type. This further allows simplifying
get_buffer_size.
The debug message in startup_tpm is more useful if it prints out the
actual size than the wanted size.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
In order to comply with latest TDX version, we rely onto the branch
kvm-upstream-2022.08.07-v5.19-rc8 from https://github.com/intel/tdx
repository. Updates are based on changes that happened in
arch/x86/include/uapi/asm/kvm.h headers file.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Dependabot will create a branch on the repo for it's updates this
triggers the release action (because it's the same event as a tag) which
will then fail leading to dependabot PRs not being automerged. Instead
only run the release check test on PRs or tag creation.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
KSM doesn't work with MSHV stack since guest memory is pinned
(`pin_user_pages`) and pinned pages cannot be merged.
So, don't run the test for mshv.
Signed-off-by: Anirudh Rayabharam <anrayabh@linux.microsoft.com>
This makes it much more convenient to build binaries for profiling
without having to modify the Cargo.toml file.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
A few breaking changes:
1. `-vvv` needs to be written as `-v -v -v`.
2. `--disk D1 D2` and others need to be written as `--disk D1 --disk D2`.
3. `--option=value` needs to be written as `--option value`
Change integration tests to adapt to the breaking changes.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Since argh does not support `--option=value`, we need to change the
integration test code to become `--option value`.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Since we run "cargo clean" before running the aarch64 build we need to
create the release and upload the x86-64 assets before the clean.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Some crates don't need it at all.
Some crates are using it for a simple functionality which can be
replaced easily.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This was causing issues with the release build process but we now have a
fix to clean residual build assets.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Adjust the release workflow to move the conditional check on the tag
creation into the steps that create the release/upload the assets.
This allows us to ensure we're always in a releaseable state.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This reverts commit 998fb48ff2.
This is breaking the release build process (cross build for aarch64
musl) so temporarily reverting until we can identify the cause to allow
the release proceeed.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This change is important to do a proper resource cleanup. We decided
to do this repetitive approach as VirtioCommon can't implement Drop
without major changes to the corresponding code. Also, devices such as
Net can't easily use the epoll_threads-abstraction from VirtioCommon as
it has multiple threads with different semantics.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Add new configuration for offloading features, including
Checksum/TSO/UFO, and set these offloading features as
enabled by default.
Fixes: #4792.
Signed-off-by: Yong He <alexyonghe@tencent.com>
There was an unnecessary change in previous PR #5077.
This is the follow-up clean up patch.
Right now there is no use case of the drive of
Eq and PartialEq.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add new latency counters for virtio-block device, including
minimal latency, maximal latency, and average latency for block
read and write.
The average latency is calculated based on cumulative average.
Signed-off-by: Yong He <alexyonghe@tencent.com>
MSHV does not require to ensure MMIO/PIO exits complete
before pausing. This patch makes sure the above requirement
by checking the hypervisor type run-time.
Fixes#5037
Signed-off-by: Muminul Islam <muislam@microsoft.com>
e.g. on QEMU on KVM:
cloud-hypervisor: 17.079406ms: <vmm> INFO:arch/src/x86_64/mod.rs:565 -- Running under nested virtualisation. Hypervisor string: KVMKVMKVM
Or under Azure:
cloud-hypervisor: 3.881263ms: <vmm> INFO:arch/src/x86_64/mod.rs:565 -- Running under nested virtualisation. Hypervisor string: Microsoft Hv
Fixes: #5067
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
For the synchronous backends efficiently preserve the order for
completion requests through the use of VecDequeue. Preserving the order
is not required but is beneficial as it matches the existing
optimisation that looks to match completions and requests.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Rather than aggregate the completion list into an intermediate vector
instead adjust the API to provide one completion item at a time.
With DHAT this shows the number of heap allocations has decreased.
Before:
dhat: Total: 623,852 bytes in 8,157 blocks
After:
dhat: Total: 380,444 bytes in 3,469 blocks
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The number of buffers in the request is usually one so by using SmallVec
the number of heap allocations can be significantly reduced.
DHAT reports:
Before:
dhat: Total: 1,166,412 bytes in 40,383 blocks
After:
dhat: Total: 623,852 bytes in 8,157 blocks
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Rather than passing the vector of iovecs for the I/O to act on pass a
reference to the slice of values inside them. This removes the explicit
container type from the API allowing the use of e.g. SmallVec.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This functionality has been obsoleted by our native support for
hugepages and shared memory.
See: #5082
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The number of entries in the refcount table was incorrectly calculated
given there was no need for dividing the number of refblock clusters.
The number of refblock clusters is the number of entries in the refcount
table.
Suggested-by: lv_mz <lv.mengzhao@zte.com.cn>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
During analysis of the asynchrous block I/O handling it was observed
that the majority of the time the completion events occur in the same
order as submissions. Further the maximum number of inflight requests
during the boot time is much lower than the size of the queue.
Through the use of a double ended queue (VecDequeue) with a reasonable
pre-allocation capacity we can have O(1) allocation free addition of
items to the list of inflight requests and mostly O(1) matching of
completed requests to submissions.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since SGX testing doesn't rely on a custom guest image anymore, there's
no need to keep the custom filename around as it's already not in use.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When using DHAT[1] for analysis the amount heap allocations change from:
dhat: Total: 3,186,536 bytes in 41,452 blocks
to
dhat: Total: 1,059,816 bytes in 34,747 blocks
When running against virtio-block; this still more allocations than
virtio-pmem but a significant improvement without any cost.
[1] https://docs.rs/dhat/latest/dhat/
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
With this change, all log messages will have the same width for the
timestamp. The number of ms is rounded to 6 decimal places.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
To align the logging messages with the rest of the code, this
message should be aligned with another similar occurrence in
epoll_helper.rs
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Some crates don't need it at all.
Some crates are using it for a simple functionality which can be
replaced easily.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
While measuring UDP PPS, we saturate the link, so there are packets
lost. We only account for the packets that are not lost.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Change fio_ops to fio_control and add a new field to indicate whether it
should report bandwidth or IOPS.
All existing tests are bandwidth tests. Adapt the code accordingly. Add
a set of new tests to report IOPS.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
This OS is EOL this year and is well tested by the Rust Hypervisor
Firmware CI so there is no need to duplicate this effort.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
With this bump there was a change in one of the externally exposed
variable. Thus, the use of that variable in CLH must be adjusted
accordingly.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The double underscore made it different from how other projects would
name this particular macro.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
There is duplicated code when handlin queue events in handle_event()
refactor and introduce a new helper function.
Signed-off-by: Hao Xu <howeyxu@tencent.com>
Between musl and glibc there is a difference in the signature of the
ioctl libc function. Use an anonymous cast to force the type coversion.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
after testing the build briefly, it seems upstream tianocore EDK2 works fine,
no need for the separate cloud-hypervisor edk2 repo.
Signed-off-by: Claudio Fontana <claudio.fontana@gmail.com>
Remove from the documentation and API definition but continue support
using the field (with a deprecation warning.)
See: #4837
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since the refactoring of the vm-migration crate broke the backward
compatibility, we must disable the live upgrade tests until next
release.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This simplifies the Snapshot creation as we expect a SnapshotData to be
provided most of the time.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The information about the identifier related to a Snapshot is only
relevant from the BTreeMap perspective, which is why we can get rid of
the duplicated identifier in every Snapshot structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
There's no reason to carry a HashMap of SnapshotDataSection per
Snapshot. And given we now provide at most one SnapshotDataSection per
Snapshot, there's no need to keep the id part of the SnapshotDataSection
structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Without breaking the former way of declaring them. This is simply based
on the presence of the GUID TDX Metadata offset. If not present, we
consider the firmware is quite old and therefore we fallback onto the
previous way to expose memory resources.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In particular update to latest linux-loader release and point to latest
vfio repository for both crates hosted there.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The preferred way of retrieving the offset where to find the TDVF
descriptor structure is by going through a table of GUIDs that can be
found at a specific offset in the firmware file. If the expected GUIDs
can't be found, we can fallback onto the former way, which is to read
directly the value at a specific offset in the file.
This patch implements the new mechanism without breaking compatibility
for older firmwares as it keeps supporting the previous mechanism.
As a reference, here is the documentation from the EDK2 code, and
particularly from the OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm file:
```
GUIDed structure. To traverse this you should first verify the
presence of the table footer guid
(96b582de-1fb2-45f7-baea-a366c55a082d) at 0xffffffd0. If that
is found, the two bytes at 0xffffffce are the entire table length.
The table is composed of structures with the form:
Data (arbitrary bytes identified by guid)
length from start of data to end of guid (2 bytes)
guid (16 bytes)
so work back from the footer using the length to traverse until you
either find the guid you're looking for or run off the beginning of
the table.
```
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The datatype used for the ioctl() C library call is different between it
and the glibc toolchains. The easiest solution is to have the compiler
type cast to type of the parameter.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The coredump functionality is only implemented for x86_64 so it should
only be compiled in there.
Fixes: #4964
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
TDX was broken by the recent refactoring moving the vCPU creation
earlier than before. The simple and correct way to fix this problem is
by moving the TDX initialization right before the vCPUs creation. The
rest of the TDX setup can remain where it is.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This removes the storage of the GuestMemoryMmap on the CpuManager
further allowing the decoupling of the CpuManager from the
MemoryManager.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
When configuring the vCPUs it is only necessary to provide the guest
memory when booting fresh (for populating the guest memory). As such
refactor the vCPU configuration to remove the use of the
GuestMemoryMmap stored on the CpuManager.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Thanks to the new way of restoring Vm, we can now create the Vm object
directly with the appropriate VmState rather than having to patch it at
a later time.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
No need to provide a boolean to know if the VM is being restored given
we already have this information from the Option<Snapshot>.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now the entire codebase has been moved to the new restore design, we can
complete the work by creating a dedicated restore() function for the Vm
object and get rid of the method restore() from the Snapshottable trait.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
On aarch64 there is no modification of the app struct however
refactoring to remove this would be very intrusive.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The snapshot and restore of AArch64 Gic was done in Vm. Now it is moved
to DeviceManager.
The benefit is that the restore can be done while the Gic is created in
DeviceManager.
While the moving of state data from Vm snapshot to DeviceManager
snapshot breaks the compatability of migration from older versions.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Given the recent factorization that happened in vm.rs, we're now able to
merge Vm::new_from_snapshot with Vm::new.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This moves the devices creation out of the dedicated restore function
which will be eventually removed.
This factorizes the creation of all devices into a single location.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This allows the clock restoration to be moved out of the dedicated
restore function, which will eventually be removed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Based on all the work that has already been merged, it is now possible
to fully move DeviceManager out of the previous restore model, meaning
there's no need for a dedicated restore() function to be implemented
there.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Every Vcpu is now created with the right state if there's an available
snapshot associated with it. This simplifies the restore logic.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Following the new restore design, it is not appropriate to set every
virtio device threads into a paused state after they've been started.
This is why we remove the line of code pausing the devices only after
they've been restored, and replace it with a small patch in every virtio
device implementation. When a virtio device is created as part of a
restored VM, the associated "paused" boolean is set to true. This
ensures the corresponding thread will be directly parked when being
started, avoiding the thread to be in a different state than the one it
was on the source VM during the snapshot.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
To synthesize the interactions between the virtio-net device and the tap
interface, the fuzzer utilizes a pair of unix domain sockets: one socket
(e.g. the dummy tap frontend) is used to construct the 'net_util::Tap'
instance for creating a virtio-net device; the other socket (e.g. the
dummy tap backend) is used in a epoll loop for handling the tx and rx
requests from the virtio-net device.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Three functions are added:
* 'Tap::new_for_fuzzing()' a custom constructor that creates a dummy
`Tap` interface directly from `File` backed by Unix domain socket;
* 'Tap::mtu()' a custom function that returns hard-coded mtu;
* 'Net::wait_for_epoll_threads()'.
Two functions are reused with modifications to work with the dummy 'Tap'
interface:
* 'Net::new_with_tap()' is made public for fuzzing;
* 'Net::activate()' is modified to not call into 'Tap::set_offload()'
for fuzzing.
Signed-off-by: Bo Chen <chen.bo@intel.com>
To me the most logical place to document the policy is right next to the
version itself.
Fixes: #4318
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Now that VirtioPciDevice, VfioPciDevice and VfioUserPciDevice have all
been moved to the new restore design, there's no need to keep the old
way around, therefore the restore() implementations for MsiConfig,
MsixConfig and PciConfiguration can be removed.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This is some preliminatory work for moving both VfioUser and Vfio to the
new restore design.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This integer overflow was triggered with fuzzing on the virtio-net
device. The integer overflow is from the wrong assumption that the
packets read from or written to the tap device is always larger than the
size of a virtio-net header.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Moving the Ioapic object to the new restore design, meaning the Ioapic
is created directly with the right state, and it shares the same
codepath as when it's created from scratch.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Extend the existing `custom-image.md` document with a new section on how
to create a custom image that contains NVIDIA drivers that are required
for our VFIO baremetal CI.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Re-enable the VFIO integration now the machine is back online.
The image has been updated to rely on Ubuntu 22.04 (Jammy) and it's
smaller given only the NVIDIA drivers along with the nvidia-smi tool are
installed.
The test to verify the GPU is functional has been simplified given it
only relies on nvidia-smi to validate it has been able to find the Tesla
T4 card, meaning the associated driver was loaded correctly.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This uncouples it from the features used for building the binary under
test allowing it to use the default build features.
This change also removes the feature control from the test scripts where
it was never used (e.g. run_integration_tests_sgx.sh)
This allows the combined binary to be used for all testing but allows
the disabling of tests known not to work under mshv.
Fixes: #4915
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This will help with debugging issues like #4917 since we can find which
ioctl causes the error.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Creating a dedicated Result type for VirtioPciDevice, associated with
the new VirtioPciDeviceError enum. This allows for a clearer handling of
the errors generated through VirtioPciDevice::new().
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The code for restoring a VirtioPciDevice has been updated, including the
dependencies VirtioPciCommonConfig, MsixConfig and PciConfiguration.
It's important to note that both PciConfiguration and MsixConfig still
have restore() implementations because Vfio and VfioUser devices still
rely on the old way for restore.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This allows the unification of the same testing methodology with aarch64
and removes a user of the Bionic image.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In the new process, `device::Gic::new()` covers additional actions:
1. Creating `hypervisor::vGic`
2. Initializing interrupt routings
The change makes the vGic device ready in the beginning of
`DeviceManager::create_devices()`. This can unblock the GIC related
devices initialization in the `DeviceManager`.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Moving the creation of the vCPUs before the DeviceManager gets created
will allow for the aarch64 vGIC to be created before the DeviceManager
as well in a follow up patch. The end goal being to adopt the same
creation sequence for both x86_64 and aarch64, and keeping in mind that
the vGIC requires every vCPU to be created.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Split the vCPU creation into two distincts parts. On the one hand we
create the actual Vcpu object with the creation of the hypervisor::Vcpu.
And on the other hand, we configure the existing Vcpu, setting registers
to proper values (such as setting the entry point).
This will allow for further work to move the creation earlier in the
boot, so that the hypervisor::Vcpu will be already created when the
DeviceManager gets created.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
The CpuManager is now created before the DeviceManager. This is required
as preliminary work for creating the vCPUs before the DeviceManager,
which is required to ensure both x86_64 and aarch64 follow the same
sequence.
It's important to note the optimization for faster PIO accesses on the
PCI config space had to be removed given the VmOps was required by the
CpuManager and by the Vcpu by extension. But given the PciConfigIo is
created as part of the DeviceManager, there was no proper way of moving
things around so that we could provide PciConfigIo early enough.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This optimisation provided some peformance improvement when measured by
perf however when considered in terms of boot time peformance this
optimisation doesn't have any impact measurable using our
peformance-metrics tooling.
Removing this optimisation helps simplify the VMM internals as it allows
the reordering of the VM creation process permitting refactoring of the
restore code path.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This panic was triggered with fuzzing on the virtio-net device. This
commits handles the error explicitly to avoid the panic, which also
makes the fuzzer happy (as panic is treated as bugs).
Signed-off-by: Bo Chen <chen.bo@intel.com>
If the KVM version is too old (pre Linux 5.7) then fetch the CPUID
information from the host and use that in the guest. We prefer the KVM
version over the host version as that would use the CPUID for the
running CPU vs the CPU that runs this code which might be different due
to a hybrid topology.
Fixes: #4918
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Fixes: #4678
Currently release build is done on kvm feature only,
that makes live upgrade test on MSHV failing since
it does not find /dev/kvm. As Cloud-Hypervisor
supports both kvm and mshv in a single binary we should
make the release build with both KVM/MSHV feature enabled.
That way live upgrade test does not fail on MSHV.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
The jammy disk image has a new enough kernel to support SGX and if we
rely on just the CPUid information (which is sufficient) then we can use
the regular jammy test image for testing.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Cloud Hypervisor supports booting disk images containing all needed components
to run cloud workloads, a.k.a. cloud images.
to run cloud workloads, a.k.a. cloud images.
The following sample commands will download an Ubuntu Cloud image, converting
it into a format that Cloud Hypervisor can use and a firmware to boot the image
@@ -141,7 +140,10 @@ The Ubuntu cloud images do not ship with a default password so it necessary to
use a `cloud-init` disk image to customise the image on the first boot. A basic
`cloud-init` image is generated by this [script](scripts/create-cloud-init.sh).
This seeds the image with a default username/password of `cloud/cloud123`. It
is only necessary to add this disk image on the first boot.
is only necessary to add this disk image on the first boot. Script also assigns
default IP address using `test_data/cloud-init/ubuntu/local/network-config` details
with `--net "mac=12:34:56:78:90:ab,tap="` option. Then the matching mac address
interface will be enabled as per `network-config` details.
```shell
$ sudo setcap cap_net_admin+ep ./cloud-hypervisor
@@ -173,23 +175,31 @@ $ ./cloud-hypervisor \
#### Building your Kernel
Cloud Hypervisor also supports direct kernel boot into a `vmlinux` ELF kernel (compiled with PVH support). In order to support development there is a custom branch; however provided the required options are enabled any recent kernel will suffice.
Cloud Hypervisor also supports direct kernel boot. For x86-64, a `vmlinux` ELF kernel (compiled with PVH support) is needed. In order to support development there is a custom branch; however provided the required options are enabled any recent kernel will suffice.
Direct kernel boot to userspace should work with a rootfs from most
distributions although you may need to enable exotic filesystem types in the
@@ -331,8 +375,8 @@ are all equal and welcome means of contribution. See the
## Slack
Get an [invite to our Slack channel](https://join.slack.com/t/cloud-hypervisor/shared_invite/enQtNjY3MTE3MDkwNDQ4LWQ1MTA1ZDVmODkwMWQ1MTRhYzk4ZGNlN2UwNTI3ZmFlODU0OTcwOWZjMTkwZDExYWE3YjFmNzgzY2FmNDAyMjI)
and [join us on Slack](https://cloud-hypervisor.slack.com/).
Get an [invite to our Slack channel](https://join.slack.com/t/cloud-hypervisor/shared_invite/enQtNjY3MTE3MDkwNDQ4LWQ1MTA1ZDVmODkwMWQ1MTRhYzk4ZGNlN2UwNTI3ZmFlODU0OTcwOWZjMTkwZDExYWE3YjFmNzgzY2FmNDAyMjI),
[join us on Slack](https://cloud-hypervisor.slack.com/), and [participate in our community activities](https://cloud-hypervisor.slack.com/archives/C04R5DUQVBN).
| Create the VM | `/vm.create`| `/schemas/VmConfig` | N/A | The VM is not created yet |
| Delete the VM | `/vm.delete`| N/A | N/A | N/A |
| Boot the VM | `/vm.boot`| N/A | N/A | The VM is created but not booted |
| Shut the VM down | `/vm.shutdown`| N/A | N/A | The VM is booted |
| Reboot the VM | `/vm.reboot`| N/A | N/A | The VM is booted |
| Trigger power button of the VM | `/vm.power-button`| N/A | N/A | The VM is booted |
| Pause the VM | `/vm.pause`| N/A | N/A | The VM is booted |
| Resume the VM | `/vm.resume`| N/A | N/A | The VM is paused |
| Task a snapshot of the VM | `/vm.snapshot`| `/schemas/VmSnapshotConfig` | N/A | The VM is paused |
| Perform a coredump of the VM* | `/vm.coredump` | `/schemas/VmCoredumpData` | N/A | The VM is paused |
| Restore the VM from a snapshot | `/vm.restore`| `/schemas/RestoreConfig` | N/A | The VM is created but not booted |
| Add/remove CPUs to/from the VM | `/vm.resize`| `/schemas/VmResize` | N/A | The VM is booted |
| Add/remove memory from the VM | `/vm.resize`| `/schemas/VmResize` | N/A | The VM is booted |
| Add/remove memory from a zone | `/vm.resize-zone`| `/schemas/VmResizeZone` | N/A | The VM is booted |
| Dump the VM information | `/vm.info`| N/A | `/schemas/VmInfo` | The VM is created |
| Add VFIO PCI device to the VM | `/vm.add-device`| `/schemas/VmAddDevice` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add disk device to the VM | `/vm.add-disk`| `/schemas/DiskConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add fs device to the VM | `/vm.add-fs`| `/schemas/FsConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add pmem device to the VM | `/vm.add-pmem`| `/schemas/PmemConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add network device to the VM | `/vm.add-net`| `/schemas/NetConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add userspace PCI device to the VM | `/vm.add-user-device` | `/schemas/VmAddUserDevice` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add vdpa device to the VM | `/vm.add-vdpa`| `/schemas/VdpaConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add vsock device to the VM | `/vm.add-vsock`| `/schemas/VsockConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Remove device from the VM | `/vm.remove-device`| `/schemas/VmRemoveDevice` | N/A | The VM is booted |
| Dump the VM counters | `/vm.counters`| N/A | `/schemas/VmCounters` | The VM is booted |
| Prepare to receive a migration | `/vm.receive-migration` | `/schemas/ReceiveMigrationData` | N/A | N/A |
| Start to send migration to target | `/vm.send-migration` | `/schemas/SendMigrationData` | N/A | The VM is booted and (shared mem or hugepages enabled) |
### REST API Examples
* The `vmcoredump` action is available exclusively for the `x86_64`
architecture and can be executed only when the `guest_debug` feature is
enabled. Without this feature, the corresponding [REST API](#rest-api) or
[D-Bus API](#d-bus-api) endpoints are not available.
#### REST API Examples
For the following set of examples, we assume Cloud Hypervisor is started with
the REST API available at `/tmp/cloud-hypervisor.sock`:
@@ -122,7 +134,7 @@ Cloud Hypervisor Guest
Disk(s): None
```
#### Create a Virtual Machine
##### Create a Virtual Machine
We want to create a virtual machine with the following characteristics:
@@ -135,7 +147,7 @@ We want to create a virtual machine with the following characteristics:
@@ -4,7 +4,7 @@ Currently Cloud Hypervisor supports hot plugging of CPUs devices (x86 only), PCI
## Kernel support
For hotplug on Cloud Hypervisor ACPI GED support is needed. This can either be achieved by turning on `CONFIG_ACPI_REDUCED_HARDWARE_ONLY`
For hotplug on Cloud Hypervisor ACPI GED support is needed. This can either be achieved by turning on `CONFIG_ACPI_REDUCED_HARDWARE_ONLY`
or by using this kernel patch (available in 5.5-rc1 and later): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/patch/drivers/acpi/Makefile?id=ac36d37e943635fc072e9d4f47e40a48fbcdb3f0
## CPU Hot Plug
@@ -159,7 +159,7 @@ The same API can also be used to reduce the desired RAM for a VM. It is importan
Extra PCI devices can be added and removed from a running `cloud-hypervisor` instance. This is controlled by making a HTTP API request to the VMM to ask for the additional device to be added, or for the existing device to be removed.
Note: On AArch64 platform, PCI device hotplug can only be achieved using ACPI. Please refer to the [documentation](arm64.md#uefi-booting) for more information.
Note: On AArch64 platform, PCI device hotplug can only be achieved using ACPI. Please refer to the [documentation](uefi.md#building-uefi-firmware-for-aarch64) for more information.
To use PCI device hotplug start the VM with the HTTP server.
@@ -7,13 +7,13 @@ The target audience of this document is both:
## Control
The number of `-v` parameters passed to the `cloud-hypervisor` binary will determine the log level. Currenly the default is log messages up to `WARN:` (`warn!`) are included by default. The `--log-file` allows the log to be sent to a location other than `stderr`.
The number of `-v` parameters passed to the `cloud-hypervisor` binary will determine the log level. Currently the default is log messages up to `WARN:` (`warn!`) are included by default. The `--log-file` allows the log to be sent to a location other than `stderr`.
## Levels
### `error!()`
For immediate, unrecoverable errors where it does not make sense for the execution to continue as the behaviour of the VM is considerablely impacted.
For immediate, unrecoverable errors where it does not make sense for the execution to continue as the behaviour of the VM is considerably impacted.
Cloud Hypervisor should exit shortly after reporting this error (with a non-zero exit code). Generally this should be used during initial construction of the VM state before the virtual CPUs have begun running code.
@@ -23,7 +23,7 @@ Users should react to this error by checking their initial VM configuration.
### `warn!()`
A serious problem has occured but the execution of the VM can continue although some functionality might be impacted.
A serious problem has occurred but the execution of the VM can continue although some functionality might be impacted.
A typical example of where this level of message should be generated is during an API call request that cannot be fulfilled.
--memory-zone <memory-zone> User defined memory zone parameters "size=<guest_memory_region_size>,file=<backing_file>,shared=on|off,hugepages=on|off,hugepage_size=<hugepage_size>,host_numa_node=<node_id>,id=<zone_identifier>,hotplug_size=<hotpluggable_memory_size>,hotplugged_size=<hotplugged_memory_size>,prefault=on|off"
```
This parameter expects one or more occurences, allowing for a list of memory
This parameter expects one or more occurrences, allowing for a list of memory
zones to be defined. It must be used with `--memory size=0`, clearly indicating
that the memory will be described through advanced parameters.
@@ -266,11 +266,8 @@ _Example_
### `file`
Path to the file backing the memory zone. This can be either a file or a
directory. In case of a file, it will be opened and used as the backing file
for the `mmap(2)` operation. In case of a directory, a temporary file with no
hard link on the filesystem will be created. This file will be used as the
backing file for the `mmap(2)` operation.
Path to the file backing the memory zone. The file will be opened and used as
the backing file for the `mmap(2)` operation.
This option can be particularly useful when trying to back a part of the guest
RAM with a well known file. In the context of the snapshot/restore feature, and
@@ -575,7 +572,16 @@ _Example_
### PCI bus
Cloud Hypervisor supports only one PCI bus, which is why it has been tied to
the NUMA node 0 by default. It is the user responsibility to organize the NUMA
nodes correctly so that vCPUs and guest RAM which should be located on the same
NUMA node as the PCI bus end up on the NUMA node 0.
Cloud Hypervisor supports guests with one or more PCI segments. The default PCI segment always
has affinity to NUMA node 0. Be default, all other PCI segments have afffinity to NUMA node 0.
The user may configure the NUMA affinity for any additional PCI segments.
The Microsoft Hypervisor is a Type 1 hypervisor which runs on x64 and ARM64 architectures. As the foundation of the Hyper-V virtualization stack, it runs millions of Linux and Windows guests in Azure and on-premises deployments. It supports nested virtualization, and security features like AMD's SEV-SNP. It also supports various features in Windows such as [Device guard and confidential guard](https://techcommunity.microsoft.com/t5/iis-support-blog/windows-10-device-guard-and-credential-guard-demystified/ba-p/376419), and [WSL2](https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq)
Since 2020, Microsoft has been releasing open-source components to support Linux running as root partition on the Microsoft Hypervisor.
1. Kernel patches to support Linux booting as root partition
2. A Linux kernel driver exposing an IOCTL interface for managing guest partitions, via a device node - /dev/mshv
3. Rust bindings and IOCTL wrappers
4. IGVM related crates
## Components
The following components are related to MSHV support with Cloud-Hypervisor:
* [mshv-crates](https://github.com/rust-vmm/mshv) : Rust crates to interact with kernel module (/dev/mshv)
* [igvm-tooling](https://github.com/microsoft/igvm-tooling) : Tool to generate IGVM file
## IGVM
Independent Guest Virtual Machine (IGVM) file format.The format specification can be found in the igvm_defs crate, with a Rust implementation of the binary format in the igvm crate.
The IGVM file format is designed to encapsulate all information required to launch a virtual machine on any given virtualization stack, with support for different isolation technologies such as AMD SEV-SNP and Intel TDX.
At a conceptual level, this file format is a set of commands created by the tool that generated the file, used by the loader to construct the initial guest state. The file format also contains measurement information that the underlying platform will use to confirm that the file was loaded correctly and signed by the appropriate authorities.
Cloud Hypervisor can be built using igvm feature flag along with mshv and/or sev-snp. IGVM only works with MSHV.
## SEV-SNP
AMD's [Secure Encrypted Virtualization (SEV)](https://www.amd.com/en/developer/sev.html) and extensions such as Secure Nested Paging (SEV-SNP) encrypt memory and restrict access to a guest VM's memory and registers, securing it against a compromised hypervisor or VMM. They utilize the Platform Security Processor (PSP) to store keys and encrypt/decrypt the data. Microsoft has been continuously adding/improving support for SEV-SNP on Microsoft Hyper-V. Cloud-Hypervisor can be built with the sev_snp feature including mshv and igvm feature.
## Use Cases
Cloud Hypervisor can be built to run on an MSHV root partition by enabling the mshv feature, e.g.:
Cloud Hypervisor supports UEFI boot through the utilization of the EDK II based UEFI firmware.
## Building UEFI Firmware
## Building UEFI Firmware for x86-64
To avoid any unnecessary issues, it is recommended to use Ubuntu 18.04 and its default toolset. Any other compatible Linux distribution is otherwise suitable, however it is suggested to use a temporary Docker container with Ubuntu 18.04 for a quick build on an existing Linux machine.
Please note that nasm-2.15 is required for the build to succeed.
The commands below will compile an OVMF firmware suitable for Cloud Hypervisor.
Any UEFI capable image can be booted using the Cloud Hypervisor specific firmware. Windows guests under Cloud Hypervisor only support UEFI boot, therefore OVMF is mandatory there.
To make Cloud Hypervisor use UEFI boot, pass the `CLOUDHV.fd` file path as an argument to the `--kernel` option. The firmware file will be opened in read only mode.
To make Cloud Hypervisor use UEFI boot, pass the `CLOUDHV.fd` (for x86-64) / `CLOUDHV_EFI.fd` (for AArch64) file path as an argument to the `--kernel` option. The firmware file will be opened in read only mode.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
