Update the vhost-user-backend crate version used along with related
crates (vhost and virtio-queue.) This requires minor changes to the
types used for the memory in the backends with the use of the
BitmapMmapRegion type for the Bitmap implementation.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit d10f20eb71)
Replace the specific Bitmap implementation from the type signature used
for functions that take memory. This allows more flexibility when using
these functions in particular when these functions are used by the
vhost-user-block backend. An updated vhost-user-backend crate requires
extra constraints on the Bitmap implementation used (it must support
BitmapReplace which AtomicBitmap does not.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 6c0dedd560)
Replace the specific Bitmap implementation from the type signature used
for functions that take memory. This allows more flexibility when using
these functions in particular when these functions are used by the
vhost-user-net backend. An updated vhost-user-backend crate requires
extra constraints on the Bitmap implementation used (it must support
BitmapReplace which AtomicBitmap does not.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit b29edfbee8)
Our Azure VM for x86_64 workers are now much smaller, and does not have
enough RAM to run the "test_live_upgrade_numa" test. Instead, this test
will still be tested on the aarch64 worker, and the "local" upgrade
variation of the same test will also be tested on all workers. So we
should be good from test coverage point of view.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Use 'kvm_vcpu_events_old' and 'kvm_clock_data_old' to support
deserialization from legacy definitions of kvm structs, so that we can
support live-upgrade from previous point releases.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Beta clippy fix
warning: useless use of `vec!`
--> tests/integration.rs:5845:23
|
5845 | let kernels = vec![direct_kernel_boot_path()];
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: you can use an array directly: `[direct_kernel_boot_path()]`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_vec
= note: `#[warn(clippy::useless_vec)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 61afd93a50)
Beta clippy fix:
warning: initializer for `thread_local` value can be made `const`
--> vmm/src/sigwinch_listener.rs:27:40
|
27 | static TX: RefCell<Option<File>> = RefCell::new(None);
| ^^^^^^^^^^^^^^^^^^ help: replace with: `const { RefCell::new(None) }`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#thread_local_initializer_can_be_made_const
= note: `#[warn(clippy::thread_local_initializer_can_be_made_const)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 9dfc39d336)
Beta clippy fix:
warning: this call to `as_ref.map(...)` does nothing
--> vmm/src/device_manager.rs:1234:9
|
1234 | self.console_resize_pipe.as_ref().map(Arc::clone)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `self.console_resize_pipe.clone()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_asref
= note: `#[warn(clippy::useless_asref)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit e70bf59809)
The following tests have been temporarily disabled:
1. Live upgrade/migration test with ovs-dpdk (#5532);
2. Disk hotplug tests on windows guests (#6037);
This patch has been tested with PR #6048.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Tested-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 24f384d239)
Bumping anyhow crate from 1.0.75 to 1.0.79 will cause seccomp
failures through integration tests. Newly added backtrace support
relies on readlink and many other syscalls.
Issue noticed with test_api_http_pause_resume test, where second time
of VM PAUSE or VM RESUME prints error and causes panic.
Noticed that panic message in a thread which is not allowed to write
output triggered the issue.
So implementing Display trait for HttpError and ApiError enums to avoid
adding many syscalls to seccomp filter section.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
(cherry picked from commit 895dc12a74)
The `ByteValued` trait implementations for the data structures from the
'linux_loader' crate are no longer needed, and hence their wrappers can
be removed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 9b0b881351)
Fix beta clippy issue:
error: file opened with `create`, but `truncate` behavior not defined
--> src/main.rs:624:26
|
624 | .create(true)
| ^^^^^^^^^^^^- help: add: `.truncate(true)`
|
= help: if you intend to overwrite an existing file entirely, call `.truncate(true)`
= help: if you instead know that you may want to keep some parts of the old file, call `.truncate(false)`
= help: alternatively, use `.append(true)` to append to the file instead of overwriting it
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#suspicious_open_options
= note: `-D clippy::suspicious-open-options` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::suspicious_open_options)]`
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit c1f4a7b295)
The embedded error in the enum will be read on debug output of the
error.
Fixes beta clippy issue:
warning: field `0` is never read
--> vhost_user_block/src/lib.rs:64:23
|
64 | CreateKillEventFd(io::Error),
| ----------------- ^^^^^^^^^
| |
| field in this variant
|
= note: `#[warn(dead_code)]` on by default
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
64 | CreateKillEventFd(()),
| ~~
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 107f4bdc12)
The embedded error in the enum will be read on debug output of the
error.
Fixes beta clippy issue:
warning: field `0` is never read
--> performance-metrics/src/performance_tests.rs:25:11
|
25 | Infra(InfraError),
| ----- ^^^^^^^^^^
| |
| field in this variant
|
= note: `#[warn(dead_code)]` on by default
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
25 | Infra(()),
| ~~
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 8899ebd63c)
warning: field `0` is never read
--> block/src/qcow/vec_cache.rs:139:21
|
139 | struct NumCache(pub u64);
| -------- ^^^^^^^
| |
| field in this struct
|
= note: `#[warn(dead_code)]` on by default
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field
|
139 | struct NumCache(());
| ~~
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit c19c73cb99)
CI reports errors:
error: writing `&Vec` instead of `&[_]` involves a new object where a slice will do
--> arch/src/x86_64/mod.rs:1351:19
|
1351 | epc_sections: &Vec<SgxEpcSection>,
| ^^^^^^^^^^^^^^^^^^^ help: change this to: `&[SgxEpcSection]`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
= note: `-D clippy::ptr-arg` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::ptr_arg)]`
Signed-off-by: Yi Wang <foxywang@tencent.com>
(cherry picked from commit 3d6594a594)
CI reports clippy errors:
error: argument to `Path::join` starts with a path separator
--> tests/integration.rs:4076:58
|
4076 | let serial_socket = guest.tmp_dir.as_path().join("/tmp/serial.socket");
| ^^^^^^^^^^^^^^^^^^^^
|
= note: joining a path starting with separator will replace the path instead
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#join_absolute_paths
Signed-off-by: Yi Wang <foxywang@tencent.com>
(cherry picked from commit ee2f0c3cb4)
CI reports clippy errors:
error: in a `match` scrutinee, avoid complex blocks or closures with blocks; instead, move the block or closure higher and bind it with a `let`
--> test_infra/src/lib.rs:93:51
|
93 | match (|| -> Result<(), WaitForBootError> {
| ___________________________________________________^
94 | | let listener =
95 | | TcpListener::bind(listen_addr.as_str()).map_err(WaitForBootError::Listen)?;
96 | | listener
... |
145 | | }
146 | | })() {
| |_________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#blocks_in_conditions
= note: `-D clippy::blocks-in-conditions` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::blocks_in_conditions)]`
Signed-off-by: Yi Wang <foxywang@tencent.com>
(cherry picked from commit 9c2d650cb8)
Consistent with the other data structures and constants used in TDX
support code import the necessary structures from the kernel for
accessing the vmcall structure.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 3993663e5c)
According to PCIe specification, a 64-bit MMIO BAR should be
naturally aligned. In addition to being more compliant with
the specification, natural aligned BARs are mapped with
the largest possible page size by the host iommu driver, which
should speed up boot time and reduce IOTLB thrashing for virtual
machines with VFIO devices.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
(cherry picked from commit c9f94be7ab)
The socket is nonblocking, so it's not guaranteed that it will be
possible to read the whole connect command in a single iteration of
the event loop. To reproduce:
(echo -n 'CONNECT '; sleep 1; echo 1234; cat) | socat STDIO UNIX-CONNECT:vsock.sock
This would produce the error:
cloud-hypervisor: 5.509209s: <_vsock4> INFO:virtio-devices/src/vsock/unix/muxer.rs:446 -- vsock: error adding local-init connection: UnixRead(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" })
To fix this, if we only get a partial command, we need to save it for
future iterations of the event loop, and only proceed once we've read
a complete command.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
(cherry picked from commit 48de800756)
The VIRTIO specification[1] says:
> The upper 32 bits of the CID are reserved and zeroed.
We should therefore not allow the user to supply a VSOCK CID with
those bits set. To accomplish this, limit the public API of the
virtio-vsock device to only accept 32-bit CIDs, while still using
64-bit CIDs internally since that's how virtio-vsock works.
[1]: https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-4400004
Signed-off-by: Alyssa Ross <hi@alyssa.is>
(cherry picked from commit 451d3fb2f0)
I accidentally ran a VM with CID 2 (VMADDR_CID_HOST), and very strange
and difficult to debug behavior ensued. I don't think a virtio-vsock
device should be allowed to have any of the special CIDs
(VMADDR_CID_ANY, VMADDR_CID_HYPERVISOR, VMADDR_CID_LOCAL, VMADDR_CID_HOST).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
(cherry picked from commit 7d0b85d727)
This PR addresses a bug in which the cpu topology of a guest
with non power-of-two number of cores is incorrect. For example,
in some contexts, a virtual machine with 2-sockets and 12-cores
will incorrectly believe that 16 cores are on socket 1 and 8
cores are on socket 2. In other cases, common topology enumeration
software such as hwloc will crash.
The root of the problem was the way that cloud-hypervisor generates
apic_id. On x86_64, the (x2) apic_id embeds information about cpu
topology. The cpuid instruction is primarily used to discover the
number of sockets, dies, cores, threads, etc. Using this information,
the (x2) apic_id is masked to determine which {core, die, socket} the
cpu is on. When the cpu topology is not a power of two
(e.g. a 12-core machine), this requires non-contiguous (x2) apic_id.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
(cherry picked from commit 5c0b66529a)
According to crates.io the crc-any crate is actively maintained which
avoids issues with the crc32c crate and the nightly compiler.
Fixes: #6168
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit d516374c39)
When the HTT flag CPUID.1.EDX[HTT] is 0, it indicates that there is
only a single logical processor in the package. When HTT is 1, it
indicates that CPUID.1.EBX[23:16] contains the number of logical
processors in the package.
When this information is not included in CPUID leaf 0x1, some cpu
topology enumeration software such as hwloc are known to crash.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
(cherry picked from commit 5ec47d4883)
When using amd topology, the svm feature flag on cpuid leaf
0x8000_0001.ecx is overwritten. We update the amd cpu topology
logic to use the flag values that originated in
KVM_GET_SUPPORTED_CPUID ioctl and override as necessary.
Signed-off-by: Thomas Barrett <tbarrett@crusoeenergy.com>
(cherry picked from commit 7bc764d4e0)
When running on the merge group this workflow is run twice - once for
the create event (merge queue creates a new branch) and once for the
merge_group event. Unfortunately the second event would cause the first
to be cancelled - unfortunately sometimes that second event is the
create event where the job in the workflow only runs if it is also a
tag.
By creating distinct concurrency groups for each event type then the
cross cancellation can be avoided.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 6f49d7f192)
The workers share a common public IP address and often GitHub will
reject attempts to access the API due to exceeding the anonymous rate
limit threshold.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 0f71956d6d)
Run these workflows as part of the merge queue to help improve testing
coverage.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit cdafe5344d)
When a bare-metal worker is canceled, its workspace can be left with
files owned by the root user as a result of running tests from our
container. This patch add a step to fix workspace permissions for such
case before checking out code.
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit f48942ce3f)
It takes longer time to restore a VM on a VM with 16 cores comparing
with ones with 64 cores.
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 0718067851)
If the PR updated cancel outstanding jobs to conserve resources.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 1db30405e1)
The DCO tool doesn't understand merge_groups but we still need to have a
valid status check to allow the merge group to proceed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 96cc1ba76c)
This takes a long time and duplicates existing checks on the pull
requests.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 022f375ef8)
When running with the merge queue the tests will be fully executed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit f15ca1aec3)
And clean up some of the whitespace formatting so that the "name" and
"on" are grouped away from the "jobs".
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit d9f48505fe)
Most of our CI workers are now running form GitHub actions, so we are
ready to disable Jenkins CI workers.
See: #6231
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 1d098949b9)
Add top-level timeout for the jobs and also more agressive per step
timeouts.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 1fe2771a0d)
To reduce issues caused by flaky tests split the musl and glibc jobs
into separate jobs. This means fewer jobs will need to be restarted for
flaky tests. This will also increase CI throughput since the musl builds
account for ~40% of the total CI time when run together with glibc.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 2e4079becb)
This will help handle flakiness in the builds by requiring the minimum
number of restarts.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit d32de07be7)
Use the matrix to add a build runnind on the AMD variant of the garm
runner.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 84a6da5e93)
The bionic image was being downloaded and converted but no test uses
this image any longer.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 6930370a03)
The 'test_vfio_user' is prone to fail when the system is under high
workloads with errors:
```
Error while connecting to /var/tmp/spdk.sock
Is SPDK application running?
Error details: Invalid or non-existing address: '/var/tmp/spdk.sock'
```
This is because SPDK is not fully functional before we request to
create a nvme device using the vfio_user protocol. This patch stabilize
this test with allowing retires to execute host commands.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch defines a new function 'generate_ram_ranges', to generate
usable physical memory ranges for the guest based on the existing guest
memory managed by VMM. This function is also made public, so that it can
be reused, say by the IGVM loader in the future [1].
No functional change.
See: #6020
Signed-off-by: Bo Chen <chen.bo@intel.com>
This patch adds missing new lines after functions,
fixes few typos in the comments, adds few missing
comments to SNP related functions.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
There is no requirement to call copy_from_slice, since all the member
variables are identical and we can directly assign them value.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
There is a mix of v2, v3 and v4 in the codebase. Let's move to v4
everywhere because v2 seems to be using a deprecated version of nodejs.
This is throwing warnings when the Github action CI is running.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Currently there are some inconsistencies in Cargo.toml which is causing
the following warnings during the build process:
Error parsing Cargo.toml manifest, fallback to caching entire file:
Invalid TOML document: expected key-value, found comma
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
vmm: Add igvm module and loader module
Add a separate module named igvm to the vmm crate
with definitions to parse and load igvm to the guest memory.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add necessary API to retrieve cpuid leaf on MSHV.
This API is used to update cpuid information
during the parsing of the igvm file.
Microsoft hypervisor does not provide common
CpuID like KVM. That's why we need to call this API
during the IGVM parsing.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch adds igvm to the Vm config and params as well as
the command line argument to pass igvm file to load into
guest memory. The file must maintain the IGVM format.
The CLI option is featured guarded by igvm feature gate.
The IGVM(Independent Guest Virtual Machine) file format
is designed to encapsulate all information required to
launch a virtual machine on any given virtualization stack,
with support for different isolation technologies such as
AMD SEV-SNP and Intel TDX.
At a conceptual level, this file format is a set of commands created
by the tool that generated the file, used by the loader to construct
the initial guest state. The file format also contains measurement
information that the underlying platform will use to confirm that
the file was loaded correctly and signed by the appropriate authorities.
The IGVM file is generated by the tool:
https://github.com/microsoft/igvm-tooling
The IGVM file is parsed by the following crates:
https://github.com/microsoft/igvm
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Random failures of downloading `cloud-hypervisor-static-aarch64` with
`wget` were seen. The commit applies a workaround to retry the download
for a few times.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
This commit changes existing behavior of named TAP interfaces.
When booting a VM with configuration for a named TAP interface,
cloud-hypervisor will create the interface and apply a given
IP configuration to that interface. If the named interface
already exists on the system, the configuration is NOT overwritten.
Setting the ip and netmask fields in a tap interface configuration
for a named tap interface now works by handing this configuration
to the virtio_devices::Net object when it is created with a name.
This commit also touches net_util to make sure that the ip configuration
of existing TAP interfaces is not modified with ip or netmask handed to
open_tap.
Signed-off-by: Markus Sütter <markus.suetter@secunet.com>
The wget has been causing frequent CI failure for the past few days,
while it can't be reproduced manually. Let's show the wget details in
our CI pipeline to understand better future errors.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Running on host where vdpa_sim_blk module is not correctly loaded
test_vdpa_block passes.
"test common_parallel::test_vdpa_block ... ok"
This commit fixes the vdpa_sim_blk test to fail in that case.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Valid title format:
component1[, component2, componentN]: submodule: summary
Title should have at least one component
Components are separated by comma+space: ", "
Components are validated to be in valid_components
Components list is ended by a colon
Submodules are not validated
See: #5846
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
We can skip the CI worker build if the changes are only from gitlint, or
the hidden files in the root folder, such as .github, .gitignore,
.gitlint, .rustfmt.toml, and .typos.toml.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This is particularly useful for commits with long component names. Of
course, it is better to have concise subject that is less than 50 char.
Signed-off-by: Bo Chen <chen.bo@intel.com>
As part SMP bringup for a SEV-SNP guest, BSP sets up the VMSA page for
each AP threads and informs hypervisor about the same using a VMGEXIT.
Thus, extend the current GHCB interface to handle this scenario.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
SEV-SNP guest can request AMD's secure co-processor i.e., PSP to
generate an runtime attesation report. During this process guest needs
to inform PSP about the request and response GPAs where that report
would be generated by the PSP. This is handled via a VMGEXIT request.
Thus, extend the current GHCB handling to add support for it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
We found that it's slow to load JSON when reading snap files. As
described in [1], using from_slice instead of from_reader can fix
this.
Also, fix the error type being returned.
1. https://github.com/serde-rs/json/issues/160
Signed-off-by: Yi Wang <foxywang@tencent.com>
Since the 'write()' to the event file was moved to its own thread
(see #5633), we have no reliable way to read the latest contents of
the event file from our integration tests, since we can't ensure the
'read()' from our test always happen after 'write()' is completed from
Cloud Hypervisor. This is also why we started to see random failures on
snapshot_restore tests (particularly when the system workload is high).
This patch adds a 1s sleep before reading the event file to mitigate the
random failures.
Signed-off-by: Bo Chen <chen.bo@intel.com>
error: use of a fallible conversion when an infallible one could be used
Error: --> virtio-devices/src/vhost_user/vu_common_ctrl.rs:206:51
|
206 | let actual_size: usize = queue.size().try_into().unwrap();
| ^^^^^^^^^^^^^^^^^^^ help: use: `into()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_fallible_conversions
= note: `-D clippy::unnecessary-fallible-conversions` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::unnecessary_fallible_conversions)]`
error: could not compile `virtio-devices` (lib) due to previous error
Error: warning: build failed, waiting for other jobs to finish...
error: could not compile `virtio-devices` (lib test) due to previous error
Error: The process '/home/runner/.cargo/bin/cargo' failed with exit code 101
Signed-off-by: Bo Chen <chen.bo@intel.com>
When the script is invoked via dev_cli.sh, it always gets AUTH_DOWNLOAD_TOKEN
from the environment. The original test always returns true.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Instead of relying on "wc" and "grep", this patch provides helper
functions for checking line counts and searching/counting keywords.
To understand assertion failures better, it also generate logs for the
L1/L2 VM commands when checks fail.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Add a test that supports configuring serial and console as TTY mode
at the same time. With this configuration, the VM can set up a legacy
serial device as an early printk console device, and then change to a
virito console device after the virito console device is initialized.
In this case, we can capture the logs printed by legacy serial on early
boot, and later by the virtio console.
Signed-off-by: Yong He <alexyonghe@tencent.com>
Cloud Hypovrisor supports legacy serial device and virito console device
for VMs. Using legacy serial device, CH can capture full VM console logs,
but its implementation is based on KVM PIO emulation and has poor
performance. Using the virtio console device, the VM console logs will
be sent to CH through the virtio ring, the performance is better, but CH
will only capture the VM console logs after the virtio console device is
initialized, the VM early startup logs will be discarded.
This patch provides a way to enable both the legacy serial device and the
virtio console device as a TTY mode by setting the leagcy serial port as
the VM's early printk device and setting the virtio console as the VM's
main console device.
Then CH can capture early boot logs from the legacy serial device and
capture later logs from the virito console device with better performance.
Signed-off-by: Yong He <alexyonghe@tencent.com>
The script "run_integration_tests_vfio.sh" requires to be ran on a
specific machine with specific hardware (e.g. the "vfio" bera-metal
worker equipped with a Nvidia Tesla T4 card). So the provisioning of the
running machine is out of the scope of this script, including the custom
guest image with Nvidia drivers installed, and properly configured
Nvidia Tesla T4 card.
See: #4405, #5509
Signed-off-by: Bo Chen <chen.bo@intel.com>
With #4324 being resolved, the nested VFIO test (e.g. "test_vfio") is
now a part of the general Azure VM-based workers. No need to run it on
the bare-metal worker.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Now, default values for vcpu topology are 0s, that is not correct and may
lead to bug. Fix it by setting default value to 1s. Also add check in
case one or more of these values are zero.
Fixes: #5892
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
virtiofsd must be built with the default gnu toolchain so clear the
environment variables that may poisoned by the alternative C library
support.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The seccompiler v0.4.0 started to use `seccomp` syscall instead of the
`prctl` syscall. Also, threads for virtio-deivces should not need any of
these syscalls anyway.
Signed-off-by: Bo Chen <chen.bo@intel.com>
MMIO emulation is also performed via VMGEXIT in case of SEV-SNP guest.
Emulation is done in a very similar way like a regular guest. Just need
to make sure that guest memory is access via read/write GPA hypercall
instead of directly accessing it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Legacy port emulation requires reading RAX register from GHCB page for
SEV-SNP guest. This is the major difference between a regular guest and
SEV-SNP enabled guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Currently MSHV does not support fetching extended guest report and thus
return an appropriate error stating the NAE event is not valid.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
As part of this handling there are 4 different operations:
1. Getting the hypervisor preffered doorbell page GPA.
2. Informing hypervisor about the doorbell page chosen by the guest
3. Querying the GPA of the doorbell page
4. Clearing the GPA of the doorbell page from hypervisor
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
As part of handling this request, hypervisor is expected to three
things:
1. Maximum GHCB protocol version supported.
2. Minimum GHCB protocol version supported.
3. SEV-page table encryption bit number.
If the guest cannot support the protocol range supplied by the
hypervisor, it should terminate
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
SEV-SNP guest allocates a GHCB page and in order to update hypervisor
about the same, there is a vmgexit which allows registering GHCB page
with the hypervisor.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
A VMGEXIT exit occurs for any of the listed NAE events in the GHCB
specification [1] (e.g. CPUID, RDMSR/WRMSR, MMIO, port IO, etc.). Some
of these events are handled by hypervisor while other are handled by
VMM. Currently, we are adding support for one such request i.e.,
report supported SEV-SNP features by hypervisor.
[1] GHCB protocol specification:
https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/specifications/56421.pdf
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This is the function that needs to be called by the VMM
to inform the MSHV that isolation is complete and inform
PSP about this completion.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Add hypervisor VM specific API to import the isolated
pages. Hypervisor adds those pages for PSP measurement.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This patch fixes following warnings:
error: boolean to int conversion using if
--> vmm/src/vm.rs:866:42
|
| .create_vm_with_type(if sev_snp_enabled.into() {
| __________________________________________^
| | 1 // SEV_SNP_ENABLED
| | } else {
| | 0 // SEV_SNP_DISABLED
| | })
| |_____________________^ help: replace with from: `u64::from(sev_snp_enabled.into())`
|
= note: `-D clippy::bool-to-int-with-if` implied by `-D warnings`
= note: `sev_snp_enabled.into() as u64` or `sev_snp_enabled.into().into()` can also be valid options
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#bool_to_int_with_if
error: useless conversion to the same type: `bool`
--> vmm/src/vm.rs:866:45
|
| .create_vm_with_type(if sev_snp_enabled.into() {
| ^^^^^^^^^^^^^^^^^^^^^^ help: consider removing `.into()`: `sev_snp_enabled`
|
= note: `-D clippy::useless-conversion` implied by `-D warnings`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
error: could not compile `vmm` due to 2 previous errors
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Partially revert 111225a2a5
and add the new dbus and pvpanic arguments.
As we are switching back to clap observe the following changes.
A few examples:
1. `-v -v -v` needs to be written as`-vvv`
2. `--disk D1 --disk D2` and others need to be written as `--disk D1 D2`.
3. `--option value` needs to be written as `--option=value.`
Change integration tests to adapt to the breaking changes.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
There is no need to set them in the test scripts while the main script
already has them.
The consolidates how things are done.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
There is no need to reconstruct it from within the scripts since the
main script already constructed it once.
Drop the previously useless setting of BUILD_TARGET from various
scripts. The value was always overwritten at a later point.
No functional change intended.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Sev-Snp on MSHV uses igvm crate to parse the igvm file.
igvm crate needs minimum rust version 1.65 to build.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This struct contains all configuration fields that controls the way how
we generate CPUID for the guest on x86_64. This allows cleaner extension
when adding new configuration fields.
Signed-off-by: Bo Chen <chen.bo@intel.com>
For a SEV-SNP enabled partition on MSHV, some of the VMGEXITS are
offloaded for Hypervisor to handle while the rest are handled by VMM.
By setting this additional partition property hypervisor is informed
about the VMGEXITs it needs to take care off, rest all would be handled
by the CloudHypervisor.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The lock to `vm_config` is held for accessing `cpus.kvm_hyperv` passing
as a reference to `arch::generate_common_cpuid()`, so acquiring the same
lock again while calling to the same function is a deadlock.
Fixes: 3793ffe888
Reported-by: Yi Wang <foxywang@tencent.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
As part of this initialization for a SEV-SNP VM on MSHV, it is required
that we transition the guest state to secure state using partition
hypercall. This implies all the created VPs will transition to secure
state and could access the guest encrypted memory.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
When performing integration testing on the Ampere One server, an error occurred when compiling spdk and not recognizing the CPU ID.
The latest spdk already contains fixes.
Signed-off-by: dom.song <dom.song@amperecomputing.com>
It's a requirement that a SEV-SNP enabled guest on MSHV must have
isolation policy set before launching the guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Cloud-Hypervisor takes a path for Unix socket, where it will listen
on. Users can connect to the other end of the socket and access serial
port on the guest.
"--serial socket=/path/to/socket" is the cmdline option to pass to
cloud-hypervisor.
Users can use socat like below to access guest's serial port once the
guest starts to boot:
socat -,crnl UNIX-CONNECT:/path/to/socket
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The 'derive' feature of `zerocopy` crate now is optional and requires to
be enabled explicitly [1]. Also, a version bump on `acpi_tables` is
needed to reply on a single version of `zerocopy` to avoid compilation
errors.
[1] https://github.com/google/zerocopy/pull/176
Signed-off-by: Bo Chen <chen.bo@intel.com>
Include the TSC frequency as part of the KVM state so that it will be
restored at the destination.
This ensures migration works correctly between hosts that have a
different TSC frequency if the guest is running with TSC as the source
of timekeeping.
Fixes: #5786
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The bare metal machines are now not reachable from our Jenkins
controller, and causing CI pipeline timeout for the past few days.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The cumulative average formula [1] requires to use signed integers
for proper calculations, while calculated result (e.g. cumulative
average) is always positive. This patch reflects the above requirements
in our code.
[1] https://en.wikipedia.org/wiki/Moving_average#Cumulative_averageFixes: #5745
Signed-off-by: Bo Chen <chen.bo@intel.com>
Fix make error:
$ make -C BaseTools -j `nproc`
/bin/sh: line 1: python: command not found
make[1]: *** [GNUmakefile:11: test] Error 127
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Bump these crates need to be bumped together to avoid build issues from
shared structures.
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
EntryPoint had an optional entry_addr, but there is no usage of this
struct that makes it necessary that the address is optional.
Remove the Option to avoid being able to express things that are not
useful.
Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
This fixes all typos found by the typos utility with respect to the config file.
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
There is a "LATENCY_SCALE" being used for calculating cumulative average
latency, so it should also be used for the latency of the first op.
See: #5712
Signed-off-by: Bo Chen <chen.bo@intel.com>
Update to the latest vm-memory and all the crates that also depend upon
it.
Fix some deprecation warnings.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
We need to make sure that SEV-SNP and TDX are not enabled at the same
time. As these two features belong to mutually exclusive hardware
vendors. So, we should make sure that these two features are not enabled
at the same. Thus, a compile time check for it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Creating some brief documentation for SEV-SNP, summarizing the links on
where to find more information about SEV-SNP, as well as how to build
and run Cloud Hypervisor on it.
This document is a work in progress and will be updated in future PRs
as we add support for it.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This feature flag gates the development for SEV-SNP enabled guest.
Also add a helper function to identify if SNP should be enabled for the
guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
The worker is running focal not bionic - this fixes build issues on that
worker machine due to the bionic version not being installable.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is necessary for O_DIRECT based use of raw block devices which may
require access at a larger block size than that of a sector (512 bytes.)
Fixes: #5722
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
It doens't make sense to limit this to the async_io module and import
across modules so place in default create module.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Assume rax is 0xfee003e0 and the displacement is negative 0x60. The effective
address is then 0xfee00380. This is perfectly valid.
Example instruction:
c7 40 a0 00 10 00 00 movl $0x1000,-0x60(%rax)
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
Logically until we have handled the first operation the latency is
infinite; this logic was applied to the minimum latency originally but
this patch extends that logic to the maximum and average latency.
To prevent the initial average latency being skewed by the inclusion of
infinity the average value is initally seeded with the first measured
latency.
Fixes: #5704
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
In the documentation of function check_latest_events_exact use same events
argument name as in the implementation
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Since kernel v6.3 the vsock packet is not split over two descriptors
and is instead included in a single one.
This change is based on the discovery and fix identified by Stefano
Garzarella for the vm-virtio vsock implementation and adapted for our
very different codebase.
Fixes: #5691
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This commit builds on top of the `Monitor::subscribe` function and
makes it possible to broadcast events published from `event-monitor`
over D-Bus.
The broadcasting functionality is enabled if the D-Bus API is enabled
and users who wish to also enable the file based `event-monitor` can do
so with the CLI arg `--event-monitor`.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
There is no need for this struct to be public and since it is used in
this module the #[allow(dead_code)] invocation can be removed.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> vmm/src/device_manager.rs:1908:35
|
1908 | fn set_raw_mode(&mut self, f: &mut dyn AsRawFd) -> vmm_sys_util::errno::Result<()> {
| ^^^^^^^^^^^^^^^^ help: consider changing to: `&dyn AsRawFd`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
= note: `#[warn(clippy::needless_pass_by_ref_mut)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> src/bin/ch-remote.rs:397:52
|
397 | fn dbus_api_do_command(toplevel: &TopLevel, proxy: &mut DBusApi1ProxyBlocking<'_>) -> ApiResult {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider changing to: `&DBusApi1ProxyBlocking<'_>`
|
= note: this is cfg-gated and may require further changes
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
= note: `#[warn(clippy::needless_pass_by_ref_mut)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> virtio-devices/src/transport/pci_common_config.rs:100:17
|
100 | queues: &mut [Queue],
| ^^^^^^^^^^^^ help: consider changing to: `&[Queue]`
|
= warning: changing this function will impact semver compatibility
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
= note: `#[warn(clippy::needless_pass_by_ref_mut)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> hypervisor/src/arch/x86/emulator/instructions/mod.rs:22:15
|
22 | platform: &mut dyn PlatformEmulator<CpuState = T>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider changing to: `&dyn PlatformEmulator<CpuState = T>`
|
= note: this is cfg-gated and may require further changes
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> net_util/src/queue_pair.rs:167:14
|
167 | tap: &mut Tap,
| ^^^^^^^^ help: consider changing to: `&Tap`
|
= warning: changing this function will impact semver compatibility
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> block/src/async_io.rs:68:28
|
68 | fn query_block_size(f: &mut File, block_size_type: BlockSize) -> std::io::Result<u64> {
| ^^^^^^^^^ help: consider changing to: `&File`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: boolean expression will never evaluate to 'true'
--> block/src/vhdx/vhdx_metadata.rs:136:20
|
136 | if disk_spec.block_size < BLOCK_SIZE_MIN && disk_spec.block_size > BLOCK_SIZE_MAX {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: since `BLOCK_SIZE_MIN` < `BLOCK_SIZE_MAX`, the expression evaluates to false for any value of `disk_spec.block_size`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#impossible_comparisons
= note: `#[warn(clippy::impossible_comparisons)]` on by default
&& was used when || should have been used
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Similar to balloon inflation, memory allocation is also constrained to
align with the page size. Therefore, memory is allocated in units of the
host page size, one page at a time, until all host pages that the memory
range requested by the guest are managed. If the requested size is
smaller than the page size, the entire page will still be allocated
because smaller allocations are not possible due to the page size
limitation.
Fixes: cloud-hypervisor#5369
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Currently, virtio-balloon can't work well with page size other than 4k.
The virtio-balloon always works in units of 4kiB (BALLOON_PAGE_SIZE), but
we can only actually discard memory in units of the host page size.
We get some idea from [1] to solve this issue.
What has been done in this commit:
For balloon inflation:
A bitmap is employed to track the memory range to be released in 4k
granularity. Once it accumulates to one host page size, the corresponding
page is released, and the bitmap is cleared to handle the next record.
This process continues until all the memory range is managed. Memory will
only be released when a consecutive set of balloon request entries from
the same host page reaches the full host page size. If a balloon request
entry from a different host page is encountered, the bitmap and the base
host page address will be reset. Consequently, memory is released in
units of the page size, ensuring efficient memory management. That's say
if memory range length to be released smaller than page size or if the
guest scatters requests each of whose size is smaller than page size
across different host pages no memory will be released.
[1] https://patchwork.kernel.org/project/qemu-devel/patch/20190214043916.22128-6-david@gibson.dropbear.id.au/
Fixes: cloud-hypervisor#5369
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Add pending removed vcpu check according to VcpuState.removing, which
can avoid cloud hypervisor hangup during continual vcpu resize.
Fix#5419
Signed-off-by: Yi Wang <foxywang@tencent.com>
These had to be previously disabled because a released binary has to be
used in course of the test scenario. The necessary functionality is now
supported with the release 34.0, thus these tests should now pass.
Fixes: #5660
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
We fixed the L2 and L3 level cache sharing issues and confirmed that the
L2 level cache is independent, while the L3 level cache is shared per-socket.
See:#5505
Signed-off-by: zhongbingnan <zhongbingnan@bytedance.com>
This patch modifies `event_monitor` to ensure that concurrent access to
`event_log` from multiple threads is safe. Previously, the `event_log`
function would acquire a reference to the event log file and write
to it without doing any synchronization, which made it prone to
data races. This issue likely went under the radar because the
relevant `SAFETY` comment on the unsafe block was incomplete.
The new implementation spawns a dedicated thread named `event-monitor`
solely for writing to the file. It uses the MPMC channel exposed by
`flume` to pass messages to the `event-monitor` thread. Since
`flume::Sender<T>` implements `Sync`, it is safe for multiple threads
to share it and send messages to the `event-monitor` thread.
This is not possible with `std::sync::mpsc::Sender<T>` since it's
`!Sync`, meaning it is not safe for it to be shared between different
threads.
The `event_monitor::set_monitor` function now only initializes
the required global state and returns an instance of the
`Monitor` struct. This decouples the actual logging logic from the
`event_monitor` crate. The `event-monitor` thread is then spawned by
the `vmm` crate.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
These tests require a binary from a previous release to perform a
migration. That binary doesn't yet have the shutdown fixes and thus
these tests are supposed to fail. The tests are to be re-enabled after
the next release.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
The fixup_msix_region() function added in
a718716831
made the assumption that MSI-X was always available. This is the case
with many VFIO devices and all our virtio devices but created regression
with MSI devices.
Simply return the existing region size if MSI-X is not supported by the
device.
Fixes: #5649
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
With the addition of the spinning waiting for the exit event to be
received in the CMOS device a regression was introduced into the CMOS
fuzzer. Since there is nothing to receive the event in the fuzzer and
there is nothing to update the bit the that the device is looping on;
introducing an infinite loop.
Use an Option<> type so that when running the device in the fuzzer no
Arc<AtomicBool> is provided effectively disabling the spinning logic.
Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61165
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
As the container image can be used on both Intel and AMD,
ensure the SPDK binaries are compatible. This implies -march=skylake
passed to the underlaying toolchain.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Now feature "mshv" can be built together with "kvm". There is no need to
use "--no-default-features" any more.
Fixes: #5647
Signed-off-by: Bo Chen <chen.bo@intel.com>
The reset system is asynchronous with an I/O event (PIO or MMIO) for
ACPI/i8042/CMOS triggering a write to the reset_evt event handler. The
VMM thread will pick up this event on the VMM main loop and then trigger
a shutdown in the CpuManager. However since there is some delay between
the CPU threads being marked to be killed (through the
CpuManager::cpus_kill_signalled bool) it is possible for the guest vCPU
that triggered the exit to be re-entered when the vCPU KVM_RUN is called
after the I/O exit is completed.
This is undesirable and in particular the Linux kernel will attempt to
jump to real mode after a CMOS based exit - this is unsupported in
nested KVM on AMD on Azure and will trigger an error in KVM_RUN.
Solve this problem by spinning in the device that has triggered the
reset until the vcpus_kill_signalled boolean has been updated
indicating that the VMM thread has received the event and called
CpuManager::shutdown(). In particular if this bool is set then the vCPU
threads will not re-enter the guest.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Currently if container registry is inaccessible the image will be built
locally and that takes time. This patch adds support to use mirror
registry. To use a different registry CTR_IMAGE environment variable
must be set. For example:
CTR_IMAGE="registry/cloud-hypervisor" scripts/dev_cli.sh
or
export CTR_IMAGE="registry/cloud-hypervisor"
scripts/dev_cli.sh
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Split interrupt source group restore into two steps, first restore
the irqfd for each interrupt source entry, and second restore the
GSI routing of the entire interrupt source group.
This patch will reduce restore latency of interrupt source group,
and in a 200-concurrent restore test, the patch reduced the
average IOAPIC restore time from 15ms to 1ms.
Signed-off-by: Yong He <alexyonghe@tencent.com>
The VFIO tests based on the NVMe emulation framework require cause a
high number of file descriptors to be opened on the system. On systems
with a low limit on opened files, tests like test_vfio_user can possibly
fail with "too many open files" exeption. This issue is fixed by raising
the corresponding limit.
A tricky detail here is, that the limit has to be changed in the test
container and not on the host. As the the container is executed in the
privileged mode, the setting in it will override the host value.
Related: #5426
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Add integration test of coredump with no need pause.
As file of coredump has been tested in test_coredump(), so this
patch only test vm state after coredump.
Signed-off-by: Yi Wang <foxywang@tencent.com>
If the VMM is not already paused then pause the VM prior to executing
the coredump and then resume it after. If the VM is already paused then
the original state is maintained.
Signed-off-by: Yi Wang <foxywang@tencent.com>
As the commit 2f15d027c05f (KVM: x86: Properly handle APF vs disabled
LAPIC situation) has been merged into kernel 5.13, so update the minimum
performance host kernel version to that.
Signed-off-by: Yi Wang <foxywang@tencent.com>
The commit b92fe648e9 (vmm: cpu: Disable KVM_FEATURE_ASYNC_PF_INT in
CPUID) disabled APF (Asynchronous Page Fault) mechanism to address
problem that makes vcpu thread spin 100%. As the actual issue is in
KVM, which has been merged in commit 2f15d027c05f (KVM: x86: Properly
handle APF vs disabled LAPIC situation) since 2021, so it's okay to
re-enable APF now.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Add MSHV_CREATE_DEVICE, MSHV_SET_DEVICE_ATTR ioctls to filters. These
ioctls are required to passthrough PCI devices on mshv.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The pause of vcpu is async now, which makes the vm pause is not
synchronised. As virtio device calls paused_sync wait() to make
sure device_manager pause synchronously, if we make vcpu pause
synchronously, the vm pause can be synchronously then. After
vm.pause() returns the vm is really paused now.
This patch adds a AtomicBool variable to mark vcpu paused state,
to make sure the pause of CpuManager is synchronised.
Signed-off-by: Yi Wang <foxywang@tencent.com>
This test case creates a new qcow2 file using the image of ubuntu as
its backing file, and boot a virtual machine with this image file.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This preserves any data that the backing file had on a cluster when
doing a write to a subset of that cluster. These writes cause a
performance penalty on creating new clusters if a backing file is
present.
This commit is based on crosvm implementation:
5ad3bc3459
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Reads to qcow files with backing files will fall through to the backing
file if there is no allocated cluster. As of this change, a write will
still trash the cluster and hide any data already present.
This commit is based on crosvm implementation:
d8144a56e2
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit allows opening qcow with a backing file, which supports any
type implementing `BlockBackend`.
This commit is based on crosvm implementation:
9ca6039b03
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit introduces the trait `BlockBackend` with generic ops
including read, write and seek, which can be used for common I/O
interfaces for the block types without using `DiskFile` and `AsyncIo`.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit merges crates `qcow`, `vhdx` and `block_util` into the
crate `block`, which can allow `qcow` to use functions from `block_util`
without introducing a circular crate dependency.
This commit is based on crosvm implementation:
f2eecc4152
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This commit introduces a generic type `FixedVhd` for its sync and async
disk I/O types, and the disk I/O types will use it directly instead of
maintaining a file themselves.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Using this over the LocalApic supports APIC IDs (and hence number of
vCPUs) above 256 (size increases from u8 to u32.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
If the VM has been configured but not yet booted, all we need to do to
support removing a device is to remove it from the config, so it will
never be created.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This fixes the valid VM config unit tests, which would otherwise fail
to deserialize their expected JSON config due to the missing "gdb" field.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
error: private item shadows public glob re-export
Error: --> hypervisor/src/mshv/mod.rs:42:27
|
42 | CpuIdEntry, FpuState, LapicState, MsrEntry, SpecialRegisters, StandardRegisters,
| ^^^^^^^^^^
|
note: the name `LapicState` in the type namespace is supposed to be publicly re-exported here
--> hypervisor/src/mshv/mod.rs:16:9
|
16 | pub use mshv_bindings::*;
| ^^^^^^^^^^^^^^^^
note: but the private item here shadows it
--> hypervisor/src/mshv/mod.rs:42:27
|
42 | CpuIdEntry, FpuState, LapicState, MsrEntry, SpecialRegisters, StandardRegisters,
| ^^^^^^^^^^
= note: `-D hidden-glob-reexports` implied by `-D warnings`
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
warning: usage of `Arc<T>` where `T` is not `Send` or `Sync`
--> virtio-devices/src/vsock/device.rs:376:22
|
376 | backend: Arc::new(RwLock::new(backend)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: consider using `Rc<T>` instead or wrapping `T` in a std::sync type like `Mutex<T>`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#arc_with_non_send_sync
= note: `#[warn(clippy::arc_with_non_send_sync)]` on by default
The vsock backend may be shared between threads, so the type `B` in
`Vsock` should be `VsockBackend` and `Sync`.
Considering that `api_receiver` and `gdb_receiver` are only used in vmm
threads, the `Arc` can be replaced by `Rc`.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
warning: useless use of `vec!`
--> test_infra/src/lib.rs:111:30
|
111 | let mut events = vec![epoll::Event::new(epoll::Events::empty(), 0); 1];
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: you can use an array directly: `[epoll::Event::new(epoll::Events::empty(), 0); 1]`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_vec
= note: `#[warn(clippy::useless_vec)]` on by default
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
warning: casting raw pointers to the same type and constness is unnecessary (`*const protocol::MemoryRange` -> `*const protocol::MemoryRange`)
--> vm-migration/src/protocol.rs:280:17
|
280 | self.data.as_ptr() as *const MemoryRange as *const u8,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `self.data.as_ptr()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
= note: `#[warn(clippy::unnecessary_cast)]` on by default
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This gives users the chance to reduce the number of dependencies
included, which is generally good practice and also reduces code size.
Furthermore, `io_uring` specifically is a strong contender for something
one may wish to disable due to the syscall API's many security issues[1]
[1]: https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html
Signed-off-by: Manish Goregaokar <manishsmail@gmail.com>
test_cpu_affinity needs the number of host CPUs. Since it is possible
for the host to have more than 255 CPUs; increase the size of the
datatype used for parsing the string to accomodate this.
Signed-off-by: dom.song <dom.song@amperecomputing.com>
Add integration test for pvpanic, by two methods:
- the vendor id and device id of pci device in guest
- triggering a guest panic and check event-monitor.
Also, to support pvpanic-pci driver, add pvpanic config
in resources.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Implemetion of BusDevice and PciDevice trait for pvpanic device.
Pvpanic device can be implemented as an ISA device or as a
PCI device, we choose PCI device for now.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Introduce emulation of pvpanic device to allow cloud hypervisor to get
the notify from guest's pvpanic driver when guest kernel crash.
Signed-off-by: Yi Wang <foxywang@tencent.com>
This reverts commit efb579b224.
This PR was mistakenly merged due to the confusion around nightly
builds. The vm-memory update needs to be done in both the root workspace
and fuzz workspace together.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
On x86-64, when the underlying hypervisor platform is KVM, no
instruction emulator is necessary. KVM handles instruction boundaries
internally.
This change allows to skip the iced-x86 dependency on KVM, improving
build times, prunes the dependency graph and reduces network traffic
during the initial build.
For Hyper-V, the emulator is still necessary on x86-64, so nothing
changes there.
Signed-off-by: Christian Blichmann <cblichmann@google.com>
Otherwise, CONFIG_DEBUG_INFO will be enabled automatically bloating the
size of the kernel image.
Now the size of kernel image is reduced from 635M to 60M on x86_64.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Remove clippy:all and update it with correct clippy lint,
so that it gives proper information.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Between musl and glibc there is a difference in the signature of the
ioctl libc function. Use an anonymous cast to force the type coversion.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
According the std docs implementing From<..> is preferred since it
gives you Into<..> for free where the reverse isn’t true.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Add a Default implementation that delegates to new, so that clippy
can be removed.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Fix these warnings:
SC2086 info: Double quote to prevent globbing and word splitting.
DL3047 info: Avoid use of wget without progress bar. Use `wget --progress=dot:giga <url>`. Or consider using `-q` or `-nv` (shorthands for `--quiet` or `--no-verbose`).
SC2006 style: Use $(...) notation instead of legacy backticks `...`.
Ignore these warning cause they break the build or they do not apply:
DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
DL3003 warning: Use WORKDIR to switch to a directory
SC2016 info: Expressions don't expand in single quotes, use double quotes for that.
SC2046 warning: Quote this to prevent word splitting.
DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check
SC2155 warning: Declare and assign separately to avoid masking return values.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
hadolint error:
DL3015 info: Avoid additional packages by specifying `--no-install-recommends`
Add required packages ca-certificates unzip iproute2 dbus
Without these packages build or integration tests fail if
--no-install-recommends is specified. Previously these packages were
installed as part of recommended dependency packages.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Setting environment variable
DEBIAN_FRONTEND=noninteractive
Makes unnecessary setting it in all apt-get install commands
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Some parts of Dockerfile are indented with spaces others with tabs.
Unify indentation to use spaces.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Test was failing due to regression caused by commit
d5558aea2a
Failing command:
sudo /mnt/ch-remote --api-socket /tmp/ch_api.sock resize --memory=1073741824"
Fixes#5190
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
This does the same thing as df2a7c17 ("vmm: Ignore and warn TAP FDs
sent via the HTTP request body"), but for the vm.create endpoint,
which also previously would accept file descriptors in the body, and
try to use whatever fd occupied that number as a TAP device.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Port of df2a7c17 ("vmm: Ignore and warn TAP FDs sent via the HTTP
request body"), but for the vm.create endpoint, which would previously
accept file descriptors in the body, and try to use whatever fd
occupied that number as a TAP device.
Since I had to move the wrapping of the net config in an Arc until
after it was modified, I made the same change to all other endpoints,
so the style stays consistent.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Using the data from sysfs forward the host host cache layout to the
guest using the FDT tables.
TEST=The host cache layout (from sysfs) can be seen in inside the guest
using lscpu.
Signed-off-by: zhongbingnan <zhongbingnan@bytedance.com>
Currently, vfio device fails to initialize as the msix-cap region in BAR
is mapped as RW region.
To resolve the initialization issue, this commit avoids mapping the
msix-cap region in the BAR. However, this solution introduces another
problem where aligning the msix table offset in the BAR to the page
size may cause overlap with the MMIO RW region, leading to reduced
performance. By enlarging the entire region in the BAR and relocating
the msix table to achieve page size alignment, this problem can be
overcomed effectively.
Fixes: #5292
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
To avoid code duplication extract page related functions to their
own module and add utility functions for manipulating addresses
related to page sizes
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
This patch clarifies the assumptions we have regarding the guest address
space layout while creating memory mapping in E820 on x86_64 and fdt on
aarch64. It also explicitly checks on these assumptions and report
errors if these assumptions do not hold.
Signed-off-by: Bo Chen <chen.bo@intel.com>
This commit renames `ram_region_sub_size` to `ram_region_available_size`
and make its value align down to the default page size or hugepage
size of the current memory zone, which can prevent the memory zone from
being split into misaligned parts. And if the available size of ram
region is zero, this region will be marked as consumed even it has
unused space.
Note that there is two methods to use hugepages.
1. Specify `hugepages` for `memory` or `memory-zone`, if the
`hugepage_size` is not specified, the value can be got by `statfs`
for `/dev/hugepages`.
2. Specify a `file` in hugetlbfs for `memory-zone`, the hugepage size
can also be got by `statfs` for the file.
The value for alignment will be the hugepage size if this memory zone
is using hugepages, otherwise the value will be default page size of
system.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The previous `arch_memory_regions` function will provide some memory
regions with the specified memory size and fill all the previous
regions before using the next one, but sometimes there may be no need
to fill up the previous one, e.g., the previous one should be aligned
with hugepage size.
This commit make `arch_memory_regions` function not take any
parameters and return the max available regions, the memory manager
can use them on demand.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The original codes did not consider that the previous memory region
might not be full and always set it to the maximum size.
This commit fixes this problem by creating memory mappings based on
the actual memory details in both E820 on x86_64 and fdt on aarch64.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The device manager will set tty or pty to raw mode, all the `\n` will
be LF without CR, which makes the output difficult to read.
This commit solves it by using `write` with `\r\n` instead of
`writeln`, which can print CR and LF explicitly.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Remove "enum_variant_names" clippy. Enumeration variant names should
specify their variant, not repeat the enumeration name.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Download firmware and focal image to fix issue of running vfio tests
with empty worlkoads directory:
$ scripts/dev_cli.sh tests --integration-vfio
+ cp /root/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw /root/workloads/vfio
cp: cannot stat '/root/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw': No such file or directory
+ cp /root/workloads/hypervisor-fw /root/workloads/vfio
cp: cannot stat '/root/workloads/hypervisor-fw': No such file or directory
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Program the APIC ID (CPUID leaf 0x1 EBX) with the CPU id. This resolves
an issue where the EDKII firmware expects the APIC ID to vary per-CPU.
Fixes: #5475
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Script `create-cloud-init.sh" uses the same cloud init data as the the CI
but this means it is somewhat overloaded with unhelpful network configuration
entries and scripts that are only needed for the CI.
For local testing data kept in test_data/cloud-init/ubuntu/local folder.
This contains minimum configuration data for user to test it locally.
Script assigns default IP address using "network-config" details
with --net "tap=" option. The default network interface will be "ens4".
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Script `create-cloud-init.sh" uses the same cloud init data as the the CI
but this means it is somewhat overloaded with unhelpful network configuration
entries and scripts that are only needed for the CI.
CI related data moved to test_data/cloud-init/ubuntu/ci folder.
Fixes: #4773
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
SerialBuffer uses VecDeque::extend, which calls realloc, which a
maximum buffer size of 1 MiB. Starting at allocation sizes of
128 KiB, musl's mallocng allocator will use mremap for the allocation.
Since this was not permitted by the seccomp rules, heavy write load
could crash cloud-hypervisor with a seccomp failure. (Encountered
using virtio-console, but I don't see any reason it wouldn't happen
for the legacy serial device too.)
Signed-off-by: Alyssa Ross <hi@alyssa.is>
In this way, our integration tests exercise the same set of build
features (e.g. "kvm,mshv") being used for releases.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Bump to the latest rust-vmm crates, including vm-memory, vfio,
vfio-bindings, vfio-user, virtio-bindings, virtio-queue, linux-loader,
vhost, and vhost-user-backend,
Signed-off-by: Bo Chen <chen.bo@intel.com>
Implemented a `TargetApi` enum to make the process of implementing
tests for the D-Bus and HTTP API more convenient.
Refactored `test_api_{create_boot, shutdown, pause_resume, delete}` tests
with the `TargetApi` enum to also implement tests for the D-Bus API.
Added a new test named `test_api_dbus_and_http_interleaved` that uses
both the HTTP and D-Bus API at the same time.
Modified integration test scripts to enable the `dbus_api` feature when
compiling and start a dbus-session when integration tests are run.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
Introduces three new CLI options, `dbus-service-name`,
`dbus-object-path` and `dbus-system-bus` to configure the DBus API.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit applies the previously created seccomp filter
to the `DbusApi` thread.
Also encloses the main loop of the `DBusApi` thread using
`std::panic::catch_unwind` and `AssertUnwindSafe` in order to mirror
the behavior of the HTTP API.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit adds support for graceful shutdown of the DBusApi thread
using `futures::channel::oneshot` channels. By using oneshot channels,
we ensure that the thread has enough time to send a response to the
`VmmShutdown` method call before it is terminated. Without this step,
the thread may be terminated before it can send a response, resulting
in an error message on the client side stating that the message
recipient disconnected from the message bus without providing a reply.
Also changes the default values for DBus service name, object path
and interface name.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit introduces three new dependencies: `zbus`, `futures`
and `blocking`. `blocking` is used to call the Internal API in zbus'
async context which is driven by `futures::executor`. They are all
behind the `dbus_api` feature flag.
The D-Bus API implementation is behind the same `dbus_api` feature
flag as well.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
Rust documentation comments follows 80 char per line. But some of
these help options doesn't have space in between. Longlines of
text is starting from beginning of the next line instead of
proper format.
Before fix:
Options:
--cpus boot=<boot_vcpus>,max=<max_vcpus>,topology=<threads_per_core>:<cores_per_die>:<dies_per_package>:<packages>,kvm_hyperv=on|off,max_phys_bits=<maximum_number_of_physical_bits>,affinity=<list_of_vcpus_with_their_associated_cpuset>,features=<list_of_features_to_enable>
After fix:
Options:
--cpus boot=<boot_vcpus>, max=<max_vcpus>,
topology=<threads_per_core>:<cores_per_die>:<dies_per_package>:<packages>,
kvm_hyperv=on|off,
max_phys_bits=<maximum_number_of_physical_bits>,
affinity=<list_of_vcpus_with_their_associated_cpuset>,
features=<list_of_features_to_enable>
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Commit 21d40d7 ("main: reset tty if starting the VM fails") changed
start_vmm() to join the vmm thread if an error happens after the vmm
thread is started. The implementation put all the error-prone code that
is run after the vmm is started in a closure, to be able to always join
the vmm thread, regardless of any error happening. However, it missed
propagating the error that might happen inside the closure back to the
main function, after joining the vmm thread.
For some cmd line options, the above issue inhibits proper error
reporting when starting a VM with invalid commands, as many parameters
are parsed after the vmm is started, thus if such parsing fails, no
error will be reported back to the user.
See: #5435
Fixes: 21d40d7 ("main: reset tty if starting the VM fails")
Signed-off-by: Rafael Mendonca <rafaelmendsr@gmail.com>
The refactoring on deferring address space allocation (#5169) broke TDX,
as TDX initialization needs to access guest memory for encryption and
measurement of guest pages.
Signed-off-by: Bo Chen <chen.bo@intel.com>
In current implementation, memory region used in vfio is assumed to
align to 4k which may cause error when the PAGE_SIZE is not 4k, like on
Arm, it can be 16k and 64k.
Remove this assumption and align memory resource used by vfio to
PAGE_SIZE then vfio can run on host with 64k PAGE_SIZE.
Fixes: #5292
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The current implementation of breadth first traversal for device tree
uses a temporary vector, therefore causes unnecessary memory copy.
Remove it and do it within vector nodes.
Signed-off-by: Hao Xu <howeyxu@tencent.com>
Unlike KVM, there's no internal handling for topoolgy under MSHV. Thus,
if no topology has been passed during the CH launch, use the boot CPUs
count to construct the topology struct.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Passing the CPUID leafs with the topology is integrated into the common
mechanism of setting and patching CPUID in Cloud Hypervisor. All the
CPUID values will be passed to the hypervisor through the register
intercept call.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Originally the AML only accepted one hex number for PCI segment
numbering. Change it to accept two numbers. That makes it possible to
add up to 256 PCI segments.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When I refactored this to centralise resetting the tty into
DeviceManager::drop, I tested that the tty was reset if an error
happened on the vmm thread, but not on the main thread. It turns out
that if an error happened on the main thread, the process would just
exit, so drop handlers on other threads wouldn't get run.
To fix this, I've changed start_vmm() to write to the VMM's exit
eventfd and then join the thread if an error happens after the vmm
thread is started.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
In particular the Std::write() API requires that the value implements
AsBytes and copies the slice representation into the table data. This
avoids unaligned writes which can cause a panic with the updated
toolchain.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
For structures that are used in SDT ACPI tables it is necessary for them
to implement this trait for the newly safe Std::write() API.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is used on older kernels where close_range() is not available.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Fixes: 505f4dfa ("vmm: close all unused fds in sigwinch listener")
On KVM this is provided by an ioctl, on MSHV this is constant. Although
there is a HV_MAXIMUM_PROCESSORS constant the MSHV ioctl API is limited
to u8.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Don't import via glob to avoid (unused) objects colliding in the
namespace. This fixes a beta clippy issue.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This fixes the following tests that have been consistently failing on
the CI:
[2023-04-22T07:00:53.760Z] failures:
[2023-04-22T07:00:53.760Z] common_parallel::test_focal_hypervisor_fw
[2023-04-22T07:00:53.760Z] common_parallel::test_focal_ovmf
I'm not sure of the origin of this check but it obviously dependent on
the underlying platform as the guest OS has not changed. Since it
depends on the host environment it doesn't make sense to assert for it.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
In this way, we can cover the scenario where a VM with hotplugged net
device using FDs can work properly with reboot.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The custom 'clone' duplicates 'preserved_fds' so that the validation
logic can be safely carried out on the clone of the VmConfig.
The custom 'drop' ensures 'preserved_fds' are safely closed when the
holding VmConfig instance is destroyed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Preserved FDs are the ones that share the same life-time as its holding
VmConfig instance, such as FDs for creating TAP devices.
Preserved FDs will stay open as long as the holding VmConfig instance is
valid, and will be closed when the holding VmConfig instance is destroyed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
If the VM is shut down, either it's going to be started again, in
which case we still want to be in raw mode, or the process is about to
exit, in which case canon mode will be set at the end of main.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Having PMU in guests isn't critical, and not all hardware supports
it (e.g. Apple Silicon).
CpuManager::init_pmu already has a fallback for if PMU is not
supported by the VCPU, but we weren't getting that far, because we
would always try to initialise the VCPU with KVM_ARM_VCPU_PMU_V3, and
then bail when it returned with EINVAL.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Cloud Hypervisor's vhost-user implementation will reconnect if it gets
disconnected from the backend. That means connections happen inside
the vhost-user seccomp sandbox, so all syscalls used in reconnecting
have to be allowed in that sandbox.
clock_nanosleep is used by Glibc, and nanosleep is used by musl.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Recently generated mshv-bindings has most of the registers
renamed. This patch renames some of the MSHV registers.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Currently we are overwriting the RUST_BACKTRACE if set
explicitly by user while running performance metrics tests
using dev_cli.sh
This change will allow user to set the RUST_BACKTRACE
while running performance metrics tests with dev_cli.sh
which invokes run_metrics.sh to run the performance binary.
We will set RUST_BACKTRACE to 1 if not set explicitly.
Signed-off-by: smit-gardhariya <gardhariya.smit@gmail.com>
Previously, we were only using it for PTYs, because for PTYs there's
no alternative. But since we have to have it for PTYs anyway, if we
also use it for TTYs, we can eliminate all of the code that handled
SIGWINCH for TTYs.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Now that the SIGWINCH listener has fallbacks for older kernels, we
don't expect it to routinely fail, so if there's an error setting it
up, we want to know about it.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This will allow the SIGWINCH listener to run on kernels older than
5.5, although on those kernels it will have to make 64 syscalls to
reset all the signal handlers.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The PTY main file descriptor had to be introduced as a parameter to
start_sigwinch_listener, so that it could be closed in the child.
Really the SIGWINCH listener process should not have any file
descriptors open, except for the ones it needs to function, so let's
make it more robust by having it close all other file descriptors.
For recent kernels, we can do this very conveniently with
close_range(2), but for older kernels, we have to fall back to closing
open file descriptors one at a time.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
It seems like these examples were always intended to be doctests,
since there are lines marked with "#" so that they are excluded from
the generated documentation, but they were not recognised as doc tests
because they were not formatted correctly.
The code needed some adjustments so that it would actually compile and
run as doctests.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
When doctests are built, the crate is built with itself as a
dependency via --extern. This causes a compiler error if using a
module with the name same as the crate, because it's ambiguous whether
it's referring to the module, or the extern version of the crate, so
it's necessary to disambiguate when using the hypervisor module here.
Fixes running cargo test --doc --workspace.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Doc comments are Markdown, and can include HTML tags. Anything in
angle brackets will therefore be inserted as an HTML tag into
rustdoc's output. If that's not intentional, the left angle bracket
needs to be escaped.
I haven't fixed the doc comments in src/main.rs, because argh doesn't
understand the escaping, so the backslashes would show up in the
--help output. I've opened https://github.com/google/argh/issues/159
about that.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
These need to be //! comments, because they apply to the module as a
whole, not to whatever directly follows the comment. Using ///
comments here resulted in documentation being attached to the wrong
thing, or not rendered at all.
I've also checked the Markdown formatting of these comments as
rendered by rustdoc, and fixed it where appropriate.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This doesn't need to be rendered in the HTML API documentation, and
wouldn't be formatted correctly if it was.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
It was not possible to build just hypervisor with Cargo's -p flag,
because it was not properly specifying the features it requires from
vfio-ioctls.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
balloon_free_page_reporting test case should not
work as expected. The reason is that MSHV pins
all the pages during the memory map for the guest.
Those pages can not be altered without unpinning the pages.
MSHV does not support modifying the pages during the guest
life cycle. This test case can be enabled once we add
VA backed VM support.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Unfortunately the build had a transient error and so needed to be
restarted which generated a build with a different date tag.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This commit will add timeout parameter to performance metrics
testcases as runtime argument. Testcases are facing the timeout
sometimes when we run with MSHV hypervisor. This change will
provide mechanism to pass the timeout as per the need while
running the testcases. If nothing is passed, default timeout or
timeout set as per the testcase will be followed.
Signed-off-by: smit-gardhariya <gardhariya.smit@gmail.com>
Tests of rate-limiter and vfio are running on bare-metal, so we need to
ensure the custom kernel is updated and rebuilt for them.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The tests require a fixed amount of huge page memory to run, rather than
encode that as fixed number of huge pages it should be calculated from
the size of memory required and the huge page size
Signed-off-by: Dom <peng6662001@163.com>
Significant API changes have occured, most significantly is the switch
to an approach which does not require vm-memory and can run no_std.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This latest stable Rust toolchain can help reduce the final binary size
around ~300k when comparing with building with Rust 1.62. Specifically,
the binary size now is 3.3M (previously 3.6M) with default feature, and
4.0M (previously 4.2M) with "kvm+mshv" feature (e.g. our release action
build) .
Signed-off-by: Bo Chen <chen.bo@intel.com>
Now cloud hypervisor will start signal thread to catch
SIGWINCH signal, cloud hypervisor then will resize the
guest console via vconsole.
This patch skip starting signal thread when there is no
need to resize guest console, such as console is not
configured.
Signed-off-by: Yong He <alexyonghe@tencent.com>
The PR #2333 added I/O rate limiter on block device, with some options
in `DiskConfig`. And the PR #2401 added rate limiter on virtio-net
device with same options, but it still throws `Error::ParseDisk`.
This commit fixes it with correct values.
Fixes: #2401
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Unused import 'xml' is redefined at:
> xml = ET.ElementTree(element=svg)
Hence, remove unused xml import.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
Any error from the backend or from the protocol handling code will now
result in an error reply being sent. This is cleanly achieved by
splitting the command handling out into its own method and using the
Rust Result<> based error handling to trigger the generation of the
error reply.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This allows the implementation of PCI devices in a different process
using the vfio-user protocol.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Replace the use of an enum with a bitfield representation which means
that is now possible to logical OR flags together.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Once error occur, vcpu thread may exit, this should
be critical event for the whole VM, we should fire
exit event and set vcpu state.
If we don't set vcpu state, the shutdown process
will hang at signal_thread, which is waiting the
vcpu state to change.
Signed-off-by: Yong He <alexyonghe@tencent.com>
This test (which relies on nesting) is failing on the VFIO worker. The tests that use the
dedicated hardware pass fine.
See: #5190
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
We need to provide valid FDs while creating 'NetConfig' instances even
for unit tests. Closing invalid FDs would cause random unit test
failures.
Also, two identical 'NetConfig' instances are not allowed any more,
because it would lead to close the same FD twice. This is consistent
with the fact that a clone of a "NetConfig" instance is no
longer *equal* to the instance itself.
Fixes: #5203
Signed-off-by: Bo Chen <chen.bo@intel.com>
These are owned by the config (and are duplicated before being used to
create the `Tap` for the virtio-net device.)
By implementing Drop on NetConfig we have issues with moving out of
members that don't implement the Copy trait. This requires a small
adjustment to the unit tests that use the Default::default() function.
Fixes: #5197
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The custom version duplicates any FDs that have been provided so that
the validation logic used on hotplug, which takes a clone of the config,
can be safely carried out.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This code is indentical to what is in this repository. When a release
gets made we can then switch to that.
Fixes: #5122
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If swtpm becomes unresponsive, guest gets blocked at "recvmsg" on tpm's
data FD. This change adds a timeout to the data fd socket. If swtpm
becomes unresponsive guest waits for "timeout" (secs) and continues to
run after returning an I/O error to tpm commands.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The updated image is configured in a same way as the previously used
2019, it has same
- Credentials
- Services configured, like SAC, SSH, RDP
- Size
All the Windows updates are applied so the state is current to the date.
Also, the latest stable version 0.1.229 of the VirtIO Windows drivers
is installed.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
We can ideally defer the address space allocation till we start the
vCPUs for the very first time. Because the VM will not access the memory
until the CPUs start running. Thus there is no need to allocate the
address space eagerly and wait till the time we are going to start the
vCPUs for the first time.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This hypervisor leaf includes details of the TSC frequency if that is
available from KVM. This can be used to efficiently calculate time
passed when there is an invariant TSC.
TEST=Run `cpuid` in the guest and observe the frequency populated.
Fixes: #5178
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Updates include:
- Add references to 'TDX Tools'
- Expand instructions on buidling and using TDShim
- Add version information of guest/host kernel, TDVF, TDShim being tested
Signed-off-by: Bo Chen <chen.bo@intel.com>
This is required for booting Linux:
From: https://lore.kernel.org/all/20221028141220.29217-3-kirill.shutemov@linux.intel.com/
"""
Virtualization Exceptions (#VE) are delivered to TDX guests due to
specific guest actions such as using specific instructions or accessing
a specific MSR.
Notable reason for #VE is access to specific guest physical addresses.
It requires special security considerations as it is not fully in
control of the guest kernel. VMM can remove a page from EPT page table
and trigger #VE on access.
The primary use-case for #VE on a memory access is MMIO: VMM removes
page from EPT to trigger exception in the guest which allows guest to
emulate MMIO with hypercalls.
MMIO only happens on shared memory. All conventional kernel memory is
private. This includes everything from kernel stacks to kernel text.
Handling exceptions on arbitrary accesses to kernel memory is
essentially impossible as handling #VE may require access to memory
that also triggers the exception.
TDX module provides mechanism to disable #VE delivery on access to
private memory. If SEPT_VE_DISABLE TD attribute is set, private EPT
violation will not be reflected to the guest as #VE, but will trigger
exit to VMM.
Make sure the attribute is set by VMM. Panic otherwise.
There's small window during the boot before the check where kernel has
early #VE handler. But the handler is only for port I/O and panic as
soon as it sees any other #VE reason.
SEPT_VE_DISABLE makes SEPT violation unrecoverable and terminating the
TD is the only option.
Kernel has no legitimate use-cases for #VE on private memory. It is
either a guest kernel bug (like access of unaccepted memory) or
malicious/buggy VMM that removes guest page that is still in use.
In both cases terminating TD is the right thing to do.
"""
With this change Cloud Hypervisor can boot the current Linux guest
kernel.
Reported-By: Jiaqi Gao <jiaqi.gao@intel.com
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Do the following:
1. Use from_be_bytes to drop mutable slices.
2. Check for the exact buffer size throughout.
3. Simplify ptm_to_request where possible.
4. Make error messages style consistent.
Fix a typo in code comment while at it.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
There is no guarantee that the write can send the whole buffer at once.
In those rare occasions, we should return a sensible error.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The largest possible PTM response is only 16 bytes. Size the output
buffer correctly.
In the socket read function, rely on the caller to provide a
sufficiently large buffer. That eliminates another large stack variable.
In total this saves almost 8KB stack space.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Change "thead" to "thread".
Also make sure the two messages are distinguishable by adding "vmm" and
"vm" prefix.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
The number of aligned operations can not be larger than the number of
descriptors. Initializing the capacity to 1 is good enough per the
observation that most of time there is only one data descriptor in a
given request.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Before Linux v6.0, AArch64 didn't support "socket" in "cpu-map"
(CPU topology) of FDT.
We found that clusters can be used in the same way of sockets. That is
the way we implemented the socket settings in Cloud Hypervisor. But in
fact it was a bug.
Linux commit 26a2b7 fixed the mistake. So the cluster nodes can no
longer act as sockets. And in a following commit dea8c0, sockets were
supported.
This patch fixed the way to configure sockets. In each socket, a default
cluster was added to contain all the cores, because cluster layer is
mandatory in CPU topology on AArch64.
This fix will break the socket settings on the guests where the kernel
version is lower than v6.0. In that case, if socket number is set to
more than 1, the kernel will treat that as FDT mistake and all the CPUs
will be put in single cluster of single socket.
The patch only impacts the case of using FDT, not ACPI.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Fix lowercase label to avoid "mkfs.fat: Warning: lowercase labels
might not work properly on some systems".
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
This patch adds a global execution timeout to the Jenkinsfile to avoid
infinite pending Jenkins pipelines, such as when certain worker nodes
are not available. The global execution timeout is now set to 4 hours
which is derived from total timeout of our longest stage (e.g. the
`Worker build`).
Fixes: #5148
Signed-off-by: Bo Chen <chen.bo@intel.com>
As a first time user of cloud-hypervisor and Rust environment
you get build errors with out this.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Right now integration test fails during the test run if
/dev/mshv or /dev/kvm does not exist. We should not
progress and exit early if not present.
Signed-off-by: Muminul Islam <muislam@microsoft.com>
Make the code more idiomatic by wrapping the actual size configured in
the returning Result type. This further allows simplifying
get_buffer_size.
The debug message in startup_tpm is more useful if it prints out the
actual size than the wanted size.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
In order to comply with latest TDX version, we rely onto the branch
kvm-upstream-2022.08.07-v5.19-rc8 from https://github.com/intel/tdx
repository. Updates are based on changes that happened in
arch/x86/include/uapi/asm/kvm.h headers file.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Dependabot will create a branch on the repo for it's updates this
triggers the release action (because it's the same event as a tag) which
will then fail leading to dependabot PRs not being automerged. Instead
only run the release check test on PRs or tag creation.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
KSM doesn't work with MSHV stack since guest memory is pinned
(`pin_user_pages`) and pinned pages cannot be merged.
So, don't run the test for mshv.
Signed-off-by: Anirudh Rayabharam <anrayabh@linux.microsoft.com>
This makes it much more convenient to build binaries for profiling
without having to modify the Cargo.toml file.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
A few breaking changes:
1. `-vvv` needs to be written as `-v -v -v`.
2. `--disk D1 D2` and others need to be written as `--disk D1 --disk D2`.
3. `--option=value` needs to be written as `--option value`
Change integration tests to adapt to the breaking changes.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Since argh does not support `--option=value`, we need to change the
integration test code to become `--option value`.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
sh 'scripts/dev_cli.sh tests --metrics -- -- --report-file /root/workloads/metrics.json'
}
}
stage('Upload metrics report') {
steps {
sh 'curl -X PUT https://cloud-hypervisor-metrics.azurewebsites.net/api/publishmetrics -H "x-functions-key: $METRICS_PUBLISH_KEY" -T ~/workloads/metrics.json'
}
}
}
}
stage('Worker build - Rate Limiter') {
agent { node { label 'focal-metrics' } }
when {
branch 'main'
beforeAgent true
expression {
return runWorkers
}
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Run rate-limiter integration tests') {
options {
timeout(time: 10, unit: 'MINUTES')
}
steps {
sh 'scripts/dev_cli.sh tests --integration-rate-limiter'
}
}
}
}
stage('Worker build - SGX') {
agent { node { label 'jammy-sgx' } }
when {
beforeAgent true
allOf {
branch 'main'
expression {
return runWorkers
}
}
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Run SGX integration tests') {
options {
timeout(time: 1, unit: 'HOURS')
}
steps {
sh 'scripts/dev_cli.sh tests --integration-sgx'
}
}
stage('Run SGX integration tests for musl') {
options {
timeout(time: 1, unit: 'HOURS')
}
steps {
sh 'scripts/dev_cli.sh tests --integration-sgx --libc musl'
}
}
}
post {
always {
sh "sudo chown -R jenkins.jenkins ${WORKSPACE}"
deleteDir()
}
}
}
stage('Worker build - VFIO') {
agent { node { label 'jammy-vfio' } }
when {
beforeAgent true
allOf {
branch 'main'
expression {
return runWorkers
}
}
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Run VFIO integration tests') {
options {
timeout(time: 1, unit: 'HOURS')
}
steps {
sh 'scripts/dev_cli.sh tests --integration-vfio'
}
}
stage('Run VFIO integration tests for musl') {
options {
timeout(time: 1, unit: 'HOURS')
}
steps {
sh 'scripts/dev_cli.sh tests --integration-vfio --libc musl'
}
}
}
post {
always {
sh "sudo chown -R jenkins.jenkins ${WORKSPACE}"
deleteDir()
}
}
}
}
}
}
post {
regression {
script {
if (env.BRANCH_NAME == 'main') {
slackSend(color: '#ff0000', message: '"main" branch build is now failing', channel: '#jenkins-ci')
}
}
}
fixed {
script {
if (env.BRANCH_NAME == 'main') {
slackSend(color: '#00ff00', message: '"main" branch build is now fixed', channel: '#jenkins-ci')
}
}
}
}
}
def cancelPreviousBuilds() {
// Check for other instances of this particular build, cancel any that are older than the current one
@@ -372,8 +375,8 @@ are all equal and welcome means of contribution. See the
## Slack
Get an [invite to our Slack channel](https://join.slack.com/t/cloud-hypervisor/shared_invite/enQtNjY3MTE3MDkwNDQ4LWQ1MTA1ZDVmODkwMWQ1MTRhYzk4ZGNlN2UwNTI3ZmFlODU0OTcwOWZjMTkwZDExYWE3YjFmNzgzY2FmNDAyMjI)
and [join us on Slack](https://cloud-hypervisor.slack.com/).
Get an [invite to our Slack channel](https://join.slack.com/t/cloud-hypervisor/shared_invite/enQtNjY3MTE3MDkwNDQ4LWQ1MTA1ZDVmODkwMWQ1MTRhYzk4ZGNlN2UwNTI3ZmFlODU0OTcwOWZjMTkwZDExYWE3YjFmNzgzY2FmNDAyMjI),
[join us on Slack](https://cloud-hypervisor.slack.com/), and [participate in our community activities](https://cloud-hypervisor.slack.com/archives/C04R5DUQVBN).
| Create the VM | `/vm.create`| `/schemas/VmConfig` | N/A | The VM is not created yet |
| Delete the VM | `/vm.delete`| N/A | N/A | N/A |
| Boot the VM | `/vm.boot`| N/A | N/A | The VM is created but not booted |
| Shut the VM down | `/vm.shutdown`| N/A | N/A | The VM is booted |
| Reboot the VM | `/vm.reboot`| N/A | N/A | The VM is booted |
| Trigger power button of the VM | `/vm.power-button`| N/A | N/A | The VM is booted |
| Pause the VM | `/vm.pause`| N/A | N/A | The VM is booted |
| Resume the VM | `/vm.resume`| N/A | N/A | The VM is paused |
| Task a snapshot of the VM | `/vm.snapshot`| `/schemas/VmSnapshotConfig` | N/A | The VM is paused |
| Perform a coredump of the VM* | `/vm.coredump` | `/schemas/VmCoredumpData` | N/A | The VM is paused |
| Restore the VM from a snapshot | `/vm.restore`| `/schemas/RestoreConfig` | N/A | The VM is created but not booted |
| Add/remove CPUs to/from the VM | `/vm.resize`| `/schemas/VmResize` | N/A | The VM is booted |
| Add/remove memory from the VM | `/vm.resize`| `/schemas/VmResize` | N/A | The VM is booted |
| Add/remove memory from a zone | `/vm.resize-zone`| `/schemas/VmResizeZone` | N/A | The VM is booted |
| Dump the VM information | `/vm.info`| N/A | `/schemas/VmInfo` | The VM is created |
| Add VFIO PCI device to the VM | `/vm.add-device`| `/schemas/VmAddDevice` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add disk device to the VM | `/vm.add-disk`| `/schemas/DiskConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add fs device to the VM | `/vm.add-fs`| `/schemas/FsConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add pmem device to the VM | `/vm.add-pmem`| `/schemas/PmemConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add network device to the VM | `/vm.add-net`| `/schemas/NetConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add userspace PCI device to the VM | `/vm.add-user-device` | `/schemas/VmAddUserDevice` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add vdpa device to the VM | `/vm.add-vdpa`| `/schemas/VdpaConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Add vsock device to the VM | `/vm.add-vsock`| `/schemas/VsockConfig` | `/schemas/PciDeviceInfo` | The VM is booted |
| Remove device from the VM | `/vm.remove-device`| `/schemas/VmRemoveDevice` | N/A | The VM is booted |
| Dump the VM counters | `/vm.counters`| N/A | `/schemas/VmCounters` | The VM is booted |
| Prepare to receive a migration | `/vm.receive-migration` | `/schemas/ReceiveMigrationData` | N/A | N/A |
| Start to send migration to target | `/vm.send-migration` | `/schemas/SendMigrationData` | N/A | The VM is booted and (shared mem or hugepages enabled) |
### REST API Examples
* The `vmcoredump` action is available exclusively for the `x86_64`
architecture and can be executed only when the `guest_debug` feature is
enabled. Without this feature, the corresponding [REST API](#rest-api) or
[D-Bus API](#d-bus-api) endpoints are not available.
#### REST API Examples
For the following set of examples, we assume Cloud Hypervisor is started with
the REST API available at `/tmp/cloud-hypervisor.sock`:
@@ -122,7 +134,7 @@ Cloud Hypervisor Guest
Disk(s): None
```
#### Create a Virtual Machine
##### Create a Virtual Machine
We want to create a virtual machine with the following characteristics:
@@ -4,7 +4,7 @@ Currently Cloud Hypervisor supports hot plugging of CPUs devices (x86 only), PCI
## Kernel support
For hotplug on Cloud Hypervisor ACPI GED support is needed. This can either be achieved by turning on `CONFIG_ACPI_REDUCED_HARDWARE_ONLY`
For hotplug on Cloud Hypervisor ACPI GED support is needed. This can either be achieved by turning on `CONFIG_ACPI_REDUCED_HARDWARE_ONLY`
or by using this kernel patch (available in 5.5-rc1 and later): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/patch/drivers/acpi/Makefile?id=ac36d37e943635fc072e9d4f47e40a48fbcdb3f0
## CPU Hot Plug
@@ -159,7 +159,7 @@ The same API can also be used to reduce the desired RAM for a VM. It is importan
Extra PCI devices can be added and removed from a running `cloud-hypervisor` instance. This is controlled by making a HTTP API request to the VMM to ask for the additional device to be added, or for the existing device to be removed.
Note: On AArch64 platform, PCI device hotplug can only be achieved using ACPI. Please refer to the [documentation](arm64.md#uefi-booting) for more information.
Note: On AArch64 platform, PCI device hotplug can only be achieved using ACPI. Please refer to the [documentation](uefi.md#building-uefi-firmware-for-aarch64) for more information.
To use PCI device hotplug start the VM with the HTTP server.
> The latest version of TDShim being tested is [_66bb334_](https://github.com/confidential-containers/td-shim/tree/66bb33451befbf1291abe3cfea7ee9e99d922b0d).
This is a lightweight version of the TDVF, written in Rust and designed for
direct kernel boot, which is useful for containers use cases.
You can find the instructions for building the firmware directly from the
@@ -7,13 +7,13 @@ The target audience of this document is both:
## Control
The number of `-v` parameters passed to the `cloud-hypervisor` binary will determine the log level. Currenly the default is log messages up to `WARN:` (`warn!`) are included by default. The `--log-file` allows the log to be sent to a location other than `stderr`.
The number of `-v` parameters passed to the `cloud-hypervisor` binary will determine the log level. Currently the default is log messages up to `WARN:` (`warn!`) are included by default. The `--log-file` allows the log to be sent to a location other than `stderr`.
## Levels
### `error!()`
For immediate, unrecoverable errors where it does not make sense for the execution to continue as the behaviour of the VM is considerablely impacted.
For immediate, unrecoverable errors where it does not make sense for the execution to continue as the behaviour of the VM is considerably impacted.
Cloud Hypervisor should exit shortly after reporting this error (with a non-zero exit code). Generally this should be used during initial construction of the VM state before the virtual CPUs have begun running code.
@@ -23,7 +23,7 @@ Users should react to this error by checking their initial VM configuration.
### `warn!()`
A serious problem has occured but the execution of the VM can continue although some functionality might be impacted.
A serious problem has occurred but the execution of the VM can continue although some functionality might be impacted.
A typical example of where this level of message should be generated is during an API call request that cannot be fulfilled.
--memory-zone <memory-zone> User defined memory zone parameters "size=<guest_memory_region_size>,file=<backing_file>,shared=on|off,hugepages=on|off,hugepage_size=<hugepage_size>,host_numa_node=<node_id>,id=<zone_identifier>,hotplug_size=<hotpluggable_memory_size>,hotplugged_size=<hotplugged_memory_size>,prefault=on|off"
```
This parameter expects one or more occurences, allowing for a list of memory
This parameter expects one or more occurrences, allowing for a list of memory
zones to be defined. It must be used with `--memory size=0`, clearly indicating
that the memory will be described through advanced parameters.
@@ -266,11 +266,8 @@ _Example_
### `file`
Path to the file backing the memory zone. This can be either a file or a
directory. In case of a file, it will be opened and used as the backing file
for the `mmap(2)` operation. In case of a directory, a temporary file with no
hard link on the filesystem will be created. This file will be used as the
backing file for the `mmap(2)` operation.
Path to the file backing the memory zone. The file will be opened and used as
the backing file for the `mmap(2)` operation.
This option can be particularly useful when trying to back a part of the guest
RAM with a well known file. In the context of the snapshot/restore feature, and
@@ -575,7 +572,16 @@ _Example_
### PCI bus
Cloud Hypervisor supports only one PCI bus, which is why it has been tied to
the NUMA node 0 by default. It is the user responsibility to organize the NUMA
nodes correctly so that vCPUs and guest RAM which should be located on the same
NUMA node as the PCI bus end up on the NUMA node 0.
Cloud Hypervisor supports guests with one or more PCI segments. The default PCI segment always
has affinity to NUMA node 0. Be default, all other PCI segments have afffinity to NUMA node 0.
The user may configure the NUMA affinity for any additional PCI segments.
The Microsoft Hypervisor is a Type 1 hypervisor which runs on x64 and ARM64 architectures. As the foundation of the Hyper-V virtualization stack, it runs millions of Linux and Windows guests in Azure and on-premises deployments. It supports nested virtualization, and security features like AMD's SEV-SNP. It also supports various features in Windows such as [Device guard and confidential guard](https://techcommunity.microsoft.com/t5/iis-support-blog/windows-10-device-guard-and-credential-guard-demystified/ba-p/376419), and [WSL2](https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq)
Since 2020, Microsoft has been releasing open-source components to support Linux running as root partition on the Microsoft Hypervisor.
1. Kernel patches to support Linux booting as root partition
2. A Linux kernel driver exposing an IOCTL interface for managing guest partitions, via a device node - /dev/mshv
3. Rust bindings and IOCTL wrappers
4. IGVM related crates
## Components
The following components are related to MSHV support with Cloud-Hypervisor:
* [mshv-crates](https://github.com/rust-vmm/mshv) : Rust crates to interact with kernel module (/dev/mshv)
* [igvm-tooling](https://github.com/microsoft/igvm-tooling) : Tool to generate IGVM file
## IGVM
Independent Guest Virtual Machine (IGVM) file format.The format specification can be found in the igvm_defs crate, with a Rust implementation of the binary format in the igvm crate.
The IGVM file format is designed to encapsulate all information required to launch a virtual machine on any given virtualization stack, with support for different isolation technologies such as AMD SEV-SNP and Intel TDX.
At a conceptual level, this file format is a set of commands created by the tool that generated the file, used by the loader to construct the initial guest state. The file format also contains measurement information that the underlying platform will use to confirm that the file was loaded correctly and signed by the appropriate authorities.
Cloud Hypervisor can be built using igvm feature flag along with mshv and/or sev-snp. IGVM only works with MSHV.
## SEV-SNP
AMD's [Secure Encrypted Virtualization (SEV)](https://www.amd.com/en/developer/sev.html) and extensions such as Secure Nested Paging (SEV-SNP) encrypt memory and restrict access to a guest VM's memory and registers, securing it against a compromised hypervisor or VMM. They utilize the Platform Security Processor (PSP) to store keys and encrypt/decrypt the data. Microsoft has been continuously adding/improving support for SEV-SNP on Microsoft Hyper-V. Cloud-Hypervisor can be built with the sev_snp feature including mshv and igvm feature.
## Use Cases
Cloud Hypervisor can be built to run on an MSHV root partition by enabling the mshv feature, e.g.:
@@ -50,14 +50,14 @@ The examples use __socat__ `>=1.7.4` to illustrate the VSOCK functionality. Howe
### Connecting from Host to Guest
The host starts to listen on the defined port:
The guest starts to listen on the defined port:
`$ socat - VSOCK-LISTEN:1234`
Once the host is listening, the guest can send data:
Once the guest is listening, the host can send data:
`echo -e "CONNECT 1234\\nHello from host!" | socat - UNIX-CONNECT:/tmp/ch.vsock
Note the string `CONNECT <port>` prepended to the actual data. It is possible for the guest to start listening on different ports, thus the specific command is needed to instruct VSOCK to which listener the host wants to connect. It needs to be sent once per connection. Once the connection established, data transfers can take place directly.
@@ -135,7 +135,7 @@ This section illustrates the Windows specific aspects of the VM network configur
### Basic Networking
As the simplest option, using `--net tap=` in the Cloud Hypervisor command line will create a `vmtapX` device on the host with the default IPv4 adress `192.168.249.1`. After SAC becomes available, the guest configuration can be set with
As the simplest option, using `--net tap=` in the Cloud Hypervisor command line will create a `vmtapX` device on the host with the default IPv4 address `192.168.249.1`. After SAC becomes available, the guest configuration can be set with
@@ -203,13 +203,13 @@ The Windows guest debugging process relies heavily on QEMU and [socat](http://ww
The connection between both guests happens over TCP, whereby on the guest side it is automatically translated to a COM port. Because the VMs are connected through TCP, the debugging infrastructure can be distributed over the network. The serial port, while slowly transferring data, is common enough to support a wide range of cases and tools.
In this excercise, [WinDbg](https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/) is used. Any other debugger of choice with the ability to use serial connection can be used instead.
In this exercise, [WinDbg](https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/) is used. Any other debugger of choice with the ability to use serial connection can be used instead.
### Debugger and Debuggee
#### WinDbg VM
For simplicity, the debugger VM is supposed to be only running under QEMU. It will require VGA and doesn't neccessarily depend on UEFI. As an OS, it can carry any supported Windows OS where the debugger of choice can be installed. The simplest way is to follow the image preparation instructions from the previous chapter, but avoid using the OVMF firmware. It is also not required to use VirtIO drivers, whereby it might be useful in some case. Though, while creating the image file for the debugger VM, be sure to choose a sufficient disk size that counts in the need to save the corresponding debug symbols and sources.
For simplicity, the debugger VM is supposed to be only running under QEMU. It will require VGA and doesn't necessarily depend on UEFI. As an OS, it can carry any supported Windows OS where the debugger of choice can be installed. The simplest way is to follow the image preparation instructions from the previous chapter, but avoid using the OVMF firmware. It is also not required to use VirtIO drivers, whereby it might be useful in some case. Though, while creating the image file for the debugger VM, be sure to choose a sufficient disk size that counts in the need to save the corresponding debug symbols and sources.
To create the debugger Windows VM, the following command can be used:
@@ -244,7 +244,7 @@ This will configure the debugging to be enabled and instruct to use the serial p
bcdedit /dbgsettings serial debugport:1 baudrate:115200
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
