Commit Graph

69 Commits

Author SHA1 Message Date
Anatol Belski
c0a81bc903 performance-metrics: Settle host before each test
Flush host writeback queues, drop the page cache and sleep 1s
for kernel housekeeping before each test run.

The cloud-hypervisor block backend does buffered I/O on the host
side, so dirty pages from prior write tests can accumulate and
compete for I/O bandwidth with subsequent tests. Dropping caches
ensures cold read tests get a consistent baseline rather than
benefiting from data cached by prior tests. The brief cooldown
lets the kernel finish tearing down KVM state and freeing pages
from the previous VM before the next one starts.

Requires root, which the metrics container provides. Silently
fails otherwise.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2026-03-12 12:27:24 +00:00
Anatol Belski
57e64a0848 performance-metrics: Kill stale processes on test timeout
When a test times out, the spawned thread containing the
cloud-hypervisor child process, iperf3/ethr sub processes,
and all associated resources (TAP devices, file descriptors,
hugepage reservations) is abandoned without cleanup. This
attaches a cleanup routine that kills cloud-hypervisor,
iperf3, and ethr processes on timeout, then waits
briefly for the kernel to reclaim their resources. This
prevents leaked processes from interfering with subsequent
tests.

Removes the existing TODO comment.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2026-03-12 12:27:24 +00:00
Anatol Belski
16d6a16e5c performance-metrics: Add standalone qcow2 perf tests
Add performance tests for standalone qcow2 images without backing
files - uncompressed, zlib and zstd compressed. Each variant
includes single queue and multiqueue tests for sequential
read, random read and warmed up sequential read.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2026-02-16 11:50:23 +00:00
Anatol Belski
333db1acb3 performance-metrics: Add multi-queue overlay block I/O tests
Add multiqueue num_queues=4 performance tests for qcow2 overlay
images with both qcow2 and raw backing files - sequential read,
random read, and warm read variants.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2026-02-16 11:50:23 +00:00
Anatol Belski
cde7856d9e performance-metrics: Add warmup support for warm cache testing
Add warmup_iterations field to run iterations before measuring
performance. This complements existing cold start tests
by separating cache effects from steady state throughput.

New tests with 2 warmup iterations:
- block_qcow2_backing_qcow2_read_warm_MiBps
- block_qcow2_backing_raw_read_warm_MiBps

Results show warm cache is much faster and more consistent:
- QCOW2: 1766 MiB/s (4% variance) vs cold 960 MiB/s (73% variance)
- RAW: 1822 MiB/s (6% variance) vs cold 1300 MiB/s (55% variance)

RAW backing is 3% faster than QCOW2 in steady state.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2026-01-16 18:54:22 +00:00
Anatol Belski
10394da0c4 performance-metrics: Add RAW backing file performance tests
Add sequential and random read tests for QCOW2 overlays with
RAW backing files.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2026-01-15 16:19:15 +00:00
Anatol Belski
d75bd1675c performance-metrics: Add QCOW2 backing file performance test
Add sequential and random read performance tests for QCOW2 overlays
with QCOW2 backing files.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2025-12-16 21:49:54 +00:00
Anatol Belski
d696cea024 performance-metrics: Refactor fio_control to BlockControl
Introduce a new BlockControl struct to encapsulate fio operation
parameters. This replaces the tuple-based fio_control with a more
extensible structure that includes:
- fio_ops: The FIO operation type
- bandwidth: Whether to measure bandwidth or IOPS
- test_file: The file path to test against

This refactoring enables reusing performance_block_io with different
test files.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2025-12-16 21:49:54 +00:00
Philipp Schuster
2c1eccc6bc misc: performance-metrics: drop extern crate, use modern rust
This commit is part of a series of similar commits.

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
2025-11-24 22:36:46 +00:00
Philipp Schuster
fed010fcd1 misc: clippy: add manual_string_new
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
2025-11-21 09:32:11 +00:00
Philipp Schuster
7cb73e9e56 misc: clippy: add unnecessary_semicolon
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
2025-11-21 09:32:11 +00:00
Philipp Schuster
363273111a build: treewide: fmt for edition 2024
`cargo +nightly fmt`

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
On-behalf-of: SAP philipp.schuster@sap.com
2025-09-10 18:35:38 +00:00
Wei Liu
8a26380657 performance-metrics: Add all supported formats to block tests
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2025-07-24 23:37:09 +00:00
Wei Liu
76d8d47f6a performance-metrics: Initialize tests after setting overrides
No functional change.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2025-07-24 23:37:09 +00:00
Ruoqing He
61e57e1cb1 misc: Further improve imports styling
By introducing `imports_granularity="Module"` format strategy,
effectively groups imports from the same module into one line or block,
improving maintainability and readability.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2024-09-29 16:13:48 +00:00
Rob Bradford
88a9f79944 misc: Adapt consistent import style formatting
Historically the Cloud Hypervisor coding style has been to ensure that
all imports are ordered and placed in a single group. Unfortunately
cargo fmt has no support for ensuring that all imports are in a single
group so if whitespace lines were added as part of the import statements
then they would only be odered correctly in the group.

By adopting "group_imports="StdExternalCrate" we can enforce a style
where imports are placed in at most three groups for std, external
crates and the crate itself. Choosing a style enforceable by the tooling
reduces the reviewer burden.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
2024-09-29 13:08:12 +01:00
Songqian Li
e18d32bac2 performance-metrics: add restore latency performance metric
This patch calculates the recovery latency from the start of
recovery program to the end of restore.

Signed-off-by: Songqian Li <sionli@tencent.com>
2024-08-02 08:29:03 +00:00
Ravi kumar Veeramally
ce902c19ec performance-metrics: switch to clap
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
2023-10-20 11:44:28 -07:00
Omer Faruk Bayram
59012ccc6e build: rename BUILT_VERSION to BUILD_VERSION
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
2023-04-14 12:13:46 -07:00
smit-gardhariya
7e47464a48 tests: performance-metrics: Add timeout parameter as runtime argument
This commit will add timeout parameter to performance metrics
testcases as runtime argument. Testcases are facing the timeout
sometimes when we run with MSHV hypervisor. This change will
provide mechanism to pass the timeout as per the need while
running the testcases. If nothing is passed, default timeout or
timeout set as per the testcase will be followed.

Signed-off-by: smit-gardhariya <gardhariya.smit@gmail.com>
2023-03-14 22:05:55 -07:00
Wei Liu
11ef495e6b performance-metrics: share build.rs from project root
No need to duplicate the same content in two places.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-30 21:10:02 +00:00
Wei Liu
1ba995d952 performance-metrics: switch to argh
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
0389190c64 build: drop the need to import macros from Clap
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 13:06:16 -08:00
Wei Liu
03d8a3ebcf performance-metrics: add metrics for UDP PPS
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
Wei Liu
e16817ea14 performance-metrics: add IOPS tests for FIO
Change fio_ops to fio_control and add a new field to indicate whether it
should report bandwidth or IOPS.

All existing tests are bandwidth tests. Adapt the code accordingly. Add
a set of new tests to report IOPS.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-04 09:47:44 +00:00
Rob Bradford
5e52729453 misc: Automatically fix cargo clippy issues added in 1.65 (stable)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-12-14 14:27:19 +00:00
Sebastien Boeuf
89677c3181 build: Bump clap from 3.2.22 to 4.0.9
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.22 to 4.0.9.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](clap-rs/clap@v3.2.22...v4.0.9)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Moving to the major version 4 introduced some breaking changes which had
to be handled manually.

Fixes #4709

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-10-05 12:59:14 +01:00
Rob Bradford
f32487f8e8 misc: Automatic beta clippy fixes
e.g. cargo clippy --all --tests --all-targets --fix --features=..

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-09-20 10:59:48 +01:00
Bo Chen
d8a6725995 performance-metrics: Move some helper functions to test_infra
The helper functions for measuring and parsing the performance of
virtio-net and virtio-block devices are moved to the `test_infra` crate
so that they be reused for integration tests of rate limiter.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-14 15:01:43 +01:00
Rob Bradford
cac42301f8 performance-metrics: Produce some error messages if git commands fail
It is reasonable for these to fail as it the tool could be run outside
of a git repository but by not giving any error message we cannot see
issues when we expect the report to have the git details.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-06-10 17:20:13 +01:00
Rob Bradford
025447e8cc performance-metrics: Add ability to override test iterations
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-05-23 11:14:53 +01:00
Maksym Pavlenko
3a0429c998 cargo: Clean up serde dependencies
There is no need to include serde_derive separately,
as it can be specified as serde feature instead.

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-05-18 08:21:19 +02:00
Rob Bradford
3c0817a1b7 performance-metrics: Add boot time metrics using multiple vCPUs
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-05-10 13:10:11 +01:00
Henry Wang
8dda052c7d performance-metrics: disable net latency test on AArch64
As ethr is not stable on AArch64, disable it until we find other tool.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2022-03-23 18:25:48 +08:00
Rob Bradford
c922bf23f6 performance-metrics: Use non-zero exit code on report file issues
This removes the need to wrap an error that is never used.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
f99c09fab7 performance-metrics: Successful output should be on stdout not stderr
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
a6d86b9496 performance-metrics: Use Write::write_all() for report
Write::write() is not guaranteed to write all the contents.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
1004f870e1 performance-metrics: Refactor report file opening code
It should be closer to the use of the opened file.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
d3be855b4b performance-metrics: Exit binary with non-zero exit code on test failure
This ensures that partial reports are not generated.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
0cbdbc8122 performance-metrics: Remove unnecessary prelude on --list-tests
The user knows they have requested the list of tests.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
959be92189 performance-metrics: Fix typo in function name
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
5934319953 performance-metrics: Remove obsolete TODO
We upload from the file written to disk.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
638c4a5781 performance-metrics: Use friendlier metric units
* Boot time in ms
* Network throughput in Gbps
* Block throughput in MiBps

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-03 11:39:55 -08:00
Rob Bradford
9592accb46 performance-metrics: Rename virtio_net_latency_ns->virtio_net_latency_us
The unit from ethr is microseconds.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-01 08:44:57 -08:00
Rob Bradford
8458696094 performance-metrics: Rename git_committer_date to git_commit_date
The former implies a person rather than a thing.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
4cc44f8cc7 performance-metrics: Trim whitespace from git command output
This removes trailing "\n"s from the strings saved in the JSON file.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
9978aac402 performance-metrics: Consistently rename queue_num to num_queues
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
082872423f performance-metrics: Fix test label to remove duplicated "bps"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
acafda6739 performance-metrics: Consistently use "test_timeout" over "test_time"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
6db3d89837 performance-metrics: Remove "performance_" prefix from report results
We know that these are all performance results.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00