Add command line flags -x and -X to print counter values in
hexadecimal format with leading 0x prefix (-X) and without (-x).
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line flags -i (read interval) and -l (loops) to
show the counter values for the specified number of iterations.
The CPUs to read the counter values from can be specified
on the command line. If none are specified, all online CPUs
are used.
For example the command '# lspai -l 2 -i 10 -t nnpa'
displays all PAI NNPA counter values two times with 10 second
interval.
Acked-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Co-developed-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Relocated the perf_event_open logic into a shared helper to eliminate
redundant implementations across multiple files. Enhances consistency,
reduces maintenance overhead, and lowers structural complexity.
Acked-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
With version 1.23 groff/troff disabled the non-portable font CW and
started to complain about a missing CW font when previewing some of the
man pages, with messages like
$ man --warnings cpumf/man/pai.8 > /dev/null
troff:<standard input>:244: warning: cannot select font 'CW'
Use CR to replace CW.
Closes: https://github.com/ibm-s390-linux/s390-tools/pull/187
[hoeppner@linux.ibm.com: Adapt commit message]
Signed-off-by: Dan Horák <dan@danny.cz>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Introduce a new command-line option --format FORMAT to control the output
format of lshwc. Supported formats include CSV, JSON, JSON-SEQ, and key-value
PAIRS. The output is now generated using the shared util_fmt infrastructure,
enabling consistent and machine-readable output.
This change also includes:
- Label generation for counters
- Integration of util_fmt for structured formatting
- Updated man page with documentation for the new option
- Updated man page with JSON output structure documentation
- Add option (-q) to force quoting all elements.
Reference: https://www.ietf.org/rfc/rfc4180.txt
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add support for IBM z17 machine types 9175 and 9176 CPU Measurement
facility basic, problem, crypto and extended counter sets.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Use util_path_sysfs() to form a fully qualified sysfs file name
for files S390_CPUMF_CF, S390_CPUMF_CFDIAG, S390_CPUMF_SF,
S390_SYSFS_PAI_CRYPTO and S390_SYSFS_PAI_EXT.
No functional change.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add a command line flag to specify run time of lshwc command in
terms of days, hours, minutes and seconds. The command line flag
is named -t and has a number as argument followed by one of the
letters 's' for seconds (default), 'm' for minutes, 'h' for hours
and 'd' for days. Default is seconds when no trailing letter is
detected.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add a flag to display the counter value in form of a delta
value. This format shows the increase of that counter value
in comparison to the previous readout.
The first line is the base for the delta calculation and always
shows 'Total'. The remaining lines show 'Delta' in the third
column.
# ./lshwc -d -i 3 -l 5 -x -s :p
Date,Time,CPU,P32,P33
2024-11-27,15:45:55,Total,d7b,172
2024-11-27,15:45:58,Delta,2be403,2d58bb
2024-11-27,15:46:01,Delta,43e3b,22c41
2024-11-27,15:46:04,Delta,58e3a,35319
2024-11-27,15:46:07,Delta,5080e,2b81c
#
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Free all buffers allocated with malloc before program exit.
Handle memory leak in libcpumf_cpuset().
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Allocate the memory required to save the per CPU CPUMF counter value
at the beginning of program execution when the involved number of
CPUs is known. This a bit faster then checking for memory allocation
each time a counter value is retrieved.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line option -H to hide counter value numbers from counters
which are not defined in a counter set. They are usually all zero and
are of no interest.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add option -s or --short to display the header using a short
counter name. With this option the counter symbol names, which can be
very long as in IDCW_ON_DRAWER_DRAWER_HIT are replaced by a shorter
name. That name consists of an abbrevation for the counter set
this counter belongs to and the counter number in that set.
The abbrevations are:
B --> Basic counter set
P --> Problem state counter set
C --> Crypto counter set
E --> Extended counter set
M --> MT_Diagnostic counter set
U --> Undefined counter.
Display E165 for counter name IDCW_ON_DRAWER_DRAWER_HIT
which is counter number 165 from the extended counter set.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Move defines from a source file lscpumf.c to the header
file in the include directory to be reused in a follow on
patch. Also reshuffle the list of defines in proper
sequence of increasing numbers.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Fix whatis-entries in the lshwc, lspai, and pai manpages by inlining the
tool name. The following command line can be used to test a local
manpage:
$ lexgrog -w -m cpumf/man/lshwc.8
cpumf/man/lshwc.8: "lshwc - extract CPU Measurement Facilities counter sets"
Github-ID: https://github.com/ibm-s390-linux/s390-tools/issues/170
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
When pai records data it may fail in select() system call.
This error is not reported and the pai program exits with success.
Change this and exit with proper exit code.
Fixes: d7b1cbad8b ("cpumf/pai: Add Processor Activity Instrumentation tool")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Running command pai -r <file> might encounter entries of type
PERF_RECORD_SWITCH. Entries of that type are currently not handled
and cause an error message:
# pai -r ~/paicrypto.000
unknown header-type 14 unknown header-type 14 \
unknown header-type 14 ...
The error message is not terminated by a newline.
Handle entries of this type PERF_RECORD_SWITCH. These records
do not carry any payload at all, just a bit is set in the
header::misc member. This bit set determines context switch out.
Output after:
# ./pai -r ~/paicrypto.000
0x4b814018f4f3 6 cs-out
0x4b817bc3c936 6 cs-in
0x4b817bc5246c 6 cs-out
0x4b817bd90e9a 6 cs-in
....
#
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Pai collects data from per CPU ring buffers and stores them in the
memory mapped output file. When data is collected from many CPUs at
the same time, writing data to output file can be slow.
Improve this and allow the pai recording to run with higher
real time priority. This is the same approach as done by the perf tool.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add lspai program and man page to display Processor Activity
Information (PAI) facility counter sets in the same way as
lscpumf for the CPU Measurement Facility counter sets.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Replace memory allocation functions malloc() and calloc()
by counterparts provided in libutil.a library. Also remove error
handling when no memory could be allocated as those functions do
not return.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Linux kernel commits
a029a4eab39 ("s390/cpumf: Allow concurrent access for CPU Measurement Counter Facility")
453380318ed ("s390/cpumf: Allow multiple processes to access /dev/hwc")
removed this restriction. Both commits are included in linux 5.16.
Therefore remove the restriction in the man page.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
lshwc displays counter values extracted from the /dev/hwctr device.
The counter values are retrieved using a loop and incremented instead
of just displayed on output.
This leads to the situation where a counter is incremented in large
numbers while the counter triggering program is active. When this
program terminates, the counter should not increase but remain steady.
This is not the case as this example shows:
# lshwc -i 2 -l90 -a :a | fgrep Total | \
awk -F ',' '{print $2 "," $152 "," $157}'
Report time and values of DFLT_ACCESS and DFLT_CC counters
sleep 10
# DFLTCC=1 python3 -c "import zlib; zlib.compress(b'A'*6000000000);
Increment counter DFLT_ACCESS and DFLT_CC, runs about 45 seconds
Output before:
15:24:20,0,0
15:24:22,0,0
15:24:24,0,0
15:24:26,0,0
15:24:28,0,0
15:24:30,0,0
15:24:32,2067027,592996481 <--- python3 program start
15:24:34,6953101,2039012580
15:24:36,14494585,4264340060
15:24:38,24759568,7346128396
....
15:24:56,232929262,70526362509
15:24:58,269037963,81464038124
15:25:00,307757248,93171809261
15:25:02,348900718,105620730492
15:25:04,392646075,118841031269 <--- python3 program stop
15:25:06,438956016,132853968029 <-- Value column two increased by 49148217
15:25:08,488104233,147709011658 <-- Similar high increase
The reason is that lshwc takes the last read counter value and keeps
adding it to the total count, even if there zero delta to the previous
read value. Fix this by clearing the total count value between ioctl()
read operations.
Output after:
15:39:26,0,0
15:39:28,0,0
15:39:30,0,0
15:39:32,0,0
15:39:34,0,0
15:39:36,1367250,380721127 <--- python3 program start
15:39:38,4104791,1108181711
15:39:40,6740994,1858843730
15:39:42,9277545,2646351211
15:39:44,11895180,3444375804
....
15:40:16,54060774,16315924126
15:40:18,57830349,17513712210
15:40:20,61628170,18690791102
15:40:22,65410470,19864086006
15:40:24,69070765,21028646754 <--- python3 program stop
15:40:26,70429758,21472772340 <-- No increase of value in column two
15:40:28,70429758,21472772340
15:40:30,70429758,21472772340
15:40:32,70429758,21472772340
15:40:34,70429758,21472772340
15:40:36,70429758,21472772340
15:40:38,70429758,21472772340
Fixes: 27a562da0a ("cpumf/lshwc: Program to extract complete counter sets")
Reported-by: Axel Busch <Axel.Busch@ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
pai dumps core when invoked with summary flag -S and
certain painnpa.XXXXX data files:
# ~/s390-tools/cpumf/pai -r -S
Segmentation fault (core dumped)
#
Fix this and check the existence of event raw data before
reading it. This avoid accessing NULL pointer trying to read raw
data when no counter values have been appended as raw data by the
kernel.
Fixes: dc73c77d73 ("cpumf/pai: Omit detailed event information when summary flag selected")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Use PMU named pai_ext to extract the PMU type number required
for the installation of PAI NNPA counters using perf_event_open()
system call.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
When the summary flag is selected do not print detailed information
on every single event. Just print the summary information on each
non-zero counter value.
Suggested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
During recording, progress information is displayed on the screen
on how many event have been processed and at which offset these
events have been extracted from the ring buffer.
This information is usually of no interest to the user. Omit this
data and only print them when requested via verbose flag.
Suggested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Use util_file_read_va() function to read out the event number
given the event file. Also use util_strdup() function to save
the event name. This function terminates on Out Of Memory condition.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Use util_file_read_va() function to read out the event number
given the event file. Also use util_strdup() function to save
the event name. This function terminates on Out Of Memory condition.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add IBM z16 extended counter set. For each counter in this set
add the counter number, the short and long description and the
counter number.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
The Processor Activity Instrumentation facility (PAI) Extension 1
adds support for the counter set for Neural Network Processing Assist
(NNPA) counters.
NNPA counter values are appended as raw data to the data report.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
The Processor Activity Instrumentation facility (PAI) uses the
perf_event_open system call to report data and the crypto counter
values. The data can be inspected with the perf tool in a very
generic way using raw data report and a hex dump utility.
The pai tool also uses the perf_event_open system call to record the
data, but uses specialized event attributes for recording detailed
information such as the recording of fork, exit, exec system call
events and context switch events.
Cryptography counter values are appended as raw data attachments.
The pai command can be used for recording and reporting.
Also a list of CPUs can be specified.
The recording ring buffers are created per CPU
and are named paicrypto.XXX where XXX stands for the CPU number.
The numbering scheme has leading zeros, such as 007 for CPU number 7.
For invocation and possible command line options see the man page.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
When lshwc is invoked with
# ./lshwc -a :P
lshwc: ioctl S390_HWCTR_START: Invalid argument
#
it returns an error instead of listing all problem state counters
of all online CPUs. The reason is an empty CPU list when it is
omitted and only a counter set is specified. Fix this.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
With the introduction of libcpumf/libcpumf.a library there is no need
for this file anymore. Its defines have been moved to
include/lib/libcpumf.h are obsolete.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
lshwc allows to specify a list of CPUs, for example
# ./lshwc -a 2-3,66-68:b
Date,Time,CPU,CPU_CYCLES(0),INSTRUCTIONS(1),L1I_DIR_WRITES(2),....
2022-01-21,12:51:54,CPU2,68762,20999,283,14157,601,87255
2022-01-21,12:51:54,CPU3,70514,21179,288,10654,586,90188
2022-01-21,12:51:54,CPU4,48504,21018,141,7831,438,78494
2022-01-21,12:51:54,Total,187780,63196,712,32642,1625,255937
#
However, as seen on the output, the CPU list is incorrect. It shows
CPUs 2, 3 and 4 even when only CPU 2 and 3 was requested. CPUs 66, 67
and 68 have not been online and can not be displayed.
What happens is a wrong parameter conversion for the device driver.
CPUs 66, 67 and 68 are passed to the device driver as CPUs 2 3 and 4
and that data is returned.
Fix this and submit a CPU list with correct bit ordering:
# ./lshwc -a 2-3,66-68:b
Date,Time,CPU,CPU_CYCLES(0),INSTRUCTIONS(1),L1I_DIR_WRITES(2),...
2022-01-21,12:59:18,CPU2,50753,18042,246,10972,494,60345
2022-01-21,12:59:18,CPU3,54002,20390,232,9219,511,66033
2022-01-21,12:59:18,Total,104755,38432,478,20191,1005,126378
#
Reported-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
