The structure perf_event_attr sometimes gets new members appended
at the end of the structure. Then the size of the structure increases.
This may lead to the situation where sizeof(struct perf_event_attr)
calculated at the pai program compile time during s390-tools build
differs from the one used to build application programs on the target
system.
The report files written by various tools write the contents of the
perf_event_attr structure followed by indivual samples to a binary file.
The second member perf_event_attr::size contains the size of that
structure in bytes.
Use that perf_event_attr::size from the target system to scan the
report files given as command line argument. This ensure the correct
size of structure perf_event_attr is used.
Output before:
Current situation: The pai program uses a perf_event_attr structure
which contains 8 bytes less than the one used on the target system.
The first sample header (8 bytes) then actually refers to the last
eight bytes of structure perf_event_attr, which are all zeroes. This
is an invalid sample entry and the program terminates with error.
# pai -r -V painnpa.0004043; echo $?
painnpa.0004043 size:1344
[0x000088] type 0 misc 0 size 0
1
Output after:
Use the correct size of structure perf_event_attr, which was read
from be binary file, written by the tools compiled on the target system:
# pai -r -V painnpa.0004043; echo $?
painnpa.0004043 size:1344
size perf_event_attr mismatch 136/144
[0x000090] type 9 misc 1 size 58 0x13537f71715 18 event 6144 \
sample pid 4043/4043 15:0x8a,16:0x88,22:0x112,25:0xc0
...
0
Also show an debug message when verbose mode is turned on.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Remove ctype.h and errno.h since they are not referenced in this
file.
Add err.h to explicitly declare the dependency on warnx().
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Remove lib/util_base.h and errno.h since they are not referenced in this
file.
Add err.h to explicitly declare the dependency on err() and warn().
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command-line option --counters "AAA,BBB,...,ZZZ" to specify a
comma-separated list of counter names to display. Counter names must
match exactly, ignoring case. All counters not listed are excluded
from output.
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Move ctr_in_list() from cpumf/lspai.c to a shared location to enable
reuse in other binaries that require counter list filtering.
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line option --counters XXX,YYY to specify a comma
separated list of counter names to be displayed. The counter names
have to match exactly beside case sensitivity.
All counters not listed in the list specified by the --counters option
are not shown.
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Co-developed-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add a command line flag '-a' to include the counter set values
from each individual CPU. Up to now only the total sum from
all CPUs was printed.
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line flag -d to display the counter value in form of
a delta value. This format shows the increase of that counter value
compared to the previous value.
The first line is the base for the delta calculation and always
shows 'Total'. The remaining lines show 'Delta' in the third
column.
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line flag -s to display the counter name in an
abbreviated form. The abbreviated form is a character for the
PAI counter set followed by a number.
PAI crypto counter set is abbreviated to 'C'.
PAI NNPA counter set is abbreviated to 'N'.
The number refers to the defined counter number ranging from
1 to X.
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line flags -x and -X to print counter values in
hexadecimal format with leading 0x prefix (-X) and without (-x).
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line flags -i (read interval) and -l (loops) to
show the counter values for the specified number of iterations.
The CPUs to read the counter values from can be specified
on the command line. If none are specified, all online CPUs
are used.
For example the command '# lspai -l 2 -i 10 -t nnpa'
displays all PAI NNPA counter values two times with 10 second
interval.
Acked-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Co-developed-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Relocated the perf_event_open logic into a shared helper to eliminate
redundant implementations across multiple files. Enhances consistency,
reduces maintenance overhead, and lowers structural complexity.
Acked-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
With version 1.23 groff/troff disabled the non-portable font CW and
started to complain about a missing CW font when previewing some of the
man pages, with messages like
$ man --warnings cpumf/man/pai.8 > /dev/null
troff:<standard input>:244: warning: cannot select font 'CW'
Use CR to replace CW.
Closes: https://github.com/ibm-s390-linux/s390-tools/pull/187
[hoeppner@linux.ibm.com: Adapt commit message]
Signed-off-by: Dan Horák <dan@danny.cz>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Introduce a new command-line option --format FORMAT to control the output
format of lshwc. Supported formats include CSV, JSON, JSON-SEQ, and key-value
PAIRS. The output is now generated using the shared util_fmt infrastructure,
enabling consistent and machine-readable output.
This change also includes:
- Label generation for counters
- Integration of util_fmt for structured formatting
- Updated man page with documentation for the new option
- Updated man page with JSON output structure documentation
- Add option (-q) to force quoting all elements.
Reference: https://www.ietf.org/rfc/rfc4180.txt
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Reviewed-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add support for IBM z17 machine types 9175 and 9176 CPU Measurement
facility basic, problem, crypto and extended counter sets.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Use util_path_sysfs() to form a fully qualified sysfs file name
for files S390_CPUMF_CF, S390_CPUMF_CFDIAG, S390_CPUMF_SF,
S390_SYSFS_PAI_CRYPTO and S390_SYSFS_PAI_EXT.
No functional change.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add a command line flag to specify run time of lshwc command in
terms of days, hours, minutes and seconds. The command line flag
is named -t and has a number as argument followed by one of the
letters 's' for seconds (default), 'm' for minutes, 'h' for hours
and 'd' for days. Default is seconds when no trailing letter is
detected.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add a flag to display the counter value in form of a delta
value. This format shows the increase of that counter value
in comparison to the previous readout.
The first line is the base for the delta calculation and always
shows 'Total'. The remaining lines show 'Delta' in the third
column.
# ./lshwc -d -i 3 -l 5 -x -s :p
Date,Time,CPU,P32,P33
2024-11-27,15:45:55,Total,d7b,172
2024-11-27,15:45:58,Delta,2be403,2d58bb
2024-11-27,15:46:01,Delta,43e3b,22c41
2024-11-27,15:46:04,Delta,58e3a,35319
2024-11-27,15:46:07,Delta,5080e,2b81c
#
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Free all buffers allocated with malloc before program exit.
Handle memory leak in libcpumf_cpuset().
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Allocate the memory required to save the per CPU CPUMF counter value
at the beginning of program execution when the involved number of
CPUs is known. This a bit faster then checking for memory allocation
each time a counter value is retrieved.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add command line option -H to hide counter value numbers from counters
which are not defined in a counter set. They are usually all zero and
are of no interest.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Add option -s or --short to display the header using a short
counter name. With this option the counter symbol names, which can be
very long as in IDCW_ON_DRAWER_DRAWER_HIT are replaced by a shorter
name. That name consists of an abbrevation for the counter set
this counter belongs to and the counter number in that set.
The abbrevations are:
B --> Basic counter set
P --> Problem state counter set
C --> Crypto counter set
E --> Extended counter set
M --> MT_Diagnostic counter set
U --> Undefined counter.
Display E165 for counter name IDCW_ON_DRAWER_DRAWER_HIT
which is counter number 165 from the extended counter set.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Move defines from a source file lscpumf.c to the header
file in the include directory to be reused in a follow on
patch. Also reshuffle the list of defines in proper
sequence of increasing numbers.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Fix whatis-entries in the lshwc, lspai, and pai manpages by inlining the
tool name. The following command line can be used to test a local
manpage:
$ lexgrog -w -m cpumf/man/lshwc.8
cpumf/man/lshwc.8: "lshwc - extract CPU Measurement Facilities counter sets"
Github-ID: https://github.com/ibm-s390-linux/s390-tools/issues/170
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
When pai records data it may fail in select() system call.
This error is not reported and the pai program exits with success.
Change this and exit with proper exit code.
Fixes: d7b1cbad8b ("cpumf/pai: Add Processor Activity Instrumentation tool")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Running command pai -r <file> might encounter entries of type
PERF_RECORD_SWITCH. Entries of that type are currently not handled
and cause an error message:
# pai -r ~/paicrypto.000
unknown header-type 14 unknown header-type 14 \
unknown header-type 14 ...
The error message is not terminated by a newline.
Handle entries of this type PERF_RECORD_SWITCH. These records
do not carry any payload at all, just a bit is set in the
header::misc member. This bit set determines context switch out.
Output after:
# ./pai -r ~/paicrypto.000
0x4b814018f4f3 6 cs-out
0x4b817bc3c936 6 cs-in
0x4b817bc5246c 6 cs-out
0x4b817bd90e9a 6 cs-in
....
#
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Pai collects data from per CPU ring buffers and stores them in the
memory mapped output file. When data is collected from many CPUs at
the same time, writing data to output file can be slow.
Improve this and allow the pai recording to run with higher
real time priority. This is the same approach as done by the perf tool.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add lspai program and man page to display Processor Activity
Information (PAI) facility counter sets in the same way as
lscpumf for the CPU Measurement Facility counter sets.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Replace memory allocation functions malloc() and calloc()
by counterparts provided in libutil.a library. Also remove error
handling when no memory could be allocated as those functions do
not return.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Linux kernel commits
a029a4eab39 ("s390/cpumf: Allow concurrent access for CPU Measurement Counter Facility")
453380318ed ("s390/cpumf: Allow multiple processes to access /dev/hwc")
removed this restriction. Both commits are included in linux 5.16.
Therefore remove the restriction in the man page.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
lshwc displays counter values extracted from the /dev/hwctr device.
The counter values are retrieved using a loop and incremented instead
of just displayed on output.
This leads to the situation where a counter is incremented in large
numbers while the counter triggering program is active. When this
program terminates, the counter should not increase but remain steady.
This is not the case as this example shows:
# lshwc -i 2 -l90 -a :a | fgrep Total | \
awk -F ',' '{print $2 "," $152 "," $157}'
Report time and values of DFLT_ACCESS and DFLT_CC counters
sleep 10
# DFLTCC=1 python3 -c "import zlib; zlib.compress(b'A'*6000000000);
Increment counter DFLT_ACCESS and DFLT_CC, runs about 45 seconds
Output before:
15:24:20,0,0
15:24:22,0,0
15:24:24,0,0
15:24:26,0,0
15:24:28,0,0
15:24:30,0,0
15:24:32,2067027,592996481 <--- python3 program start
15:24:34,6953101,2039012580
15:24:36,14494585,4264340060
15:24:38,24759568,7346128396
....
15:24:56,232929262,70526362509
15:24:58,269037963,81464038124
15:25:00,307757248,93171809261
15:25:02,348900718,105620730492
15:25:04,392646075,118841031269 <--- python3 program stop
15:25:06,438956016,132853968029 <-- Value column two increased by 49148217
15:25:08,488104233,147709011658 <-- Similar high increase
The reason is that lshwc takes the last read counter value and keeps
adding it to the total count, even if there zero delta to the previous
read value. Fix this by clearing the total count value between ioctl()
read operations.
Output after:
15:39:26,0,0
15:39:28,0,0
15:39:30,0,0
15:39:32,0,0
15:39:34,0,0
15:39:36,1367250,380721127 <--- python3 program start
15:39:38,4104791,1108181711
15:39:40,6740994,1858843730
15:39:42,9277545,2646351211
15:39:44,11895180,3444375804
....
15:40:16,54060774,16315924126
15:40:18,57830349,17513712210
15:40:20,61628170,18690791102
15:40:22,65410470,19864086006
15:40:24,69070765,21028646754 <--- python3 program stop
15:40:26,70429758,21472772340 <-- No increase of value in column two
15:40:28,70429758,21472772340
15:40:30,70429758,21472772340
15:40:32,70429758,21472772340
15:40:34,70429758,21472772340
15:40:36,70429758,21472772340
15:40:38,70429758,21472772340
Fixes: 27a562da0a ("cpumf/lshwc: Program to extract complete counter sets")
Reported-by: Axel Busch <Axel.Busch@ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
pai dumps core when invoked with summary flag -S and
certain painnpa.XXXXX data files:
# ~/s390-tools/cpumf/pai -r -S
Segmentation fault (core dumped)
#
Fix this and check the existence of event raw data before
reading it. This avoid accessing NULL pointer trying to read raw
data when no counter values have been appended as raw data by the
kernel.
Fixes: dc73c77d73 ("cpumf/pai: Omit detailed event information when summary flag selected")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Use PMU named pai_ext to extract the PMU type number required
for the installation of PAI NNPA counters using perf_event_open()
system call.
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
When the summary flag is selected do not print detailed information
on every single event. Just print the summary information on each
non-zero counter value.
Suggested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
