The function copy_table_entry_write() maps/unmaps /proc/vmcore for each
64kB of the file in order to write its content to a dump partition.
This requires a page allocation of order 2 in kernel (crst_table_alloc)
each time a mmap() syscall is performed and which cannot always be
fulfilled due to external memory fragmentation. And this in turn
results in a OOM kernel panic while writing /proc/vmcore content to
a dump partition. To make zfcpdump more robust in the face of such
problems with page allocations of order > 0, we use a simple file
read/write loop to transfer the content of /proc/vmcore to a dump
partition. This solution is less efficient (by not much) than the old
one with mmap() because it requires additional user space copy but
we trade off here efficiency for robustness which is more important
for zfcpdump.
We use a statically allocated buffer to read data into from /proc/vmcore
and write to a dump partition to avoid repeated use of the memory
mapping kernel path which would be required for a buffer allocated
dynamically at each entry of copy_table_entry_write().
Another possible and very convenient solution would be to use sendfile()
which would avoid any user space copies but, unfortunately, sendfile()
doesn't work with large /proc files.
Example of failed mmap() call for /proc/vmcore
==============================================
[ 26.568654] init invoked oom-killer: gfp_mask=0x40cc0(GFP_KERNEL|__GFP_COMP), order=2, oom_score_adj=0
[ 26.568665] CPU: 0 PID: 1 Comm: init Not tainted 6.9.0-20240504.rc6.git0.9986ea583f39.300.fc39.s390x+zfcpdump #1
[ 26.568668] Hardware name: IBM 3931 A01 704 (LPAR)
[ 26.568669] Call Trace:
[ 26.568671] [<000003ffe03277c6>] dump_stack_lvl+0x76/0x98
[ 26.568676] [<000003ffe00f15c8>] dump_header+0x58/0x2d0
[ 26.568680] [<000003ffe00f219a>] out_of_memory+0x252/0x348
[ 26.568683] [<000003ffe0132f3e>] __alloc_pages_slowpath.constprop.0+0x65e/0x7c0
[ 26.568686] [<000003ffe01331a4>] __alloc_pages+0x104/0x128
[ 26.568688] [<000003ffe001ab46>] crst_table_alloc+0x2e/0xa0
[ 26.568693] [<000003ffe01164c4>] __pmd_alloc+0x24/0x160
[ 26.568696] [<000003ffe0116650>] pmd_alloc+0x50/0x68
[ 26.568698] [<000003ffe0117be4>] remap_pfn_range_notrack+0x194/0x310
[ 26.568701] [<000003ffe0117d7c>] remap_pfn_range+0x1c/0x28
[ 26.568703] [<000003ffe0015442>] remap_oldmem_pfn_range+0xb2/0x108
[ 26.568705] [<000003ffe01a0544>] mmap_vmcore+0x414/0x448
[ 26.568707] [<000003ffe01937c4>] proc_reg_mmap+0x84/0xa0
[ 26.568709] [<000003ffe011f264>] mmap_region+0x22c/0x6f8
[ 26.568711] [<000003ffe011fae8>] do_mmap+0x3b8/0x410
[ 26.568713] [<000003ffe01015cc>] vm_mmap_pgoff+0x9c/0xe8
[ 26.568715] [<000003ffe011d006>] ksys_mmap_pgoff+0x8e/0xb8
[ 26.568716] [<000003ffe011d0f4>] __s390x_sys_old_mmap+0x74/0x90
[ 26.568718] [<000003ffe0341a2a>] __do_syscall+0x1aa/0x220
[ 26.568720] [<000003ffe03478b0>] system_call+0x70/0x98
[ 26.568724] Mem-Info:
[ 26.568725] active_anon:1 inactive_anon:12 isolated_anon:0
active_file:83590 inactive_file:93939 isolated_file:0
unevictable:201 dirty:17845 writeback:4
slab_reclaimable:4259 slab_unreclaimable:1482
mapped:174 shmem:0 pagetables:810
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:2694 free_pcp:105 free_cma:0
[ 26.568729] Node 0 active_anon:4kB inactive_anon:48kB active_file:334360kB inactive_file:375756kB unevictable:804kB isolated(anon):0kB isolated(file):0kB mapped:696kB dirty:71380kB writeback:16kB shmem:0kB writeback_tmp:0kB kernel_stack:544kB pagetables:3240kB sec_pagetables:0kB all_unreclaimable? no
[ 26.568732] DMA free:10776kB boost:0kB min:3496kB low:4368kB high:5240kB reserved_highatomic:0KB active_anon:4kB inactive_anon:48kB active_file:334360kB inactive_file:375756kB unevictable:804kB writepending:71360kB present:786428kB managed:764664kB mlocked:0kB bounce:0kB free_pcp:420kB local_pcp:420kB free_cma:0kB
[ 26.568735] lowmem_reserve[]: 0 0 0
[ 26.568738] DMA: 1069*4kB (UH) 757*8kB (UH) 11*16kB (H) 6*32kB (H) 1*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 10764kB
[ 26.568747] 177773 total pagecache pages
[ 26.568748] 196607 pages RAM
[ 26.568748] 0 pages HighMem/MovableOnly
[ 26.568749] 5441 pages reserved
[ 26.568749] Tasks state (memory values in pages):
[ 26.568750] [ pid ] uid tgid total_vm rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name
[ 26.568751] Out of memory and no killable processes...
[ 26.568752] Kernel panic - not syncing: System is deadlocked on memory
[ 26.568755] CPU: 0 PID: 1 Comm: init Not tainted 6.9.0-20240504.rc6.git0.9986ea583f39.300.fc39.s390x+zfcpdump #1
[ 26.568758] Hardware name: IBM 3931 A01 704 (LPAR)
[ 26.568759] Call Trace:
[ 26.568760] [<000003ffe03277c6>] dump_stack_lvl+0x76/0x98
[ 26.568763] [<000003ffe001f14e>] panic+0x10e/0x2e8
[ 26.568765] [<000003ffe00f21c8>] out_of_memory+0x280/0x348
[ 26.568768] [<000003ffe0132f3e>] __alloc_pages_slowpath.constprop.0+0x65e/0x7c0
[ 26.568771] [<000003ffe01331a4>] __alloc_pages+0x104/0x128
[ 26.568773] [<000003ffe001ab46>] crst_table_alloc+0x2e/0xa0
[ 26.568776] [<000003ffe01164c4>] __pmd_alloc+0x24/0x160
[ 26.568779] [<000003ffe0116650>] pmd_alloc+0x50/0x68
[ 26.568781] [<000003ffe0117be4>] remap_pfn_range_notrack+0x194/0x310
[ 26.568784] [<000003ffe0117d7c>] remap_pfn_range+0x1c/0x28
[ 26.568787] [<000003ffe0015442>] remap_oldmem_pfn_range+0xb2/0x108
[ 26.568789] [<000003ffe01a0544>] mmap_vmcore+0x414/0x448
[ 26.568791] [<000003ffe01937c4>] proc_reg_mmap+0x84/0xa0
[ 26.568793] [<000003ffe011f264>] mmap_region+0x22c/0x6f8
[ 26.568795] [<000003ffe011fae8>] do_mmap+0x3b8/0x410
[ 26.568797] [<000003ffe01015cc>] vm_mmap_pgoff+0x9c/0xe8
[ 26.568799] [<000003ffe011d006>] ksys_mmap_pgoff+0x8e/0xb8
[ 26.568801] [<000003ffe011d0f4>] __s390x_sys_old_mmap+0x74/0x90
[ 26.568803] [<000003ffe0341a2a>] __do_syscall+0x1aa/0x220
[ 26.568805] [<000003ffe03478b0>] system_call+0x70/0x98
Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reviewed-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
For vr-kernel dumps use the offsets stored in os_info entries for
virt to phys address conversion when dump virtual address is to be
read, (e.g. vmcoreinfo symbols) using similar method as implemented
in crash-utility.
It is mainly required for reading "init_uts_ns" symbol and, in case
of crashed kdump, "lowcore_ptr" symbol along with a pointers to the
lowcore of every CPU.
Acked-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Support vmcore based dumps (ngdump, zfcpdump) with uncoupled physical and
virtual addresses. For that we just remove p_paddr != p_vaddr sanity check
for elf load headers.
Acked-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Output generated using libutil's util_fmt_*() functions contains
meta-data such as hostname and time of invocation. When writing test
cases that compare expected and actual tool output, this run-time
variable information needs to be filtered out, resulting in increased
complexity.
To address this, add a global option to suppress meta-data output via
environment variable FMT_NOMETA. If set to '1', no meta-data will be
generated.
Example:
$ FMT_NOMETA=1 chpstat --format json
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
The previous implementation did stop after the first download was
successful, even if it did not contain a CRL. This behavior renders a
second, third, ... link as backup location useless as the code ignores
them if the URI e.g. contains a error message. That results in not
having a CRL and probably a failed certificate verification.
Fix this by trying again if the download was successful but did not
contain a CRL
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
In order to allow loop devices usage for DASD dump emulation in fvt-tests,
do not bail out on ZG_TYPE_DASD device type returned by zg_type() for
s390_ext dump.
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
There are still a few scripts using the 'which' command to determine
either the full path or the mere existence of an executable.
Some of these scripts might run in minimal environments where 'which' is
not available due to dependency restriction. 'which' is also considered
unreliable for historical implementation details.
Use the POSIX defined [1] built-in 'command -v' instead to reduce
package dependencies and improve reliability.
[1] https://pubs.opengroup.org/onlinepubs/9699919799/
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
The new parameter 'SCP_DATA' can be used to pass additional kernel cmdline
parameters to a stand-alone SCSI/NVMe/ECKD dumper for debugging purposes.
Configuration example
=====================
ON_PANIC=dump_reipl
DUMP_TYPE=fcp
DEVICE=0.0.4711
WWPN=0x5005076303004711
LUN=0x4711000000000000
BOOTPROG=0
BR_LBA=0
SCP_DATA="dump_debug=6 ignore_loglevel earlyprintk=sclp panic_print=3"
Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reviewed-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
When dump is copied to the filesystem in s390 format, follow on
'zgetdump -i' can fail with ioctl error:
# zgetdump /dev/dasdb1 -f s390 dump.s390
Format Info:
Source: s390_ext
Target: s390
Copying dump:
00000001 / 00008192 MB
00003688 / 00008192 MB
00006646 / 00008192 MB
00008192 / 00008192 MB
Success: Dump has been copied
# zgetdump -iVVVV dump.s390
zgetdump: Operation "BLKSSZGET" failed on "dump.s390" (Inappropriate ioctl for device)
Call ioctl(BLKSSZGET) only for s390_ext dump format (dump can be
stored on DASD partition only, not on the filesystem). For s390 format
a blocksize is not required for dump processing since s390 dump data is
not compressed.
Fixes: 271b809495 ("zdump/dfi_s390: Support reading compressed s390_ext dumps")
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
pvattest-C had a -a option for --arpk which the Rust implementation
missed.
Add this short option.
Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
The commit fb0b6263d1 ("zipl/src: Fix problems when target parameters
are specified by user") breaks the case where the user has to provide
all target attributes via the zipl command line, because the target
device cannot be used to retrieve those attributes.
$ zipl -V --blsdir /tmp/tmp.xHmFUdgBCi/boot//loader/entries/ --config /tmp/tmp.dSv9MJ3svs
Looking for components in '/lib/s390-tools'
Using config file '/tmp/tmp.dSv9MJ3svs' (from command line)
Using BLS config file '/tmp/tmp.xHmFUdgBCi/boot//loader/entries//50_normal.conf'
Using BLS config file '/tmp/tmp.xHmFUdgBCi/boot//loader/entries//45_normal_swiotlb.conf'
Using BLS config file '/tmp/tmp.xHmFUdgBCi/boot//loader/entries//40_pv.conf'
Using BLS config file '/tmp/tmp.xHmFUdgBCi/boot//loader/entries//30_pv_reboot.conf'
Secure boot support: yes
Target device information
Device..........................: 2b:00
Device name.....................: nbd0 *)
Device driver name..............: nbd
Type............................: disk device
Disk layout.....................: SCSI disk layout *)
Geometry - start................: 2048 *)
File system block size..........: 4096
Physical block size.............: 512 *)
Device size in physical blocks..: 2095071
*) Data provided by user.
Building bootmap in '/tmp/tmp.xHmFUdgBCi/boot/'
Building menu 'zipl-automatic-menu'
Adding #1: IPL section 'Normal Guest' (default)
initial ramdisk...: /tmp/tmp.xHmFUdgBCi/boot//ramdisk-s390x
kernel image......: /tmp/tmp.xHmFUdgBCi/boot//vmlinux-s390x
zIPL environment block content:
Error: Could not add image file '/tmp/tmp.xHmFUdgBCi/boot//vmlinux-s390x': Could not get disk geometry
This reverts commit fb0b6263d1.
Acked-by: Eduard Shishkin <edward6@linux.ibm.com>
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Having timestamps in gzip leads to lintian warning
'package-contains-timestamped-gzip' and will harm reproducible builds.
Gzip option '-n' avoids saving original file name and especially timestamps
by default.
Fixes: https://github.com/ibm-s390-linux/s390-tools/pull/169
Signed-off-by: Frank Heimes <frank.heimes@canonical.com>
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
[seiden@linux.ibm.com: Fix commit message]
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Do some formatting that are in experimental stage but improve the code
readability.
Use rustfmt with a nightly toolchain and enable:
format_code_in_doc_comments = true
reorder_impl_items = true
comment_width = 100
wrap_comments = true
normalize_comments = true
(see .rustfmt.toml)
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Remove all the code just pvattest-C used from libpv.
z(get)dump is the only user as of now.
Acked-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
This will reduce code dependency to otherwise unused code in libpv.
This code will be removed with the next patch.
Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add a CLI compatible Rust implementation of pvattest-C.
- All (non-experimental) options are supported and work exactly as in
the C implementation. For some options/parameters new variants are
available.
- `perform` now also accepts positional arguments, while keep accepting
-i and -o that was mandatory in the C implementation.
- `version` may also be a command instead of an option now.
- -V is deprecated
- -v increases verbosity instead of showing the version
- all experimental options are dropped
Acked-by: Qi Feng Huo <huoqif@cn.ibm.com>
Acked-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Renames pv crate to s390_pv and pv_core to s390_pv_core. pv was already
taken on crates.io.
Bump the versions of all crates to 0.10.0. From now on we follow Semver
compatibility rules when it comes to updates. patch-level updates will
not introduce any backwards incompatible changes. For now all crates in
this directory will have the same version number. A version update may,
therefore, not add any new things.
Library users in this repository still use the non prefixed names and
rename the crate in the Cargo.toml. Doc-tests have to use the new name
however.
Add some Cargo metadata to the Cargo.toml.
Acked-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Use `AsRef<Path>` instead of `&Path`, &str, .... to be more versatile
and accept more input types. In addition, use `PathBuf` and `Path` for
paths instead of `String` and `str`.
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Denies compiling if one of the following lints find something in pv or
pv_core:
missing_docs,
missing_debug_implementations,
trivial_numeric_casts,
unstable_features,
unused_import_braces,
unused_qualifications
Those lint force developers to avoid unnecessary code and providing
debuggability & documentation for each public symbol.
Fix the compile time error introduced with those lints.
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
crates.io does not like sub-crates in a crate. Unpack the
openssl-extensions sub-crate into a (private) module.
While at it, fix some styling issues.
Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add functionality for:
* sending attestation requests to the uvdevice and retrieve the
response
* create/read the attestation exchange format file format
Acked-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add a function to decrypt a block with aes_gcm.
Add functionality to perform HMAC operations.
Acked-by: Qi Feng Huo <huoqif@cn.ibm.com>
Acked-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add a thin wrapper around [u8] to be able to represent an u8-slice as a
hex-string for Display and Serialize.
Acked-by: Qi Feng Huo <huoqif@cn.ibm.com>
Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Remove !#[allow(unused)] in pv_core and fix all unused warnings from the
Rust compiler.
Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Prepare pv & pv_core crates to be released on crates.io:
* Remove any unused API to stay flexible
* Remove utils dependency
* Move cli, tmpfile and version utilities to local utils crate
* Use the new utilities in the pv tools
* Rename Secret into Confidential to avoid confusion of Secret (now
Confidential) and AddSecret requests.
* Move the uvsecret module out of the request module and change the name
to secret.
* Cleanup dependencies
* Precise and correct minimal dependency versions
* Inline `Aes256Key::from_digest`
The cleanup ensures that the code also compiles with the dependencies
resolved to their minimal versions using:
$ cargo +nightly -Z minimal-versions update
$ cargo build
For more information refer to this blog post:
https://users.rust-lang.org/t/psa-please-specify-precise-dependency-versions-in-cargo-toml/71277/8
Signed-off-by: Marc Hartmayer <mhartmay@de.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add subcommands in the man description. Add description for the help option.
Fix some minor wording issues.
Add the curve type in the --user-data option of create.
Acked-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add a new tool named chpstat that can be used to view channel-path
statistics such as utilization and I/O throughput, and to query and
control the status of the channel-path statistics function.
Note: Channel-path statistics are only available on systems running in
an LPAR or DPM partition.
When run without further options, data for all channel-paths is
displayed repeatedly with a 5 second delay in table format.
Example output:
CHANNEL-PATH UTILIZATION(%) READ(B/s) WRITE(B/s)
ID TYP CMG SHR SPEED PART TOTAL BUS PART TOTAL PART TOTAL
1d 25 2 1 - 7.16 7.50 7.50 129M 129M 0.00 161K
21 1b 2 1 32G 0.00 0.00 0.00 0.00 0.00 0.00 0.00
34 1b 2 1 32G 0.00 0.00 0.00 0.00 0.00 0.00 0.00
61 25 2 1 - 0.00 0.01 0.00 0.00 2.00K 0.00 307K
63 25 2 1 - 0.00 0.01 0.00 0.00 0.00 0.00 381K
bd 11 2 1 10G - - - 529.8 532.1 616.3 616.3
Reviewed-by: Vineeth Vijayan <vneethv@linux.ibm.com>
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add helper functions for converting structured key-value data into
different formats such as JSON, text pairs, and CSV.
Using these functions the resulting output format can be dynamically
configured at run-time without the need to duplicate output-generating
code for each format type. Also format-specific requirements such as
quoting, indentation, and comma-placement are automatically taken care
of.
Basic API calling sequence:
util_fmt_init() => Select output format
util_fmt_obj_start() => Start a new object or list
util_fmt_pair() => Emit a key-value pair
util_fmt_obj_end() => End the most recent object or list
util_fmt_exit() => Cleanup
Notes:
- Supported data elements are objects, lists and key-value pairs
(mappings)
- Scalars are only supported as part of a mapping
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add function util_concatf() that appends the result of a format string
expansion to the end of an existing string while taking care of the
required memory allocations.
Usage example:
char *str = NULL;
util_concatf(&str, "list:");
for (int i = 1; i <= 3; i++)
util_concatf(&str, "%spart%d", (i > 1 ? "," : ""), i);
printf("%s\n", str); /* list:part1,part2,part3 */
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
