zpcictl: Add option to trigger firmware reset

With "zpcictl --reset DDDD:BB:FF.F" now causing a fully Linux driven
reset where the Linux kernel does an explicit device driver unbind,
disable and re-enable, let's also expose a way to instead have firmware
perform a device reset by issuing an SCLP with SCLP_ERRNOTIFY_RESET.

When firmware is done resetting the device it will then issue an error
notification with PCI Error Code 0x3a indicating successful reset, which
will subsequently cause the new kernel based automatic recovery
mechanism to perform recovery in coordination with the device driver.
This allows resetting devices without unbinding them from their device
driver and thus without losing related block devices or network
interfaces. This may also be used to test the automatic recovery
mechanism.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>

zpcictl/zpcictl.8

@@ -1,4 +1,4 @@
-.\" Copyright IBM Corp. 2018
+.\" Copyright IBM Corp. 2022
 .\" s390-tools is free software; you can redistribute it and/or modify
 .\" it under the terms of the MIT license. See LICENSE for details.
 .\"
@@ -17,7 +17,7 @@
 .  PD
 ..
 .
-.TH zpcictl 8 "Oct 2018" s390-tools zpcictl
+.TH zpcictl 8 "Mar 2022" s390-tools zpcictl
 .
 .SH NAME
 zpcictl - Manage PCI devices on IBM Z
@@ -52,7 +52,45 @@ device (e.g.
 .SH OPTIONS
 .SS Error Handling Options
 .OD reset "" "DEVICE"
-Reset and re-initialize the PCI device.
+Reset and re-initialize the PCI device and report a device error to the Support
+Element (SE). The reset consists of a controlled shutdown and a subsequent
+re-enabling of the device from the shut off state. This process destroys and
+then re-creates higher level interfaces such as network interfaces and block
+devices. This reset is disruptive and often requires manual intervention on
+multiple layers. In particular, network interfaces that are part of a bonded
+interface must be re-added to the bond after the reset. Similarly, block
+devices backed by an NVMe that are part of a software RAID must be re-synced by
+re-adding to the RAID after resetting the NVMe.
+
+Use this reset option only if the less disruptive automatic recovery mechanism
+is not supported by your kernel or it failed to restore the device's
+functionality. Unsuccessful automatic recovery can result in kernel messages
+indicating required manual intervention. If the device is malfunctioning
+without automatic recovery being triggered, consider using the \fB--reset-fw\fR
+option to trigger a less disruptive automatic recovery through
+a firmware-driven reset.
+.PP
+.
+.OD reset-fw "" "DEVICE"
+Reset the PCI device using a firmware-driven reset that also reports a device
+error on the Support Element (SE). If supported by your kernel, automatic recovery
+re-initializes the device after the firmware reports a successful device reset.
+
+Use this option if the device is malfunctioning and automatic recovery is
+supported by the kernel but was not triggered. This condition can occur if the
+error is not detected by the low level PCI interfaces. A successful automatic
+recovery after the firmware-driven reset, is less disruptive than the full
+reset that is performed by the \fB--reset\fR option. Other than the full reset,
+the automatic recovery does not completely shut down the device and re-create
+it from the shut down state. Instead, it works with the device driver to
+restore the device in place. Thus, higher level interfaces such as network
+interfaces and block devices remain intact. In particular, with this type of
+reset high availability mechanisms like a bonded network interface or
+a software RAID can transparently re-integrate the recovered device. For
+example, after a failure and recovery, a software RAID can resync a stroage
+device or a network interface can be re-integrated in a bond. In contrast to
+a complete shut down, the device driver remains active and informs higher
+layers of both the occurence of an error state and the eventual recovery.
 .PP
 .
 .OD deconfigure "" "DEVICE"

zpcictl/zpcictl.c

@@ -46,12 +46,27 @@ static const struct util_prg prg = {
 #define OPT_RESET	128
 #define OPT_DECONF	129
 #define OPT_REPORT_ERR	130
+#define OPT_RESET_FW	131
 
 static struct util_opt opt_vec[] = {
 	UTIL_OPT_SECTION("ERROR HANDLING OPTIONS"),
 	{
 		.option = { "reset", no_argument, NULL, OPT_RESET },
-		.desc = "Reset the device",
+		.desc = "Reset the device and report an error to the Support Element (SE). "
+			"The reset consists of a controlled shutdown and a subsequent "
+			"re-enabling of the device. As a result, higher level interfaces such "
+			"as network interfaces and block devices are destroyed and re-created.\n"
+			"Manual configuration steps might be required to re-integrate the device, "
+			"for example, in bonded interfaces or software RAIDs.\n"
+			"Use this option only if the automatic recovery failed, or if it did "
+			"not succeed to restore regular operations of the device and manual "
+			"intervention is required.\n",
+		.flags = UTIL_OPT_FLAG_NOSHORT,
+	},
+	{
+		.option = { "reset-fw", no_argument, NULL, OPT_RESET_FW },
+		.desc = "Reset the device through a firmware driven reset that triggers "
+			"automatic recovery and reports an error to the Support Element (SE).\n",
 		.flags = UTIL_OPT_FLAG_NOSHORT,
 	},
 	{
@@ -341,6 +356,15 @@ static void sclp_reset_device(struct zpci_device *pdev)
 	sysfs_write_value(pdev, "recover", 1);
 }
 
+/*
+ * Reset the PCI device via firmware and let auto recovery handle
+ * re-initialization
+ */
+static void sclp_reset_device_fw(struct zpci_device *pdev)
+{
+	sclp_issue_action(pdev, SCLP_ERRNOTIFY_AQ_RESET);
+}
+
 /*
  * De-Configure/repair PCI device. Moves the device from configured
  * to reserved state.
@@ -372,6 +396,9 @@ static void parse_cmdline(int argc, char *argv[], struct options *opts)
 		case OPT_RESET:
 			opts->reset = 1;
 			break;
+		case OPT_RESET_FW:
+			opts->reset_fw = 1;
+			break;
 		case OPT_DECONF:
 			opts->deconfigure = 1;
 			break;
@@ -411,6 +438,8 @@ int main(int argc, char *argv[])
 
 	if (opts.reset)
 		sclp_reset_device(&pdev);
+	if (opts.reset_fw)
+		sclp_reset_device_fw(&pdev);
 	else if (opts.deconfigure)
 		sclp_deconfigure(&pdev);
 	else if (opts.report)

zpcictl/zpcictl.h

@@ -23,6 +23,7 @@
 
 struct options {
 	unsigned int reset;
+	unsigned int reset_fw;
 	unsigned int deconfigure;
 	unsigned int report;
 };