JSON Lines text format (JSONL) [1] is a line-delimited JSON format where
objects are separated by the new line character (\n, LF) as opposed to
the JSON Sequence text format (json-seq) where JSON text is encapsulated
in an ASCII Record Separator (0x1E, RS) and ASCII Line Feed character
(0x0A, LF).
Whilst JSONL is also used for data streaming, this simpler format is
better suited for logging and works also well with traditional
line-oriented Unix tooling (e.g. grep or sed).
Add this format to util_fmt so that users have more choice and control
over formats that are required for their usecases.
Add helper functions that let the user determine whether a given format
type is JSON in general or a JSON streaming format (such as json-seq or
jsonl).
For better readability and more clarity use the helper function
util_fmt_is_json_stream() where the same decision is made for both
JSON streaming formats FMT_JSONSEQ and FMT_JSONL.
[1] https://jsonlines.org/
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Reviewed-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>
This patch modifies the default behavior of CSV header generation:
header cells are no longer enclosed in double quotes unless the
FMT_QUOTEALL flag is explicitly set. According to RFC 4180, quoting is
only required when a cell contains control characters, commas, or double
quotes.
The goal of this change is to produce cleaner and more readable CSV
output by default, and to avoid unnecessary quoting in header rows. It
also simplifies algorithms that rely on FMT_UTIL and improves
compatibility with downstream tools expecting unquoted headers.
Tools that expect unquoted headers include:
* SQL tools: Headers optional, usually unquoted
* R (read.csv): Uses header=TRUE, no quotes needed
* Pandas: Assumes headers, quoting not required
* Excel: Detects headers; quotes only for special chars
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Output generated using libutil's util_fmt_*() functions contains
meta-data such as hostname and time of invocation. When writing test
cases that compare expected and actual tool output, this run-time
variable information needs to be filtered out, resulting in increased
complexity.
To address this, add a global option to suppress meta-data output via
environment variable FMT_NOMETA. If set to '1', no meta-data will be
generated.
Example:
$ FMT_NOMETA=1 chpstat --format json
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Add helper functions for converting structured key-value data into
different formats such as JSON, text pairs, and CSV.
Using these functions the resulting output format can be dynamically
configured at run-time without the need to duplicate output-generating
code for each format type. Also format-specific requirements such as
quoting, indentation, and comma-placement are automatically taken care
of.
Basic API calling sequence:
util_fmt_init() => Select output format
util_fmt_obj_start() => Start a new object or list
util_fmt_pair() => Emit a key-value pair
util_fmt_obj_end() => End the most recent object or list
util_fmt_exit() => Cleanup
Notes:
- Supported data elements are objects, lists and key-value pairs
(mappings)
- Scalars are only supported as part of a mapping
Reviewed-by: Jan Höppner <hoeppner@linux.ibm.com>
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
