History

Patrick Ohly 02efe09abe component-base/logs: improve handling of re-applying a configuration Normal binaries should never have to do this. It's not safe when there are already some goroutines running which might do logging. Therefore the new default is to return an error when a binary accidentally re-applies. A few unit ensure that there are no goroutines and have to call the functions more then once. The new ResetForTest API gets used by those to enable changing the logging settings more than once in the same process. Integration tests use the same code as the normal binaries. To make reuse of that code safe, component-base/logs can be configured to silently ignore any additional calls. This addresses data races that were found when enabling -race for integration tests. To catch cases where the integration test does want to modify the config, the old and new config get compared and an error is raised when it's not the same. To avoid having to modify all integration tests which start test servers, reconfiguring component-base/logs is done by the test server packages.		2023-07-05 19:08:54 +02:00
..
contextual-logging	logs: add micro benchmarks for argument passing	2022-05-31 08:07:37 +02:00
data	test/integration/logs: use stable struct for unit test	2023-03-07 16:04:32 +01:00
.gitignore	test/integration/logs: update benchmark support	2023-03-07 16:03:48 +01:00
benchmark_test.go	chore: os.SEEK_END os.SEEK_SET and use b.Logf(...) instead of b.Log(fmt.Sprintf(...))	2023-06-06 15:23:19 +08:00
common_test.go	test/integration/logs: benchmark using logsapi	2023-03-07 16:04:32 +01:00
get-logs.sh	test/integration/logs: update benchmark support	2023-03-07 16:03:48 +01:00
load_test.go	component-base/logs: improve handling of re-applying a configuration	2023-07-05 19:08:54 +02:00
load.go	test/integration/logs: use stable struct for unit test	2023-03-07 16:04:32 +01:00
README.md	test/integration/logs: remove useless stats case	2023-03-07 16:04:32 +01:00

README.md

Benchmarking logging

Any major changes to the logging code, whether it is in Kubernetes or in klog, must be benchmarked before and after the change.

Running the benchmark

go test -v -bench=. -benchmem -benchtime=10s .

Real log data

The files under data define test cases for specific aspects of formatting. To test with a log file that represents output under some kind of real load, copy the log file into data/<file name>.log and run benchmarking as described above. -bench=BenchmarkLogging/<file name without .log suffix> can be used to benchmark just the new file.

When using data/v<some number>/<file name>.log, formatting will be done at that log level. Symlinks can be created to simulating writing of the same log data at different levels.

No such real data is included in the Kubernetes repo because of their size. They can be found in the "artifacts" of this https://testgrid.kubernetes.io/sig-instrumentation-tests#kind-json-logging-master Prow job:

artifacts/logs/kind-control-plane/containers
artifacts/logs/kind-*/kubelet.log

With sufficient credentials, gsutil can be used to download everything for a job directly into a directory that then will be used by the benchmarks automatically:

kubernetes$ test/integration/logs/benchmark/get-logs.sh
++ dirname test/integration/logs/benchmark/get-logs.sh
+ cd test/integration/logs/benchmark
++ latest_job
++ gsutil cat gs://kubernetes-jenkins/logs/ci-kubernetes-kind-e2e-json-logging/latest-build.txt
+ job=1618864842834186240
+ rm -rf ci-kubernetes-kind-e2e-json-logging
+ mkdir ci-kubernetes-kind-e2e-json-logging
...

This sets up the data directory so that additional test cases are available (BenchmarkEncoding/v3/kind-worker-kubelet/, BenchmarkEncoding/kube-scheduler/, etc.).

To clean up, use

git clean -fx test/integration/logs/benchmark

Analyzing log data

While loading a file, some statistics about it are collected. Those are shown when running with:

go test -v -bench=BenchmarkEncoding/none -run=none  .