github/kubernetes
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
114,612 Commits 1 Branch 31 Tags 1,019 MiB
01b96e7704
Commit Graph

255 Commits

Author SHA1 Message Date
Wei Huang
c9bc2f98d0
fix: remove SchedulingMigratedInTreePVs feature gate in sched perf test 2023-03-08 08:34:44 -08:00
Patrick Ohly
cc4bcd1d8e scheduler_perf: report data items as benchmark results 
This replaces the pretty useless us/op metric (useless because it includes
setup and teardown times) with the same values that also get stored in the JSON
file.

The main advantage is that benchstat can be used to analyze and compare
results.
 2023-02-28 23:08:23 +01:00
Patrick Ohly
961129c5f1 scheduler_perf: add logging flags 
This enables testing of different real production configurations (JSON
vs. text, different log levels, contextual logging).
 2023-02-28 23:08:17 +01:00
Kante Yin
3d0894fabf
Fix failure(context canceled) in scheduler_perf benchmark (#114843) 
* Fix failure in scheduler_perf benchmark

Signed-off-by: Kante Yin <kerthcet@gmail.com>

* Fatal when error in cleaning up nodes in scheduler perf tests

Signed-off-by: Kante Yin <kerthcet@gmail.com>

* Use derived context to better organize the codes

Signed-off-by: Kante Yin <kerthcet@gmail.com>

* Change log level to 2 in scheduler perf-test

Signed-off-by: Kante Yin <kerthcet@gmail.com>

---------

Signed-off-by: Kante Yin <kerthcet@gmail.com>
 2023-01-30 16:21:00 -08:00
Kensei Nakada
e8092cc885 cleanup(scheduler_perf): remove all removed feature gates 2023-01-04 01:07:47 +00:00
Patrick Ohly
2f6c4f5eab e2e: use Ginkgo context 
All code must use the context from Ginkgo when doing API calls or polling for a
change, otherwise the code would not return immediately when the test gets
aborted.
 2022-12-16 20:14:04 +01:00
Mark Rossetti
534f052a8d
Updating pause image refernces to 3.9 
Signed-off-by: Mark Rossetti <marosset@microsoft.com>
 2022-11-14 10:24:54 -08:00
kerthcet
d6ffb47832 Replace klog with benchmark log in scheduler_perf 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-09 09:11:55 +08:00
Kubernetes Prow Robot
73f6b96f0a
Merge pull request #113615 from kerthcet/feat/add-benchmark-tests 
Add nodeInclusionPolicy benchmark tests to scheduler_perf
 2022-11-07 09:18:28 -08:00
kerthcet
bc15aca26d Refactor SchedulerConfigFile 
Rename to SchedulerConfigPath and make it a pointer
to be consist with other fields

Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-05 00:30:34 +08:00
kerthcet
48f2c9ec20 Add benchmark tests for nodeInclusionPolicy 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-05 00:13:43 +08:00
kerthcet
cfc53ee524 Refactor code and annotations for readability 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-01 17:44:45 +08:00
kerthcet
21e8a69a22 Use operationCode instead of string directly 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-01 17:01:22 +08:00
Patrick Ohly
41619ace15 stop using deprecated klog flags 
Some scripts and tools still relied on the deprecated flags, the ones
which are about to be removed.

This is intentionally not a complete removal of all those flags in the entire
repo. This would lead to much more code churn also in places where commands
still accept the flags because they use klog directly.
 2022-09-04 21:02:43 +02:00
Davanum Srinivas
a9593d634c
Generate and format files 
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
 2022-07-26 13:14:05 -04:00
Wojciech Tyczyński
5b042f0bf4 Remove RunAnAPIServer from integration tests 2022-07-25 17:52:31 +02:00
Mark Rossetti
40f3e624a6 Switching everything to use pause:3.8 
Signed-off-by: Mark Rossetti <marosset@microsoft.com>
 2022-07-21 14:53:15 -07:00
Kensei Nakada
b0d47cb380
scheduler_perf: allow users to specify default pod and node specs (#101799) 
* scheduler_perf: default pod and node spec

* Fix: un-support DefaultNodeTemplatePath
 2022-06-29 11:44:07 -07:00
Davanum Srinivas
50bea1dad8
Move from k8s.gcr.io to registry.k8s.io 
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
 2022-05-31 10:16:53 -04:00
Kubernetes Prow Robot
570f1092f4
Merge pull request #109542 from sanposhiho/fix-test-case-scheduler-perf 
scheduler_perf: Remove test cases for Preemption which always fail
 2022-05-07 03:33:29 -07:00
Kubernetes Prow Robot
71df3e819b
Merge pull request #109545 from sanposhiho/fix-nun-on-scheduler_perf 
Skip adding data to avoid "json: unsupported value: NaN" panic when data is NaN
 2022-05-05 11:53:45 -07:00
Kensei Nakada
4af3c5efeb Skip adding data to avoid "json: unsupported value: NaN" panic when data is NaN 2022-05-05 16:11:22 +00:00
Kubernetes Prow Robot
3bef1692ef
Merge pull request #109696 from Huang-Wei/rm-sched-perf-legacy 
Cleanup legacy scheduler perf tests
 2022-05-04 02:35:43 -07:00
Kubernetes Prow Robot
629706e0fe
Merge pull request #109546 from sanposhiho/replace-metrics 
Replace scheduler_e2e_scheduling_duration_seconds with scheduler_scheduling_attempt_duration_seconds in scheduler_perf
 2022-05-04 01:29:22 -07:00
sanposhiho
1c2c20e6bd Change test cases for Preemption to create fewer Pods 2022-05-04 07:47:46 +00:00
Kubernetes Prow Robot
f0cd3725d3
Merge pull request #101835 from sanposhiho/scheduler_perf/feature/op-sleep 
scheduler_perf: create sleep operation
 2022-05-03 17:17:11 -07:00
Wei Huang
846ebf7814
Cleanup legacy scheduler perf tests 2022-04-27 09:57:17 -07:00
sanposhiho
b7b94b6b39 scheduler_perf: create sleep operation 2022-04-25 23:02:09 +00:00
sanposhiho
6e0da69632 Replace scheduler_e2e_scheduling_duration_seconds with scheduler_scheduling_attempt_duration_seconds in scheduler_perf 2022-04-20 00:48:12 +09:00
Davanum Srinivas
f7ad09c447
Switch to pause 3.7 
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
 2022-03-29 15:36:38 -04:00
Kubernetes Prow Robot
546e4fa1ef
Merge pull request #107771 from sanposhiho/fix-tiny 
make scheduler_perf stable
 2022-03-04 17:22:52 -08:00
sanposhiho
4c3a1000c7 fix by gofmt 2022-02-25 00:23:01 +09:00
sanposhiho
1080c2d717 Make scheduler_perf stable 2022-02-24 01:29:38 +09:00
Kubernetes Prow Robot
21c0f6f6ff
Merge pull request #107677 from pohly/scheduler-integration-benchmark 
scheduler integration benchmark improvements
 2022-02-14 01:23:28 -08:00
Patrick Ohly
e1e84c8e5f scheduler_perf: run with -v=0 by default 
This provides a mechanism for overriding the forced increase of the klog
verbosity to 4 when starting the apiserver and uses that for the scheduler_perf
benchmark. Other tests run as before.

A global variable was used because adding an explicit parameter to several
helper functions would have caused a lot of code churn (test ->
integration/util.StartApiserver ->
integration/framework.RunAnAPIServerUsingServer ->
integration/framework.startAPIServerOrDie).
 2022-02-11 16:58:33 +01:00
Patrick Ohly
c62d7407c8 scheduler_perf: dump test data when writing it failed 
Occasionally, writing as JSON failed because a NaN float couldn't be
encoded. The extended log message helps understand where that comes from, for
example:

F0120 20:24:45.515745  511835 scheduler_perf_test.go:540] BenchmarkPerfScheduling: unable to write measured data {Version:v1 DataItems:[{Data:map[Average:35.714285714285715 Perc50:2 Perc90:36 Perc95:412 Perc99:412] Unit:pods/s Labels:map[Metric:SchedulingThroughput Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2]} {Data:map[Average:27.863967530999993 Perc50:13.925925925925926 Perc90:30.06711409395973 Perc95:31.85682326621924 Perc99:704] Unit:ms Labels:map[Metric:scheduler_e2e_scheduling_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2]} {Data:map[Average:11915.651577744 Perc50:15168.796680497926 Perc90:19417.759336099585 Perc95:19948.87966804979 Perc99:20373.77593360996] Unit:ms Labels:map[Metric:scheduler_pod_scheduling_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2]} {Data:map[Average:1.1865832049999983 Perc50:0.7636363636363637 Perc90:2.891903719912473 Perc95:3.066958424507659 Perc99:5.333333333333334] Unit:ms Labels:map[Metric:scheduler_framework_extension_point_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2 extension_point:Filter]} {Data:map[Average:NaN Perc50:NaN Perc90:NaN Perc95:NaN Perc99:NaN] Unit:ms Labels:map[Metric:scheduler_framework_extension_point_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2 extension_point:Score]}]}: json: unsupported value: NaN
 2022-02-07 08:59:19 +01:00
Patrick Ohly
8d44b819b3 scheduler_perf: avoid ambiguous test names 
"-bench=PerfScheduling/Preemption/500Nodes" ran both the
PerfScheduling/Preemption/500Nodes and the
PerfScheduling/PreemptionPVs/500Nodes benchmark.

This can be avoided by choosing names where none is the prefix of another.
 2022-02-07 08:59:19 +01:00
Patrick Ohly
259a8ad0b7 test: allow controlling etcd log level 
When running an integration test that measures performance, like for example
test/integration/scheduler_perf, running etcd with debug level output is
undesirable because it creates additional load on the system and isn't
realistic.

The default is still "debug", but ETCD_LOGLEVEL=warn can be used to override
that.
 2022-02-07 08:59:19 +01:00
ahrtr
fe95aa614c io/ioutil has already been deprecated in golang 1.16, so replace all ioutil with io and os 2022-02-03 05:32:12 +08:00
sanposhiho
d8840405e2 Create namespace for Pod not to occur error log of namespace not-found 2022-01-26 00:39:12 +09:00
Kubernetes Prow Robot
ca4af7a981
Merge pull request #104716 from sanposhiho/feature/scheduler_perf/unused-template-params 
test/integration/scheduler_perf: check for unused template parameters
 2022-01-10 16:21:16 -08:00
Davanum Srinivas
9405e9b55e
Check in OWNERS modified by update-yamlfmt.sh 
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
 2021-12-09 21:31:26 -05:00
Hanna Lee
07a883d8e6 Remove //lint:ignore pragmas that aren't being used anymore 2021-11-17 08:56:54 +01:00
Hanna Lee
c8fde197f5 Add more //nolint:staticcheck for failures caught in PR tests 2021-11-17 08:56:02 +01:00
kerthcet
75a255d2ed remove scheduler component config v1beta1 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2021-09-28 13:13:17 +08:00
Kubernetes Prow Robot
86d23cf441
Merge pull request #105206 from pohly/test-integration-help 
test/integration: skip etcd startup for -help flag
 2021-09-24 10:29:23 -07:00
Patrick Ohly
81b4a695b3 test/integration: skip etcd startup for -help flag 
By parsing flags in the test's main function before starting etcd we bail out
early without ever starting etcd when the test was invoked with -help.

Otherwise etcd must be available, gets started and then hangs because
flag.Parse itself exits when called by testing.go. This bypasses the code in
EtcdMain which normally stops etcd.
 2021-09-24 11:51:58 +02:00
Kubernetes Prow Robot
857d4c107c
Merge pull request #104808 from chendave/indent 
Format json file with proper indentation
 2021-09-21 19:14:00 -07:00
sanposhiho
1318f74609 Fix: use Fatalf and list all unused params in one error 2021-09-09 07:34:30 +09:00
sanposhiho
6bf6e424a1 Fix: rename getParams→get 2021-09-09 07:11:36 +09:00