github/kubernetes
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
115,481 Commits 1 Branch 31 Tags 1,019 MiB
76a94fd836
Commit Graph

264 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
aa026a6b30
Merge pull request #117202 from pohly/scheduler-perf-zero-count 
scheduler perf: allow creating 0 items
 2023-04-11 21:18:20 -07:00
Kubernetes Prow Robot
251d4a00e6
Merge pull request #117201 from pohly/scheduler-perf-node-labels 
test/integration: create nodes directly with kubernetes.io/hostname label
 2023-04-11 21:18:12 -07:00
Kubernetes Prow Robot
69b59b9d42
Merge pull request #117199 from pohly/scheduler-perf-race-fix 
scheduler_perf: fix race condition
 2023-04-11 21:18:05 -07:00
Kubernetes Prow Robot
2bfaaf21c1
Merge pull request #117197 from pohly/scheduler-perf-cleanup 
scheduler perf: remove cleanup func
 2023-04-11 21:17:57 -07:00
Patrick Ohly
464edfe6f6 test/integration: create nodes directly with kubernetes.io/hostname label 
By generating the unique name in advance, the label also can be set to a
matching value directly in the Create request. This makes test startup in
test/integration/scheduler_perf a bit faster because the extra patching can be
avoided.

It also leads to a better label because previously, the unique label value
didn't match the node name. This is required for simulating dynamic resource
allocation, which relies on the label to track where an allocated claim is
available.
 2023-04-11 16:35:37 +02:00
Patrick Ohly
aa73f06e56 scheduler perf: allow creating 0 items 
It makes sense to define a test where, depending on the parameters, some
operation creations zero pods, namespaces or nodes. The validation didn't allow
that previously due to the way how it was implemented although the underlying
code works fine with zero as count.
 2023-04-11 09:59:16 +02:00
Patrick Ohly
49bbf7c268 scheduler_perf: fix race condition 
collector.collect got called without ensuring that collector.run had
terminated, so it could have happened that collector.run adds another sample
while collector.collect is reading them.
 2023-04-11 09:46:34 +02:00
Patrick Ohly
a869a89825 scheduler perf: remove cleanup func 
b.Cleanup may as well get called inside the function instead
of leaving that to the caller.
 2023-04-11 09:43:45 +02:00
sarab
8d18ae6fc2 Use the generic Set in scheduler 2023-04-09 11:34:17 +05:30
Wei Huang
c9bc2f98d0
fix: remove SchedulingMigratedInTreePVs feature gate in sched perf test 2023-03-08 08:34:44 -08:00
Patrick Ohly
cc4bcd1d8e scheduler_perf: report data items as benchmark results 
This replaces the pretty useless us/op metric (useless because it includes
setup and teardown times) with the same values that also get stored in the JSON
file.

The main advantage is that benchstat can be used to analyze and compare
results.
 2023-02-28 23:08:23 +01:00
Patrick Ohly
961129c5f1 scheduler_perf: add logging flags 
This enables testing of different real production configurations (JSON
vs. text, different log levels, contextual logging).
 2023-02-28 23:08:17 +01:00
Kante Yin
3d0894fabf
Fix failure(context canceled) in scheduler_perf benchmark (#114843) 
* Fix failure in scheduler_perf benchmark

Signed-off-by: Kante Yin <kerthcet@gmail.com>

* Fatal when error in cleaning up nodes in scheduler perf tests

Signed-off-by: Kante Yin <kerthcet@gmail.com>

* Use derived context to better organize the codes

Signed-off-by: Kante Yin <kerthcet@gmail.com>

* Change log level to 2 in scheduler perf-test

Signed-off-by: Kante Yin <kerthcet@gmail.com>

---------

Signed-off-by: Kante Yin <kerthcet@gmail.com>
 2023-01-30 16:21:00 -08:00
Kensei Nakada
e8092cc885 cleanup(scheduler_perf): remove all removed feature gates 2023-01-04 01:07:47 +00:00
Patrick Ohly
2f6c4f5eab e2e: use Ginkgo context 
All code must use the context from Ginkgo when doing API calls or polling for a
change, otherwise the code would not return immediately when the test gets
aborted.
 2022-12-16 20:14:04 +01:00
Mark Rossetti
534f052a8d
Updating pause image refernces to 3.9 
Signed-off-by: Mark Rossetti <marosset@microsoft.com>
 2022-11-14 10:24:54 -08:00
kerthcet
d6ffb47832 Replace klog with benchmark log in scheduler_perf 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-09 09:11:55 +08:00
Kubernetes Prow Robot
73f6b96f0a
Merge pull request #113615 from kerthcet/feat/add-benchmark-tests 
Add nodeInclusionPolicy benchmark tests to scheduler_perf
 2022-11-07 09:18:28 -08:00
kerthcet
bc15aca26d Refactor SchedulerConfigFile 
Rename to SchedulerConfigPath and make it a pointer
to be consist with other fields

Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-05 00:30:34 +08:00
kerthcet
48f2c9ec20 Add benchmark tests for nodeInclusionPolicy 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-05 00:13:43 +08:00
kerthcet
cfc53ee524 Refactor code and annotations for readability 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-01 17:44:45 +08:00
kerthcet
21e8a69a22 Use operationCode instead of string directly 
Signed-off-by: kerthcet <kerthcet@gmail.com>
 2022-11-01 17:01:22 +08:00
Patrick Ohly
41619ace15 stop using deprecated klog flags 
Some scripts and tools still relied on the deprecated flags, the ones
which are about to be removed.

This is intentionally not a complete removal of all those flags in the entire
repo. This would lead to much more code churn also in places where commands
still accept the flags because they use klog directly.
 2022-09-04 21:02:43 +02:00
Davanum Srinivas
a9593d634c
Generate and format files 
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
 2022-07-26 13:14:05 -04:00
Wojciech Tyczyński
5b042f0bf4 Remove RunAnAPIServer from integration tests 2022-07-25 17:52:31 +02:00
Mark Rossetti
40f3e624a6 Switching everything to use pause:3.8 
Signed-off-by: Mark Rossetti <marosset@microsoft.com>
 2022-07-21 14:53:15 -07:00
Kensei Nakada
b0d47cb380
scheduler_perf: allow users to specify default pod and node specs (#101799) 
* scheduler_perf: default pod and node spec

* Fix: un-support DefaultNodeTemplatePath
 2022-06-29 11:44:07 -07:00
Davanum Srinivas
50bea1dad8
Move from k8s.gcr.io to registry.k8s.io 
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
 2022-05-31 10:16:53 -04:00
Kubernetes Prow Robot
570f1092f4
Merge pull request #109542 from sanposhiho/fix-test-case-scheduler-perf 
scheduler_perf: Remove test cases for Preemption which always fail
 2022-05-07 03:33:29 -07:00
Kubernetes Prow Robot
71df3e819b
Merge pull request #109545 from sanposhiho/fix-nun-on-scheduler_perf 
Skip adding data to avoid "json: unsupported value: NaN" panic when data is NaN
 2022-05-05 11:53:45 -07:00
Kensei Nakada
4af3c5efeb Skip adding data to avoid "json: unsupported value: NaN" panic when data is NaN 2022-05-05 16:11:22 +00:00
Kubernetes Prow Robot
3bef1692ef
Merge pull request #109696 from Huang-Wei/rm-sched-perf-legacy 
Cleanup legacy scheduler perf tests
 2022-05-04 02:35:43 -07:00
Kubernetes Prow Robot
629706e0fe
Merge pull request #109546 from sanposhiho/replace-metrics 
Replace scheduler_e2e_scheduling_duration_seconds with scheduler_scheduling_attempt_duration_seconds in scheduler_perf
 2022-05-04 01:29:22 -07:00
sanposhiho
1c2c20e6bd Change test cases for Preemption to create fewer Pods 2022-05-04 07:47:46 +00:00
Kubernetes Prow Robot
f0cd3725d3
Merge pull request #101835 from sanposhiho/scheduler_perf/feature/op-sleep 
scheduler_perf: create sleep operation
 2022-05-03 17:17:11 -07:00
Wei Huang
846ebf7814
Cleanup legacy scheduler perf tests 2022-04-27 09:57:17 -07:00
sanposhiho
b7b94b6b39 scheduler_perf: create sleep operation 2022-04-25 23:02:09 +00:00
sanposhiho
6e0da69632 Replace scheduler_e2e_scheduling_duration_seconds with scheduler_scheduling_attempt_duration_seconds in scheduler_perf 2022-04-20 00:48:12 +09:00
Davanum Srinivas
f7ad09c447
Switch to pause 3.7 
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
 2022-03-29 15:36:38 -04:00
Kubernetes Prow Robot
546e4fa1ef
Merge pull request #107771 from sanposhiho/fix-tiny 
make scheduler_perf stable
 2022-03-04 17:22:52 -08:00
sanposhiho
4c3a1000c7 fix by gofmt 2022-02-25 00:23:01 +09:00
sanposhiho
1080c2d717 Make scheduler_perf stable 2022-02-24 01:29:38 +09:00
Kubernetes Prow Robot
21c0f6f6ff
Merge pull request #107677 from pohly/scheduler-integration-benchmark 
scheduler integration benchmark improvements
 2022-02-14 01:23:28 -08:00
Patrick Ohly
e1e84c8e5f scheduler_perf: run with -v=0 by default 
This provides a mechanism for overriding the forced increase of the klog
verbosity to 4 when starting the apiserver and uses that for the scheduler_perf
benchmark. Other tests run as before.

A global variable was used because adding an explicit parameter to several
helper functions would have caused a lot of code churn (test ->
integration/util.StartApiserver ->
integration/framework.RunAnAPIServerUsingServer ->
integration/framework.startAPIServerOrDie).
 2022-02-11 16:58:33 +01:00
Patrick Ohly
c62d7407c8 scheduler_perf: dump test data when writing it failed 
Occasionally, writing as JSON failed because a NaN float couldn't be
encoded. The extended log message helps understand where that comes from, for
example:

F0120 20:24:45.515745  511835 scheduler_perf_test.go:540] BenchmarkPerfScheduling: unable to write measured data {Version:v1 DataItems:[{Data:map[Average:35.714285714285715 Perc50:2 Perc90:36 Perc95:412 Perc99:412] Unit:pods/s Labels:map[Metric:SchedulingThroughput Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2]} {Data:map[Average:27.863967530999993 Perc50:13.925925925925926 Perc90:30.06711409395973 Perc95:31.85682326621924 Perc99:704] Unit:ms Labels:map[Metric:scheduler_e2e_scheduling_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2]} {Data:map[Average:11915.651577744 Perc50:15168.796680497926 Perc90:19417.759336099585 Perc95:19948.87966804979 Perc99:20373.77593360996] Unit:ms Labels:map[Metric:scheduler_pod_scheduling_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2]} {Data:map[Average:1.1865832049999983 Perc50:0.7636363636363637 Perc90:2.891903719912473 Perc95:3.066958424507659 Perc99:5.333333333333334] Unit:ms Labels:map[Metric:scheduler_framework_extension_point_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2 extension_point:Filter]} {Data:map[Average:NaN Perc50:NaN Perc90:NaN Perc95:NaN Perc99:NaN] Unit:ms Labels:map[Metric:scheduler_framework_extension_point_duration_seconds Name:BenchmarkPerfScheduling/PreemptionPVs/500Nodes/namespace-2 extension_point:Score]}]}: json: unsupported value: NaN
 2022-02-07 08:59:19 +01:00
Patrick Ohly
8d44b819b3 scheduler_perf: avoid ambiguous test names 
"-bench=PerfScheduling/Preemption/500Nodes" ran both the
PerfScheduling/Preemption/500Nodes and the
PerfScheduling/PreemptionPVs/500Nodes benchmark.

This can be avoided by choosing names where none is the prefix of another.
 2022-02-07 08:59:19 +01:00
Patrick Ohly
259a8ad0b7 test: allow controlling etcd log level 
When running an integration test that measures performance, like for example
test/integration/scheduler_perf, running etcd with debug level output is
undesirable because it creates additional load on the system and isn't
realistic.

The default is still "debug", but ETCD_LOGLEVEL=warn can be used to override
that.
 2022-02-07 08:59:19 +01:00
ahrtr
fe95aa614c io/ioutil has already been deprecated in golang 1.16, so replace all ioutil with io and os 2022-02-03 05:32:12 +08:00
sanposhiho
d8840405e2 Create namespace for Pod not to occur error log of namespace not-found 2022-01-26 00:39:12 +09:00
Kubernetes Prow Robot
ca4af7a981
Merge pull request #104716 from sanposhiho/feature/scheduler_perf/unused-template-params 
test/integration/scheduler_perf: check for unused template parameters
 2022-01-10 16:21:16 -08:00