Commit Graph

789 Commits

Author SHA1 Message Date
Kubernetes Prow Robot
8aaea8e24c
Merge pull request #107787 from Jiawei0227/volumelimit
CSI migration doesn't count inline volumes for attach limit
2022-05-11 23:09:13 -07:00
Steve Kuznetsov
3939f3003e
storage: move the APIObjectVersioner definition to storage
The means by which we extract and parse the version of an API object is
not specific to etcd3. In order to allow for a generic suite of tests
against any storage.Interface imlpementation, we need this logic to live
outside of the etcd3 package, or import cycles will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
2022-05-11 08:19:32 -07:00
kerthcet
790d983867 feat: implement NodeInclusionPolicy in TopologySpreadConstraint
Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-05-11 07:51:18 +08:00
Jiawei Wang
9d01f0bf52 Count inline volume for NodeVolumeLimit when CSI migration enabled
Previsouly, when kube-scheduler schedule a pod, it does
not take inline intree volume into account when CSI
migration is enabled. This could lead to failures where
pod scheduled to a node but volume attachment fails.
2022-05-10 07:43:28 +00:00
Yibo Zhuang
bc8f3198d5 cleanup: move scheduler plugin tests to use PodWrapper
Move scheduler plugin unit tests use testing PodWrapper
where applicable to reduce duplicating pod creation
code and shorten number of lines.

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-05 10:48:55 -07:00
Kubernetes Prow Robot
39021f66ef
Merge pull request #109715 from yibozhuang/volumerestrictions-refactor
Remove parallel node processing in PreFilter stage in volumerestrictions scheduler plugin
2022-05-04 06:49:51 -07:00
Kubernetes Prow Robot
7ed4b59200
Merge pull request #109043 from Abirdcfly/fixtestpvc
fix volumebinding test in scheduler
2022-05-03 18:24:27 -07:00
Yibo Zhuang
6cfc9901ce Remove parallel node processing in volumerestrictions
Leverage the usedPVCSet in snapshot to determine
whether a PVC with ReadWriteOncePod access mode is being
used by another scheduled pod to achieve O(1) look up in
PreFilter and avoid needing to parallel process all nodes.

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-03 18:01:22 -07:00
Abirdcfly
a7cfbb3e6c
fix volumebinding test in scheduler
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-04-28 16:22:02 +08:00
Abdullah Gharaibeh
6d499ee9ea Correct event registration for multiple scheduler plugins. 2022-04-12 18:32:24 -04:00
Wei Huang
6532bad659
compact scheduling failure msg for taint/toleration mismatch 2022-03-29 15:29:19 -07:00
Carlos Eduardo Arango Gutierrez
d2f67b42b7
Fix typos under pkg/scheduler
./scheduler/framework/plugins/noderesources/most_allocated_test.go:227: negtive ==> negative
./scheduler/framework/plugins/noderesources/most_allocated_test.go:228: negtive ==> negative
./scheduler/framework/plugins/volumebinding/binder.go:402: everytime ==> every time
./scheduler/framework/preemption/preemption_test.go:67: staus ==> status
./scheduler/framework/runtime/framework_test.go:1252: followings ==> following
./scheduler/internal/cache/cache_test.go:932: datas ==> data
./scheduler/internal/cache/node_tree_test.go:471: unknow ==> unknown
./scheduler/internal/queue/scheduling_queue_test.go:442: actived ==> activated
./scheduler/internal/queue/scheduling_queue_test.go:2209: covert ==> convert

Signed-off-by: Carlos Eduardo Arango Gutierrez <carangog@redhat.com>
2022-03-24 08:01:25 -04:00
Kubernetes Prow Robot
2d46f1bc30
Merge pull request #103062 from ikeeip/component_helper_storage
Move volume helpers to "k8s.io/component-helpers/storage/volume".
2022-03-23 13:21:20 -07:00
Kubernetes Prow Robot
24a71990e0
Merge pull request #108445 from pohly/storage-capacity-ga
storage capacity GA
2022-03-23 08:06:21 -07:00
Kubernetes Prow Robot
f979b4094e
Merge pull request #99292 from yangjunmyfm192085/run-test23
replace all occurrences of "node", nodeName to "node", klog.KRef("", nodeName)
2022-03-17 12:22:42 -07:00
Kubernetes Prow Robot
a504daa048
Merge pull request #108441 from pacoxu/pod-overload-ga
mark PodOverhead to GA in v1.24; remove in v1.26
2022-03-17 06:33:22 -07:00
Paco Xu
acd696266e mark PodOverhead to GA in v1.24; remove in v1.26 2022-03-17 09:30:14 +08:00
Konstantin Misyutin
438d224f0e Cleanup k8s.io/component-helpers/storage/volume package
Signed-off-by: Konstantin Misyutin <konstantin.misyutin@huawei.com>
2022-03-16 15:43:09 +08:00
Konstantin Misyutin
4ba98a8610 cleanup: remove unnecessary import aliases 2022-03-16 15:43:09 +08:00
Konstantin Misyutin
1d7cefe9c4 Move volume helpers to "k8s.io/component-helpers/storage/volume".
This patch aims to simplify decoupling "pkg/scheduler/framework/plugins"
from internal "k8s.io/kubernetes" packages. More described in
issue #89930 and PR #102953.

Some helpers from "k8s.io/kubernetes/pkg/controller/volume/persistentvolume"
package moved to "k8s.io/component-helpers/storage/volume" package:

- IsDelayBindingMode
- GetBindVolumeToClaim
- IsVolumeBoundToClaim
- FindMatchingVolume
- CheckVolumeModeMismatches
- CheckAccessModes
- GetVolumeNodeAffinity

Also "CheckNodeAffinity" from "k8s.io/kubernetes/pkg/volume/util"
package moved to "k8s.io/component-helpers/storage/volume" package
to prevent diamond dependency conflict.

Signed-off-by: Konstantin Misyutin <konstantin.misyutin@huawei.com>
2022-03-16 15:43:09 +08:00
Kubernetes Prow Robot
aa383570f0
Merge pull request #108705 from denkensk/refine-preempt-msg
make the preempt failed message per node compressible in event
2022-03-15 18:09:55 -07:00
Kubernetes Prow Robot
2b1b849d6a
Merge pull request #108362 from sanposhiho/implement-mindomains
Implement MinDomains on Pod Topology Spread
2022-03-15 10:34:11 -07:00
Alex Wang
ddd62202fb make the preempt message per node compressible 2022-03-15 22:02:19 +08:00
Kubernetes Prow Robot
d7bd0d4f52
Merge pull request #108648 from ahg-g/ahg-ds
Update PreFilter interface to return a PreFilterResult
2022-03-14 18:50:20 -07:00
Abdullah Gharaibeh
da7f085dcb Added a NodeAffinity PreFilter that looks for node.Name MatchField terms; if exist, the pod is only evaluated against the matching nodes. 2022-03-14 15:37:03 -04:00
Abdullah Gharaibeh
5b20b68bc9 Updated the scheduler framework's PreFilter interface to return a PreFilterResult 2022-03-14 15:36:52 -04:00
Patrick Ohly
f84f4fa291 storage capacity: use V1 API 2022-03-14 20:05:45 +01:00
Patrick Ohly
3a7deaa141 storage capacity: GA, always enabled, remove feature check
The code and tests for scenarios where the feature is disabled are no longer
needed because the feature is graduating to GA.
2022-03-14 20:05:45 +01:00
sanposhiho
6141aa53f9 Implement MinDomains 2022-03-15 01:38:44 +09:00
Kubernetes Prow Robot
5b52c4d127
Merge pull request #107674 from sanposhiho/api-min-domains
Add MinDomains API to TopologySpreadConstraints field
2022-03-14 09:33:57 -07:00
Kubernetes Prow Robot
d0d9a69414
Merge pull request #108484 from Abirdcfly/fixctx
fix: some function should pass context parameter
2022-03-11 13:26:34 -08:00
sanposhiho
3b13e9445a Add MinDomains API to TopologySpreadConstraints field 2022-03-09 20:57:12 +09:00
Abirdcfly
e35cfbb5a7 fix: some function should pass context parameter
Change-Id: Ib509573a72c8bd0c61233ade415fef470c61bf5f
2022-03-04 00:42:45 +08:00
Abirdcfly
52c87654a1 simplify function MatchProvisioner
Change-Id: I15b7507fed50032d0df7a91553ba58f764d92aac
2022-03-03 16:30:20 +08:00
sanposhiho
0b16a7fefa Support ExtenderName in FakeExtender 2022-02-23 12:14:39 +09:00
Abdullah Gharaibeh
8a1c70b48c Graduate PodAffinityNamespaceSelector to GA 2022-02-18 12:07:29 -05:00
Kubernetes Prow Robot
21c0f6f6ff
Merge pull request #107677 from pohly/scheduler-integration-benchmark
scheduler integration benchmark improvements
2022-02-14 01:23:28 -08:00
Kubernetes Prow Robot
0dcd6eaa0d
Merge pull request #103934 from boenn/tainttoleration
De-duplicate predicate (known as filter now) logic shared in kubelet and scheduler
2022-02-09 16:53:46 -08:00
Patrick Ohly
1f341ee7b5 kube-scheduler: downgrade namespace log message from "error" to "info"
GetNamespaceLabelsSnapshot has a fallback when it gets errors when looking up a
namespace, therefore reporting the error is more informational than a real
error. In particular, not finding the namespace is normal when running
test/integration/scheduler_perf and happens so frequently that there is a lot
of output on stderr:

E0120 12:19:09.204768   95305 plugin.go:138] "getting namespace, assuming empty set of namespace labels" err="namespace \"namespace-1\" not found" namespace="namespace-1"
2022-02-07 08:59:19 +01:00
Kubernetes Prow Robot
6410ddaba9
Merge pull request #107623 from bbarnes52/podtopologyoptimization
Optimize pod topology spread performance
2022-02-04 15:58:58 -08:00
Brian Barnes
4222d3a48e optimize pod topology spread 2022-02-04 10:27:58 -08:00
Kubernetes Prow Robot
2fb24de42b
Merge pull request #107775 from denkensk/add-postfilter-info-to-event
Add details about preemption in the event for scheduling failed
2022-01-31 09:38:23 -08:00
Alex Wang
52275fc831 add preemption info in scheduling failed event 2022-02-01 00:37:16 +08:00
Wei Huang
939e98135c
sched: fix goroutine leak in unit tests 2022-01-22 22:53:11 -08:00
Kubernetes Prow Robot
0c8074e1b1
Merge pull request #107445 from mengjiao-liu/goroutine_leak_TestSelectorSpreadScore
Fix goroutine leaks in package `selectorspread`
2022-01-21 11:23:21 -08:00
Kubernetes Prow Robot
184daed0db
Merge pull request #107559 from liggitt/invalid-selectors
Handle invalid selectors properly
2022-01-19 14:49:31 -08:00
Jordan Liggitt
c0af728f43 Handle invalid selectors properly 2022-01-14 12:11:02 -05:00
Wojciech Tyczyński
551790729f Remove selflink references in different testing-related files 2022-01-14 12:58:05 +01:00
Kubernetes Prow Robot
42436c877e
Merge pull request #107462 from dims/next-step-in-csi-migration-for-openstack-take-2
Next step in CSI migration for openstack
2022-01-12 19:57:07 -08:00
Patrick Ohly
9eaa2dc554 avoid klog Info calls without verbosity
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:

   if klog.V(5).Enabled() {
       klog.Info("hello world")
   }

Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.

Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.
2022-01-12 07:48:36 +01:00
Mengjiao Liu
d0ec334605 Fix goroutine leaks in package selectorspread 2022-01-12 10:20:32 +08:00
Kubernetes Prow Robot
39136784a4
Merge pull request #107009 from kerthcet/feature/fix-wrong-calculate-in-podTopologySpread
nodeAffinity filtered nodes should be excluded when calculating skew …
2022-01-11 08:33:16 -08:00
Davanum Srinivas
7fd97433f0
Next step in CSI migration for openstack
delete/modify tests that use intree cinder as well.

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-01-10 22:07:44 -05:00
kerthcet
935cbc8e62 nodeAffinity filtered nodes should be excluded when calculating skew in PodTopologySpread
Signed-off-by: kerthcet <kerthcet@gmail.com>
2022-01-07 10:25:10 +08:00
sanposhiho
85816089a0 Use math round when convert float to int in Pod Topology Spread 2022-01-07 11:11:47 +09:00
JUN YANG
13fa2b1949 replace all occurrences of "node", nodeName to "node", klog.KRef("", nodeName) 2022-01-06 21:19:29 +08:00
JunYang
b41c2881be Optimize the use of klog.Errors
Signed-off-by: JunYang <yang.jun22@zte.com.cn>
2022-01-06 08:47:33 +08:00
Wei Huang
2433b083a9
clear pod's .status.nominatedNodeName when necessary 2021-12-16 10:55:13 -08:00
Davanum Srinivas
497e9c1971
Cleanup OWNERS files (No Activity in the last year)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-12-15 10:34:02 -05:00
Davanum Srinivas
9405e9b55e
Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-12-09 21:31:26 -05:00
Kubernetes Prow Robot
d7f8234b6d
Merge pull request #106747 from ahg-g/ahg-test
Added an integration test for NodeResourcesFit scoring
2021-12-07 19:28:06 -08:00
Kubernetes Prow Robot
75109026d0
Merge pull request #106447 from hyschumi/fix-noderesources
cleanup duplicated method `makeNodeWithExtendedResource` in noderesources unit test
2021-12-07 17:27:10 -08:00
Abdullah Gharaibeh
33a04dc5f5 Added an integration test for NodeResourcesFit scoring 2021-11-30 12:13:30 -05:00
boenn
cec2aae1e5 rebase master 2021-11-25 11:21:12 +08:00
hyschumi
7ad629c864 rm makeNodeWithExtendedResource method in noderesources unit test && doc: update least_allocated strategy detail doc 2021-11-18 09:15:26 +08:00
Oksana Naumov
3af11fc12d Add support for Portworx to csi-translation lib
Signed-off-by: Oksana Naumov <trierra.dev@gmail.com>
2021-11-16 13:26:09 -08:00
Kubernetes Prow Robot
66c342ba63
Merge pull request #95361 from humblec/rbd-migration
RBD in-tree plugin migration to CSI driver using migration translation lib
2021-11-15 19:53:25 -08:00
Humble Chirammal
7c40eb9ae0 Add support for rbd plugin to csi-translation-lib
In support of csi-migration proposal here:
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-migration.md

Will help with migration of in-tree RBD plugin ( kubernetes.io/rbd)
to RBD CSI driver ( rbd.csi.ceph.com ).

Fixes https://github.com/kubernetes/enhancements/issues/2923

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-15 23:46:29 +05:30
Shivanshu Raj Shrivastava
3142acbc8c
migrated remaining lines of server.go and csi.go to structured logging 2021-11-10 21:24:34 +05:30
Kubernetes Prow Robot
a41d166a90
Merge pull request #105904 from mengjiao-liu/structured_logging_scheduler
Migrate assume_cache.go to structured logging
2021-11-09 11:28:00 -08:00
Kubernetes Prow Robot
6ac2d8edc8
Merge pull request #105967 from shivanshu1333/feature2/master/105841
Migrated scheduler files `preemption.go`, `stateful.go`, `resource_allocation.go` to structured logging
2021-11-09 10:28:01 -08:00
Shivanshu Raj Shrivastava
f4aad52885
migrated preemption.go, stateful.go, resource_allocation.go to structured logging 2021-11-08 22:52:47 +05:30
Kubernetes Prow Robot
53addf3ba3
Merge pull request #105858 from jyz0309/migrate-log
Migrated scheduler files binder.go binder_test.go to structured logging
2021-11-02 19:01:09 -07:00
Kubernetes Prow Robot
ec8e6e8778
Merge pull request #106002 from kerthcet/feature/refactor-NodeResourcesFit-plugin
refactor scheudler's node resource plugins
2021-11-02 10:52:34 -07:00
kerthcet
e5dd375b14 refactor scheudler's node resource plugins
Signed-off-by: kerthcet <kerthcet@gmail.com>
2021-11-03 00:52:23 +08:00
Mengjiao Liu
e262e3ad2f Migrate assume_cache.go to structured logging 2021-11-01 17:43:39 +08:00
jyz0309
07bf08690c migrate log to structure log
Signed-off-by: jyz0309 <45495947@qq.com>

add klog.Kobj

Signed-off-by: jyz0309 <45495947@qq.com>

use KObj

Signed-off-by: jyz0309 <45495947@qq.com>

address comment

Signed-off-by: jyz0309 <45495947@qq.com>

remove useless var

Signed-off-by: jyz0309 <45495947@qq.com>

format code

Signed-off-by: jyz0309 <45495947@qq.com>

address comment

Signed-off-by: jyz0309 <45495947@qq.com>

use err key

Signed-off-by: jyz0309 <45495947@qq.com>

use PVC

Signed-off-by: jyz0309 <45495947@qq.com>

improve log message

Signed-off-by: jyz0309 <45495947@qq.com>

address comment

Signed-off-by: jyz0309 <45495947@qq.com>

use pod instead podName

Signed-off-by: jyz0309 <45495947@qq.com>
2021-10-31 21:11:26 +08:00
Ahmad Diaa
a2c37bfd09
use original requests in NodeResourcesBalancedAllocation instead of NonZero (#105845) 2021-10-29 19:04:14 -07:00
Kubernetes Prow Robot
adff4a75ad
Merge pull request #105931 from mengjiao-liu/structured_logging_scheduler_part2
Migrate scheduler files `interpodaffinity/filtering.go`,`podtopologyspread/filtering.go`, `volume_zone.go` to structured logging
2021-10-29 12:13:23 -07:00
Mengjiao Liu
2783ddc227 Migrate scheduler files interpodaffinity/filtering.go,podtopologyspread/filtering.go, volume_zone.go to structured logging 2021-10-29 11:41:02 +08:00
Kubernetes Prow Robot
c592bd40f2
Merge pull request #105609 from pohly/generic-ephemeral-volume-ga
generic ephemeral volume GA
2021-10-28 17:36:50 -07:00
Kubernetes Prow Robot
8e8bf06f15
Merge pull request #105796 from sanposhiho/move/parallelizer-to-non-internal
Fix: move parallelize package to non-internal file
2021-10-28 15:31:00 -07:00
Kubernetes Prow Robot
87b0412232
Merge pull request #105727 from chendave/wrong_status
Fix the inaccurate status when a plugin internal status is found
2021-10-27 19:45:02 -07:00
Dave Chen
468a6005a6 Fix the return status when a plugin internal status is found
Currently, the status code returned is `Unschedulable` when an internal error
found, the `Unschedulable` status is built from a `FitError` which means no
fit nodes found without a internal error.

Instead of build an Unschedulable status from the `FitError`, return the
Error status directly.

Signed-off-by: Dave Chen <dave.chen@arm.com>
2021-10-28 09:43:31 +08:00
kerthcet
2c5b5533bf remove scheduler NodeLabel plugin
Signed-off-by: kerthcet <kerthcet@gmail.com>
2021-10-27 10:07:35 +08:00
Kubernetes Prow Robot
3141c984a1
Merge pull request #105907 from kerthcet/feature/remove-scheduler-plugin-nodePreferAvoidPods
remove scheduler NodePreferAvoidPods plugin
2021-10-26 18:07:41 -07:00
kerthcet
a139da6b04 remove scheduler NodePreferAvoidPods plugin
Signed-off-by: kerthcet <kerthcet@gmail.com>
2021-10-27 08:05:44 +08:00
Shivanshu Raj Shrivastava
3c87c43cef
Migrated scheduler files server.go, node_label.go, csi.go, non_csi.go to structured logging (#105855)
* migrated server.go

* fixed migration

* resolving review comments

* added storageClass

* review comments

* review comments
2021-10-26 13:21:22 -07:00
Abdullah Gharaibeh
faf0ce68a0 Remove scheduler's legacy policy config 2021-10-24 21:58:12 -04:00
sanposhiho
9dc0489c1a Fix: move parallelize package to non-internal file 2021-10-21 02:39:23 +09:00
kerthcet
fc9533e72f remove scheduler ServiceAffinity plugin
Signed-off-by: kerthcet <kerthcet@gmail.com>
2021-10-15 22:10:31 +08:00
Patrick Ohly
a8c930ef46 generic ephemeral volume: graduation to GA
The feature gate gets locked to "true", with the goal to remove it in two
releases.

All code now can assume that the feature is enabled. Tests for "feature
disabled" are no longer needed and get removed.

Some code wasn't using the new helper functions yet. That gets changed while
touching those lines.
2021-10-11 20:54:20 +02:00
Patrick Ohly
bc263f3ba5 scheduler: use generic ephemeral volume helper functions
The name concatenation and ownership check were originally considered small
enough to not warrant dedicated functions, but the intent of the code is more
readable with them.
2021-10-11 17:33:57 +02:00
Wei Huang
b7d90ca991
sched: adjust events to register for VolumeBinding plugin 2021-10-07 08:51:04 -07:00
Kubernetes Prow Robot
6c292ce270
Merge pull request #105245 from yibozhuang/lost-pvc-prefilter-optimization
Scheduler volumebinding plugin - handle Lost PVC as UnschedulableAndUnresolvable
2021-10-05 08:23:09 -07:00
Kubernetes Prow Robot
04f747d09f
Merge pull request #104782 from kerthcet/cleanup/remove-cc-v1beta1
remove scheduler component config v1beta1
2021-10-04 08:53:08 -07:00
Kubernetes Prow Robot
e414cf7641
Merge pull request #100482 from pohly/generic-ephemeral-volume-checks
generic ephemeral volume checks
2021-10-01 10:47:22 -07:00
Patrick Ohly
1d181ad84f scheduler: fail the volume attach limit when PVC is missing
Previously, the situation was ignored, which might have had the effect that Pod
scheduling continued (?) even though the Pod+PVC weren't known to be in an
acceptable state.
2021-10-01 17:03:44 +02:00
Patrick Ohly
1e26115df5 consider ephemeral volumes for host path and node limits check
When adding the ephemeral volume feature, the special case for
PersistentVolumeClaim volume sources in kubelet's host path and node
limits checks was overlooked. An ephemeral volume source is another
way of referencing a claim and has to be treated the same way.
2021-10-01 17:03:44 +02:00
Yibo Zhuang
b8fe514232 Scheduler volumebinding plugin - handle Lost PVC as
UnschedulableAndUnresolvable

This change adds an additional check in the volumebinding scheduler
plugin to handle PVC with phase ClaimLost which will allow the
scheduler to return UnschedulableAndUnresolvable during the PreFilter
stage and skip the rest of the node evaluation since the PVC is
bound to a PV that does not exist.

Without this change, the FailedScheduling error message would look like:

0/10 nodes are available: 2 node(s) had taint {node/test: true},
that the pod didn't tolerate, 6 node(s) had taint {node/unhealthy: true},
that the pod didn't tolerate, 2 pvc(s) bound to non-existent pv(s)

Which is still evaluating every single node to determine that the pod
cannot be scheduled because the PVC is bound to a non-existent PV

With this change, the FailedScheduling error message would look like:

0/10 nodes are available: 1 persistentvolumeclaim "foo" bound
to non-existent persistentvolume "bar"

Signed-off By: Yibo Zhuang <yibzhuang@gmail.com>
2021-09-30 21:49:46 -07:00