Commit Graph

263 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
bcbdcd17f3 Merge pull request #34685 from resouer/eclass-2
Automatic merge from submit-queue

Refactor scheduler to enable switch on equivalence cache

Part 2 of #30844 

Refactoring to enable easier switch on of equivalence cache.
2016-10-18 04:21:33 -07:00
Harry Zhang
50eaeaa7bd Update ecache and add scheduler method 2016-10-17 11:42:16 -04:00
Kubernetes Submit Queue
669efd72a4 Merge pull request #34041 from pigmej/fix_typos_plugin
Automatic merge from submit-queue

Fix typos and englishify plugin/pkg

**What this PR does / why we need it**:  Just typos

**Which issue this PR fixes**: `None`

**Special notes for your reviewer**: Just typos

**Release note**: `NONE`
2016-10-15 02:13:06 -07:00
Jess Frazelle
d51962e1bb
vet fixes
Signed-off-by: Jess Frazelle <me@jessfraz.com>
2016-10-13 21:12:15 -07:00
jayunit100
182e89b84e Part 1 of pr #33763: cleanup CheckServiceAffinity in preparation for
predicate injection support, Update metadata struct
2016-10-13 10:38:45 -04:00
yarntime
f21ff43368 add podAntiAffinity test cases 2016-10-13 17:01:58 +08:00
Kubernetes Submit Queue
5509e50db4 Merge pull request #31605 from resouer/eclass-1
Automatic merge from submit-queue

[Part 1] Implementation of equivalence pod

Part 1 of #30844

This PR: 

- Refactored `predicate.go`, so that `GetResourceRequest` can be used in other places to `GetEquivalencePod`.
- Implement a `equivalence_cache.go` to deal with all information we need to calculate an equivalent pod.
- Define and register a `RegisterGetEquivalencePodFunction `.

Work in next PR:
- The update of `equivalence_cache.go`
- Unit test
- Integration/e2e test

I think we can begin from the `equivalence_cache.go`? Thanks.  cc @wojtek-t @davidopp 

If I missed any other necessary part, please point it out.
2016-10-10 02:46:00 -07:00
Harry Zhang
204dbe7fdd Update provider and cache
Update equivalent class & remove priority

Use controller ref

Directly clear the cache
2016-10-10 09:25:15 +01:00
David Oppenheimer
cd4e08e7ec Revert "Add kubelet awareness to taint tolerant match caculator." 2016-10-07 12:10:55 -07:00
Kubernetes Submit Queue
21188cadeb Merge pull request #26501 from resouer/scheduler
Automatic merge from submit-queue

Add kubelet awareness to taint tolerant match caculator.

Add kubelet awareness to taint tolerant match caculator.

Ref: #25320

This is required by `TaintEffectNoScheduleNoAdmit` & `TaintEffectNoScheduleNoAdmitNoExecute `, so that node will know if it should expect the taint&tolerant
2016-10-07 12:05:35 -07:00
Kubernetes Submit Queue
0f40c1a6d0 Merge pull request #34149 from timothysc/prio-hotloop
Automatic merge from submit-queue

Turn down hootloop logs in priorities.  

Excessive spam is output once we near cluster capacity, sometimes a panic is triggered but that data is clipped in the logs see #33935 for more details.
2016-10-06 06:55:00 -07:00
Timothy St. Clair
f2b8b08651 Turn down hootloop logs in priorities. Hit once cluster is near capacity 2016-10-05 16:58:23 -05:00
deads2k
c30b2efc46 update replicaset lister 2016-10-05 15:20:27 -04:00
Kubernetes Submit Queue
96fde0fe8d Merge pull request #33834 from wojtek-t/scheduler_map_reduce_6
Automatic merge from submit-queue

Migrate EqualPriority in scheduler to map-reduce-like framework

Ref #24246
2016-10-04 09:23:01 -07:00
Jedrzej Nowak
d2161c21d7 Fix typos and englishify plugin/pkg 2016-10-04 17:51:14 +02:00
Wojciech Tyczynski
f8632e2203 Migrate EqualPriority to MapReduce-like framework. 2016-10-04 12:19:08 +02:00
David Ashpole
0c8a664e50 addressed comments 2016-10-03 11:42:56 -07:00
David Ashpole
fed3f37eef Split NodeDiskPressure into NodeInodePressure and NodeDiskPressure 2016-10-03 11:42:56 -07:00
Harry Zhang
c2cf5bbaf6 Setup e2e test for no admit 2016-10-01 01:07:18 -04:00
Wojciech Tyczynski
daac9a1869 Remove unnecessary podLister from SelectorSpreading 2016-09-30 15:15:14 +02:00
Wojciech Tyczynski
6598c181f7 Support metadata producer with underlying data. 2016-09-28 09:11:18 +02:00
Wojciech Tyczynski
4ccb27202c Migrate priority functions with non-trivial reduce function. 2016-09-27 11:27:38 +02:00
Jordan Liggitt
1812fe70a3
Log test error 2016-09-26 11:21:38 -04:00
deads2k
500959b70c fix RC lister 2016-09-23 08:12:03 -04:00
Kubernetes Submit Queue
b2aed32578 Merge pull request #33269 from deads2k/client-15-svc-lister
Automatic merge from submit-queue

simplify svc lister

trying to track down what killed the e2e tests.
2016-09-23 03:10:57 -07:00
Daniel Smith
db3e549bd1 more usages of go2idl 2016-09-22 13:42:46 -07:00
Harry Zhang
c735921b6f Add no admit on node side
Update generated code

Refactored predicates & restore helper
2016-09-22 10:12:44 -04:00
deads2k
483af28944 fix up service lister 2016-09-22 09:12:37 -04:00
Antoine Pelisse
938872582e Revert "simplify RC and SVC listers" 2016-09-21 15:49:38 -07:00
Kubernetes Submit Queue
2d9d84dc64 Merge pull request #32888 from deads2k/client-10-fixup-remaining-listers
Automatic merge from submit-queue

simplify RC and SVC listers

Make the RC and SVC listers use the common list functions that more closely match client APIs, are consistent with other listers, and avoid unnecessary copies.
2016-09-21 04:13:56 -07:00
Ivan Shvedunov
f758cb418d Fix possible panic in PodAffinityChecker 2016-09-20 15:53:13 +03:00
deads2k
16fbb47189 fix up service lister 2016-09-20 08:24:33 -04:00
deads2k
185a7adf84 fix RC lister 2016-09-20 08:24:32 -04:00
Kubernetes Submit Queue
4b5c74eed8 Merge pull request #31859 from resouer/fix-nits
Automatic merge from submit-queue

Fix minor nits in test cases

Found a group of nits when doing #30844, fixed them in a this PR since 30844 requires a long time to review.
2016-09-18 06:27:12 -07:00
Mike Danese
a765d59932 move informer and controller to pkg/client/cache
Signed-off-by: Mike Danese <mikedanese@google.com>
2016-09-15 12:50:08 -07:00
Wojciech Tyczynski
ea943d825e Migrate a bunch of priority functions to map-reduce framework 2016-09-13 13:20:29 +02:00
Wojciech Tyczynski
c8c938c3e0 AvoidPriority using ControllerRef 2016-09-12 08:45:02 +02:00
Harry Zhang
608e7ce0d4 Fix minor nits in test cases 2016-09-01 05:21:17 -04:00
Wojciech Tyczynski
33c710adf0 MapReduce-like scheduler priority functions 2016-08-31 15:16:10 +02:00
Wojciech Tyczynski
e9d5be628a Don't validate selector that is already validated 2016-08-22 09:39:32 +02:00
Marcin Wielgus
de2fea95ca ClusterAutoscaler-friendly scheduler priority function that promotes well-used nodes. 2016-08-18 08:34:10 +02:00
Jess Frazelle
7e9d82129e
fix go vet errors
Signed-off-by: Jess Frazelle <jessfraz@google.com>

fix composites

Signed-off-by: Jess Frazelle <me@jessfraz.com>
2016-08-10 16:45:41 -07:00
mksalawa
2749ec7555 Create PredicateFailureReason, modify scheduler predicate interface. 2016-08-09 14:01:46 +02:00
Kubernetes Submit Queue
faffbe4e18 Merge pull request #29622 from rootfs/rbd-ro
Automatic merge from submit-queue

allow a read-only rbd image mounted by multiple pods

allow pod to run read-only rbd volume 
fix #27725
2016-08-07 17:03:39 -07:00
Huamin Chen
730db45eab allow a read-only rbd image mounted by multiple pods
Signed-off-by: Huamin Chen <hchen@redhat.com>
2016-08-07 10:32:26 -04:00
Kubernetes Submit Queue
711b63e78b Merge pull request #28104 from ping035627/ping035627-patch-4
Automatic merge from submit-queue

SchedulerExtender: add failedPredicateMap in Filter() returns

Fix #25797. modify extender.Filter for adding extenders information to “failedPredicateMap” in findNodesThatFit.
When all the filtered nodes that passed "predicateFuncs" don’t pass the extenders filter, the failedPredicateMap hasn’t the extenders information, should add it, I think. So when the length of the “filteredNodes.Items” is 0, we can know the integral information. (The length of the “filteredNodes.Items” is 0, may be because the extenders filter failed.)
2016-08-06 20:50:33 -07:00
Kubernetes Submit Queue
2ff8280123 Merge pull request #29915 from wojtek-t/prepare_for_controller_ref_in_scheduler
Automatic merge from submit-queue

Prepare for using "ControllerRef" in scheduler

This is part of a PR that I already have to avoid a bunch of rebases in the future (controller ref probably won't happen in 1.4 release).

@davidopp
2016-08-04 02:23:31 -07:00
PingWang
4106eb70b0 modify extends.Filter
Signed-off-by: PingWang <wang.ping5@zte.com.cn>

Add docs

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

add docs for ExtenderFilterResult.FailedNodes in types.go

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

Modify the extender.Filter test.

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

Update extender_test.go

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

modify the comments

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

gofmt -s scheduler_interface.go

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update the comments

Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2016-08-04 14:03:32 +08:00
Wojciech Tyczynski
c092e15edf SelectorSpreading using controllerRef. 2016-08-02 16:24:48 +02:00
Wojciech Tyczynski
022719b323 Enable PodAffinity by default in scheduler 2016-08-02 15:06:45 +02:00
Wojciech Tyczynski
4bc410e47a Speedup pod affintiy predicate function 2016-08-02 08:01:04 +02:00
k8s-merge-robot
821ff657f9 Merge pull request #27199 from derekwaynecarr/disk_eviction
Automatic merge from submit-queue

Initial support for pod eviction based on disk

This PR adds the following:

1. node reports disk pressure condition based on configured thresholds
1. scheduler does not place pods on nodes reporting disk pressure
1. kubelet will not admit any pod when it reports disk pressure
1. kubelet ranks pods for eviction when low on disk
1. kubelet evicts greediest pod

Follow-on PRs will need to handle:

1. integrate with new image gc PR (https://github.com/kubernetes/kubernetes/pull/27199)
1. container gc policy should always run (will not be launched from eviction, tbd who does that)
  1. this means kill pod is fine for all eviction code paths since container gc will remove dead container
1. min reclaim support will just poll summary provider (derek will do follow-on)
1. need to know if imagefs is same device as rootfs from summary (derek follow-on)

/cc @vishh @kubernetes/sig-node
2016-07-28 20:18:54 -07:00
derekwaynecarr
0de1e62b30 modify fsStats to fsStatsType to avoid confusion with cadvisor types 2016-07-28 16:01:38 -04:00
derekwaynecarr
9604b47c13 Scheduler does not place pods on nodes that have disk pressure 2016-07-28 16:01:38 -04:00
Wojciech Tyczynski
d3b9d583a2 Optimize PodAffinity priority function. 2016-07-28 16:57:28 +02:00
Wojciech Tyczynski
898a6444e3 Return pointer for Affinity in api helper 2016-07-28 16:57:28 +02:00
Wojciech Tyczynski
fad876b6f9 PodAffinity code refinements 2016-07-22 08:49:28 +02:00
Wojciech Tyczynski
cab7db3a64 Use []*api.Node instead of NodeLister in priority functions. 2016-07-21 08:21:07 +02:00
Wojciech Tyczynski
dcb5a6d1a6 Reuse existing Resource struct instead of new resourceRequest 2016-07-19 12:21:09 +02:00
k8s-merge-robot
a049a97820 Merge pull request #28803 from lukaszo/ds
Automatic merge from submit-queue

Make Daemonset use GeneralPredicates

fixes: #21454 #22205
2016-07-18 22:12:14 -07:00
Davanum Srinivas
2b0ed014b7 Use Go canonical import paths
Add canonical imports only in existing doc.go files.
https://golang.org/doc/go1.4#canonicalimports

Fixes #29014
2016-07-16 13:48:21 -04:00
k8s-merge-robot
4374b090c3 Merge pull request #28958 from wojtek-t/optimize_priorities_5
Automatic merge from submit-queue

Few more optimizations of priority functions in scheduler

Ref #28590

@davidopp
2016-07-15 09:32:04 -07:00
k8s-merge-robot
8b2d994153 Merge pull request #28952 from wojtek-t/optimize_pod_affinity
Automatic merge from submit-queue

First step of optimizing PodAffinity priority function

Ref #26144 

This is obviously only a first step - I will continue working on this code. However, this is changing the general scheme of computations to what is described in: https://github.com/kubernetes/kubernetes/issues/26144#issuecomment-232612384
2016-07-15 08:59:17 -07:00
Wojciech Tyczynski
a538045d7b Cleanup and prepare for optimizing PodAffinity priority function. 2016-07-15 10:06:36 +02:00
Wojciech Tyczynski
a23f7bd556 Few more optimizations of priority functions in scheduler 2016-07-14 15:32:56 +02:00
k8s-merge-robot
74c5b4e4be Merge pull request #28459 from ping035627/ping035627-patch-0704
Automatic merge from submit-queue

Optimise the process of the CalculateSpreadPriority in selector_spreading.go

It had better inspect if the nodeLister is normal first in the CalculateSpreadPriority in selector_spreading.go. If the nodeLister.List return error, the function return directly, not need deal the serviceLister and controllerLister and replicaSetLister.
2016-07-14 04:54:32 -07:00
k8s-merge-robot
06939c57aa Merge pull request #20699 from jiangyaoguo/add-AvoidPreviousNode
Automatic merge from submit-queue

Implement alpha version of PreferAvoidPods

This is part of #18853

<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/20699)
<!-- Reviewable:end -->
2016-07-14 03:30:25 -07:00
PingWang
e4ab32cde2 Modify CalculateAntiAffinityPriority function
Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2016-07-14 09:10:50 +08:00
PingWang
437e133b34 Optimise the process of the CalculateSpreadPriority in selector_spreading.go
Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2016-07-14 09:09:05 +08:00
Łukasz Oleś
528bf7af3a Make Daemonset use GeneralPredicates
fixes #21454, fixes #22205
2016-07-13 14:50:29 +02:00
Wojciech Tyczynski
c929d95884 Cache Allocatable Resources 2016-07-13 12:57:18 +02:00
Wojciech Tyczynski
58c201834c Unify resources 2016-07-13 12:57:07 +02:00
jiangyaoguo
4e91166bc6 Use PreferAvoidPods annotation to avoid pods being scheduled to specific node.
1. define PreferAvoidPods annotation
2. add PreferAvoidPodsPriority
3. validate AvoidPods in node annotations
2016-07-13 17:07:46 +08:00
Wojciech Tyczynski
ea1d797f98 Avoid computing ports multiple times 2016-07-13 11:02:26 +02:00
Wojciech Tyczynski
ae6b66207a Optimize selector spreading 2016-07-13 11:02:26 +02:00
Wojciech Tyczynski
b9d13c5dbd Avoid computing pod resources multiple times 2016-07-13 11:01:53 +02:00
kevin-wangzefeng
ff6280fa5b update inter-pod affinity predicates, check PodAntiAffinity even when pod has no AntiAffinity constraints 2016-07-13 03:55:01 +00:00
k8s-merge-robot
5894dc4615 Merge pull request #28829 from gmarek/hooks
Automatic merge from submit-queue

Add hooks for cluster health detection

Separate a function that decides if zone is healthy. First real commit for preventing massive pod eviction.
Ref. #28832

cc @davidopp
2016-07-12 08:12:04 -07:00
k8s-merge-robot
98030ded05 Merge pull request #28781 from wojtek-t/optimize_priorities_2
Automatic merge from submit-queue

Change storeToNodeConditionLister to return []*api.Node instead of api.NodeList for performance



Currently copies that are made while copying/creating api.NodeList are significant part of scheduler profile, and a bunch of them are made in places, that are not-parallelizable.
Ref #28590
2016-07-12 06:18:15 -07:00
gmarek
fd600ab65c Add hooks for cluster health detection 2016-07-12 15:10:58 +02:00
k8s-merge-robot
9b5827691a Merge pull request #28669 from lixiaobing10051267/masterTestPodFitsResources
Automatic merge from submit-queue

Add test case to TestPodFitsResources() of scheduler algorithm

File "plugin\pkg\scheduler\algorithm\predicates", function "TestPodFitsResources()", line 199, only provide test case "one resource cpu fits but memory not", it should add test case "one resource memory fits but cpu not".
2016-07-11 22:46:09 -07:00
Wojciech Tyczynski
d14fe0f269 Change storeToNodeConditionLister to return []*api.Node instead of api.NodeList for performance 2016-07-11 21:02:33 +02:00
k8s-merge-robot
5067af159e Merge pull request #28715 from euank/kubelet-trim-dead-code
Automatic merge from submit-queue

kubelete: delete a few bits of dead code

Less is more.
2016-07-11 11:40:47 -07:00
k8s-merge-robot
9b74e24fa3 Merge pull request #28769 from wojtek-t/optimize_priorities
Automatic merge from submit-queue

Optimize priorities in scheduler

Ref #28590

It's probably easier to review it commit by commit, since those changes are kind of independent from each other.

@davidopp - FYI
2016-07-11 07:49:23 -07:00
Wojciech Tyczynski
d02e8d2885 Avoid unnecessary conversions 2016-07-11 15:41:27 +02:00
Wojciech Tyczynski
dcb2ca54ad Avoid unnecessary copies 2016-07-11 15:41:27 +02:00
Wojciech Tyczynski
989202c384 Cache AllowedPodNumber to avoid conversions. 2016-07-11 15:41:26 +02:00
Wojciech Tyczynski
4eed5e07a5 Precompute pod resources 2016-07-11 15:41:26 +02:00
Wojciech Tyczynski
e8e8e2d086 Remove unneeded factories 2016-07-11 15:41:26 +02:00
lixiaobing10051267
ce83c04251 Error info "scheduler" modify 2016-07-11 15:17:43 +08:00
PingWang
4fde2ff3a5 Update CheckServiceAffinity in predicates.go
Signed-off-by: PingWang <wang.ping5@zte.com.cn>

gofmt -s -w

Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2016-07-11 09:57:57 +08:00
Euan Kemp
26e0f50504 kubelet: delete unused code 2016-07-08 15:15:43 -07:00
Wojciech Tyczynski
0e6dc8a932 Avoid unnecessary copies 2016-07-08 13:54:06 +02:00
Wojciech Tyczynski
b1d53895a4 Check only predicate functions in test 2016-07-08 13:01:41 +02:00
Wojciech Tyczynski
b92eadfd1a Avoid computing QOS multiple times 2016-07-08 12:57:40 +02:00
Wojciech Tyczynski
6890868823 Add meta field to predicates signature 2016-07-08 12:25:48 +02:00
lixiaobing10051267
b4c106426e Add test case to TestPodFitsResources() 2016-07-08 15:43:03 +08:00
Wojciech Tyczynski
31538db145 Avoid growing slices 2016-07-07 15:54:39 +02:00
Wojciech Tyczynski
6c77c01f24 Avoid copying nodes in priority functions 2016-07-07 14:38:09 +02:00
Wojciech Tyczynski
58c95c136f Quick returns in predicates 2016-07-07 12:34:54 +02:00