Commit Graph

1141 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
34b258ca4b Merge pull request #55933 from bsalamat/starvation3
Automatic merge from submit-queue (batch tested with PRs 54316, 53400, 55933, 55786, 55794). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add support to take nominated pods into account during scheduling to avoid starvation of higher priority pods

**What this PR does / why we need it**:
When a pod preempts lower priority pods, the preemptor gets a "nominated node name" annotation. We call such a pod a nominated pod. This PR adds the logic to take such nominated pods into account when scheduling other pods on the same node that the nominated pod is expected to run. This is needed to avoid starvation of preemptor pods. Otherwise, lower priority pods may fill up the space freed after preemption before the preemptor gets a chance to get scheduled.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #54501

**Special notes for your reviewer**: This PR is built on top of #55109 and includes all the changes there as well.

**Release note**:

```release-note
Add support to take nominated pods into account during scheduling to avoid starvation of higher priority pods.
```

/sig scheduling
ref/ #47604
2017-11-21 15:04:28 -08:00
Bobby (Babak) Salamat
eda3df8732 Autogenerated files 2017-11-20 22:17:06 -08:00
Bobby (Babak) Salamat
e3a0e0eb40 fixup! Add logic to account for pods nominated to run on nodes, but are not running yet. Add tests for the new logic. 2017-11-20 22:17:06 -08:00
Bobby (Babak) Salamat
8a17ae241d Add logic to account for pods nominated to run on nodes, but are not running yet.
Add tests for the new logic.
2017-11-20 22:17:05 -08:00
Yang Guo
035e73b6c3 Suppress warning when a pod in binding cannot be expired 2017-11-20 16:16:42 -08:00
Kubernetes Submit Queue
d0301aa6e8 Merge pull request #54208 from resouer/rm-alpha
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move of unreachable taint key out of alpha

**What this PR does / why we need it**:

Move of unreachable taint key out of alpha, which already happened in community doc.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #54198

**Special notes for your reviewer**:
Please see #54198 for the context of this inconsistency.

**Release note**:

```release-note
Move unreachable taint key out of alpha. 
Please note the existing pods with the alpha toleration should be updated by user himself to tolerate the GA taint.
```
2017-11-20 10:50:17 -08:00
Bobby (Babak) Salamat
79ac37d8b3 Autogenerated files 2017-11-19 11:32:14 -08:00
Bobby (Babak) Salamat
daf8492bc5 Added a new scheduling queue based on priority queue. 2017-11-19 11:29:23 -08:00
Kubernetes Submit Queue
ab203e54ed Merge pull request #51192 from guangxuli/scheduler_priority_functions_map_reduce
Automatic merge from submit-queue (batch tested with PRs 51192, 55010). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Refactoring of priority function(CaculateSpreadPriority) by using map/reduce pattern

**What this PR does / why we need it**:
Ref #24246. exactly ref https://github.com/kubernetes/kubernetes/issues/51455, the PR aim to unify priority functions(deprecated) by using map/reduce pattern.
This is the first step, my todo list(WIP):
- interpod-affnity priority funciton refactoring 
- the priority funcitons register pattern
- deprecated priority function definition and all related logic. etc.

**Which issue this PR fixes**:

no issue, just unify the priority functions pattern.

**Special notes for your reviewer**:
none
**Release note**:
none
2017-11-19 05:22:23 -08:00
Gavin
8fa59aa9b0 address review comments 2017-11-19 13:29:21 +08:00
Kubernetes Submit Queue
3ec7487c0f Merge pull request #55263 from anfernee/refactor_reduce
Automatic merge from submit-queue (batch tested with PRs 55254, 55525, 50108, 54674, 55263). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Refactor Priority Reduce functions

- Reducing the duplicated reduce function by introducing a reduce
  function generator that generates common reduce functions.
- Remove logs from reduce function, so it's purely calculating scores.
- Optimize the reduce functions by removing unnecessary conversion to
  float64.

**Release note**:
```release-note
None
```
2017-11-17 13:34:17 -08:00
Kubernetes Submit Queue
a00b766ab0 Merge pull request #54134 from chentao1596/plugin-pkg-scheduler-util-non-zero-unit-test
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adding unit tests to methods of 'scheduler/algorithm/priorities/util'

What this PR does / why we need it:

 1) Adding unit tests to methods of 'non_zero.go'
 2) Adding unit tests to methods of 'util.go'
 3) Adding/Modifing unit tests to methods of 'topologies.go'

thank you!
2017-11-17 01:13:03 -08:00
Harry Zhang
df8c92ac12 Replace node's alpha taint key with GA 2017-11-17 15:35:35 +08:00
Connor Doyle
80ac705ef3 Removed opaque integer resources. 2017-11-16 10:47:40 -08:00
Gavin
71a0350239 httpserver should be close since the issue has been fixed 2017-11-16 15:03:08 +08:00
Kubernetes Submit Queue
b623026d2a Merge pull request #52421 from WIZARD-CXY/fixpredicate
Automatic merge from submit-queue (batch tested with PRs 53780, 55663, 55321, 52421, 55659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add hostip and protocol to the hostport predicates

**What this PR does / why we need it**:
This PR adds "hostIP and protocol" to scheduler hostport predicate procedure
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fix #51950 
**Special notes for your reviewer**:
- [x] basic implementation, need review
- [x] e2e test
- [x] update doc (will be done in seperate PR)

**Release note**:

```release-note
add hostIP and protocol to the original hostport predicates procedure in scheduler.
```
2017-11-15 09:30:36 -08:00
Doug MacEachern
7950609b31 Enforce use of fixed size int types in the API
Changes 'int' to 'int32', enforced by import_known_versions_test

Follow up to PR #53402
2017-11-13 11:28:59 -08:00
Kubernetes Submit Queue
fe5b8fac86 Merge pull request #52148 from wackxu/addmtcase
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add test case  in metadata_test.go

**What this PR does / why we need it**:

Add test case in metadata_test.go

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #55567 

**Release note**:

```release-note
NONE
```
2017-11-13 00:50:11 -08:00
Yuhao Fang
12fc5a5612 fixtypo 2017-11-13 10:26:31 +08:00
Dr. Stefan Schimanski
1e79dfb959 Update generated code 2017-11-10 18:26:46 +01:00
Dr. Stefan Schimanski
72809a08b9 deepcopy: remove deepcopy register tags 2017-11-10 18:25:26 +01:00
chenxingyu
954c97fe6d add e2e test on the hostport predicates 2017-11-10 15:44:23 +08:00
chenxingyu
2d44ef9dfa add hostip protocol to the hostport predicates and make unit test adapt to the code change 2017-11-10 15:41:52 +08:00
Gavin
bed435deca compute pod selectors in priority meta data producer 2017-11-09 20:21:17 +08:00
Gavin
4aa92bac73 Refactoring of priority function(CaculateSpreadPriority) by using map/reduce pattern 2017-11-09 20:21:13 +08:00
Dr. Stefan Schimanski
bec617f3cc Update generated files 2017-11-09 12:14:08 +01:00
Dr. Stefan Schimanski
012b085ac8 pkg/apis/core: mechanical import fixes in dependencies 2017-11-09 12:14:08 +01:00
Kubernetes Submit Queue
17f7183e74 Merge pull request #55172 from hongshibao/master
Automatic merge from submit-queue (batch tested with PRs 54493, 52501, 55172, 54780, 54819). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix divide by zero issue in calculating spread priority for zones

```release-note
NONE
```
2017-11-08 15:41:18 -08:00
Yongkun Anfernee Gui
4be6a60df2 Refactor Priority Reduce functions
- Reducing the duplicated reduce function by introducing a reduce
  function generator that generates common reduce functions.
- Remove logs from reduce function, so it's purely calculating scores.
- Optimize the reduce functions by removing unnecessary conversion to
  float64.
2017-11-08 10:00:23 -08:00
Kubernetes Submit Queue
33f873dbbe Merge pull request #55262 from liggitt/schedulercache
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix 'Schedulercache is corrupted' error

Fixes #50916

If an Assume()ed pod is Add()ed with a different nodeName, the podStates view of the pod is not corrected to reflect the actual nodeName. On the next Update(), the scheduler observes the mismatch and process exits.

```release-note
Fixed 'Schedulercache is corrupted' error in kube-scheduler
```
2017-11-07 23:23:33 -08:00
Jordan Liggitt
a366e6ced0 Fix 'Schedulercache is corrupted' error 2017-11-07 18:49:49 -05:00
Dan Mace
efb2bb71cd Refactor scheduler config API
Refactor the kube-scheduler configuration API, command setup, and server
setup according to the guidelines established in #32215 and using the
kube-proxy refactor (#34727) as a model of a well factored component
adhering to said guidelines.

* Config API: clarify meaning and use of algorithm source by replacing
modality derived from bools and string emptiness checks with an explicit
AlgorithmSource type hierarchy.
* Config API: consolidate client connection config with common structs.
* Config API: split and simplify healthz/metrics server configuration.
* Config API: clarify leader election configuration.
* Config API: improve defaulting.
* CLI: deprecate all flags except `--config`.
* CLI: port all flags to new config API.
* CLI: refactor to match kube-proxy Cobra command style.
* Server: refactor away configurator.go to clarify application wiring.
* Server: refactor to more clearly separate wiring/setup from running.

Fixes #52428.
2017-11-07 09:41:39 -05:00
Kubernetes Submit Queue
32b761edf3 Merge pull request #54871 from anfernee/clean
Automatic merge from submit-queue (batch tested with PRs 55114, 52976, 54871, 55122, 55140). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

haveSame is suboptimal, fix it as well as the name

**What this PR does / why we need it**: Make the util function simpler, also faster by reducing a for loop.

**Special notes for your reviewer**:

**Release note**:
```release-note
None
```
2017-11-06 23:19:18 -08:00
hongshibao
2141b9edd5 Fix divide by zero issue in calculating spread priority for zones 2017-11-06 23:39:05 +08:00
Kubernetes Submit Queue
aed9ad4b08 Merge pull request #54778 from stewart-yu/addexplain
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add comment for preempt sunction.

**What this PR does / why we need it**:
Add explain for preempt sunction.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-11-04 15:42:20 -07:00
Yongkun Anfernee Gui
5d268eb1d3 haveSame is suboptimal, fix it as well as the name 2017-11-02 22:50:32 -07:00
Kubernetes Submit Queue
2084f7f4f3 Merge pull request #54488 from lichuqiang/plugin_base
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add admission handler for device resources allocation

**What this PR does / why we need it**:
Add admission handler for device resources allocation to fail fast during pod creation

**Which issue this PR fixes** 
fixes #51592

**Special notes for your reviewer**:
@jiayingz Sorry, there is something wrong with my branch in #51895. And I think the existing comments in the PR might be too long for others to view. So I closed it and opened the new one, as we have basically reach an agreement on the implement :)
I have covered the functionality and unit test part here, and would set about the e2e part ASAP

/cc @jiayingz @vishh @RenaudWasTaken 

**Release note**:

```release-note
NONE
```
2017-11-02 17:24:06 -07:00
lichuqiang
ebd445eb8c add admission handler for device resources allocation 2017-11-02 09:17:48 +08:00
Yongkun Anfernee Gui
8465625bd7 Optimize the suboptimal image locality algorithm 2017-11-01 10:18:59 -07:00
Stewart-YU
bc2845ee08 Add explain for preempt sunction. 2017-10-31 08:27:52 +08:00
Dr. Stefan Schimanski
ce6ecbbc54 Remove all api.Scheme references by using explicit package aliases 2017-10-30 19:54:02 +01:00
Kubernetes Submit Queue
7c96feb298 Merge pull request #54047 from kuramal/my
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Optimize Repeated registration of AlgorithmProvider when ApplyFeatureGates

**What this PR does / why we need it**:
modified ApplyFeatureGates() just add/del features, cancel the register of all AlgorithmProvider.

 there is Repeated registration of all AlgorithmProvider when ApplyFeatureGates() runs;
AlgorithmProvider have already registered when  package defaults loaded;
I think ApplyFeatureGates() is just add/del features, it needn't  register all AlgorithmProvider again
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```/release-note-none
```
2017-10-29 20:42:48 -07:00
Kubernetes Submit Queue
7d628db7cb Merge pull request #54657 from anfernee/sched
Automatic merge from submit-queue (batch tested with PRs 54635, 54250, 54657, 54696, 54700). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove unused pods in genericScheduler

**What this PR does / why we need it**: source code cleanup. less confusion.

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```
2017-10-27 14:38:26 -07:00
Kubernetes Submit Queue
3a5eab236f Merge pull request #54538 from WIZARD-CXY/updatehostPort
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

delete the hostport from usedmap

**What this PR does / why we need it**:
delete the hostport record when pod is not on the host

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Facilitate the further pr https://github.com/kubernetes/kubernetes/pull/52421. Because the code which detects the conflict between wantports and existingports is not quite clean now.
Besides remove the unused port from map will save the memory.

**Special notes for your reviewer**:

I and the original coder @k82cn agreed to make this change

**Release note**:

```release-note
NONE
```
2017-10-27 02:18:41 -07:00
Yongkun Anfernee Gui
6ef9cf4936 Remove unused pods in genericScheduler 2017-10-26 15:23:26 -07:00
Kubernetes Submit Queue
32847aab1d Merge pull request #51783 from jiulongzaitian/myfeature3
Automatic merge from submit-queue (batch tested with PRs 54081, 54271, 51783, 54600, 54594). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

move getMaxVols function to predicates.go and add some NewVolumeCount…

…Predicate funcs

Signed-off-by: zhangjie <zhangjie0619@yeah.net>



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
move getMaxVols function to predicates.go and add some NewVolumeCountPredicate funcs
```
2017-10-26 02:43:48 -07:00
zhangjie
968df828a0 move getMaxVols function to predicates.go and change the params of NewMaxPDVolumeCountPredicate funcs
Signed-off-by: zhangjie <zhangjie0619@yeah.net>
2017-10-26 12:11:09 +08:00
chentao1596
222470684c Adding unit tests to methods of 'plugin/pkg/scheduler/algorithm/priorities/util' 2017-10-26 11:22:42 +08:00
kuramal
a48cc26443 Optimize Repeated registration of AlgorithmProvider when ApplyFeatureGates
Add InsertPredicateKeyToAlgorithmProviderMap() and RemovePredicateKeyFromAlgorithmProviderMap() to insert/remove fit predicate key of all algorithmProviders which in algorithmProviderMap
Add Func RemovePredicateKeyFromAlgoProvider() AND InsertPredicateKeyToAlgoProvider() which can insert/remove fit predicate key to specific algorithmProvider
2017-10-26 10:10:48 +08:00
chenxingyu
195de933f1 delete the hostport from usedmap 2017-10-25 15:49:18 +08:00