Commit Graph

93 Commits

Author SHA1 Message Date
Hemant Kumar
45b8107378 Fix volume limit for EBS on m5 and c5 instances 2018-07-19 16:27:52 -04:00
Silvery Fu
2003a0db97 Rework image locality with spread-based scoring 2018-07-11 23:58:23 -07:00
Jeff Grafton
23ceebac22 Run hack/update-bazel.sh 2018-06-22 16:22:57 -07:00
Kubernetes Submit Queue
b45ba959c0
Merge pull request #64693 from xiechengsheng/fix-typos
Automatic merge from submit-queue (batch tested with PRs 65024, 65287, 65345, 64693, 64941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix some typos in code comments.

Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>



**What this PR does / why we need it**:
Fix some typos in code comments.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-06-22 06:10:21 -07:00
xiechengsheng
66e0b53c3c fix some typos
Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>
2018-06-22 09:26:14 +08:00
Kubernetes Submit Queue
1712073165
Merge pull request #63660 from xchapter7x/pkg-scheduler-algorithm-predicates
Automatic merge from submit-queue (batch tested with PRs 64285, 63660, 63661, 63662, 64883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg/scheduler/algorithm/predicates)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-21 01:19:18 -07:00
Kubernetes Submit Queue
e486e8f1f9
Merge pull request #63659 from xchapter7x/pkg-scheduler-algorithm-priorities-util
Automatic merge from submit-queue (batch tested with PRs 65230, 57355, 59174, 63698, 63659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg-scheduler-algorithm-priorities-util)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-19 20:19:25 -07:00
xiechengsheng
cd65afdf05 fix some typos
Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>
2018-06-06 13:11:26 +08:00
John Calabrese
8bbf182695 use subtest for table units
apply subtest to predicates
2018-06-05 09:07:57 -04:00
Silvery Fu
8792f99bf8 Fix the handling of untagged images 2018-06-04 13:57:09 -07:00
Silvery Fu
73970a5027 Update generated bazel 2018-06-04 13:56:51 -07:00
Łukasz Osipiuk
4684d5aac6 Register RequestedToCapacityRatioPriority priority function 2018-06-04 11:30:57 +02:00
Hemant Kumar
a0a9ccfa87 Implement scheduler changes for volume limits 2018-06-01 19:17:31 -04:00
Guoliang Wang
761cf41427 Move pkg/scheduler/schedulercache -> pkg/scheduler/cache 2018-05-31 22:55:34 +08:00
John Calabrese
d1abc5c824 use subtest for table units
employ consitent table var naming

  - https://github.com/kubernetes/kubernetes/pull/63659#discussion_r187736533
2018-05-28 09:33:42 -04:00
Kubernetes Submit Queue
d057795f3b
Merge pull request #64278 from silveryfu/master
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adding tests for ImageLocalityPriority

**What this PR does / why we need it**:

This PR adds tests for ImageLocalityPriority scheduling policy, as follow-ups of [#63842](https://github.com/kubernetes/kubernetes/issues/63842) and [#63345](https://github.com/kubernetes/kubernetes/issues/63345). It includes the unit test for ImageSizes function of NodeInfo in the scheduler cache.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

@resouer 

**Release note**:

```release-note
NONE
```
2018-05-26 12:46:57 -07:00
Mayank Kumar
83818ee0c8 simplify else 2018-05-25 00:38:54 -07:00
Silvery Fu
096dda3768 Rename and add unit test for ImageSizes 2018-05-24 23:29:23 -07:00
Guoliang Wang
9219a76266 remove unused code of (pkg/scheduler) 2018-05-18 23:48:43 +08:00
Kubernetes Submit Queue
9a15ee7035
Merge pull request #63749 from wenlxie/githubupstream.master.minofixforgetstorageclass
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

minor fix for VolumeZoneChecker predicate 

storageclass can be in annotation and spec.

```release-note
minor fix for VolumeZoneChecker predicate, storageclass can be in annotation and spec.
```
2018-05-17 05:51:05 -07:00
wenlxie
38106e7f0b storageclass can be in annotation and spec 2018-05-16 20:49:09 +08:00
Kubernetes Submit Queue
af9531b8a7
Merge pull request #63658 from xchapter7x/pkg-scheduler-algorithm-priorities
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg-scheduler-algorithm-priorities)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-05-15 14:12:39 -07:00
John Calabrese
3a9e988015 use subtest for table units 2018-05-10 07:12:20 -04:00
Kubernetes Submit Queue
aea6addb82
Merge pull request #62934 from wackxu/scto
Automatic merge from submit-queue (batch tested with PRs 62354, 62934, 63502). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Refactor GetResourceRequest and GetResourceLimit

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/assign @bsalamat 

**Release note**:

```release-note
NONE
```
2018-05-09 09:30:11 -07:00
Kubernetes Submit Queue
0949cefd7b
Merge pull request #62453 from k82cn/k8s_61410_2
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Supported nodeSelector.matchFields in scheduler.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
part of  #61410

**Special notes for your reviewer**:

**Release note**:
```release-note
Supported nodeSelector.matchFields (node's `metadata.node`) in scheduler.
```
2018-05-09 02:47:21 -07:00
wackxu
adea3733a8 add SetMaxResource for Resource 2018-05-09 15:33:24 +08:00
Kubernetes Submit Queue
13f3c7c7ca
Merge pull request #63120 from wgliang/master.remove-todo
Automatic merge from submit-queue (batch tested with PRs 58580, 63120). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

-Remove TODO comment of GetNonzeroRequests function

**What this PR does / why we need it**:

-Remove TODO comment of GetNonzeroRequests function

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-05-08 21:45:13 -07:00
Silvery Fu
4087ff32bc Build image size map upon node info updates 2018-05-08 15:02:18 -07:00
Da K. Ma
b1a7f85c9e Supported matchField for NodeAffinity.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
2018-04-28 11:10:29 +08:00
Kubernetes Submit Queue
6b64c07baf
Merge pull request #59735 from wgliang/master.predicates_test
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add test for scheduler:VolumeCountConflicts

**What this PR does / why we need it**:
Add test for scheduler:VolumeCountConflicts

**Special notes for your reviewer**:
2018-04-26 20:41:33 -07:00
Kubernetes Submit Queue
0cf3788419
Merge pull request #63174 from misterikkit/equivHash
Automatic merge from submit-queue (batch tested with PRs 62937, 63105, 63031, 63174). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "Revert "Revert revert of equivalence class hash calculation i…

…n scheduler""

This reverts commit 4386751b5d.



**What this PR does / why we need it**:
This re-introduces the change from https://github.com/kubernetes/kubernetes/pull/58555 which changes how the scheduler computes equivalence classes of pods. I believe we have fixed the flakiness observed previously (https://github.com/kubernetes/kubernetes/issues/61512, https://github.com/kubernetes/kubernetes/issues/62921). I have run the test in question a few dozen times without a failure.

```bash
make test-integration WHAT="./test/integration/scheduler" KUBE_TEST_ARGS="-run TestPreemptionStarvation" GOFLAGS="-v"
```

/ref https://github.com/kubernetes/kubernetes/issues/58222

**Special notes for your reviewer**:
I had to resolve several merge conflicts. I think I resolved them correctly, but keep an eye out for anything silly.

**Release note**:

```release-note
NONE
```
/sig scheduling
2018-04-26 16:40:19 -07:00
Da K. Ma
2c10d15ae5 Do not schedule pod to the node under PID pressure.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
2018-04-26 10:07:42 +08:00
Jonathan Basseri
eace2d08d0 Revert "Revert "Revert revert of equivalence class hash calculation in scheduler""
This reverts commit 4386751b5d.
2018-04-25 16:11:59 -07:00
Kubernetes Submit Queue
e270b8aace
Merge pull request #63067 from k82cn/k8s_63027
Automatic merge from submit-queue (batch tested with PRs 62982, 63075, 63067, 62877, 63141). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Removed e2e test on empty NodeAffinity.

Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #63027 

**Special notes for your reviewer**:
In #62448, we removed the validation on empty `nodeAffinity` which is already handled in scheduler: select no objects.

**Release note**:
```release-note
None
```
2018-04-25 11:53:17 -07:00
Da K. Ma
793ed98715 Added more UT for invalid case.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
2018-04-25 11:14:24 +08:00
Guoliang Wang
e075309b0d -Remove TODO comment of GetNonzeroRequests function 2018-04-25 10:44:43 +08:00
Jesse Haka
de967b717d PR #59323, fix bug and remove one api call, add node util dependency to cloud controller 2018-04-22 20:32:26 +03:00
Kubernetes Submit Queue
98968c36d8
Merge pull request #62715 from bsalamat/fix_antiaffinity
Automatic merge from submit-queue (batch tested with PRs 62761, 62715). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix inter-pod anti-affinity check to consider a pod a match when all the anti-affinity terms match

**What this PR does / why we need it**:
Inter-pod anti-affinity check used to incorrectly consider a pod a match when any of the anti-affinity terms matched the pod. This PR fixes the logic to consider a pod a match when all the terms match the pod.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #62567

**Special notes for your reviewer**:

**Release note**:

```release-note
Fix inter-pod anti-affinity check to consider a pod a match when all the anti-affinity terms match.
```

/sig scheduling
2018-04-17 22:38:13 -07:00
Bobby (Babak) Salamat
eb213e68c2 Fix anti-affinity issue that caused a pod to be considered a match if any of the terms matched (as opposed to all terms matched) 2018-04-16 23:47:46 -07:00
Bobby (Babak) Salamat
4f2155ae22 Addressed reviewer comments 2018-04-16 16:09:24 -07:00
Bobby (Babak) Salamat
c590ec7ae9 Fix an issue in inter-pod affinity predicate that cause affinity to self being processed incorrectly 2018-04-14 19:46:38 -07:00
Bobby (Babak) Salamat
4d7fff1257 Add test to ensure anti-affinity matches against all terms 2018-04-14 19:32:58 -07:00
Kubernetes Submit Queue
3cdf5eecd7
Merge pull request #62211 from bsalamat/affinity_performance
Automatic merge from submit-queue (batch tested with PRs 62467, 62482, 62211). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improve performance of affinity/anti-affinity predicate by 20x in large clusters

**What this PR does / why we need it**:
Improves performance of affinity/anti-affinity predicate by over 20x in large clusters. Performance improvement is smaller in small clusters, but it is still very significant and is about 4x. Also, before this PR, performance of the predicate was dropping quadratically with increasing size of nodes and pods. As the results shows, the slow down is now linear in larger clusters.

Affinity/anti-affinity predicate was checking all pods of the cluster for each node in the cluster to determine feasibility of affinit/anti-affinity terms of the pod being scheduled. This optimization first finds all the pods in a cluster that match the affinity/anti-affinity terms of the pod being scheduled once and stores the metadata. It then only checks the topology of the matching pods for each node in the cluster. 
 This results in major reduction of the search space per node and improves performance significantly. 

Below results are obtained by running scheduler benchmarks:
```
make test-integration WHAT=./test/integration/scheduler_perf KUBE_TEST_ARGS="-run=xxx -bench=.*BenchmarkSchedulingAntiAffinity"
```
```
AntiAffinity Topology: Hostname
before: BenchmarkSchedulingAntiAffinity/500Nodes/250Pods-12         	     	  37031638 ns/op
after:  BenchmarkSchedulingAntiAffinity/500Nodes/250Pods-12         	     	  10373222 ns/op

before: BenchmarkSchedulingAntiAffinity/500Nodes/5000Pods-12        	     	 134205302 ns/op
after:  BenchmarkSchedulingAntiAffinity/500Nodes/5000Pods-12        	     	  12000580 ns/op

befor: BenchmarkSchedulingAntiAffinity/1000Nodes/10000Pods-12         	     	 498439953 ns/op
after: BenchmarkSchedulingAntiAffinity/1000Nodes/10000Pods-12         	     	  24692552 ns/op


AntiAffinity Topology: Region
before: BenchmarkSchedulingAntiAffinity/500Nodes/250Pods-12         	     	  60003672 ns/op
after:  BenchmarkSchedulingAntiAffinity/500Nodes/250Pods-12         	     	  13346400 ns/op

before: BenchmarkSchedulingAntiAffinity/1000Nodes/10000Pods-12         	     	 600085491 ns/op
after: BenchmarkSchedulingAntiAffinity/1000Nodes/10000Pods-12         	     	  27783333 ns/op
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

ref/ #56032 #47318 #25319

**Release note**:

```release-note
improve performance of affinity/anti-affinity predicate of default scheduler significantly.
```

/sig scheduling
2018-04-13 07:25:21 -07:00
Bobby (Babak) Salamat
3041698e52 Addressed reviewer comments 2018-04-12 16:30:08 -07:00
Bobby (Babak) Salamat
418c7502f0 Improve performance of affinity/anti-affinity predicate 2018-04-11 21:17:29 -07:00
Kubernetes Submit Queue
71f150422c
Merge pull request #62180 from msau42/binding-predicate
Automatic merge from submit-queue (batch tested with PRs 61918, 62180, 62198). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use provided node object in volume binding predicate

**What this PR does / why we need it**:
Autoscaler creates fake node objects, so we should use the provided node object instead of looking up the node from the informer.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #62178

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-04-06 11:56:07 -07:00
Kubernetes Submit Queue
6d8df0c400
Merge pull request #61445 from resouer/extender-priority
Automatic merge from submit-queue (batch tested with PRs 62063, 62169, 62155, 62139, 61445). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Schedule even if extender is not available when using extender 

**What this PR does / why we need it**:
When using scheduler extender, if the extender is not available scheduling of all pods fail.
We should let the scheduling happen but display error message that extender is failing.

`IsIgnorable()`  is added to extender to indicate: if scheduling of all pods should fail when it's unavailable

**Backward compabtiility:**

We use `IsIgnorable` instead of `IsCritical` so that when this flag is not set, the default value will be `false`, i.e. not ignorable, which consistent with the current behavior in existing extenders.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes: #60616

**Special notes for your reviewer**:
kindly cc @ravisantoshgudimetla to see if this meets your expectation

TODO: update the examples in kubernetes/examples, but the strategy there is not clear to me for now

**Release note**:

```release-note
Schedule even if extender is not available when using extender 
```
2018-04-05 17:42:13 -07:00
Michelle Au
8d1cd819ec Use provided node object in volume binding predicate 2018-04-05 14:35:55 -07:00
Rohit Agarwal
87dda3375b Delete in-tree support for NVIDIA GPUs.
This removes the alpha Accelerators feature gate which was deprecated in 1.10.
The alternative feature DevicePlugins went beta in 1.10.
2018-04-02 20:17:01 -07:00
Kubernetes Submit Queue
1f69c34478
Merge pull request #61860 from mindprince/kubernetes.io-resources
Automatic merge from submit-queue (batch tested with PRs 60073, 58519, 61860). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Resources prefixed with *kubernetes.io/ should remain unscheduled if they are not exposed on the node.

Currently, resources prefixed with `*kubernetes.io/` get scheduled to any
node whether it's exposing that resource or not.

On the other hand, resources prefixed with `someother.domain/` don't get
scheduled to a node until that node is exposing that resource (or if the
resource is ignored because of scheduler extender).

This commit brings the behavior of `*kubernetes.io/` prefixed resources in
line with other extended resources and they will remain unscheduled
until some node exposes these resources.

Fixes #50658

```release-note
Pods requesting resources prefixed with `*kubernetes.io` will remain unscheduled if there are no nodes exposing that resource.
```

/sig scheduling
/assign jiayingz vishh bsalamat ConnorDoyle k82cn
2018-04-02 17:07:05 -07:00