Commit Graph

638 Commits

Author SHA1 Message Date
deads2k
518d5500c7 remove testapi.Default.GroupVersion 2016-10-07 10:10:54 -04:00
gmarek
bb71f9473f Small update to scheduler benchmark 2016-10-06 17:26:39 +02:00
Kubernetes Submit Queue
0f40c1a6d0 Merge pull request #34149 from timothysc/prio-hotloop
Automatic merge from submit-queue

Turn down hootloop logs in priorities.  

Excessive spam is output once we near cluster capacity, sometimes a panic is triggered but that data is clipped in the logs see #33935 for more details.
2016-10-06 06:55:00 -07:00
Timothy St. Clair
f2b8b08651 Turn down hootloop logs in priorities. Hit once cluster is near capacity 2016-10-05 16:58:23 -05:00
deads2k
c30b2efc46 update replicaset lister 2016-10-05 15:20:27 -04:00
Kubernetes Submit Queue
7856e46f1b Merge pull request #33967 from xiang90/fix_33772
Automatic merge from submit-queue

scheduler: initialize podsWithAffinity

Without initializing podsWithAffinity, scheduler panics when deleting
a pod from a node that has no pods with affinity ever scheduled to.

Fix #33772
2016-10-05 05:17:51 -07:00
Xiang Li
dc62007478 scheduler: cache.delete deletes the pod from node specified in the cached state 2016-10-05 03:33:53 -07:00
Kubernetes Submit Queue
96fde0fe8d Merge pull request #33834 from wojtek-t/scheduler_map_reduce_6
Automatic merge from submit-queue

Migrate EqualPriority in scheduler to map-reduce-like framework

Ref #24246
2016-10-04 09:23:01 -07:00
Jedrzej Nowak
d2161c21d7 Fix typos and englishify plugin/pkg 2016-10-04 17:51:14 +02:00
Wojciech Tyczynski
f8632e2203 Migrate EqualPriority to MapReduce-like framework. 2016-10-04 12:19:08 +02:00
David Ashpole
0c8a664e50 addressed comments 2016-10-03 11:42:56 -07:00
David Ashpole
fed3f37eef Split NodeDiskPressure into NodeInodePressure and NodeDiskPressure 2016-10-03 11:42:56 -07:00
Lucas Käldström
0bba65ca1a Remove old references to contrib/mesos 2016-10-01 16:46:48 +03:00
Wojciech Tyczynski
daac9a1869 Remove unnecessary podLister from SelectorSpreading 2016-09-30 15:15:14 +02:00
Wojciech Tyczynski
6598c181f7 Support metadata producer with underlying data. 2016-09-28 09:11:18 +02:00
Wojciech Tyczynski
4ccb27202c Migrate priority functions with non-trivial reduce function. 2016-09-27 11:27:38 +02:00
Kubernetes Submit Queue
8b2dfac042 Merge pull request #33489 from liggitt/log-error
Automatic merge from submit-queue

Log test error

We're seeing flakes in this unit test downstream at this point, logging the error would be helpful
2016-09-26 20:34:52 -07:00
Jordan Liggitt
1812fe70a3
Log test error 2016-09-26 11:21:38 -04:00
Wojciech Tyczynski
ff765ed43b Compute priorities in parallel 2016-09-26 14:33:20 +02:00
deads2k
500959b70c fix RC lister 2016-09-23 08:12:03 -04:00
Kubernetes Submit Queue
b2aed32578 Merge pull request #33269 from deads2k/client-15-svc-lister
Automatic merge from submit-queue

simplify svc lister

trying to track down what killed the e2e tests.
2016-09-23 03:10:57 -07:00
Daniel Smith
db3e549bd1 more usages of go2idl 2016-09-22 13:42:46 -07:00
deads2k
483af28944 fix up service lister 2016-09-22 09:12:37 -04:00
Antoine Pelisse
938872582e Revert "simplify RC and SVC listers" 2016-09-21 15:49:38 -07:00
Kubernetes Submit Queue
2d9d84dc64 Merge pull request #32888 from deads2k/client-10-fixup-remaining-listers
Automatic merge from submit-queue

simplify RC and SVC listers

Make the RC and SVC listers use the common list functions that more closely match client APIs, are consistent with other listers, and avoid unnecessary copies.
2016-09-21 04:13:56 -07:00
Ivan Shvedunov
f758cb418d Fix possible panic in PodAffinityChecker 2016-09-20 15:53:13 +03:00
deads2k
16fbb47189 fix up service lister 2016-09-20 08:24:33 -04:00
deads2k
185a7adf84 fix RC lister 2016-09-20 08:24:32 -04:00
Kubernetes Submit Queue
4b5c74eed8 Merge pull request #31859 from resouer/fix-nits
Automatic merge from submit-queue

Fix minor nits in test cases

Found a group of nits when doing #30844, fixed them in a this PR since 30844 requires a long time to review.
2016-09-18 06:27:12 -07:00
Kubernetes Submit Queue
8c48469213 Merge pull request #32353 from smarterclayton/stable_sort
Automatic merge from submit-queue

Doc page for scheduler is not stable (for man pages)

Need to sort maps that are included in help output
2016-09-16 05:44:33 -07:00
Mike Danese
a765d59932 move informer and controller to pkg/client/cache
Signed-off-by: Mike Danese <mikedanese@google.com>
2016-09-15 12:50:08 -07:00
Kubernetes Submit Queue
8c83671789 Merge pull request #32643 from smarterclayton/scheduler_retry
Automatic merge from submit-queue

Exit scheduler retry loop correctly

The error was being eaten and shadowed, which means we would never exit
this loop. This might lead to a goroutine in the scheduler being used
forever without exiting at maximum backoff.

Switched to use the real client to make later refactors easier.

@wojtek-t this may lead to scheduler informer exhaustion - not that familiar with retries
2016-09-15 05:16:16 -07:00
Clayton Coleman
4d79030b8d
Exit scheduler retry loop correctly
The error was being eaten and shadowed, which means we would never exit
this loop. This might lead to a goroutine in the scheduler being used
forever without exiting at maximum backoff.

Switched to use the real client to make later refactors easier.
2016-09-14 00:34:14 -04:00
Wojciech Tyczynski
ea943d825e Migrate a bunch of priority functions to map-reduce framework 2016-09-13 13:20:29 +02:00
Wojciech Tyczynski
c8c938c3e0 AvoidPriority using ControllerRef 2016-09-12 08:45:02 +02:00
Kubernetes Submit Queue
cfe53d14fd Merge pull request #30733 from asalkeld/compat-test
Automatic merge from submit-queue

Update scheduler config file compatibility tests

**What this PR does / why we need it**:
Added missing compatibility tests for scheduler config file options.

**Which issue this PR fixes** 
fixes #30099

**Special notes for your reviewer**:
I came up with the options based on the contents of default.go in each branch.

**Release note**:
```release-note
NONE
```
2016-09-10 03:48:49 -07:00
Clayton Coleman
adc3761bfc
Doc page for scheduler is not stable (for man pages)
Need to sort maps
2016-09-09 01:09:36 -04:00
Angus Salkeld
27f1f854ba Update scheduler config file compatibility tests
Co-Authored-By: Jordan Liggitt <jliggitt@redhat.com>
2016-09-09 09:31:46 +10:00
Harry Zhang
608e7ce0d4 Fix minor nits in test cases 2016-09-01 05:21:17 -04:00
Wojciech Tyczynski
33c710adf0 MapReduce-like scheduler priority functions 2016-08-31 15:16:10 +02:00
Kubernetes Submit Queue
d12b3429fe Merge pull request #30676 from m1093782566/improve-pod-log-output-format
Automatic merge from submit-queue

[Scheduler] Improve pod log output debuggability

Address issue is #30675

The result of my expirments shows that both `glog.Infof("%v", pod)` and `glog.Infof("%+v", pod)` can't output useful information of a pod, it can only output `kind:"" apiVersion:""`. 

`%#v` can output the whole content of pod, but it seems too much!

So, my opinion is output pod info use the format of `%v` to print`pod.Namespace/pod.Name` instead of the pod **object** in both [here](https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/scheduler.go#L96) and [here](https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/scheduler.go#L100).

@wojtek-t
2016-08-25 04:05:24 -07:00
Wojciech Tyczynski
e9d5be628a Don't validate selector that is already validated 2016-08-22 09:39:32 +02:00
Kubernetes Submit Queue
4e39800baa Merge pull request #30992 from mwielgus/cluster-autoscaler-alg
Automatic merge from submit-queue

Cluster autoscaler friendly scheduling algorithm

cc: @davidopp
2016-08-20 18:49:24 -07:00
Kubernetes Submit Queue
248e508225 Merge pull request #30609 from m1093782566/update-comment-info
Automatic merge from submit-queue

update comment info for scheduler binding fails

Since the process logic for scheduler binding failed has changed, I think we should update the comment information to avoid make people confused :)

The related issue is #30611.

@wojtek-t What do you think about it ?

Thanks!
2016-08-19 15:47:33 -07:00
Marcin Wielgus
2df92ff155 Cluster autoscaler friendly scheduling algorithm 2016-08-19 15:20:15 +02:00
Kubernetes Submit Queue
1f3a703df1 Merge pull request #30274 from mwielgus/best-fit
Automatic merge from submit-queue

ClusterAutoscaler-friendly scheduler priority function that promotes well-used nodes

It will help cluster autoscaler to put pods on nodes that are unlikely to be deleted soon due to low usage. Otherwise a pod may be frequently kicked from one node to another. A flag that enables it when CA is on will be added in a separate PR.

Fixes: #28842
2016-08-18 23:49:10 -07:00
Clayton Coleman
12a5eeea17
Introduce GroupVersioner for capturing desired target version
Convert single GV and lists of GVs into an interface that can handle
more complex scenarios (everything internal, nothing supported). Pass
the interface down into conversion.
2016-08-18 14:45:00 -04:00
Kubernetes Submit Queue
214c916045 Merge pull request #30585 from m1093782566/remotve-duplicated-assignment
Automatic merge from submit-queue

remove duplicated nodeinfo.pods assignment

There are duplicated assignments for nodeinfo.pods, one place is [here](https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/schedulercache/node_info.go#L139) and the other one is [here](https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/schedulercache/node_info.go#L147). 

I think we can remove one.

The related issue is #30610
2016-08-18 01:29:20 -07:00
Marcin Wielgus
de2fea95ca ClusterAutoscaler-friendly scheduler priority function that promotes well-used nodes. 2016-08-18 08:34:10 +02:00
m1093782566
40f5d64411 update comment info when scheduler bind fails
Change-Id: Idce89003fe408b713431d07a3300e3acd1af87a9
2016-08-16 20:23:26 +08:00
m1093782566
458f5bd7f2 improve pod log output format
Change-Id: I415dd97f14b507ebb8340eb853f935e1e90b7a53
2016-08-16 17:59:58 +08:00
Kubernetes Submit Queue
69419a145a Merge pull request #29802 from jfrazelle/fix-go-vet-errors
Automatic merge from submit-queue

fix go vet errors

<!--
Checklist for submitting a Pull Request

Please remove this comment block before submitting.

1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
   add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
   to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->

```release-note
```

This fixes the `go vet` errors brought about by go 1.7 testing re (#28742).

The are all pretty trivial and mostly related to literal composites.

also related to #16086
2016-08-15 13:10:08 -07:00
m1093782566
8f607394ee remove duplicated nodeinfo.pods assignment
Change-Id: I8b15624e1cf146ab1693938bbd6ab1c6be030724
2016-08-14 16:43:45 +08:00
Daniel Smith
f1fd638962 fix register.go files up + add test import 2016-08-11 17:06:54 -07:00
Jess Frazelle
7e9d82129e
fix go vet errors
Signed-off-by: Jess Frazelle <jessfraz@google.com>

fix composites

Signed-off-by: Jess Frazelle <me@jessfraz.com>
2016-08-10 16:45:41 -07:00
mksalawa
2749ec7555 Create PredicateFailureReason, modify scheduler predicate interface. 2016-08-09 14:01:46 +02:00
Kubernetes Submit Queue
faffbe4e18 Merge pull request #29622 from rootfs/rbd-ro
Automatic merge from submit-queue

allow a read-only rbd image mounted by multiple pods

allow pod to run read-only rbd volume 
fix #27725
2016-08-07 17:03:39 -07:00
Kubernetes Submit Queue
e19ea41a5e Merge pull request #30136 from m1093782566/master
Automatic merge from submit-queue

remove useless value copy

Copy something to values in parameters won't change them in go. So, remove it to avoid making people confused.
2016-08-07 16:06:23 -07:00
Huamin Chen
730db45eab allow a read-only rbd image mounted by multiple pods
Signed-off-by: Huamin Chen <hchen@redhat.com>
2016-08-07 10:32:26 -04:00
Kubernetes Submit Queue
711b63e78b Merge pull request #28104 from ping035627/ping035627-patch-4
Automatic merge from submit-queue

SchedulerExtender: add failedPredicateMap in Filter() returns

Fix #25797. modify extender.Filter for adding extenders information to “failedPredicateMap” in findNodesThatFit.
When all the filtered nodes that passed "predicateFuncs" don’t pass the extenders filter, the failedPredicateMap hasn’t the extenders information, should add it, I think. So when the length of the “filteredNodes.Items” is 0, we can know the integral information. (The length of the “filteredNodes.Items” is 0, may be because the extenders filter failed.)
2016-08-06 20:50:33 -07:00
m1093782566
91f6b7bc2b remove ineffective copy 2016-08-05 20:43:40 +08:00
Kubernetes Submit Queue
2ff8280123 Merge pull request #29915 from wojtek-t/prepare_for_controller_ref_in_scheduler
Automatic merge from submit-queue

Prepare for using "ControllerRef" in scheduler

This is part of a PR that I already have to avoid a bunch of rebases in the future (controller ref probably won't happen in 1.4 release).

@davidopp
2016-08-04 02:23:31 -07:00
PingWang
4106eb70b0 modify extends.Filter
Signed-off-by: PingWang <wang.ping5@zte.com.cn>

Add docs

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

add docs for ExtenderFilterResult.FailedNodes in types.go

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

Modify the extender.Filter test.

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

Update extender_test.go

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

modify the comments

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

gofmt -s scheduler_interface.go

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update the comments

Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2016-08-04 14:03:32 +08:00
Wojciech Tyczynski
c092e15edf SelectorSpreading using controllerRef. 2016-08-02 16:24:48 +02:00
Wojciech Tyczynski
022719b323 Enable PodAffinity by default in scheduler 2016-08-02 15:06:45 +02:00
Wojciech Tyczynski
4bc410e47a Speedup pod affintiy predicate function 2016-08-02 08:01:04 +02:00
k8s-merge-robot
821ff657f9 Merge pull request #27199 from derekwaynecarr/disk_eviction
Automatic merge from submit-queue

Initial support for pod eviction based on disk

This PR adds the following:

1. node reports disk pressure condition based on configured thresholds
1. scheduler does not place pods on nodes reporting disk pressure
1. kubelet will not admit any pod when it reports disk pressure
1. kubelet ranks pods for eviction when low on disk
1. kubelet evicts greediest pod

Follow-on PRs will need to handle:

1. integrate with new image gc PR (https://github.com/kubernetes/kubernetes/pull/27199)
1. container gc policy should always run (will not be launched from eviction, tbd who does that)
  1. this means kill pod is fine for all eviction code paths since container gc will remove dead container
1. min reclaim support will just poll summary provider (derek will do follow-on)
1. need to know if imagefs is same device as rootfs from summary (derek follow-on)

/cc @vishh @kubernetes/sig-node
2016-07-28 20:18:54 -07:00
derekwaynecarr
0de1e62b30 modify fsStats to fsStatsType to avoid confusion with cadvisor types 2016-07-28 16:01:38 -04:00
derekwaynecarr
9604b47c13 Scheduler does not place pods on nodes that have disk pressure 2016-07-28 16:01:38 -04:00
Wojciech Tyczynski
d3b9d583a2 Optimize PodAffinity priority function. 2016-07-28 16:57:28 +02:00
Wojciech Tyczynski
898a6444e3 Return pointer for Affinity in api helper 2016-07-28 16:57:28 +02:00
Wojciech Tyczynski
a63cccfafc Cache pods with pod (anti)affinity constraints 2016-07-27 17:31:53 +02:00
Wojciech Tyczynski
fad876b6f9 PodAffinity code refinements 2016-07-22 08:49:28 +02:00
Wojciech Tyczynski
fc6d38baa2 Avoid locking when computing predicates. 2016-07-21 08:21:07 +02:00
Wojciech Tyczynski
cab7db3a64 Use []*api.Node instead of NodeLister in priority functions. 2016-07-21 08:21:07 +02:00
mksalawa
fea8d0aebf Fix generic scheduler test 2016-07-20 14:47:44 +02:00
Davanum Srinivas
ee8507a5ae Use Infof/Warningf when appropriate
When we use a format string, we should use Infof/Warningf instead
of Info/Warning
2016-07-19 12:10:53 -04:00
k8s-merge-robot
9a7507c4db Merge pull request #29186 from wojtek-t/cleanup_resource_request
Automatic merge from submit-queue

Reuse existing Resource struct instead of new resourceRequest

@davidopp
2016-07-19 07:31:19 -07:00
k8s-merge-robot
b0bcb8f7b2 Merge pull request #29100 from wojtek-t/pods_can_never_be_scheduled_again
Automatic merge from submit-queue

Fix disappearing pods in scheduler

Fix #29098
2016-07-19 03:26:16 -07:00
Wojciech Tyczynski
dcb5a6d1a6 Reuse existing Resource struct instead of new resourceRequest 2016-07-19 12:21:09 +02:00
Wojciech Tyczynski
66009877bc Fix disappearing pods in scheduler 2016-07-19 08:18:04 +02:00
k8s-merge-robot
a049a97820 Merge pull request #28803 from lukaszo/ds
Automatic merge from submit-queue

Make Daemonset use GeneralPredicates

fixes: #21454 #22205
2016-07-18 22:12:14 -07:00
k8s-merge-robot
fa174bcdaf Merge pull request #29042 from dims/fixup-imports
Automatic merge from submit-queue

Use Go canonical import paths

Add canonical imports only in existing doc.go files.
https://golang.org/doc/go1.4#canonicalimports

Fixes #29014
2016-07-18 07:23:38 -07:00
Wojciech Tyczynski
78101ac55e Fix crashes in schedulercache 2016-07-18 10:04:36 +02:00
Davanum Srinivas
2b0ed014b7 Use Go canonical import paths
Add canonical imports only in existing doc.go files.
https://golang.org/doc/go1.4#canonicalimports

Fixes #29014
2016-07-16 13:48:21 -04:00
k8s-merge-robot
4374b090c3 Merge pull request #28958 from wojtek-t/optimize_priorities_5
Automatic merge from submit-queue

Few more optimizations of priority functions in scheduler

Ref #28590

@davidopp
2016-07-15 09:32:04 -07:00
k8s-merge-robot
8b2d994153 Merge pull request #28952 from wojtek-t/optimize_pod_affinity
Automatic merge from submit-queue

First step of optimizing PodAffinity priority function

Ref #26144 

This is obviously only a first step - I will continue working on this code. However, this is changing the general scheme of computations to what is described in: https://github.com/kubernetes/kubernetes/issues/26144#issuecomment-232612384
2016-07-15 08:59:17 -07:00
Wojciech Tyczynski
a538045d7b Cleanup and prepare for optimizing PodAffinity priority function. 2016-07-15 10:06:36 +02:00
Wojciech Tyczynski
a23f7bd556 Few more optimizations of priority functions in scheduler 2016-07-14 15:32:56 +02:00
k8s-merge-robot
6bb0995c89 Merge pull request #28886 from wojtek-t/fix_schedulercache_bug
Automatic merge from submit-queue

Add ForgetPod to SchedulerCache

Fix #28883

@gmarek @davidopp @xiang90
2016-07-14 05:30:23 -07:00
k8s-merge-robot
74c5b4e4be Merge pull request #28459 from ping035627/ping035627-patch-0704
Automatic merge from submit-queue

Optimise the process of the CalculateSpreadPriority in selector_spreading.go

It had better inspect if the nodeLister is normal first in the CalculateSpreadPriority in selector_spreading.go. If the nodeLister.List return error, the function return directly, not need deal the serviceLister and controllerLister and replicaSetLister.
2016-07-14 04:54:32 -07:00
k8s-merge-robot
ae990defcf Merge pull request #26956 from joe2far/fix-typos
Automatic merge from submit-queue

Fixed several typos
2016-07-14 04:13:15 -07:00
k8s-merge-robot
06939c57aa Merge pull request #20699 from jiangyaoguo/add-AvoidPreviousNode
Automatic merge from submit-queue

Implement alpha version of PreferAvoidPods

This is part of #18853

<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/20699)
<!-- Reviewable:end -->
2016-07-14 03:30:25 -07:00
PingWang
e4ab32cde2 Modify CalculateAntiAffinityPriority function
Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2016-07-14 09:10:50 +08:00
PingWang
437e133b34 Optimise the process of the CalculateSpreadPriority in selector_spreading.go
Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2016-07-14 09:09:05 +08:00
Wojciech Tyczynski
9745878778 Add ForgetPod to SchedulerCache 2016-07-13 16:30:05 +02:00
joe2far
5ead89b5bb Fixed several typos 2016-07-13 15:06:24 +01:00
Łukasz Oleś
528bf7af3a Make Daemonset use GeneralPredicates
fixes #21454, fixes #22205
2016-07-13 14:50:29 +02:00
Wojciech Tyczynski
c929d95884 Cache Allocatable Resources 2016-07-13 12:57:18 +02:00
Wojciech Tyczynski
58c201834c Unify resources 2016-07-13 12:57:07 +02:00