Commit Graph

46 Commits

Author SHA1 Message Date
David McMahon
ef0c9f0c5b Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
James DeFelice
3d3577b9f3 rewrite of static pod json zipper
- add busybox static pod to mesos-docker cluster
- customize static pods with binding annotations
- code cleanup
- removed hacky podtask.And func; support minimal resources for static pods when resource accounting is disabled
- removed zip archive of static pods, changed to gzip of PodList json
- pod utilities moved to package podutil
- added e2e test
- merge watched mirror pods into the mesos pod config stream
2015-11-13 23:30:26 +00:00
Dr. Stefan Schimanski
bb53cc26d3 Reorder scheduler packages
- Scheduler implemenation -> components
- types -> toplevel scheduler package
- integration_test.go -> integration
2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
835961ac43 Make scheduler implementation private 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
ef0a512c15 Move framework into components directory 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
be57bd63e5 Introduce interfaces for all scheduler components 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
29e58bab68 Move scheduler components into their own packages
- rename scheduler.{operations -> components}
- move each component into its own directory with doc.go file, for proper
  access isolation
2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
3601292cff Split MesosScheduler from actual Scheduler code holding the task registry
- rename types.Framework -> types.Scheduler
- rename MesosScheduler -> Framework
- transform MesosFramework into independent Scheduler glueing all scheduler
  components, implementing types.Scheduler and owning the task registry
2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
ae4673e86e Split binder, deleter, podScheduler initialion from NewSchedulerLoop 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
0ebfc02d16 Separate SchedulerLoop instantiation from MesosScheduler 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
2c4142494a Turn plugin into a SchedulerLoop and move to operations directory 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
e5ce6eccf9 Rename schedulerApiAlgorithmAdapter -> mesosSchedulerAlgorithm, move to algorithm.go 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
0bd1666d9b Move reconciler.go into operations 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
23fa56adb1 Move etcd code from MesosScheduler into service 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
d4675f1dc3 Rename mesos_{scheduler ->}.go, {mesos ->}framework.go 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
518a7d5f3b Rename KubernetesMesosScheduler -> MesosScheduler 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
8e3c648ca0 Rename schedcfg -> schedulerConfig 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
beaaf81ee3 Move fcfs and types to scheduler.algorithm module 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
b600e6c497 Rename KubernetesMesosScheduler.{Scheduler -> PodScheduler} 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
01a97ebc14 Rename KubernetesScheduler -> KubernetesMesosScheduler 2015-11-12 01:33:50 +01:00
Dr. Stefan Schimanski
4f13ba1af6 Revert "Merge pull request #15976 from mesosphere/sur-k8sm-475-error-checking"
The given merge will be rebased manually and appended to the scheduler refactoring.

This reverts commit 8d923afe23, reversing
changes made to d7458ddd4c.
2015-11-12 01:33:50 +01:00
Sergiusz Urbaniak
6a16fe314f scheduler: remove err param from Register 2015-11-10 14:39:57 +01:00
Daniel Smith
241551e36e set-gen example 2015-11-03 15:14:51 -08:00
Steve Milner
8da2d4002f Fixes remaining imports per #15319 2015-10-20 07:13:13 -04:00
Ravi L R
99b03baa9b handle task lost status update from executor on docker daemon crash/restart by deleting the pod, so that the RC creates a new one, if the pod is backed by RC 2015-10-15 12:03:50 -07:00
Ravi L R
0e296c3dcd while deleting pod on terminal task status update, delete with grace_period=0 as the corresponding pod is already gone 2015-10-15 12:03:50 -07:00
Yu-Ju Hong
a3e60cc32e Rename imported package local name kubeletTypes to kubetypes
According to the naming guidelines, package name should not include mixedCaps.
2015-10-09 10:24:31 -07:00
Yu-Ju Hong
098ab05997 kubelet: move common types to kubelet/types
This would faciliate tasks such as moving code in pkg/kubelet to sub packages.
2015-10-08 14:38:01 -07:00
Dr. Stefan Schimanski
39942952cd Fix hostname check logic in scheduler 2015-10-01 06:50:43 +02:00
Dr. Stefan Schimanski
4d4ebe9f18 Add Mesos slave attributes as node labels
- pre-create node api objects from the scheduler when offers arrive
- decline offers until nodes a registered
- turn slave attributes as k8s.mesosphere.io/attribute-* labels
- update labels from executor Register/Reregister
- watch nodes in scheduler to make non-Mesos labels available for NodeSelector matching
- add unit tests for label predicate
- add e2e test to check that slave attributes really end up as node labels
2015-09-26 09:46:56 +02:00
Dr. Stefan Schimanski
132f8892ed Move task-lost message prefix into messages module 2015-09-21 09:51:05 +02:00
k8s-merge-robot
a204f31516 Merge pull request #13918 from mesosphere/reconcile-ignore-mirror-pods
Auto commit by PR queue bot
2015-09-17 02:36:31 -07:00
Dr. Stefan Schimanski
eb5a5ffc28 Extract slave hostname registry code in its own module
- remove bleeding of registry-internal objects, without any locking
- rename from SlaveStorage to Registry which fits much better to what
  it actually does
2015-09-16 14:50:31 +02:00
James DeFelice
a5307305f1 avoid recovery for mirrored pods on failover 2015-09-14 20:21:19 +00:00
tummychow
78ce5da988 Move util.StringSet into its own package
A lot of packages use StringSet, but they don't use anything else from
the util package. Moving StringSet into another package will shrink
their dependency trees significantly.
2015-09-10 12:04:15 -07:00
James DeFelice
a1cea8dd87 Flexible resource accounting and pod resource containment:
- new: introduce AllocationStrategy, Predicate, and Procurement to scheduler pkg
- new: --contain-pod-resources flag (workaround for docker+systemd+mesos problems)
- new: --account-for-pod-resources flag (for testing overcommitment)
- bugfix: forward -v flag from minion controller to executor
2015-09-04 00:49:13 +00:00
James DeFelice
d1589b9571 task reconciliation must look at NamespaceAll, not NamespaceDefault 2015-09-03 18:01:45 +00:00
Karl Isenberg
158c42f4be Log Mesos task message on status update
- Log error when task state is unknown
2015-08-24 15:31:31 -07:00
Dr. Stefan Schimanski
f9635d5778 [mesos] Handle TASK_ERROR
- The TASK_ERROR task status was introduced with Mesos 0.21 and is actually used since 0.22.
  It was not handled at all before this patch, leaving errored task in the registry in phase
  "Pending". This will lead to task status updates from the Mesos Master on reconciliation with empty
  slaveId fields, leading to scheduler crashes eventually.
- Handle terminal task with empty slaveId.
  The slave id can be empty for TASK_ERROR.
  The modified code path does not use the slaveId.
2015-08-24 15:31:31 -07:00
Kris Rousey
ae6c64d9bb Moving everyone to unversioned client 2015-08-18 10:23:03 -07:00
Dr. Stefan Schimanski
f1a560718c Make slave assignment before binding persistent
- move assigned slave to T.Spec.AssignedSlave
- only create the BindingHost annoation in prepareTaskForLaunch
- recover the assigned slave from annotation and write it back to the T.Spec field

Before this patch the annotation were used to store the assign slave. But due
to the cloning of tasks in the registry, this value was never persisted in the
registry.

This patch adds it to the Spec of a task and only creates the annotation
last-minute before launching.

Without this patch pods which fail before binding will stay in the registry,
but they are never rescheduled again. The reason: the BindingHost annotation does
not exist in the registry and not on the apiserver (compare reconcilePod function).
2015-08-12 08:03:36 +02:00
Mike Danese
17defc7383 run gofmt on everything we touched 2015-08-05 17:52:56 -07:00
Mike Danese
8e33cbfa28 rewrite go imports 2015-08-05 17:30:03 -07:00
Dr. Stefan Schimanski
a2fa41b73f Implement resource accounting for pods with the Mesos scheduler
This patch

- set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec
  (these are also passed to the kubelet such that it uses them for the docker
  run limits)
- sums up the container resource limits for cpu and memory inside a pod,
- compares the sums to the offered resources
- puts the sums into the Mesos TaskInfo such that Mesos does the accounting
  for the pod.
- parses the static pod spec and adds up the resources
- sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources
- sets the cgroups in the kubelet for system containers, resource containers
  and docker to the one of the executor that Mesos assigned
- adds scheduler parameters --default-container-cpu-limit and
  --default-container-mem-limit.

The containers themselves are resource limited the Docker resource limit which
the kubelet applies when launching them.

Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
2015-07-30 21:18:04 +02:00
Wojciech Tyczynski
ee92aa3897 Prepare for extracting EtcdHelper interface 2015-07-23 09:37:39 +02:00
James DeFelice
932c58a497 Kubernetes Mesos integration
This commit includes the fundamental components of the Kubernetes Mesos
integration:

* Kubernetes-Mesos scheduler
* Kubernetes-Mesos executor
* Supporting libs

Dependencies and upstream changes are included in a separate commit for easy
review.

After this initial upstream, there'll be two PRs following.

* km (hypercube) and k8sm-controller-manager #9265
* Static pods support #9077

Fixes applied:

- Precise metrics subsystems definitions
  -  mesosphere/kubernetes-mesos#331
  - https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion_r31875232
  - https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion_r31875240
- Improve comments and add clarifications
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875208
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875226
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875227
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875228
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875239
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875243
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875234
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875256
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875255
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875251
- Clarify which Schedule function is actually called
  - Fixes https://github.com/GoogleCloudPlatform/kubernetes/pull/8882#discussion-diff-31875246
2015-06-10 20:58:39 +00:00