a305269e18
This implements phase 1 of the proposal in #3579, moving the creation of the pods, RCs, and services to the master after the apiserver is available. This is such a wide commit because our existing initial config story is special: * Add kube-addons service and associated salt configuration: ** We configure /etc/kubernetes/addons to be a directory of objects that are appropriately configured for the current cluster. ** "/etc/init.d/kube-addons start" slurps up everything in that dir. (Most of the difficult is the business logic in salt around getting that directory built at all.) ** We cheat and overlay cluster/addons into saltbase/salt/kube-addons as config files for the kube-addons meta-service. * Change .yaml.in files to salt templates * Rename {setup,teardown}-{monitoring,logging} to {setup,teardown}-{monitoring,logging}-firewall to properly reflect their real purpose now (the purpose of these functions is now ONLY to bring up the firewall rules, and possibly to relay the IP to the user). * Rework GCE {setup,teardown}-{monitoring,logging}-firewall: Both functions were improperly configuring global rules, yet used lifecycles tied to the cluster. Use $NODE_INSTANCE_PREFIX with the rule. The logging rule needed a $NETWORK specifier. The monitoring rule tried gcloud describe first, but given the instancing, this feels like a waste of time now. * Plumb ENABLE_CLUSTER_MONITORING, ENABLE_CLUSTER_LOGGING, ELASTICSEARCH_LOGGING_REPLICAS and DNS_REPLICAS down to the master, since these are needed there now. (Desperately want just a yaml or json file we can share between providers that has all this crap. Maybe #3525 is an answer?) Huge caveats: I've gone pretty firm testing on GCE, including twiddling the env variables and making sure the objects I expect to come up, come up. I've tested that it doesn't break GKE bringup somehow. But I haven't had a chance to test the other providers.
DNS in Kubernetes
SkyDNS can be configured to automatically run in a Kubernetes cluster.
What things get DNS names?
The only objects to which we are assigning DNS names are Services. Every Kubernetes Service is assigned a virtual IP address which is stable as long as the Service exists. This maps well to DNS, which has a long history of clients that, on purpose or on accident, do not respect DNS TTLs.
How do I find the DNS server?
The DNS server itself runs as a Kubernetes Service. This gives it a stable IP
address. When you run the SkyDNS service, you can assign a static IP to use for
the Service. For example, if you assign DNS_SERVER_IP (see below) as
10.0.0.10, you can configure your docker daemon with the flag --dns 10.0.0.10.
Of course, giving services a name is just half of the problem - DNS names need a
domain also. This implementation uses the variable DNS_DOMAIN (see below).
You can configure your docker daemon with the flag --dns-search.
How do I configure it?
The following environment variables are used at cluster startup to create the SkyDNS pods and configure the kubelets. If you need to, you can reconfigure your provider as necessary (e.g. cluster/gce/config-default.sh):
ENABLE_CLUSTER_DNS=true
DNS_SERVER_IP="10.0.0.10"
DNS_DOMAIN="kubernetes.local"
DNS_REPLICAS=1
How does it work?
SkyDNS depends on etcd for what to serve, but it doesn't really need all of
what etcd offers in the way we use it. For simplicty, we run etcd and SkyDNS
together in a pod, and we do not try to link etcd instances across replicas. A
helper container called kube2sky also runs in the pod and acts a bridge
between Kubernetes and SkyDNS. It finds the Kubernetes master through the
kubernetes-ro service, it pulls service info from the master, and it writes
that to etcd for SkyDNS to find.
Known issues
Kubernetes installs do not configure the nodes' resolv.conf files to use the cluster DNS by default, because that process is inherently distro-specific. This should probably be implemented eventually.