Automatic merge from submit-queue (batch tested with PRs 51682, 51546, 51369, 50924, 51827)
kubeadm: Detect kubelet readiness and error out if the kubelet is unhealthy
**What this PR does / why we need it**:
In order to improve the UX when the kubelet is unhealthy or stopped, or whatever, kubeadm now polls the kubelet's API after 40 and 60 seconds, and then performs an exponential backoff for a total of 155 seconds.
If the kubelet endpoint is not returning `ok` by then, kubeadm gives up and exits.
This will miligate at least 60% of our "[apiclient] Created API client, waiting for control plane to come up" issues in the kubeadm issue tracker 🎉, as kubeadm now informs the user what's wrong and also doesn't deadlock like before.
Demo:
```
lucas@THEGOPHER:~/luxas/kubernetes$ sudo ./kubeadm init --skip-preflight-checks
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.7.4
[init] Using Authorization modes: [Node RBAC]
[preflight] Skipping pre-flight checks
[kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0)
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [thegopher kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.115]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
[init] This often takes around a minute; or longer if the control plane images have to be pulled.
[apiclient] All control plane components are healthy after 40.502199 seconds
[markmaster] Will mark node thegopher as master by adding a label and a taint
[markmaster] Master thegopher tainted and labelled with key/value: node-role.kubernetes.io/master=""
[bootstraptoken] Using token: 5776d5.91e7ed14f9e274df
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[addons] Applied essential addon: kube-dns
[addons] Applied essential addon: kube-proxy
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run (as a regular user):
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token 5776d5.91e7ed14f9e274df 192.168.1.115:6443 --discovery-token-ca-cert-hash sha256:6f301ce8c3f5f6558090b2c3599d26d6fc94ffa3c3565ffac952f4f0c7a9b2a9
lucas@THEGOPHER:~/luxas/kubernetes$ sudo ./kubeadm reset
[preflight] Running pre-flight checks
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
[reset] Removing kubernetes-managed containers
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/etcd]
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
lucas@THEGOPHER:~/luxas/kubernetes$ sudo systemctl stop kubelet
lucas@THEGOPHER:~/luxas/kubernetes$ sudo ./kubeadm init --skip-preflight-checks
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.7.4
[init] Using Authorization modes: [Node RBAC]
[preflight] Skipping pre-flight checks
[kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0)
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [thegopher kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.115]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
[init] This often takes around a minute; or longer if the control plane images have to be pulled.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz/syncloop' failed with error: Get http://localhost:10255/healthz/syncloop: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz/syncloop' failed with error: Get http://localhost:10255/healthz/syncloop: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz/syncloop' failed with error: Get http://localhost:10255/healthz/syncloop: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz/syncloop' failed with error: Get http://localhost:10255/healthz/syncloop: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10255/healthz' failed with error: Get http://localhost:10255/healthz: dial tcp 127.0.0.1:10255: getsockopt: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by that:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
- There is no internet connection; so the kubelet can't pull the following control plane images:
- gcr.io/google_containers/kube-apiserver-amd64:v1.7.4
- gcr.io/google_containers/kube-controller-manager-amd64:v1.7.4
- gcr.io/google_containers/kube-scheduler-amd64:v1.7.4
You can troubleshoot this for example with the following commands if you're on a systemd-powered system:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
couldn't initialize a Kubernetes cluster
```
In this demo, I'm first starting kubeadm normally and everything works as usual.
In the second case, I'm explicitely stopping the kubelet so it doesn't run, and skipping preflight checks, so that kubeadm doesn't even try to exec `systemctl start kubelet` like it does usually.
That obviously results in a non-working system, but now kubeadm tells the user what's the problem instead of waiting forever.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes: https://github.com/kubernetes/kubeadm/issues/377
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: Detect kubelet readiness and error out if the kubelet is unhealthy
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @pipejakob
cc @justinsb @kris-nova @lukemarsden as well as you wanted this feature :)
We allow a kubeadm user to use an external CA by checking to see if ca.key is missing and skipping cert checks and kubeconfig generation if ca.key is missing.
Automatic merge from submit-queue (batch tested with PRs 49861, 50933, 51380, 50688, 51305)
Add configurable groups to bootstrap tokens.
**What this PR does / why we need it**:
This change adds support for authenticating bootstrap tokens into a configurable set of extra groups in addition to `system:bootstrappers`. Previously, bootstrap tokens could only ever authenticate to the `system:bootstrappers` group.
Groups are specified as a comma-separated list in the `auth-extra-groups` key of the `bootstrap.kubernetes.io/token` Secret, and must begin with the prefix `system:bootstrapper:` (and match a validation regex that checks against our normal convention). Whether or not any extra groups are configured, `system:bootstrappers` will still be added.
This also adds a `--groups` flag for `kubeadm token create`, which sets the `auth-extra-groups` key on the resulting Secret. The default is to not set the key.
`kubeadm token list` is also updated to include a `EXTRA GROUPS` output column.
**Which issue this PR fixes**: fixes#49306
**Special notes for your reviewer**:
The use case for this is in https://github.com/kubernetes/kubernetes/issues/49306. Comments on the feature itself are probably better over there. It will be part of how HA/self-hosting kubeadm bootstraps new master nodes (post 1.8).
**Release note**:
```release-note
Add support for configurable groups for bootstrap token authentication.
```
cc @luxas @kubernetes/sig-cluster-lifecycle-api-reviews @kubernetes/sig-auth-api-reviews
/kind feature
Automatic merge from submit-queue
kubeadm: Implement 'kubeadm config'
**What this PR does / why we need it**:
Implements a `kubeadm config` command for viewing the current kubeadm configuration stored as a ConfigMap in the cluster and creating that configuration for v1.7- users. kubeadm v1.8+ handles the creation of this ConfigMap at init time, but v1.7 users have to create it themselves with this command in order to be able to preserve the same config after the upgrade.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes: https://github.com/kubernetes/kubeadm/issues/406
**Special notes for your reviewer**:
**Release note**:
```release-note
Adds a new `kubeadm config` command that lets users tell `kubeadm upgrade` what kubeadm configuration to use and lets users view the current state.
```
@kubernetes/sig-cluster-lifecycle-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 51039, 50512, 50546, 50965, 50467)
kubeadm: Get kube-dns based on the kubernetes version
**What this PR does / why we need it**:
Makes the kube-dns version used dependent on the kubernetes version. This is required for upgrades as we have to be able to handle one kube-dns version per branch for instance...
Currently a no-op though, as both v1.7 and v1.8 seem to use 1.14.4
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Dependency for https://github.com/kubernetes/kubernetes/pull/48899 (kubeadm upgrades)
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews
@kubernetes/dns-maintainers FYI; next time you bump DNS version, please update this func instead of the constant there...
Automatic merge from submit-queue (batch tested with PRs 50967, 50505, 50706, 51033, 51028)
kubeadm: Tell the user when a static pod is created
**What this PR does / why we need it**:
Prints a line to notify the user of the static pod creation in order to be consistent with the other phases (one line per phase and optionally per component).
Now the phase command `controlplane all` and `etcd local` also actually outputs something.
Also renamed `[token]` to `[bootstraptoken]` to match the output below and `s/mode/modes/`
`kubeadm init` output now:
```console
$ ./kubeadm init
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.7.4
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks
[preflight] WARNING: docker service is not enabled, please run 'systemctl enable docker.service'
[preflight] Starting the kubelet service
[kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0)
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [thegopher kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.115]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
[apiclient] All control plane components are healthy after 40.002026 seconds
[markmaster] Master thegopher tainted and labelled with key/value: node-role.kubernetes.io/master=""
[bootstraptoken] Using token: cfe65e.d196614967c3ffe3
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[addons] Applied essential addon: kube-dns
[addons] Applied essential addon: kube-proxy
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run (as a regular user):
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token cfe65e.d196614967c3ffe3 192.168.1.115:6443 --discovery-token-ca-cert-hash sha256:eb3461b9b707eafc214577f36ae8c351bbc4d595ab928fc84caf1325b69cb192
$ ./kubeadm alpha phase controlplane all
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
$ ./kubeadm alpha phase etcd local
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @fabriziopandini
Automatic merge from submit-queue (batch tested with PRs 47896, 50678, 50620, 50631, 51005)
kubeadm: Adds dry-run support for kubeadm using the `--dry-run` option
**What this PR does / why we need it**:
Adds dry-run support to kubeadm by creating a fake clientset that can get totally fake values (like in the init case), or delegate GETs/LISTs to a real API server but discard all edits like POST/PUT/PATCH/DELETE
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes: https://github.com/kubernetes/kubeadm/issues/389
**Special notes for your reviewer**:
This PR depends on https://github.com/kubernetes/kubernetes/pull/50626, first three commits are from there
This PR is a dependency for https://github.com/kubernetes/kubernetes/pull/48899 (kubeadm upgrades)
I have some small things to fixup and I'll yet write unit tests, but PTAL if you think this is going in the right direction
**Release note**:
```release-note
kubeadm: Adds dry-run support for kubeadm using the `--dry-run` option
```
cc @kubernetes/sig-cluster-lifecycle-pr-reviews @kubernetes/sig-api-machinery-pr-reviews
Previously, kubeadm would use <ip>:<port> to construct a master
endpoint. This works fine for IPv4 addresses, but not for IPv6.
IPv6 requires the ip to be encased in brackets when being joined
to a port with a colon.
This patch updates kubeadm to support wrapping a v6 address with
[] to form the master endpoint url. Since this functionality is
needed in multiple areas, a dedicated util function was created.
Fixes: https://github.com/kubernetes/kubernetes/issues/48227
Automatic merge from submit-queue (batch tested with PRs 50626, 50683, 50679, 50684, 50460)
kubeadm: Centralize client create-or-update logic in one package
**What this PR does / why we need it**:
Moves all Create-or-Update logic into one package instead of duplicating that logic all around in the codebase.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
This PR depends on https://github.com/kubernetes/kubernetes/pull/50214.
Note that commit 2 is the only one that needs reviewing.
This PR is required for https://github.com/kubernetes/kubernetes/pull/48899 (kubeadm upgrade)
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @mattmoyer @fabriziopandini
Automatic merge from submit-queue
kubeadm: Implementing the controlplane phase
**What this PR does / why we need it:**
This contains implementation of controlplane phases in kubeadm, which is part of the wider effort of implementing phases in kubeadm, previously in alpha stage.
The original proposal for this activity can be found [here](https://github.com/kubernetes/kubeadm/pull/156/files) and related comments.
Kubeadm phase implementation checklist is defined [here](https://github.com/kubernetes/kubeadm/issues/267)
Common implementation guidelines and principles for all phases are defined [here](https://docs.google.com/document/d/1VQMyFIVMfRGQPP3oCUpfjiWtOr3pLxp4g7cP-hXQFXc/edit?usp=sharing)
This PR implements:
- [x] kubeadm phase controlplane: wip by @fabriziopandini
- [x] kubeadm phase controlplane all
- [x] kubeadm phase controlplane etcd
- [x] kubeadm phase controlplane apiserver
- [x] kubeadm phase controlplane scheduler
- [x] kubeadm phase controlplane controller-manager
**Which issue this PR fixes:**
https://github.com/kubernetes/kubeadm/issues/349
**Special notes for your reviewer:**
This PR implements the same approach of #49419, thus minimising rework/impacts on existing codebase.
This change adds the `--discovery-token-ca-cert-hash` and `--discovery-token-unsafe-skip-ca-verification` flags for `kubeadm join` and corresponding fields on the kubeadm NodeConfiguration struct. These flags configure enhanced TLS validation for token-based discovery.
The enhanced TLS validation works by pinning the public key hashes of the cluster CA. This is done by connecting to the `cluster-info` endpoint initially using an unvalidated/unsafe TLS connection. After the cluster info has been loaded, parsed, and validated with the existing symmetric signature/MAC scheme, the root CA is validated against the pinned public key set. A second request is made using validated/safe TLS using the newly-known CA and the result is validated to make sure the same `cluster-info` was returned from both requests.
This validation prevents a class of attacks where a leaked bootstrap token (such as from a compromised worker node) allows an attacker to impersonate the API server.
This change also update `kubeadm init` to print the correct `--discovery-token-ca-cert-hash` flag in the example `kubeadm join` command it prints at the end of initialization.
Automatic merge from submit-queue (batch tested with PRs 48572, 48838, 48931, 48783, 47090)
kubeadm: add a warning about the default token TTL changing in 1.8
**What this PR does / why we need it**:
This adds a warning to `kubeadm init` and `kubeadm token create` if they are run without the `--token-ttl` / `--ttl` flags. In 1.7 and before, the tokens generated by these commands defaulted to an infinite TTL (no expiration) in 1.8, they will generate a token with a 24 hour TTL.
The actual default change is in https://github.com/kubernetes/kubernetes/pull/48783. This change is separate so we can cherry pick the warning into the `release-1.7` branch.
**Which issue this PR fixes**: ref https://github.com/kubernetes/kubeadm/issues/343
**Special notes for your reviewer**:
This change is blocked on https://github.com/kubernetes/kubeadm/issues/343. These warnings should probably be removed in the 1.9 cycle.
**Release note**:
```release-note
Add a runtime warning about the kubeadm default token TTL changes in 1.8.
```
/assign @luxas
This adds a warning to `kubeadm init` and `kubeadm token create` if they are run without the `--token-ttl` / `--ttl` flags. In 1.7 and before, the tokens generated by these commands defaulted to an infinite TTL (no expiration) in 1.8, they will generate a token with a 24 hour TTL.
The actual default change is in https://github.com/kubernetes/kubernetes/pull/48783. This change is separate so we can cherry pick the warning into the release-1.7 branch.
This code was changed in ea196490a0 to validate that `--config` wasn't passed along with other flags. Unfortunately, the implementation was checking `PersistentFlags()`, which was not parsed at the point it was being validated. The result was that `--config` was _always_ rejected, not just when it was specified alongside other flags.
The fix is to use `Flags()` instead, which contains the expected data.
Allow to specify a node-name instead of relaying in `os.Hostname()`
This is useful where kubelet use the name given by the cloud-provider to
register the node.
Partially fix: kubernetes/kubeadm#64
Automatic merge from submit-queue
kubeadm: If `--config` is set, don't allow any other option as it won't have effect
If use config in kubeadm init, cann't get other values from other arguments.
`kubeadm init --config=../kubeadm.config --token 447ad3.96cda76e3206fca0 --apiserver-bind-port 6445`
So I think we need to allow get values from command and is prior than cofig file.
Automatic merge from submit-queue
kubeadm: Enable the Node Authorizer/Admission plugin in v1.7
**What this PR does / why we need it**:
This is similar to https://github.com/kubernetes/kubernetes/pull/46796, but for kubeadm.
Basically it was a part of https://github.com/kubernetes/kubernetes/pull/46796, but there were some other upgradability and compability concerns for kubeadm I took care of while working today.
Example:
```console
$ kubeadm init --kubernetes-version v1.7.0-beta.0
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.7.0-beta.0
[init] Using Authorization mode: [RBAC Node]
...
$ sudo kubectl --kubeconfig=/etc/kubernetes/kubelet.conf get secret foo
Error from server (Forbidden): User "system:node:thegopher" cannot get secrets in the namespace "default".: "no path found to object" (get secrets foo)
$ echo '{"apiVersion":"v1","kind":"Node","metadata":{"name":"foo"}}' | sudo kubectl create -f - --kubeconfig=/etc/kubernetes/kubelet.conf
Error from server (Forbidden): error when creating "STDIN": nodes "foo" is forbidden: node thegopher cannot modify node foo
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Depends on https://github.com/kubernetes/kubernetes/pull/46864 (uses that PR as a base, will rebase once it's merged)
Please only review the second commit. Will also fix tests in a minute.
**Release note**:
```release-note
kubeadm: Enable the Node Authorizer/Admission plugin in v1.7
```
@mikedanese @liggitt @pipejakob @roberthbailey @jbeda @timothysc
Automatic merge from submit-queue
kubeadm: Remove an outdated comment
Now that `AdvertiseAddress` is a `string` and not
`AdvertiseAddresses` a `[]string` this comment is no longer
necessary.
@k8s-mirror-cluster-lifecycle-misc RFR
**What this PR does / why we need it**
Just a little house cleaning by removing an outdated comment.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42940, 42906, 42970, 42848)
Improve kubeadm init message
Now that we are locking down the insecure port, we should give clearer instructions on how to copy out the root owned admin.conf file, chmod it and use it.
Signed-off-by: Joe Beda <joe.github@bedafamily.com>
```release-note
NONE
```
Now that we are locking down the insecure port, we should give clearer instructions on how to copy out the root owned admin.conf file, chmod it and use it.
Signed-off-by: Joe Beda <joe.github@bedafamily.com>
Automatic merge from submit-queue (batch tested with PRs 41701, 41818, 41897, 41119, 41562)
kubeadm: Secure the control plane communication and add the kubeconfig phase command
**What this PR does / why we need it**:
This generates kubeconfig files for the controller-manager and the scheduler, ref: https://github.com/kubernetes/kubeadm/issues/172
The second commit adds the `kubeadm alpha phase kubeconfig` command as described in the design doc: https://github.com/kubernetes/kubeadm/pull/156
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
@dmmcquay What kind of tests would you like for the kubeconfig phase command?
**Release note**:
```release-note
```
@jbeda @mikedanese @dmmcquay @pires @liggitt @deads2k @errordeveloper
Automatic merge from submit-queue (batch tested with PRs 41857, 41864, 40522, 41835, 41991)
kubeadm: Demote --self-hosted to master config file.
**What this PR does / why we need it**:
kubeadm init --self-hosted was meant to be a short lived hack to enable self-hosted deployments until we're ready to make them the default. Rather than shipping this in 1.6 (for the first time) we will move this to the config file as it is presently only an advanced feature, leaving us with more well supported ways to remove it in the future.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
CC @luxas @pires @errordeveloper @dmmcquay
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41540, 41808, 41710, 41838, 41840)
kubeadm: Remove the --cloud-provider flag for beta init UX
**What this PR does / why we need it**:
We decided the `--cloud-provider` flag promises way too much compared to what it really does. There is a lot you have to do as an user in order to make the current cloud provider integrations to work. And since we're promising to support the `kubeadm init` UX on a beta level in v1.6, we can't have this flag in the UX. A lot is gonna change here... see proposal: https://github.com/kubernetes/community/pull/128
Once we find a cloudprovider solution we can support (probably using the new flow), we'll just add it.
For now, we'll just document how to do cloudprovider integrations by hand instead.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: Remove the --cloud-provider flag for beta init UX
```
@jbeda @dmmcquay @mikedanese @roberthbailey @pires @errordeveloper
- Factor out token constants to kubeadmconstants.
- Move cmd/kubeadm/app/util/{,token/}tokens.go
- Use the token-id, token-secret, etc constants provided by the bootstrapapi package
- Move cmd/kubeadm/app/master/tokens.go to cmd/kubeadm/app/phases/token/csv.go
This refactor basically makes it possible to hook up kubeadm to the BootstrapSigner controller later on
Some imports dont exist yet (or so it seems) in client-go (examples
being:
- "k8s.io/kubernetes/pkg/api/validation"
- "k8s.io/kubernetes/pkg/util/initsystem"
- "k8s.io/kubernetes/pkg/util/node"
one change in kubelet to import to client-go
Adds kubeadm subcommands to create, list, and delete bootstrap tokens.
Tokens can be created with a TTL duration, or 0 for tokens that will not
expire. The create command can also be used to specify your own token
(for use when bootstrapping masters and nodes in parallel), or update an
existing token's secret or ttl.
Marked "ex" for experimental for now as the boostrap controllers are not
yet hooked up in core.
Fix boilerplates, comments in the code and make the output of kubeadm more user-friendly
Start using HostPKIPath and KubernetesDir everywhere in the code, so they can be changed for real
More robust kubeadm reset code now.
Removed old glog-things from app.Run()
Renamed /etc/kubernetes/cloud-config.json to /etc/kubernetes/cloud-config since it shouldn't be a json file
Simplification of the code
Less verbose output from master/pki.go
Cleaned up dead code
Start a small logging/output framework:
- fmt.Println("[the-stage-here] Capital first letter of this message. Tell the user what the current state is")
- fmt.Printf("[the-stage-here] Capital first letter. Maybe a [%v] in the end if an error should be displayed. Always ends with \n")
- fmt.Errorf("Never starts with []. Includes a short error message plus the underlying error in [%v]. Never ends with \n")
Hardcoded known stable version will be returned if user
didn't request specific version and kubeadm for some reason
not able to fetch latest stable information from release servers.
For now, fallback version is v1.4.6
Now, defaults can be pointing to "stable" and users will always get
latest available stable build of Kubernetes via kubeadm.
There is no need anymore to hardcode version string inside kubeadm
binary.
It is also possible to use labels like "latest" or point to exact
branch: "stable-1.4"
kubeadm during initialization of master and slave nodes need to make
several API calls directly to the node where it is running or master.
In environments with http/https proxies, user might accidentally
have configuration where connections to API would go over proxy instead
of directly.
User can re-run kubeadm with corrected NO_PROXY variable. Example:
$ NO_PROXY=* kubeadm join ...
Automatic merge from submit-queue
kubeadm implement preflight checks
Checks that user running kubeamd init and join is root and will only execute
command if user is root. Moved away from using kubectl error handling to
having kubeadm handle its own errors. This should allow kubeadm to have
more meaningful errors, exit codes, and logging for specific kubeadm use
cases.
fixes#33908
Add skip-preflight-checks to known flags.
Fix bug with preflight checks not returning system is-active as errors.
Fix error handling to use correct function.
Includes checks for verifying services exist and are enabled, ports are
open, directories do not exist or are empty, and required binaries are
in the path.
Checks that user running kubeamd init and join is root and will only execute
command if user is root. Moved away from using kubectl error handling to
having kubeadm handle its own errors. This should allow kubeadm to have
more meaningful errors, exit codes, and logging for specific kubeadm use
cases.
Includes checks for verifying services exist and are enabled, ports are
open, directories do not exist or are empty, and required binaries are
in the path.
