Automatic merge from submit-queue (batch tested with PRs 59220, 62927, 63084, 63090, 62284). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: add better test coverage to token.go
**What this PR does / why we need it**:
a PR for adding some more tests in `kubeadm/cmd` for `token.go`.
some areas of the `token.go` like listing, creating and deleting tokens can present challenges.
coverage was increased to around 87%.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
please, link issue # if you know of such.
**Special notes for your reviewer**:
none
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Make kube-proxy tolerate all taints
**What this PR does / why we need it**:
As a essential core component, kube-proxy should generally run on all
nodes even if the cluster operator taints nodes for special purposes.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#699
**Release note**:
```release-note
kubeadm creates kube-proxy with a toleration to run on all nodes, no matter the taint.
```
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Mount additional paths inside apiserver/controller-manager for working CA root
This is required for a working CA root, as /etc/ssl/certs on a few
Linux distributions just contains a bunch of symlinks.
Container Linux and Debian have symlinks pointing to
/usr/share/ca-certificates, ArchLinux has symlinks pointing
to /etc/ca-certificates.
On Debian /etc/ssl/certs can also include symlinks pointing
to /usr/local/share/ca-certificates for local CA certificates.
Fix: kubeadm/#671
---
**What this PR does / why we need it**:
Without this PR, `controller-manager` and `apiserver` would lack a CA root on some Linux distro (ex: Container Linux) which for example break flexplugins which require a CA root [[1]](https://github.com/kubernetes-incubator/external-storage/issues/571#issuecomment-360155462).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/671
**Special notes for your reviewer**:
**Release note**:
```release-note
Mount additional paths required for a working CA root, for setups where /etc/ssl/certs doesn't contains certificates but just symlink.
```
/sig sig-kubeadm
Automatic merge from submit-queue (batch tested with PRs 62655, 61711, 59122, 62853, 62390). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Modify the kubeadm upgrade DAG for the TLS Upgrade
**What this PR does / why we need it**:
This adds the necessary utilities to detect Etcd TLS on static pods from the file system and query Etcd.
It modifies the upgrade logic to make it support the APIServer downtime.
Tests are included and should be passing.
```bash
bazel test //cmd/kubeadm/... \
&& bazel build //cmd/kubeadm --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 \
&& issue=TLSUpgrade ~/Repos/vagrant-kubeadm-testing/copy_kubeadm_bin.sh
```
These cases are working consistently for me
```bash
kubeadm-1.9.6 reset \
&& kubeadm-1.9.6 init --kubernetes-version 1.9.1 \
&& kubectl apply -f https://git.io/weave-kube-1.6
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.9.6 # non-TLS to TLS
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.10.0 # TLS to TLS
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.10.1 # TLS to TLS
/vagrant/bin/TLSUpgrade_kubeadm upgrade apply 1.9.1 # TLS to TLS /w major version downgrade
```
This branch is based on top of #61942, as resolving the hash race condition is necessary for consistent behavior.
It looks to fit in pretty well with @craigtracey's PR: #62141
The interfaces are pretty similar
/assign @detiber @timothysc
**Which issue(s) this PR fixes**
Helps with https://github.com/kubernetes/kubeadm/issues/740
**Special notes for your reviewer**:
278b322a1c
[kubeadm] Implement ReadStaticPodFromDisk
c74b56372d
Implement etcdutils with Cluster.HasTLS()
- Test HasTLS()
- Instrument throughout upgrade plan and apply
- Update plan_test and apply_test to use new fake Cluster interfaces
- Add descriptions to upgrade range test
- Support KubernetesDir and EtcdDataDir in upgrade tests
- Cover etcdUpgrade in upgrade tests
- Cover upcoming TLSUpgrade in upgrade tests
8d8e5fe33b
Update test-case, fix nil-pointer bug, and improve error message
97117fa873
Modify the kubeadm upgrade DAG for the TLS Upgrade
- Calculate `beforePodHashMap` before the etcd upgrade in anticipation of
KubeAPIServer downtime
- Detect if pre-upgrade etcd static pod cluster `HasTLS()==false` to switch
on the Etcd TLS Upgrade if TLS Upgrade:
- Skip L7 Etcd check (could implement a waiter for this)
- Skip data rollback on etcd upgrade failure due to lack of L7 check
(APIServer is already down unable to serve new requests)
- On APIServer upgrade failure, also rollback the etcd manifest to
maintain protocol compatibility
- Add logging
**Release note**:
```release-note
kubeadm upgrade no longer races leading to unexpected upgrade behavior on pod restarts
kubeadm upgrade now successfully upgrades etcd and the controlplane to use TLS
kubeadm upgrade now supports external etcd setups
kubeadm upgrade can now rollback and restore etcd after an upgrade failure
```
Fix `rollbackEtcdData()` to return error=nil on success
`rollbackEtcdData()` used to always return an error making the rest of the
upgrade code completely unreachable.
Ignore errors from `rollbackOldManifests()` during the rollback since it
always returns an error.
Success of the rollback is gated with etcd L7 healthchecks.
Remove logic implying the etcd manifest should be rolled back when
`upgradeComponent()` fails
- Calculate `beforePodHashMap` before the etcd upgrade in anticipation of KubeAPIServer downtime
- Detect if pre-upgrade etcd static pod cluster `HasTLS()==false` to switch on the Etcd TLS Upgrade
if TLS Upgrade:
- Skip L7 Etcd check (could implement a waiter for this)
- Skip data rollback on etcd upgrade failure due to lack of L7 check (APIServer is already down unable to serve new requests)
- On APIServer upgrade failure, also rollback the etcd manifest to maintain protocol compatibility
- Add logging
- Test HasTLS()
- Instrument throughout upgrade plan and apply
- Update plan_test and apply_test to use new fake Cluster interfaces
- Add descriptions to upgrade range test
- Support KubernetesDir and EtcdDataDir in upgrade tests
- Cover etcdUpgrade in upgrade tests
- Cover upcoming TLSUpgrade in upgrade tests
Automatic merge from submit-queue (batch tested with PRs 62464, 62947). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
make API.ControlPlaneEndpoint accept IP
**What this PR does / why we need it**:
This PR implements one of the actions defined by https://github.com/kubernetes/kubeadm/issues/751 (checklist form implementing HA in kubeadm).
With this PR, the `API.ControlPlaneEndpoint` value in the kubeadm MasterConfiguration file now accepts both DNS and IP.
The `API.ControlPlaneEndpoint` should be used to set a stable IP address for the control plane; in an HA configuration, this should be the load balancer address (no matter if identified by a DNS name or by a stable IP).
**Special notes for your reviewer**:
/CC @timothysc
This PR is the same of https://github.com/kubernetes/kubernetes/pull/62667, that I closed by error 😥
**Release note**:
```release-note
NONE
```
Nb. first https://github.com/kubernetes/kubernetes/pull/62667 already has the release note
- Update kubeadm static pod upgrades to use the
kubetypes.ConfigHashAnnotationKey annotation on the mirror pod rather
than generating a hash from the full object info. Previously, a status
update for the pod would allow the upgrade to proceed before the
new static pod manifest was actually deployed.
Signed-off-by: Jason DeTiberus <detiber@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 62726, 60085, 62583). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor kubeadm api validation.
**What this PR does / why we need it**:
This PR refactor kubeadm api validation to use field json name. When users get a validation error, they can easily know which field is invalid in their config files.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
@fabriziopandini I remember we mentioned this refactoring in some comment. I didn't see this change was done, so I send this PR to address this.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62781, 62815). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update kubeadm upgrade message
**What this PR does / why we need it**:
Updates kubeadm upgrade message per user request
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubeadm/issues/672
**Special notes for your reviewer**:
none
**Release note**:
```
NONE
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews @kargakis
Automatic merge from submit-queue (batch tested with PRs 62481, 62643, 61877, 62515). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Not validating front proxy CA Key when using External CA.
**What this PR does / why we need it**:
"That the front ca key is not required as the front proxy client tls keypair can be managed by the third party." This PR don't validate the front CA Key but check if it already exists.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/752
**Special notes for your reviewer**:
@yanndegat @timothysc @stealthybox @fabriziopandini
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62481, 62643, 61877, 62515). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm preflight: check CRI socket path if defined or docker service otherwise
**What this PR does / why we need it**:
Currently, running `kubeadm init` without Docker installed will cause the `Service-Docker` preflight check to fail even when another CRI is installed and the CRI socket specified. This changes the preflight checks to check the CRI socket if specified, and falling back to checking the Docker service otherwise. Additionally, this deduplicates common checks between `kubeadm init` and `kubeadm join` to ensure that similar preflight checks stay in-sync going forward.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # https://github.com/kubernetes/kubeadm/issues/657 - it still has the same behavior on joins
Fixes # https://github.com/kubernetes/kubeadm/issues/749 - will check the CRI socket if specified and skip the Docker service check
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm preflight: check CRI socket path if defined, otherwise check for Docker
```
This is required for a working CA root, as /etc/ssl/certs on a few
Linux distributions just contains a bunch of symlinks.
Container Linux and Debian have symlinks pointing to
/usr/share/ca-certificates, ArchLinux has symlinks pointing
to /etc/ca-certificates.
On Debian /etc/ssl/certs can also include symlinks pointing
to /usr/local/share/ca-certificates for local CA certificates.
Fix: kubeadm/#671
Automatic merge from submit-queue (batch tested with PRs 62568, 62220, 62743, 62751, 62753). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubeadm upgrade same version
What this PR does / why we need it:
When kubeadm 1.10 came out, it inadvertently introduced a backwards incompatible config change. Because the kubeadm MasterConfiguration is written by the old version of kubeadm and read by the new one, this incompatibility causes the upgrade to fail.
To mitigate this, I've written a simple transform that operates on a map-based version of the config. This map is mutated to make it compatible with the new structure, then serialised to JSON and deserialised by the usual APIMachinery.
Because of complications with the multiple versions, this PR enforces kubeadm only being used to upgrade to kubernetes of the same minor and major versions.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes [kubeadm#744](https://github.com/kubernetes/kubeadm/issues/744#issuecomment-379045823L)
This PR is an alternate take on #62353. Instead of trying to gate migration on versions, this constrains kubeadm to only upgrade versions from the same major and minor versions.
Special notes for your reviewer:
```release-note
fixes configuration error when upgrading kubeadm from 1.9 to 1.10+
enforces kubeadm upgrading kubernetes from the same major and minor versions as the kubeadm binary.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add phase command for dynamic kubelet configuration in kubeadm.
**What this PR does / why we need it**:
As the title says.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
partially fixes this: https://github.com/kubernetes/kubeadm/issues/571
**Special notes for your reviewer**:
/cc @luxas @fabriziopandini
@kubernetes/sig-cluster-lifecycle-pr-reviews
**Release note**:
```release-note
Phase `kubeadm alpha phase kubelet` is added to support dynamic kubelet configuration in kubeadm.
```
Automatic merge from submit-queue (batch tested with PRs 56040, 62627). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support groups (organizations) to be specified in client cert.
**What this PR does / why we need it**:
Provide a flag `--org` for `kubeadm alpha phase kubeconfig user` to support groups (organizations) to be specified in client cert.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/753
**Special notes for your reviewer**:
cc @TomRK1089
**Release note**:
```release-note
`kubeadm alpha phase kubeconfig user` supports groups (organizations) to be specified in client cert.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm/phases: small grammar improvements
**What this PR does / why we need it**:
small grammar fixes and cleanup in `kubeadm/phases`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
none
**Release note**:
```release-note
NONE
```
Lubomir (VMware)
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: add better test coverage to version.go
**What this PR does / why we need it**:
This PR adds unit tests for the file `kubeadm/app/cmd/version.go` via `version_test.go`. Test coverage is increased to 90% for this file. only a couple of functions are tested.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
None
**Special notes for your reviewer**:
related PRs:
https://github.com/kubernetes/kubernetes/pull/59220https://github.com/kubernetes/kubernetes/pull/58540
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58540, 62314). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add ability to specify port for kubeadm `API.ControlPlaneEndpoint`
**What this PR does / why we need it**:
When `API.ControlPlaneEndpoint` is used, the `BindPort` of the
apiserver is currently assumed, which means a load balancer cannot
listen on a different port than the apiserver. This extends the
`ControlPlaneEndpoint` to take an optional port which may differ
from the apiserver's `BindPort`.
**Release note**:
```release-note
The kubeadm config option `API.ControlPlaneEndpoint` has been extended to take an optional port which may differ from the apiserver's bind port.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: add better test coverage to reset.go
**What this PR does / why we need it**:
a PR for adding some more tests in `kubeadm/cmd` for `reset.go`.
increasing coverage from ~11% to ~92%
the remaining 8% are kind of hard to test or not worth the complexity.
original idea to add more tests in kubeadm was suggested to me by @luxas at slack.
in time, i might add better coverage to all cmd `.go` files.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
please, link issue # if you know of such.
**Special notes for your reviewer**:
none
**Release note**:
```release-note
NONE
```
When `API.ControlPlaneEndpoint` is used, the `BindPort` of the
apiserver is currently assumed, which means a load balancer cannot
listen on a different port than the apiserver. This extends the
`ControlPlaneEndpoint` to take an optional port which may differ
from the apiserver's `BindPort`.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove invalid TODOs in kubeadm constants.
**What this PR does / why we need it**:
These TODOs are invalid now.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref https://github.com/kubernetes/kubeadm/issues/571#issuecomment-347408384
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
As a essential core component, kube-proxy should generally run on all
nodes even if the cluster operator taints nodes for special purposes.
This fixeskubernetes/kubeadm#699
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes kubeadm upgrade plan output
Closeskubernetes/kubeadm#713
Aligns `kubeadm upgrade plan` output when kubelets have different versions.
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR fixes the output of `kubeadm upgrade plan` when kubelets have different versions.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#713
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58178, 62491, 60852). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[kubeadm] Add support for clusterName in config file.
**What this PR does / why we need it**: Adds a `--cluster-name` arg to kubeadm init.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
See https://github.com/kubernetes/kube-deploy/issues/636
Code inspired by #52470
**Special notes for your reviewer**:
**Release note**:
```release-note
Adds --cluster-name to kubeadm init for specifying the cluster name in kubeconfig.
```
Automatic merge from submit-queue (batch tested with PRs 62455, 62465, 62427, 62416, 62411). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix kubeadm upgrade ignores HighAvailability feature gate
**What this PR does / why we need it**:
This PR fixes https://github.com/kubernetes/kubeadm/issues/731
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubeadm/issues/731
**Special notes for your reviewer**:
The problem is a regression introduced by [#55952](https://github.com/kubernetes/kubernetes/pull/55952) which added a --feature-gates
flag to `kubeadm upgrade`, to be used for updating feature gates during the upgrade process (so it is update + upgrade).
The original implementation always override the actual feature gates with the flag value, even when the flag is not set; this PR fixes this behaviour making `kubeadm upgrade` considers the flag value only if set, otherwise actual feature gates will be preserved.
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews @kubernetes/sig-cluster-lifecycle-bugs
**Release note**:
```release-note
Fixed#731 kubeadm upgrade ignores HighAvailability feature gate
```
