Automatic merge from submit-queue (batch tested with PRs 63011, 68089, 67944, 68132). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Kubeadm upload and fetch of kubeam config v1alpha3
**What this PR does / why we need it**:
This PR implements upload and fetch of kubeam config v1alpha3 from cluster.
More in detail:
In upload, `kubeadm-config` gets
- `ClusterConfiguration` (without components config which are already stored in separated ConfigMaps)
- `ClusterStatus`(initialised or updated with the API endpoint of the current node)
During fetch `InitConfiguration` is composed with:
- `ClusterConfiguration` from `kubeadm-config`
- The `APIEndpoint` of the current node from `ClusterStatus` in `kubeadm-config`
- Component configs from corresponding ConfigMaps
**Which issue(s) this PR fixes** :
refs https://github.com/kubernetes/kubeadm/issues/911, refs https://github.com/kubernetes/kubeadm/issues/963
**Special notes for your reviewer**:
In order to implement this it was necessary to extend current component config management with a new GetFromConfigMap operation. This is implemented in a separated commit "
implement component configs GetFromConfigMap".
The real change build on this (commi "upload and fetch kubeadm v1alpha3")
**Release note**:
```release-note
NONE
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
/sig cluster-lifecycle
/area kubeadm
/kind enhancement
/assign @luxas
/assign @timothysc
/cc @chuckha @rosti @neolit123 @liztio
Automatic merge from submit-queue (batch tested with PRs 67397, 68019). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
kubeadm: fix offline and air-gapped support
**What this PR does / why we need it**:
1.
Change the error output of getAllDefaultRoutes() so that it includes
information on which files were probed for the IP routing tables
even if such files are obvious.
Introduce a new error type which can be used to figure out of this
error is exactly of the "no routes" type.
2.
If netutil.ChooseBindAddress() fails looking up IP route tables
it will fail with an error in which case the kubeadm config
code will hard stop.
This scenario is possible if the Linux user intentionally disables
the WiFi from the distribution settings. In such a case the distro
could empty files such files as /proc/net/route and ChooseBindAddress()
will return an error.
For improved offline support, don't error on such scenarios but instead
show a warning. This is done by using the NoRoutesError type.
Also default the address to 0.0.0.0.
While doing that, prevent some commands like `init`, `join` and also
phases like `controlplane` and `certs` from using such an invalid
address.
3.
If there is no internet, label versions fail and this breaks
air-gapped setups unless the users pass an explicit version.
To work around that:
- Remain using 'release/stable-x.xx' as the default version.
- On timeout or any error different from status 404 return error
- On status 404 fallback to using the version of the client via
kubeadmVersion()
Add unit tests for kubeadmVersion().
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
refs kubernetes/kubeadm#1041
**Special notes for your reviewer**:
1st and second commits fix offline support.
3rd commit fixes air-gabbed support (as discussed in the linked issue)
the api-machinery change is only fmt.Errorf() related.
**Release note**:
```release-note
kubeadm: fix air-gapped support and also allow some kubeadm commands to work without an available networking interface
```
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
/cc @kubernetes/sig-api-machinery-pr-reviews
/assign @kad
/assign @xiangpengzhao
/area UX
/area kubeadm
/kind bug
1) Do not fail in case a bind address cannot be obtained
If netutil.ChooseBindAddress() fails looking up IP route tables
it will fail with an error in which case the kubeadm config
code will hard stop.
This scenario is possible if the Linux user intentionally disables
the WiFi from the distribution settings. In such a case the distro
could empty files such files as /proc/net/route and ChooseBindAddress()
will return an error.
For improved offline support, don't error on such scenarios but instead
show a warning. This is done by using the NoRoutesError type.
Also default the address to 0.0.0.0.
While doing that, prevent some commands like `init`, `join` and also
phases like `controlplane` and `certs` from using such an invalid
address.
Add unit tests for the new function for address verification.
2) Fallback to local client version
If there is no internet, label versions fail and this breaks
air-gapped setups unless the users pass an explicit version.
To work around that:
- Remain using 'release/stable-x.xx' as the default version.
- On timeout or any error different from status 404 return error
- On status 404 fallback to using the version of the client via
kubeadmVersion()
Add unit tests for kubeadmVersion().
Co-authored-by: Alexander Kanevskiy <alexander.kanevskiy@intel.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm - set peer urls for default etcd instance
**What this PR does / why we need it**:
Override the default peer URLs for the default etcd instance. Previously we left the defaults, which meant the peer URL was unsecured previously.
**Release note**:
```release-note
kubeadm - Ensure the peer port is secured by explicitly setting the peer URLs for the default etcd instance.
kubeadm - Ensure that the etcd certificates are generated using a proper CN
kubeadm - Update generated etcd peer certificate to include localhost addresses for the default configuration.
kubeadm - Increase the manifest update timeout to make upgrades a bit more reliable.
```
- Set peer urls for default etcd instance to avoid leaving peer port unsecured
- Update generated etcd peer certificate SANs to include localhost
- Use a proper CN for etcd server and peer certificates
- Increase the manifest update timeout
Some code in kubeadm was designed with the intent, that in the future CoreDNS
and kube-dns versions will match to specific K8s versions. This code is not
functional, since it does not perform any version matching. As of this moment,
no version matching is planned and a lot of boilerplate code is left useless.
The solution is simple - remove the unneeded parts to simplify the flow.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
When kubeadm upgrades a static pod cluster, the old manifests were previously
deleted. This patch alters this behaviour so they are now stored in a
timestamped temporary directory.
Automatic merge from submit-queue (batch tested with PRs 63969, 63902, 63689, 63973, 63978). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Remove dependency on pkg/client/clientset_generated/internalclientset
**What this PR does / why we need it**:
This PR removes dependency in kubeadm on `pkg/client/clientset_generated/internalclientset` and blacklists it in `.import-restrictions`.
The above package is pulled in transitively by the usage of the following:
- [GetFlagString](a481f4bbe8/cmd/kubeadm/app/cmd/version.go (L63)) from `k8s.io/kubernetes/pkg/kubectl/cmd/util`
- [LeaseEndpointReconcilerType](a481f4bbe8/cmd/kubeadm/app/phases/controlplane/manifests.go (L205)) from `k8s.io/kubernetes/pkg/master/reconcilers`
Dropping the unneeded dependencies from the build shaves off 4-5MB from the amd64 executable and reduces the dependency hell in kubeadm.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#806
**Special notes for your reviewer**:
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
/area kubeadm
/assign @luxas
/assign @timothysc
**Release note**:
```release-note
NONE
```
kubeadm uses LeaseEndpointReconcilerType as import from
k8s.io/kubernetes/pkg/master/reconcilers. However, this pull a huge
load of extra dependencies (among which
pkg/client/clientset_generated/internalclientset). The solution is
to copy this string constant locally in kubeadm.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove invalid TODOs in kubeadm constants.
**What this PR does / why we need it**:
These TODOs are invalid now.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref https://github.com/kubernetes/kubeadm/issues/571#issuecomment-347408384
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm 710 Switch to a dedicated CA for kubeadm etcd identities
**What this PR does / why we need it**:
On `kubeadm init`/`kubeadm upgrade`, this PR generates an etcd specific CA for signing the following certs:
- etcd serving cert
- etcd peer cert
- apiserver etcd client cert
These certs were previously signed by the kubernetes CA.
The etcd static pod in `local.go` has also been updated to only mount the `/etcd` subdir of `cfg.CertificatesDir`.
New phase command:
```
kubeadm alpha phase certs etcd-ca
```
See the linked issue for details on why this change is an important security feature.
**Which issue(s) this PR fixes**
Fixes https://github.com/kubernetes/kubeadm/issues/710
**Special notes for your reviewer**:
#### on the master
this should still fail:
```bash
curl localhost:2379/v2/keys # no output
curl --cacert /etc/kubernetes/pki/etcd/ca.crt https://localhost:2379/v2/keys # handshake error
```
this should now fail: (previously would succeed)
```
cd /etc/kubernetes/pki
curl --cacert etcd/ca.crt --cert apiserver-kubelet-client.crt --key apiserver-kubelet-client.key https://localhost:2379/v2/keys
# curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate
```
this should still succeed:
```
cd /etc/kubernetes/pki
curl --cacert etcd/ca.crt --cert apiserver-etcd-client.crt --key apiserver-etcd-client.key https://localhost:2379/v2/keys
```
**Release note**:
```release-note
On cluster provision or upgrade, kubeadm generates an etcd specific CA for all etcd related certificates.
```
- Place etcd server and peer certs & keys into pki subdir
- Move certs.altName functions to pkiutil + add appendSANstoAltNames()
Share the append logic for the getAltName functions as suggested by
@jamiehannaford.
Move functions/tests to certs/pkiutil as suggested by @luxas.
Update Bazel BUILD deps
- Warn when an APIServerCertSANs or EtcdCertSANs entry is unusable
- Add MasterConfiguration.EtcdPeerCertSANs
- Move EtcdServerCertSANs and EtcdPeerCertSANs under MasterConfiguration.Etcd
- Generate Server and Peer cert for etcd
- Generate Client cert for apiserver
- Add flags / hostMounts for etcd static pod
- Add flags / hostMounts for apiserver static pod
- Generate certs on upgrade of static-pods for etcd/kube-apiserver
- Modify logic for appending etcd flags to staticpod to be safer for external etcd
Audit logs are configurable via the MasterConfiguration file.
All options are ignored unless the FeatureGate is enabled.
Fixeskubernetes/kubeadm#623
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Automatic merge from submit-queue (batch tested with PRs 57287, 57233, 57305). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove kube-proxy 1.8 configmap and daemonset manifests in kubeadm
**What this PR does / why we need it**:
Remove unsupported manifests of kube-proxy in kubeadm of 1.10.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use kube-proxy ComponentConfig in kubeadm clusters
This change adds configuring the kube-proxy bind address to be an
IPv6 address based on the whether the API server advertise address is IPv6.
It is doing this via the kube-proxy ComponentConfig API now from v1.9
**What this PR does / why we need it**:
This PR sets the bind address for kube-proxy to be a IPv6 address. This is needed for IPv6
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#50927
Fixes https://github.com/kubernetes/kubeadm/issues/527
**Special notes for your reviewer**:
**Release note**:
```release-note
Adds kubeadm support for using ComponentConfig for the kube-proxy
```
Automatic merge from submit-queue (batch tested with PRs 55952, 49112, 55450, 56178, 56151). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support to kubeadm upgrade for CoreDNS
**What this PR does / why we need it**:
This PR enables to get CoreDNS in the kubeadm upgrade and alpha phases addons via feature-gates.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/446
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: Add CoreDNS support for kubeadm "upgrade" and "alpha phases addons".
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Write marshalled kubeletconfig object to init-config-dir
**What this PR does / why we need it**:
from @luxas :
>Write the the marshalled kubeletconfig object to /var/lib/kubelet/config/init/kubelet so that the kubelet will start up with the right params on init/join. The only params expected in the kubelet command-line after this is kubelet --init-config-dir /var/lib/kubelet/config/init --dynamic-config-dir /var/lib/kubelet/config/dynamic
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref: https://github.com/kubernetes/kubeadm/issues/28#issuecomment-345502933
**Special notes for your reviewer**:
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51192, 55010). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding etcd upgrade option to kubeadm upgrade apply
This PR adds etcd upgrade functionality to kubeadm upgrade apply.
First commit adds certain functions to be able to deal with a single component of control plane and not just with all three components (apiserver, controller-manager and scheduler). It adds granularity as a result code can be reused.
Closes: https://github.com/kubernetes/kubeadm/issues/490
```release-note
Adds to **kubeadm upgrade apply**, a new **--etcd-upgrade** keyword. When this keyword is specified, etcd's static pod gets upgraded to the etcd version officially recommended for a target kubernetes release.
```
Automatic merge from submit-queue (batch tested with PRs 53204, 53364, 53559, 53589, 53088). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
New version number for kubeadm constants.go
**What this PR does / why we need it**:
In kubeadm v1.9 the minimum kubelet & API Server version will be v1.8.0.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a label which prevents a node from being added to a cloud load balancer
There are a variety of reasons that you may not want a node in a cluster to participate in a cloud load balancer. For example workload isolation for security, or managing network throughput, or because the node is not in the appropriate virtual network (cluster's that span environments)
This PR adds a label so that you can select which nodes you want to participate.
Automatic merge from submit-queue (batch tested with PRs 46458, 50934, 50766, 50970, 47698)
kubeadm: Make the self-hosting with certificates in Secrets mode work again
**What this PR does / why we need it**:
This PR:
- makes the self-hosting with certificates in Secrets mode work
- makes the wait functions timeoutable
- fixes a race condition where the kubelet may be slow to remove the Static Pod
- cleans up some of the self-hosting logic
- makes self-hosting-with-secrets respect the feature flag
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes: https://github.com/kubernetes/kubeadm/issues/405
**Special notes for your reviewer**:
This is work in progress. I'll add unit tests, rebase upon https://github.com/kubernetes/kubernetes/pull/50762 and maybe split out some of the functionatlity here into a separate PR
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 48572, 48838, 48931, 48783, 47090)
kubeadm: change the default bootstrap token TTL to 24 hours
**What this PR does / why we need it**:
This PR changes the TTL for the default bootstrap token generated by `kubeadm init` (without the `--token-ttl` parameter) and `kubeadm token create` (without the `--ttl` flag). Previously, the default TTL was infinite. After this change it is 24 hours.
~~The reasoning for 2 hours as a default is that it's 1) long enough that someone manually using kubeadm (copy-pasting) shouldn't have any issues and 2) short enough that if something is going to break, it should break while the user/admin is still paying attention to the cluster. I'm open to bikeshedding about the exact value, 2 hours is a bit of a strawman.~~
**Edit: updated this to 24 hours instead of 2 hours.**
This is a breaking change if you rely on infinite TTL tokens (e.g., if you had an ASG group of worker nodes). The old behavior is easily restored by passing `--token-ttl 0` to `kubeadm init` or the `--ttl 0` flag to `kubeadm token create`.
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubeadm/issues/343
**Special notes for your reviewer**:
This was discussed earlier today in SIG-cluster-lifecycle
**Release note**:
```release-note
Change the default kubeadm bootstrap token TTL from infinite to 24 hours. This is a breaking change. If you require the old behavior, use `kubeadm init --token-ttl 0` / `kubeadm token create --ttl 0`.
```
cc @jbeda
Automatic merge from submit-queue
Add initializer support to admission and uninitialized filtering to rest storage
Initializers are the opposite of finalizers - they allow API clients to react to object creation and populate fields prior to other clients seeing them.
High level description:
1. Add `metadata.initializers` field to all objects
2. By default, filter objects with > 0 initializers from LIST and WATCH to preserve legacy client behavior (known as partially-initialized objects)
3. Add an admission controller that populates .initializer values per type, and denies mutation of initializers except by certain privilege levels (you must have the `initialize` verb on a resource)
4. Allow partially-initialized objects to be viewed via LIST and WATCH for initializer types
5. When creating objects, the object is "held" by the server until the initializers list is empty
6. Allow some creators to bypass initialization (set initializers to `[]`), or to have the result returned immediately when the object is created.
The code here should be backwards compatible for all clients because they do not see partially initialized objects unless they GET the resource directly. The watch cache makes checking for partially initialized objects cheap. Some reflectors may need to change to ask for partially-initialized objects.
```release-note
Kubernetes resources, when the `Initializers` admission controller is enabled, can be initialized (defaulting or other additive functions) by other agents in the system prior to those resources being visible to other clients. An initialized resource is not visible to clients unless they request (for get, list, or watch) to see uninitialized resources with the `?includeUninitialized=true` query parameter. Once the initializers have completed the resource is then visible. Clients must have the the ability to perform the `initialize` action on a resource in order to modify it prior to initialization being completed.
```
Automatic merge from submit-queue (batch tested with PRs 41954, 40528, 41875, 41165, 41877)
preflight check external etcd version when kubeadm init
**What this PR does / why we need it**:
1. preflight check if verson of external etcd server meets the demand of kubeadm, currently requires >= 3.0.14
2. support mixed http endpoints and https endpoints
**Which issue this PR fixes** : fixes https://github.com/kubernetes/kubeadm/issues/174
**Special notes for your reviewer**:
i have tested against single endpoint including http etcd server , https etcd server, but multiple endpoints not tested yet. i'll do it tomorrow
Automatic merge from submit-queue (batch tested with PRs 41701, 41818, 41897, 41119, 41562)
kubeadm: Secure the control plane communication and add the kubeconfig phase command
**What this PR does / why we need it**:
This generates kubeconfig files for the controller-manager and the scheduler, ref: https://github.com/kubernetes/kubeadm/issues/172
The second commit adds the `kubeadm alpha phase kubeconfig` command as described in the design doc: https://github.com/kubernetes/kubeadm/pull/156
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
@dmmcquay What kind of tests would you like for the kubeconfig phase command?
**Release note**:
```release-note
```
@jbeda @mikedanese @dmmcquay @pires @liggitt @deads2k @errordeveloper
Automatic merge from submit-queue (batch tested with PRs 38101, 41431, 39606, 41569, 41509)
kubeadm: Reorder the token packages more logically
**What this PR does / why we need it**:
In order to be able to implement https://github.com/kubernetes/kubernetes/pull/41417, the token functionality (which now is spread across the codebase), should be in two places: a generic token functions library, which in the future _may_ [move into client-go](https://github.com/kubernetes/kubernetes/pull/41281#discussion_r101357106) in some form, and a package for the token handling against the api server.
This commit has no large functional changes.
```
kubeadm: Aggregate the token functionality in sane packages.
- Factor out token constants to kubeadmconstants.
- Move cmd/kubeadm/app/util/{,token/}tokens.go
- Use the token-id, token-secret, etc constants provided by the bootstrapapi package
- Move cmd/kubeadm/app/master/tokens.go to cmd/kubeadm/app/phases/token/csv.go
This refactor basically makes it possible to hook up kubeadm to the BootstrapSigner controller later on
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@mikedanese @pires @errordeveloper @dmmcquay @jbeda @GheRivero
Automatic merge from submit-queue (batch tested with PRs 41505, 41484, 41544, 41514, 41022)
add front proxy to kubeadm created kube-apiservers
The front proxy authenticator configuration has been in a release or two. It allows a front proxy (secured by mutual TLS auth) to provide user information for a request. The kube-aggregator uses this to securely terminate authentication (has to terminate TLS and thus client-certs) and communicate user info to backing API servers.
Since the kube-apiserver always verifies the front-proxy via a client certificate, this isn't open for abuse unless you already have access to either the signing key or client cert which kubeadm creates locally. If you got there, you already owned the box. Therefore, this adds the authenticator unconditionally.
@luxas Are there e2e tests for `kubeadm`?
@liggitt @kubernetes/sig-auth-misc
- Factor out token constants to kubeadmconstants.
- Move cmd/kubeadm/app/util/{,token/}tokens.go
- Use the token-id, token-secret, etc constants provided by the bootstrapapi package
- Move cmd/kubeadm/app/master/tokens.go to cmd/kubeadm/app/phases/token/csv.go
This refactor basically makes it possible to hook up kubeadm to the BootstrapSigner controller later on
Automatic merge from submit-queue (batch tested with PRs 41466, 41456, 41550, 41238, 41416)
add check to authorization config
Prompt user to create the config when using abac/webhook.
