For file discovery, in case the user feeds a file for the CA
from the kubeconfig, make sure it's preloaded and embedded using
the new function EnsureCertificateAuthorityIsEmbedded().
This commit also applies cleanup:
- unroll validateKubeConfig() into ValidateConfigInfo() as this way
the default cluster can be re-used.
- in ValidateConfigInfo() reuse the variable config instead of creating
a new variable kubeconfig.
- make the Ensure* functions return descriptive errors instead of
wrapping the errors on the side of the callers.
Secure serving was already enabled for kube-controller-manager.
Do the same for kube-scheduler, by passing the flags
"authentication-kubeconfig" and "authorization-kubeconfig"
to the binary in the static Pod.
This change allows the scheduler to perform reviews on incoming
requests, such as:
- authentication.k8s.io/v1beta1 TokenReview
- authorization.k8s.io/v1 SubjectAccessReview
The authentication and authorization checks for "system:kube-scheduler"
users were previously enabled by PR 72491.
- common_test.go: use constants.CurrentKubernetesVersion
- diff_test.go: write temporary files instead of using testdata.
this allows us to not have to bump kubernetesVersions in the
testdata files (now removed)
- policy_test.go: apply fixes to tests that were previously passing,
but a bump in constants.go breaks them. these tests now work
for any version.
progagates the kubeadm FG to the individual k8scomponents
on the control-plane node.
* Note: Users who want to join worker nodes to the cluster
will have to specify the dual-stack FG to kubelet using the
nodeRegistration.kubeletExtraArgs option as part of their
join config. Alternatively, they can use KUBELET_EXTRA_ARGS.
kubeadm FG: kubernetes/kubeadm#1612
If empty "--kubernetes-version" is given (as it's not configurable now)
k8s.io/kubernetes/cmd/kubeadm/app/util/version.go.KubernetesReleaseVersion
will fetch the version from the internet.
But, this can fail:
% kubeadm init phase certs ca --cert-dir ...
unable to fetch file. URL: "https://dl.k8s.io/release/stable-1.txt", status: 502 Bad Gateway
failed to run commands: exit status 1
Can happen to other commands:
% kubeadm init phase kubeconfig controller-manager ...
% kubeadm init phase kubeconfig scheduler ...
This make "--kubernetes-version" configurable, so users can enable offline mode.
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
When adding a new etcd member the etcd cluster can enter a state
of vote, where any new members added at the exact same time will
fail with an error right away.
Implement exponential backoff retry around the MemberAdd call.
This solves a kubeadm problem when concurrently joining
control-plane nodes with stacked etcd members.
From experiment, a few retries with milliseconds apart are
sufficient to achieve the concurrent join of a 3xCP cluster.
Apply the same backoff to MemberRemove in case the concurrent
removal of members fails for similar reasons.
Nit: remove capitalization of preferred
Remove line from kubelet and add to separate PR for easier merge
nit: dependency added to separate PR
Add check to ensure strict policy cannot be set without feature gate enabled
Topology Manager runs "none" policy by default.
Added constants for policies and updated documentation.
libcni 0.7.0 caches ADD operation results and allows the runtime to
retrieve these from the cache. In case the user wants a different
cache directory than the defaul, plumb that through like we do
for --cni-bin-dir and --cni-conf-dir.
If the cluster has a PSP that blocks Pods from running as root
the DS that handles upgrade prepull will fail to create its Pods.
Workaround that by adding a PodSecurityContext with RunAsUser=999.
Instead of creating a Docker client and fetching an Info object
from the docker enpoint, call the "docker info" command
and populate a local dockerInfo struct from JSON output.
Also
- add unit tests.
- update import boss and bazel.
This change affects "test/e2e_node/e2e_node_suite_test.go"
as it consumes this Docker validator by calling
"system.ValidateSpec()".
Since we never use the cobras "SilenceErrors" or "SilenceUsage",
a command executed with "cmd.Execute()" will never return an error
without printing it.
The current behavior results in all error messages being printed twice:
Example:
$ kubectl abc
Error: unknown command "abc" for "kubectl"
Run 'kubectl --help' for usage.
unknown command "abc" for "kubectl"
This applies to all cli commands using Cobra. To verify, follow the code
path of the Execute function:
https://github.com/spf13/cobra/blob/c439c4fa0937/command.go#L793
Signed-off-by: Odin Ugedal <odin@ugedal.com>
- comment out Liz and Chuck until further notice.
Feel free to come back to kubeadm!!
- Add SataQiu as reviewer. Welcome.
- Add ereslibre as approver. Congrats!
MarshalClusterConfigurationToBytes has capabilities to output the component
configs, as separate YAML documents, besides the kubeadm ClusterConfiguration
kind. This is no longer necessary for the following reasons:
- All current use cases of this function require only the ClusterConfiguration.
- It will output component configs only if they are not the default ones. This
can produce undeterministic output and, thus, cause potential problems.
- There are only hacky ways to dump the ClusterConfiguration only (without the
component configs).
Hence, we simplify things by replacing the function with direct calls to the
underlaying MarshalToYamlForCodecs. Thus marshalling only ClusterConfiguration,
when needed.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Stop using //pkg/util/normalizer. Use local versions of LongDesc and Examples,
that do not require any external dependencies (other than the Go standard
library).
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
one test also proved it did not call the internet
but this was not fool proof as it did not return a string
and thus could be called with something expecting to fail.
updated the network calls to be package local so tests could pass their
own implementation. A public interface was not provided as it would not
be likely this would ever be needed or wanted.
When a kubeconfig file is read from disk it may lack the
propper mapping between contexts and clusters.
In such a case the kubeconfig phase backend will panic,
without throwing a sensible error.
Add nil checks for a couple of map operations in
validateKubeConfig().
This helper is used in tests and pulls in unnecessary dependency, which should
not be used if kubeadm is to move to staging.
Replace with direct use of the GroupResource type.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
ValidateDNS1123Subdomain is a simple wrapper around IsDNS1123Subdomain, however
it's the only reason for us to pull k8s.io/kubernetes/pkg/apis/core/validation
as a dependency.
To avoid unnecessary dependencies, replace the use of ValidateDNS1123Subdomain
with IsDNS1123Subdomain.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
RBAC construction helpers are part of the Kubernetes internal APIs. As such,
we cannot use them once we move to staging.
Hence, replace their use with manual RBAC rule construction.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
During the control plane joins, sometimes the control plane returns an
expected error when trying to download the `kubeadm-config` ConfigMap.
This is a workaround for this issue until the root cause is completely
identified and fixed.
Ideally, this commit should be reverted in the near future.
kube-controller-manager.
If a service CIDR that overlaps with the cluster CIDR is
specified to kube-controller-manager then kube-controller-
manager will incorrectly allocate node CIDRs that overlap
with the service CIDR. The fix ensure that kubeadm
maps the --service-cidr to --service-cluster-ip-range for use
by kube-controller-manager.
As per docs, --allocate-node-cidrs must be true for
--service-cluster-ip-range to be considered. It does not make
sense for --cluster-cidr to be unspecified but for
--service-cluster-ip-range and --allocate-node-cidrs to be
set, since the purpose of these options is to have the
controller-manager do the per node CIDR allocation. Also
note that --service-cluster-ip-range is passed to the
api-server, so the presence of *just*
--service-cluster-ip-range should not imply that
--allocate-node-cidrs should be true.
Resolves: kubernetes/kubeadm/issues/1591
This fixes possible problems when kubeadm upgrade can't load the
InitConfig properly. Some new code introduced in
https://github.com/kubernetes/kubernetes/pull/75499 is placed between
the loading of the config and the error handling, hiding possible
errors.
This error cannot be ignored (as is the case now), since the cfg ptr.
returned from the configutil function will be nil in the case of an
error.
Signed-off-by: Odin Ugedal <odin@ugedal.com>
The existing logic already creates a proper "tree"
where a CA is always generated before the certs that are signed
by this CA, however the tree is not deterministic.
Always use the default list of certs when generating the
"kubeadm init phase certs" phases. Add a unit test that
makes sure that CA always precede signed certs in the default
lists.
This solves the problem where the help screen for "kubeadm
init" cert sub-phases can have a random order.
Similar to --token, do not allow the mixture of --config and
--certificate-key.
If the user has fed a config, it is expected that the certificate
key should also be provided in the config and not from
the command line.
Ever since v1alpha3, InitConfiguration is containing ClusterConfiguration
embedded in it. This was done to mimic the internal InitConfiguration, which in
turn is used throughout the kubeadm code base as if it is the old
MasterConfiguration of v1alpha2.
This, however, is confusing to users who vendor in kubeadm as the embedded
ClusterConfiguration inside InitConfiguration is not marshalled to YAML.
For this to happen, special care must be taken for the ClusterConfiguration
field to marshalled separately.
Thus, to make things smooth for users and to reduce third party exposure to
technical debt, this change removes ClusterConfiguration embedding from
InitConfiguration.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
This commit contains only changes generated by the build process.
Nothing here was manually changed.
Changes made to:
```
cmd/kubeadm/app/apis/kubeadm/validation/BUILD
cmd/kubeadm/app/cmd/BUILD
```
were generated by running:
````
./hack/update-bazel.sh
```
currently sub phases cannot have custom arg validation and container commands can have args.
This removes phase container commands from taking args and enables custom args on the leaf phases
* fix duplicated imports of api/core/v1
* fix duplicated imports of client-go/kubernetes
* fix duplicated imports of rest code
* change import name to more reasonable
During the upgrade process, `kubeadm` will take the current
`ClusterConfiguration`, update the `KubernetesVersion` to the latest
version, and call to `UploadConfiguration`.
This change makes sure that when the mutation happens, not only the
`ClusterStatus` is mutated, but the `ClusterConfiguration` object
inside the `kubeadm-config` ConfigMap as well; it will contain the
new `KubernetesVersion`.
There are a couple of problems with regards to the `omitempty` in v1beta1:
- It is not applied to certain fields. This makes emitting YAML configuration
files in v1beta1 config format verbose by both kubeadm and third party Go
lang tools. Certain fields, that were never given an explicit value would
show up in the marshalled YAML document. This can cause confusion and even
misconfiguration.
- It can be used in inappropriate places. In this case it's used for fields,
that need to be always serialized. The only one such field at the moment is
`NodeRegistrationOptions.Taints`. If the `Taints` field is nil, then it's
defaulted to a slice containing a single control plane node taint. If it's
an empty slice, no taints are applied, thus, the cluster behaves differently.
With that in mind, a Go program, that uses v1beta1 with `omitempty` on the
`Taints` field has no way to specify an explicit empty slice of taints, as
this would get lost after marshalling to YAML.
To fix these issues the following is done in this change:
- A whole bunch of additional omitemptys are placed at many fields in v1beta2.
- `omitempty` is removed from `NodeRegistrationOptions.Taints`
- A test, that verifies the ability to specify empty slice value for `Taints`
is included.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
This change introduces config fields to the v1beta2 format, that allow
certificate key to be specified in the config file. This certificate key is a
hex encoded AES key, that is used to encrypt certificates and keys, needed for
secondary control plane nodes to join. The same key is used for the decryption
during control plane join.
It is important to note, that this key is never uploaded to the cluster. It can
only be specified on either command line or the config file.
The new fields can be used like so:
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
certificateKey: "yourSecretHere"
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: JoinConfiguration
controlPlane:
certificateKey: "yourSecretHere"
---
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Even though CreateServiceAccountKeyAndPublicKeyFiles() function is
an interface function it's not unittested. Instead it wraps a couple
of internal functions which are used only inside CreateServiceAccountKeyAndPublicKeyFiles()
and those internal functions are tested.
Rewrite the function to do only what it's intended to do and add unit
tests for it.
These are based on recommendation from
[staticcheck](http://staticcheck.io/).
- Remove unused struct fields
- Remove unused function
- Remove unused variables
- Remove unused constants.
- Miscellaneous cleanups
This unit test contains some hacks are causing the bazel-test
e2e job to flake very often. Instead of maintaining this
unit test remove it completely. It has little benefits
WRT testing app/util/chroot*.go.
kubeadm still generates RSA keys when deploying a node, but also
accepts ECDSA keys if they already exist pregenerated in the
directory specified in --cert-dir.
Add the functionality to support `CreateOrMutateConfigMap` and `MutateConfigMap`.
* `CreateOrMutateConfigMap` will try to create a given ConfigMap object; if this ConfigMap
already exists, a new version of the resource will be retrieved from the server and a
mutator callback will be called on it. Then, an `Update` of the mutated object will be
performed. If there's a conflict during this `Update` operation, retry until no conflict
happens. On every retry the object is refreshed from the server to the latest version.
* `MutateConfigMap` will try to get the latest version of the ConfigMap from the server,
call the mutator callback and then try to `Update` the mutated object. If there's a
conflict during this `Update` operation, retry until no conflict happens. On every retry
the object is refreshed from the server to the latest version.
Add unit tests for `MutateConfigMap`
* One test checks that in case of no conflicts, the update of the
given ConfigMap happens without any issues.
* Another test mimics 5 consecutive CONFLICT responses when updating
the given ConfigMap, whereas the sixth try it will work.
The function certs.NewCACertAndKey() is just a wrapper around
pkiutil.NewCertificateAuthority() which doesn't add any
additional functionality.
Instead use pkiutil.NewCertificateAuthority() directly.
Currently kubeadm produces an error upon parsing multiple
certificates stored in the cluster-info configmap. Yet it
should check all available certificates in a scenario like
CA key rotation.
Check all available CA certs against pinned certificate hashes.
Fixes https://github.com/kubernetes/kubeadm/issues/1399
In the case where newControlPlane is true we don't go through
getNodeRegistration() and initcfg.NodeRegistration.CRISocket is empty.
This forces DetectCRISocket() to be called later on, and if there is more than
one CRI installed on the system, it will error out, while asking for the user
to provide an override for the CRI socket. Even if the user provides an
override, the call to DetectCRISocket() can happen too early and thus ignore it
(while still erroring out).
However, if newControlPlane == true, initcfg.NodeRegistration is not used at
all and it's overwritten later on.
Thus it's necessary to supply some default value, that will avoid the call to
DetectCRISocket() and as initcfg.NodeRegistration is discarded, setting
whatever value here is harmless.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Windows worker nodes run kube-proxy as a Windows service.
In the future the kube-proxy DaemonSet might run on Windows nodes
too, but for now a temporary measure is needed to disable it.
Add a linux node selector in the kube-proxy manifest spec.
We have an existing helper function for this: runtime.SerializerInfoForMediaType()
This is common prep-work for encoding runtime.Objects into JSON/YAML for transmission over the wire or writing to ComponentConfigs.
If the k8s version argument passed to "upgrade plan" is missing
the logic should perform the following actions:
- fetch a "stable" version from the internet.
- if that fails, fallback to the local client version.
Currentely the logic fails because the cfg.KubernetesVersion is
defaulted to the version of the existing cluster, which
then causes an early exit without any ugprade suggestions.
See app/cmd/upgrade/common.go::enforceRequirements():
configutil.FetchInitConfigurationFromCluster(..)
Fix that by passing the explicit user value that can also be "".
This will then make the "offline getter" treat it as an explicit
desired upgrade target.
In the future it might be best to invert this logic:
- if no user k8s version argument is passed - default to the kubeadm
version.
- if labels are passed (e.g. "stable"), fetch a version from the
internet.
Used T.Run API for kubeadm tests in app/phases/*
This should improve testing output and make it more visible
which test is doing what.
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
