The new etcd balancer (>3.3.14, 3.4.0) uses an asynchronous resolver for
endpoints. Without "WithBlock", the client may return before the
connection is up.
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
A recent commit added warnings for KubeletConfiguration and
KubeProxyConfiguration fields that kubeadm cares about and
does not recommend the user modifying them. Kubelet's
"rotateCertificates" cannot be handled using this function
as there is not way to figure out if the user has set it explicitly to
"false". Hardcode the value to "true" and add a comment about that.
Also apply the following changes to warnDefaultComponentConfigValue()
calls:
- use a local "kind" variable that defines the Kind we are warning about.
- fix wrong paths to fields.
- replace all stray calls of os.Exit() to util.CheckError() instead
- CheckError() now checks if the klog verbosity level is >=5
and shows a stack trace of the error
- don't call klog.Fatal in version.go
It seems undesirable that Kubernetes as a system should be
blocking a node if it's Linux kernel is way too new.
If such a problem even occurs we should exclude versions from
the list of supported versions instead of blocking users
from trying e.g. the latest 7.0.0-beta kernel because our
validators are not aware of this new version.
Usage of github.com/blang/semver is not needed and
k8s.io/apimachinery/pkg/util/version should be used instead
for semantic version parsing and version comparison.
Etcd v3.3.0 added the --listen-metrics-urls flag which allows specifying
addition URLs to the already present /health and /metrics endpoints.
While /health and /metrics are enabled for URLS defined with
--listen-client-urls (v3+ ?) they do require HTTPS.
Replace the present etcdctl based liveness probe with a standard HTTP
GET v1.Probe that connects to http://127.0.0.1:2381/health.
These endpoints are not reachable from the outside and only available
for localhost connections.
For file discovery, in case the user feeds a file for the CA
from the kubeconfig, make sure it's preloaded and embedded using
the new function EnsureCertificateAuthorityIsEmbedded().
This commit also applies cleanup:
- unroll validateKubeConfig() into ValidateConfigInfo() as this way
the default cluster can be re-used.
- in ValidateConfigInfo() reuse the variable config instead of creating
a new variable kubeconfig.
- make the Ensure* functions return descriptive errors instead of
wrapping the errors on the side of the callers.
Secure serving was already enabled for kube-controller-manager.
Do the same for kube-scheduler, by passing the flags
"authentication-kubeconfig" and "authorization-kubeconfig"
to the binary in the static Pod.
This change allows the scheduler to perform reviews on incoming
requests, such as:
- authentication.k8s.io/v1beta1 TokenReview
- authorization.k8s.io/v1 SubjectAccessReview
The authentication and authorization checks for "system:kube-scheduler"
users were previously enabled by PR 72491.
- common_test.go: use constants.CurrentKubernetesVersion
- diff_test.go: write temporary files instead of using testdata.
this allows us to not have to bump kubernetesVersions in the
testdata files (now removed)
- policy_test.go: apply fixes to tests that were previously passing,
but a bump in constants.go breaks them. these tests now work
for any version.
progagates the kubeadm FG to the individual k8scomponents
on the control-plane node.
* Note: Users who want to join worker nodes to the cluster
will have to specify the dual-stack FG to kubelet using the
nodeRegistration.kubeletExtraArgs option as part of their
join config. Alternatively, they can use KUBELET_EXTRA_ARGS.
kubeadm FG: kubernetes/kubeadm#1612
If empty "--kubernetes-version" is given (as it's not configurable now)
k8s.io/kubernetes/cmd/kubeadm/app/util/version.go.KubernetesReleaseVersion
will fetch the version from the internet.
But, this can fail:
% kubeadm init phase certs ca --cert-dir ...
unable to fetch file. URL: "https://dl.k8s.io/release/stable-1.txt", status: 502 Bad Gateway
failed to run commands: exit status 1
Can happen to other commands:
% kubeadm init phase kubeconfig controller-manager ...
% kubeadm init phase kubeconfig scheduler ...
This make "--kubernetes-version" configurable, so users can enable offline mode.
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
When adding a new etcd member the etcd cluster can enter a state
of vote, where any new members added at the exact same time will
fail with an error right away.
Implement exponential backoff retry around the MemberAdd call.
This solves a kubeadm problem when concurrently joining
control-plane nodes with stacked etcd members.
From experiment, a few retries with milliseconds apart are
sufficient to achieve the concurrent join of a 3xCP cluster.
Apply the same backoff to MemberRemove in case the concurrent
removal of members fails for similar reasons.
Nit: remove capitalization of preferred
Remove line from kubelet and add to separate PR for easier merge
nit: dependency added to separate PR
Add check to ensure strict policy cannot be set without feature gate enabled
Topology Manager runs "none" policy by default.
Added constants for policies and updated documentation.
libcni 0.7.0 caches ADD operation results and allows the runtime to
retrieve these from the cache. In case the user wants a different
cache directory than the defaul, plumb that through like we do
for --cni-bin-dir and --cni-conf-dir.
If the cluster has a PSP that blocks Pods from running as root
the DS that handles upgrade prepull will fail to create its Pods.
Workaround that by adding a PodSecurityContext with RunAsUser=999.
Instead of creating a Docker client and fetching an Info object
from the docker enpoint, call the "docker info" command
and populate a local dockerInfo struct from JSON output.
Also
- add unit tests.
- update import boss and bazel.
This change affects "test/e2e_node/e2e_node_suite_test.go"
as it consumes this Docker validator by calling
"system.ValidateSpec()".
Since we never use the cobras "SilenceErrors" or "SilenceUsage",
a command executed with "cmd.Execute()" will never return an error
without printing it.
The current behavior results in all error messages being printed twice:
Example:
$ kubectl abc
Error: unknown command "abc" for "kubectl"
Run 'kubectl --help' for usage.
unknown command "abc" for "kubectl"
This applies to all cli commands using Cobra. To verify, follow the code
path of the Execute function:
https://github.com/spf13/cobra/blob/c439c4fa0937/command.go#L793
Signed-off-by: Odin Ugedal <odin@ugedal.com>
- comment out Liz and Chuck until further notice.
Feel free to come back to kubeadm!!
- Add SataQiu as reviewer. Welcome.
- Add ereslibre as approver. Congrats!
MarshalClusterConfigurationToBytes has capabilities to output the component
configs, as separate YAML documents, besides the kubeadm ClusterConfiguration
kind. This is no longer necessary for the following reasons:
- All current use cases of this function require only the ClusterConfiguration.
- It will output component configs only if they are not the default ones. This
can produce undeterministic output and, thus, cause potential problems.
- There are only hacky ways to dump the ClusterConfiguration only (without the
component configs).
Hence, we simplify things by replacing the function with direct calls to the
underlaying MarshalToYamlForCodecs. Thus marshalling only ClusterConfiguration,
when needed.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Stop using //pkg/util/normalizer. Use local versions of LongDesc and Examples,
that do not require any external dependencies (other than the Go standard
library).
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
one test also proved it did not call the internet
but this was not fool proof as it did not return a string
and thus could be called with something expecting to fail.
updated the network calls to be package local so tests could pass their
own implementation. A public interface was not provided as it would not
be likely this would ever be needed or wanted.
When a kubeconfig file is read from disk it may lack the
propper mapping between contexts and clusters.
In such a case the kubeconfig phase backend will panic,
without throwing a sensible error.
Add nil checks for a couple of map operations in
validateKubeConfig().
This helper is used in tests and pulls in unnecessary dependency, which should
not be used if kubeadm is to move to staging.
Replace with direct use of the GroupResource type.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
ValidateDNS1123Subdomain is a simple wrapper around IsDNS1123Subdomain, however
it's the only reason for us to pull k8s.io/kubernetes/pkg/apis/core/validation
as a dependency.
To avoid unnecessary dependencies, replace the use of ValidateDNS1123Subdomain
with IsDNS1123Subdomain.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
RBAC construction helpers are part of the Kubernetes internal APIs. As such,
we cannot use them once we move to staging.
Hence, replace their use with manual RBAC rule construction.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
During the control plane joins, sometimes the control plane returns an
expected error when trying to download the `kubeadm-config` ConfigMap.
This is a workaround for this issue until the root cause is completely
identified and fixed.
Ideally, this commit should be reverted in the near future.
kube-controller-manager.
If a service CIDR that overlaps with the cluster CIDR is
specified to kube-controller-manager then kube-controller-
manager will incorrectly allocate node CIDRs that overlap
with the service CIDR. The fix ensure that kubeadm
maps the --service-cidr to --service-cluster-ip-range for use
by kube-controller-manager.
As per docs, --allocate-node-cidrs must be true for
--service-cluster-ip-range to be considered. It does not make
sense for --cluster-cidr to be unspecified but for
--service-cluster-ip-range and --allocate-node-cidrs to be
set, since the purpose of these options is to have the
controller-manager do the per node CIDR allocation. Also
note that --service-cluster-ip-range is passed to the
api-server, so the presence of *just*
--service-cluster-ip-range should not imply that
--allocate-node-cidrs should be true.
Resolves: kubernetes/kubeadm/issues/1591
This fixes possible problems when kubeadm upgrade can't load the
InitConfig properly. Some new code introduced in
https://github.com/kubernetes/kubernetes/pull/75499 is placed between
the loading of the config and the error handling, hiding possible
errors.
This error cannot be ignored (as is the case now), since the cfg ptr.
returned from the configutil function will be nil in the case of an
error.
Signed-off-by: Odin Ugedal <odin@ugedal.com>
The existing logic already creates a proper "tree"
where a CA is always generated before the certs that are signed
by this CA, however the tree is not deterministic.
Always use the default list of certs when generating the
"kubeadm init phase certs" phases. Add a unit test that
makes sure that CA always precede signed certs in the default
lists.
This solves the problem where the help screen for "kubeadm
init" cert sub-phases can have a random order.
Similar to --token, do not allow the mixture of --config and
--certificate-key.
If the user has fed a config, it is expected that the certificate
key should also be provided in the config and not from
the command line.
Ever since v1alpha3, InitConfiguration is containing ClusterConfiguration
embedded in it. This was done to mimic the internal InitConfiguration, which in
turn is used throughout the kubeadm code base as if it is the old
MasterConfiguration of v1alpha2.
This, however, is confusing to users who vendor in kubeadm as the embedded
ClusterConfiguration inside InitConfiguration is not marshalled to YAML.
For this to happen, special care must be taken for the ClusterConfiguration
field to marshalled separately.
Thus, to make things smooth for users and to reduce third party exposure to
technical debt, this change removes ClusterConfiguration embedding from
InitConfiguration.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
This commit contains only changes generated by the build process.
Nothing here was manually changed.
Changes made to:
```
cmd/kubeadm/app/apis/kubeadm/validation/BUILD
cmd/kubeadm/app/cmd/BUILD
```
were generated by running:
````
./hack/update-bazel.sh
```
currently sub phases cannot have custom arg validation and container commands can have args.
This removes phase container commands from taking args and enables custom args on the leaf phases
* fix duplicated imports of api/core/v1
* fix duplicated imports of client-go/kubernetes
* fix duplicated imports of rest code
* change import name to more reasonable
During the upgrade process, `kubeadm` will take the current
`ClusterConfiguration`, update the `KubernetesVersion` to the latest
version, and call to `UploadConfiguration`.
This change makes sure that when the mutation happens, not only the
`ClusterStatus` is mutated, but the `ClusterConfiguration` object
inside the `kubeadm-config` ConfigMap as well; it will contain the
new `KubernetesVersion`.
There are a couple of problems with regards to the `omitempty` in v1beta1:
- It is not applied to certain fields. This makes emitting YAML configuration
files in v1beta1 config format verbose by both kubeadm and third party Go
lang tools. Certain fields, that were never given an explicit value would
show up in the marshalled YAML document. This can cause confusion and even
misconfiguration.
- It can be used in inappropriate places. In this case it's used for fields,
that need to be always serialized. The only one such field at the moment is
`NodeRegistrationOptions.Taints`. If the `Taints` field is nil, then it's
defaulted to a slice containing a single control plane node taint. If it's
an empty slice, no taints are applied, thus, the cluster behaves differently.
With that in mind, a Go program, that uses v1beta1 with `omitempty` on the
`Taints` field has no way to specify an explicit empty slice of taints, as
this would get lost after marshalling to YAML.
To fix these issues the following is done in this change:
- A whole bunch of additional omitemptys are placed at many fields in v1beta2.
- `omitempty` is removed from `NodeRegistrationOptions.Taints`
- A test, that verifies the ability to specify empty slice value for `Taints`
is included.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
This change introduces config fields to the v1beta2 format, that allow
certificate key to be specified in the config file. This certificate key is a
hex encoded AES key, that is used to encrypt certificates and keys, needed for
secondary control plane nodes to join. The same key is used for the decryption
during control plane join.
It is important to note, that this key is never uploaded to the cluster. It can
only be specified on either command line or the config file.
The new fields can be used like so:
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
certificateKey: "yourSecretHere"
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: JoinConfiguration
controlPlane:
certificateKey: "yourSecretHere"
---
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Even though CreateServiceAccountKeyAndPublicKeyFiles() function is
an interface function it's not unittested. Instead it wraps a couple
of internal functions which are used only inside CreateServiceAccountKeyAndPublicKeyFiles()
and those internal functions are tested.
Rewrite the function to do only what it's intended to do and add unit
tests for it.
These are based on recommendation from
[staticcheck](http://staticcheck.io/).
- Remove unused struct fields
- Remove unused function
- Remove unused variables
- Remove unused constants.
- Miscellaneous cleanups
This unit test contains some hacks are causing the bazel-test
e2e job to flake very often. Instead of maintaining this
unit test remove it completely. It has little benefits
WRT testing app/util/chroot*.go.
