Automatic merge from submit-queue
kubeadm: Stop assuming full ownership of /etc/kubernetes.
Packages may auto-create directories in /etc/kubernetes, and users also
need files such as cloud-config.json to be present and preserved at
their default locations in /etc/kubernetes. As such this modifies
pre-flight checks to only require the absence of the files and
directories we explicitly create in kubeadm.
Reset is similarly modified to not wipe out /etc/kubernetes entirely.
When resetting directories we also now preserve the directory itself,
but delete it's contents.
Also adds tests for reset command logic specifically for /etc/kubernetes
cleanup, to ensure user files are not inadvertently wiped out.
This will allow packages to maintain ownership of config and data
directories, which may carry selinux or other attributes that should be
preserved, but we do not wish to manage within kubeadm itself.
Packages may auto-create directories in /etc/kubernetes, and users also
need files such as cloud-config.json to be present and preserved at
their default locations in /etc/kubernetes. As such this modifies
pre-flight checks to only require the absence of the files and
directories we explicitly create in kubeadm.
Reset is similarly modified to not wipe out /etc/kubernetes entirely.
When resetting directories we also now preserve the directory itself,
but delete it's contents.
Also adds tests for reset command logic specifically for /etc/kubernetes
cleanup, to ensure user files are not inadvertently wiped out.
Automatic merge from submit-queue
rename kubelet flag mounter-path to experimental-mounter-path
```release-note
* Kubelet flag '--mounter-path' renamed to '--experimental-mounter-path'
```
The feature the flag controls is an experimental feature and this renaming ensures that users do not depend on this feature just yet.
Automatic merge from submit-queue
CRI: Add dockershim grpc server.
This PR adds a in-process grpc server for dockershim.
Flags change:
1. `container-runtime` will not be automatically set to remote when `container-runtime-endpoint` is set. @feiskyer
2. set kubelet flag `--experimental-runtime-integration-type=remote --container-runtime-endpoint=UNIX_SOCKET_FILE_PATH` to enable the in-process dockershim grpc server.
3. set node e2e test flag `--runtime-integration-type=remote -container-runtime-endpoint=UNIX_SOCKET_FILE_PATH` to run node e2e test against in-process dockershim grpc server.
I've run node e2e test against the remote cri integration, tests which don't rely on stream and log functions can pass.
This unblocks the following work:
1) CRI conformance test.
2) Performance comparison between in-process integration and in-process grpc integration.
@yujuhong @feiskyer
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Always run the podGC controller.
**What this PR does / why we need it**: The podGC controller has evolved to do more than just GC of terminated pods beyond a threshold number. It no longer makes sense to gate running it with the `terminated-pod-gc-threshold` flag. [We still ensure that it only runs the terminatedPodsGC if the threshold specified in the argument to the controller manager is > 0](6079053407/pkg/controller/podgc/gc_controller.go (L124)).
Related discussion: https://github.com/kubernetes/kubernetes/pull/34160#issuecomment-255900898
**Release note**:
```release-note
The podGC controller will now always run, irrespective of the value supplied to the "terminated-pod-gc-threshold" flag supplied to the controller manager.
The specific behavior of the podGC controller to clean up terminated pods is still governed by the flag, but the podGC's responsibilities have evolved beyond just cleaning up terminated pods.
```
Automatic merge from submit-queue
kubeadm: added unit tests for app/images pkg
Added unit tests for kubeadm/app/images package testing functionality of images.go.
This PR is part of the ongoing effort to add tests (#35025)
/cc @pires @jbeda
Automatic merge from submit-queue
enhance join arguments generation logic using template
**What this PR does / why we need it**:
this PR enhances kubeadm join arguments generation logic using template, this makes code more readable and adding arguments more easier.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Signed-off-by: 欧阳钦华10079130 <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue
Create restclient interface
Refactoring of code to allow replace *restclient.RESTClient with any RESTClient implementation that implements restclient.RESTClientInterface interface.
Automatic merge from submit-queue
kubeadm: Normalized reset command to match init and join commands.
**What this PR does / why we need it**: Overall, improves code structure. Opening single PR in order avoid big PRs in the future, when adding tests and new functionality, i.e. #34404.
```release-note
NONE
```
In order to be able to use new mounter library, this PR adds the
mounterPath flag to kubelet which passes the flag to the mount
interface. If flag is empty, mount uses default mount path.
Automatic merge from submit-queue
If token validation fails, give the user the expected format.
If a user specifies their own token to kubeadm, and it fails validation, the error they currently receive isn't the friendliest. This first change adds messaging for the expected token format, with more improvements to follow as part of #33930. It also adds some unit tests to document the behavior we have currently.
CC: @mikedanese, @krousey, @kubernetes/sig-cluster-lifecycle
Part of #33930.
Automatic merge from submit-queue
WIP: Remove the legacy networking mode
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Removes the deprecated configure-cbr0 flag and networking mode to avoid having untested and maybe unstable code in kubelet, see: #33789
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes#30589fixes#31937
**Special notes for your reviewer**: There are a lot of deployments who rely on this networking mode. Not sure how we deal with that: force switch to kubenet or just delete the old deployment?
But please review the code changes first (the first commit)
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Removed the deprecated kubelet --configure-cbr0 flag, and with that the "classic" networking mode as well
```
PTAL @kubernetes/sig-network @kubernetes/sig-node @mikedanese
Given an object that wishes to have a top level defaulter, traverse the
object looking for nested fields that have defaulters and generate a
single function for that type that invokes all defaulters. The function
will have the name `SetObjectDefaults_NAME`.
Types use `// +k8s:defaulter-gen=true` to indicate they wish a defaulter
generated. If a function already exists with the desired name
`SetObjectDefaults_NAME` then no generation will occur. At a package
level, authors can bulk select the types to generate by setting the
value of the comment to the name of a field - all objects with that
field name without `// +k8s:defaulter-gen=false` defined on the type
will get a defaulter.
Because the defaulting behavior from conversions happens recursively,
all defaulters are expected to be invoked. We call these defaulters
"non-covering" (other defaulters may be invoked beneath them). The
defaulters we generate, by comparison, are "covering" - no nested
defaulters should be invoked. To distinguish between these two types, we
introduce the `// +k8s:defaulter-gen=covers` comment on a defaulter
function which will instruct the generator that the function should
terminate recursion.
This sets the stage for future defaulter generation from comments by
subsuming our existing generators
Automatic merge from submit-queue
Remove static kubelet client, refactor ConnectionInfoGetter
Follow up to https://github.com/kubernetes/kubernetes/pull/33718
* Collapses the multi-valued return to a `ConnectionInfo` struct
* Removes the "raw" connection info method and interface, since it was only used in a single non-test location (by the "real" connection info method)
* Disentangles the node REST object from being a ConnectionInfoProvider itself by extracting an implementation of ConnectionInfoProvider that takes a node (using a provided NodeGetter) and determines ConnectionInfo
* Plumbs the KubeletClientConfig to the point where we construct the helper object that combines the config and the node lookup. I anticipate adding a preference order for choosing an address type in https://github.com/kubernetes/kubernetes/pull/34259
Automatic merge from submit-queue
make version an explicit choice so zero config and customized work
Makes `/version` key off of setting the version. This allows composers to add a version that is correct.
Automatic merge from submit-queue
Improvements to CLI usability and maintainability
Improves `kubectl` from an usability perspective by
1. Fixing how we handle terminal width in help. Some sections like the flags use the entire available width, while others like long descriptions breaks lines but don't follow a well established max width (screenshot below). This PR adds a new responsive writer that will adjust to terminal width and set 80, 100, or 120 columns as the max width, but not more than that given POSIX best practices and recommendations for better readability.
2. Adds our own normalizers for long descriptions and cmd examples which allows us better control about how things like lists, paragraphs, line breaks, etc are printed. Features markdown support. Looks like `templates.LongDesc` and `templates.Examples` instead of `dedent.Dedend`.
3. Allows simple reordering and reuse of help and usage sections.
3. Adds `verify-cli-conventions.sh` which intends to run tests to make sure cmd developers are using what we propose as [kubectl conventions](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/kubectl-conventions.md). Just a couple simple tests for now but the framework is there and it's easy to extend.
4. Update [kubectl conventions](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/kubectl-conventions.md) to use our own normalizers instead of `dedent.Dedent`.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Improves how 'kubectl' uses the terminal size when printing help and usage.
```
@kubernetes/kubectl
Automatic merge from submit-queue
+optional tag for OpenAPI spec
OpenAPI rely on "omitempty" json tag to determine if a field is optional or not. This change will add "+optional" tag to all fields with "omitempty" json tag and support the tag in OpenAPI spec generator.
Automatic merge from submit-queue
add ownerref permission checks
Adds an admission plugin that ensures that anyone adding an `ownerReference` to a resource has delete rights on the resource they're setting up a delete for.
@caesarxuchao example admission plugin that tests for ownerReference diffs and uses an authorizer to drive the decision.
@liggitt @ncdc we've talked about this before
Automatic merge from submit-queue
Add flags for alternative API and discovery ports
**What this PR does / why we need it**:
We had many issues reported due to the fact that we are using port 443 by default, and we should allow users to override whatever defaults we pick. This doesn't touch on `localhost:8080` yet, which should just generally get rid of.
**Which issue this PR fixes**: fixes#34311#34307#33638
**Special notes for your reviewer**: cc @pires
**Release note**:
```release-note
Add `kubeadm` flags `--api-port` and `--discovery-port`, change default API port to 6443
```
Automatic merge from submit-queue
clientgen: allow to pass custom apiPath when generating client sets
This PR allow to pass the `--clientset-api-path` parameter to clientgen that allows to customize the default API path set in clients. This allows projects like OpenShift to re-use the client with different API path (`/oapi` in our case).
Automatic merge from submit-queue
kubeadm join: Added support for config file.
As more behavior (#34719, #34807, fix for #33641) is added to `kubeadm join`, this will be eventually very much needed. Makes sense to go in sooner rather than later.
Also references #34501 and #34884.
/cc @luxas @mikedanese
Automatic merge from submit-queue
default serializer
Everyone uses the same serializer. Set it as the default, but still allow someone to take control if they want.
Found while trying to use genericapiserver for composition.
Automatic merge from submit-queue
kubeadm join: wait for API endpoints
**What this PR does / why we need it**: enhance kubeadm to allow for parallel provisioning of API endpoints and slave nodes, continued from https://github.com/kubernetes/kubernetes/pull/33543
**Fixes**: https://github.com/kubernetes/kubernetes/issues/33542
**Special notes for your reviewer**:
* Introduces a concurrent retry mechanism for bootstrapping with a single API endpoint during `kubeadm join` (this was left out in https://github.com/kubernetes/kubernetes/pull/33543 so that it can be implemented in a separate PR). The polling of the discovery service API itself is yet to come.
@errordeveloper @pires
Automatic merge from submit-queue
Add support for admission controller based on namespace node selectors.
This work is to upstream openshift's project node selectors based admission controller.
Fixes https://github.com/kubernetes/kubernetes/issues/17151
Automatic merge from submit-queue
Run rbac authorizer from cache
RBAC authorization can be run very effectively out of a cache. The cache is a normal reflector backed cache (shared informer).
I've split this into three parts:
1. slim down the authorizer interfaces
1. boilerplate for adding rbac shared informers and associated listers which conform to the new interfaces
1. wiring
@liggitt @ericchiang @kubernetes/sig-auth
Automatic merge from submit-queue
Merge string flag into util flag
Continuing my work on https://github.com/kubernetes/kubernetes/issues/15634
This refactoring is expected to be completely finished and then I will add a verify scripts in `hack`
Automatic merge from submit-queue
split genericapiserver configuration apart so that you can run withou…
…t flag options
@dims Take a look at this re-slicing of the `genericapiserver.Config` creation. I think this helps composers overall and resolves the chicken/egg problem you were having.
Automatic merge from submit-queue
kubeadm implement preflight checks
Checks that user running kubeamd init and join is root and will only execute
command if user is root. Moved away from using kubectl error handling to
having kubeadm handle its own errors. This should allow kubeadm to have
more meaningful errors, exit codes, and logging for specific kubeadm use
cases.
fixes#33908
Add skip-preflight-checks to known flags.
Fix bug with preflight checks not returning system is-active as errors.
Fix error handling to use correct function.
Automatic merge from submit-queue
kube-dns: Add --dns-bind-address flag
Currently, `kube-dns` always binds to `0.0.0.0` when starting the SkyDNS server. Sometimes it's useful to have it bind to a different address for testing. So add a new `--dns-bind-address` flag that can be used to configure this (default is `0.0.0.0`).
Includes checks for verifying services exist and are enabled, ports are
open, directories do not exist or are empty, and required binaries are
in the path.
Checks that user running kubeamd init and join is root and will only execute
command if user is root. Moved away from using kubectl error handling to
having kubeadm handle its own errors. This should allow kubeadm to have
more meaningful errors, exit codes, and logging for specific kubeadm use
cases.
Includes checks for verifying services exist and are enabled, ports are
open, directories do not exist or are empty, and required binaries are
in the path.
* `kubeadm join` will now retry to connect to the discovery service
API instead of exit on first failure. Allows for parallel install.
of master and slave nodes.
Automatic merge from submit-queue
WantsAuthorizer admission plugin support
The next step of PSP admission is to be able to limit the PSPs used based on user information. To do this the admission plugin would need to make authz checks for the `user.Info` in the request. This code allows a plugin to request the injection of an authorizer to allow it to make the authz checks.
Note: this could be done with a SAR, however since admission is running in the api server using the SAR would incur an extra hop vs using the authorizer directly.
@deads2k @derekwaynecarr
Automatic merge from submit-queue
Test API more extensivelly before declaring readiness
**What this PR does / why we need it**:
It's possible that first deployment kubeadm creates will hit `deployments.extensions "kube-discovery" is forbidden: not yet ready to handle request`, which comes from NamespaceLifecycle admission controller and has something to do with cache. According to @derekwaynecarr, we need to create a namespace-scoped resource to really check for this. I didn't want to make a check with deployment of whatever comes first right now, and decided to have explicit step for this in `apiclient.go`.
**Which issue this PR fixes**: fixes#34411
**Special notes for your reviewer**: @kubernetes/sig-cluster-lifecycle
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Generate unique Operation IDs for root OpenAPI spec
This PR adds a customization method GetOperationID to OpenAPI spec generation and then use it to make sure root spec has unique operation IDs by mostly adding GroupVersion to the start of operation ID.
Automatic merge from submit-queue
kubeadm: fix conversion macros and add kubeadm to round trip testing
Tests are probably broken but I'll fix. @jbeda this probably fixes your change unless we decide we need generated deep copies or conversions.
@kubernetes/sig-cluster-lifecycle
Automatic merge from submit-queue
clean api server cruft
Some cruft has developed over refactors. Remove that cruft.
@liggitt probably last in the chain so far
Automatic merge from submit-queue
convert deployment controller to shared informers
Converts the deployment controller to shared informers.
@kargakis I think you've been in here. Pretty straight forward swap.
Fixes#27687
Automatic merge from submit-queue
Enable service account signing key rotation
fixes#21007
```release-note
The kube-apiserver --service-account-key-file option can be specified multiple times, or can point to a file containing multiple keys, to enable rotation of signing keys.
```
This PR enables the apiserver authenticator to verify service account tokens signed by different private keys. This can be done two different ways:
* including multiple keys in the specified keyfile (e.g. `--service-account-key-file=keys.pem`)
* specifying multiple key files (e.g. `--service-account-key-file current-key.pem --service-account-key-file=old-key.pem`)
This is part of enabling signing key rotation:
1. update apiserver(s) to verify tokens signed with a new public key while still allowing tokens signed with the current public key (which is what this PR enables)
2. give controllermanager the new private key to sign new tokens with
3. remove old service account tokens (determined by verifying signature or by checking creationTimestamp) once they are no longer in use (determined using garbage collection or magic) or some other algorithm (24 hours after rotation, etc). For the deletion to immediately revoke the token, `--service-account-lookup` must be enabled on the apiserver.
4. once all old tokens are gone, update apiservers again, removing the old public key.
Automatic merge from submit-queue
kubeadm: mark etcd flags as deprecated
Lets mark flags that we don't plan on exposing through the command line permanently as deprecated from now on.
@kubernetes/sig-cluster-lifecycle
Automatic merge from submit-queue
Add cgroup-driver and cgroups-per-qos flags to kubelet
Add the flags needed to support pod-level cgroups to kubelet.
/cc @vishh @dchen1107 @dubstack
Automatic merge from submit-queue
decouple workqueue metrics from prometheus
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
We want to include the workqueue in client-go, but do not want to having to import Prometheus. This PR decouples the workqueue from prometheus.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Partially address https://github.com/kubernetes/kubernetes/issues/33497
User requested for `workqueue` in client-go: https://github.com/kubernetes/client-go/issues/4#issuecomment-249444848
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
The implicit registration of Prometheus metrics for workqueue has been removed, and a plug-able interface was added. If you were using workqueue in your own binaries and want these metrics, add the following to your imports in the main package: "k8s.io/pkg/util/workqueue/prometheus".
```
Automatic merge from submit-queue
Speed up dockerized builds
This PR speeds up dockerized builds. First, we make sure that we are as incremental as possible. The bigger change is that now we use rsync to move sources into the container and get data back out.
To do yet:
* [x] Add a random password to rsync. This is 128bit MD4, but it is better than nothing.
* [x] Lock down rsync to only come from the host.
* [x] Deal with remote docker engines -- this should be necessary for docker-machine on the mac.
* [x] Allow users to specify the port for the rsync daemon. Perhaps randomize this or let docker pick an ephemeral port and detect the port?
* [x] Copy back generated files so that users can check them in. This is done for `zz_generated.*` files generated by `make generated_files`
* [x] This should include generated proto files so that we can remove the hack-o-rama that is `hack/hack/update-*-dockerized.sh`
* [x] Start "versioning" the build container and the data container so that the CI system doesn't have to be manually kicked.
* [x] Get some benchmarks to qualify how much faster.
This replaces #28518 and is related to #30600.
cc @thockin @spxtr @david-mcmahon @MHBauer
Benchmarks by running `make clean ; sync ; time bash -xc 'time build/make-build-image.sh ; time sync ; time build/run.sh make ; time sync; time build/run.sh make'` on a GCE n1-standard-8 with PD-SSD.
| setup | build image | sync | first build | sync | second build | total |
|-------|-------------|----- |----------|------|--------------|------|
| baseline | 0m11.420s | 0m0.812s | 7m2.353s | 0m42.380s | 7m8.381s | 15m5.348s |
| this pr | 0m10.977s | 0m15.168s | 7m31.096s | 1m55.692s | 0m16.514s | 10m9.449s |
Automatic merge from submit-queue
Set deserialization cache size based on target memory usage
**Special notes for your reviewer**:
This is the PR we talked about yesterday.
**Release note**:
```release-note
To reduce memory usage to reasonable levels in smaller clusters, kube-apiserver now sets the deserialization cache size based on the target memory usage.
```
