Automatic merge from submit-queue (batch tested with PRs 46581, 55426, 54849). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiserver shutdown gracefully
**What this PR does / why we need it**:
apiserver shutdown gracefully and wait all non-long running requests finish before process exit.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54793
**Special notes for your reviewer**:
remove waitGroup, use atomic to count.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove Google Cloud KMS's in-tree integration
Removes the following introduced by #48574 and others:
* `kms.go` which contained the cloudkms-specific code for Google Cloud KMS service.
* Registering the Google Cloud KMS in the KMS plugin registry.
* Google's `cloudkms` API package from `vendor` folder.
The following changes are upcoming:
* Removal of KMSPluginRegistry. This would not be needed anymore, since KMS providers will be out-of-tree from now on (so no need of registering them, an address of the process would be enough).
* A service which allows encrypt/decrypt functionality (satisfies `envelope.Service` interface) if initialized with an IP/Port of an out-of-tree process serving KMS requests. Will tentatively use gRPC requests to talk to this external service.
Reference: https://github.com/kubernetes/kubernetes/pull/54439#issuecomment-340062801 and https://github.com/kubernetes/kubernetes/issues/51965#issuecomment-339333937.
```release-note
Google KMS integration was removed from in-tree in favor of a out-of-process extension point that will be used for all KMS providers.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Shutdown http handlers before shutting down audit backend
**What this PR does / why we need it**:
Currently, audit backend is shut down before http handlers stop processing requests, so some audit events can be dropped in case of batching webhook.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50781
**Special notes for your reviewer**:
**Release note**:
```release-note
Shutdown http handlers in kube-apiserver before shutting down audit backend.
```
Automatic merge from submit-queue (batch tested with PRs 54331, 54655, 54320, 54639, 54288). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ability to do object count quota for all namespaced resources
**What this PR does / why we need it**:
- Defines syntax for generic object count quota `count/<resource>.<group>`
- Migrates existing objects to support new syntax with old syntax
- Adds support to quota all standard namespace resources
- Updates the controller to do discovery and replenishment on those resources
- Updates unit tests
- Tweaks admission configuration around quota
- Add e2e test for replicasets (demonstrate dynamic generic counting)
```
$ kubectl create quota test --hard=count/deployments.extensions=2,count/replicasets.extensions=4,count/pods=3,count/secrets=4
resourcequota "test" created
$ kubectl run nginx --image=nginx --replicas=2
$ kubectl describe quota
Name: test
Namespace: default
Resource Used Hard
-------- ---- ----
count/deployments.extensions 1 2
count/pods 2 3
count/replicasets.extensions 1 4
count/secrets 1 4
```
**Special notes for your reviewer**:
- simple object count quotas no longer require writing code
- deferring support for custom resources pending investigation about how to share caches with garbage collector. in addition, i would like to see how this integrates with downstream quota usage in openshift.
**Release note**:
```release-note
Object count quotas supported on all standard resources using `count/<resource>.<group>` syntax
```
Automatic merge from submit-queue (batch tested with PRs 53000, 52870, 53569). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Allow running kube-apiserver without specifing service-cluster-ip-range
**What this PR does / why we need it**:
The flag service-cluster-ip-range has a default value which is not used by kube-apiserver, causing errors when people rely on that default value.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#52695
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed a bug which is causes kube-apiserver to not run without specifying service-cluster-ip-range
```
Automatic merge from submit-queue (batch tested with PRs 52240, 48145, 52220, 51698, 51777). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
add lease endpoint reconciler
**What this PR does / why we need it**: Adds OpenShift's LeaseEndpointReconciler to register kube-apiserver endpoints within the storage registry.
Adds a command-line argument `alpha-endpoint-reconciler-type` to the kube-apiserver.
Defaults to the old MasterCount reconciler.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixeskubernetes/community#939fixeskubernetes/kubernetes#22609
**Release note**:
```release-note
Adds a command-line argument to kube-apiserver called
--alpha-endpoint-reconciler-type=(master-count, lease, none) (default
"master-count"). The original reconciler is 'master-count'. The 'lease'
reconciler uses the storageapi and a TTL to keep alive an endpoint within the
`kube-apiserver-endpoint` storage namespace. The 'none' reconciler is a noop
reconciler that does not do anything. This is useful for self-hosted
environments.
```
/cc @lavalamp @smarterclayton @ncdc
Currently setting watch cache size for a given resource does not disable
the watch cache. This commit adds a new `default-watch-cache-size` flag
to map to the existing field, and refactors how watch cache sizes are
calculated to bring all of the code into one place. It also adds debug
logging to startup to allow us to verify watch cache enablement in
production.
Automatic merge from submit-queue
Enable batch/v1beta1.CronJobs by default
This PR re-applies the cronjobs->beta back (https://github.com/kubernetes/kubernetes/pull/51720) with the fix from @shyamjvs.
Fixes#51692
@apelisse @dchen1107 @smarterclayton ptal
@janetkuo @erictune fyi
Remove APIService OpenAPI spec when it is deleted
Add eTag support and returning httpStatus to OpenAPI spec downloader
Update aggregated OpenAPI spec periodically
Use delegate chain
Refactor OpenAPI aggregator to have separate controller and aggregation function
Enable OpenAPI spec for extensions api server
Do not filter paths. higher priority specs wins the conflicting paths
Move OpenAPI aggregation controller to pkg/controller/openapi
Automatic merge from submit-queue (batch tested with PRs 50775, 51397, 51168, 51465, 51536)
Enable batch/v1beta1.CronJobs by default
This PR moves to CronJobs beta entirely, enabling `batch/v1beta1` by default.
Related issue: #41039
@erictune @janetkuo ptal
```release-note
Promote CronJobs to batch/v1beta1.
```
The current registration of admission plug-ins happen too late so the
command line help is not yielding correct/useful information to users.
This is already causing confusion as mentioned in some issues.
This patch fixes it by moving plugins.go down to options package because
"plugins" in this context are themselves options for users. Registration
of plugins is not an expensive operation and it is already done in most
execution paths.
In future, we may want to revisit these plugins and migrate them to the
shared apiserver repo when appropriate.
Automatic merge from submit-queue (batch tested with PRs 49989, 49806, 49649, 49412, 49512)
This adds an etcd health check endpoint to kube-apiserver
addressing https://github.com/kubernetes/kubernetes/issues/48215.
**What this PR does / why we need it**:
This ensures kube-apiserver `/healthz` endpoint fails whenever connectivity cannot be established to etcd, also ensures the etcd preflight checks works with unix sockets
**Which issue this PR fixes**: fixes#48215
**Special notes for your reviewer**:
This PR does not use the etcd client directly as the client object is wrapped behind the storage interface and not exposed directly for use, so I decided to reuse what's being done in the preflight. So this will only check fail for connectivity and not etcd auth related problems. I did not write tests for the endpoint because I couldn't find examples that I could follow for writing tests for healthz related endpoints, I'll be willing to write those tests if someone can point me at a relevant one.
**Release note**:
```release-note
Add etcd connectivity endpoint to healthz
```
@deads2k please help review, thanks!
Automatic merge from submit-queue (batch tested with PRs 48012, 47443, 47702, 47178)
incluster config will be used when creating external shared informers.
**What this PR does / why we need it**:
Previously the loopback configuration was used to talk to the server.
As a consequence a custom API server was unable to talk to the root API server.
This PR changes the above by using incluster configuration to create shared informers.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47878, 47503, 47857)
restore working aggregator and avoid duplicate informers
Fixes https://github.com/kubernetes/kubernetes/issues/47866
This runs the informer all the way through and makes sure its started.
@lavalamp ptal
@kubernetes/sig-api-machinery-bugs
Automatic merge from submit-queue
Separate serviceaccount and secret storage config
Fixes#47815, and is required in order to enable the secret encryption feature with a recommended configuration
This passes distinct storage options for serviceaccounts and secrets, since secrets can now have an encrypting transformer associated with them
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
Add configuration for encryption providers
## Additions
Allows providing a configuration file (using flag `--experimental-encryption-provider-config`) to use the existing AEAD transformer (with multiple keys) by composing mutable transformer, prefix transformer (for parsing providerId), another prefix transformer (for parsing keyId), and AES-GCM transformers (one for each key). Multiple providers can be configured using the configuration file.
Example configuration:
```
kind: EncryptionConfig
apiVersion: v1
resources:
- resources:
- namespaces
providers:
- aes:
keys:
- name: key1
secret: c2vjcmv0iglzihnly3vyzq==
- name: key2
secret: dghpcybpcybwyxnzd29yza==
- identity: {}
```
Need for configuration discussed in:
#41939
[Encryption](3418b4e4c6/contributors/design-proposals/encryption.md)
**Pathway of a read/write request**:
1. MutableTransformer
2. PrefixTransformer reads the provider-id, and passes the request further if that matches.
3. PrefixTransformer reads the key-id, and passes the request further if that matches.
4. GCMTransformer tries decrypting and authenticating the cipher text in case of reads. Similarly for writes.
## Caveats
1. To keep the command line parameter parsing independent of the individual transformer's configuration, we need to convert the configuration to an `interface{}` and manually parse it in the transformer. Suggestions on better ways to do this are welcome.
2. Flags `--encryption-provider` and `--encrypt-resource` (both mentioned in [this document](3418b4e4c6/contributors/design-proposals/encryption.md) ) are not supported in this because they do not allow more than one provider, and the current format for the configuration file possibly supersedes their functionality.
3. Currently, it can be tested by adding `--experimental-encryption-provider-config=config.yml` to `hack/local-up-cluster.sh` on line 511, and placing the above configuration in `config.yml` in the root project directory.
Previous discussion on these changes:
https://github.com/sakshamsharma/kubernetes/pull/1
@jcbsmpsn @destijl @smarterclayton
## TODO
1. Investigate if we need to store keys on disk (per [encryption.md](3418b4e4c6/contributors/design-proposals/encryption.md (option-1-simple-list-of-keys-on-disk)))
2. Look at [alpha flag conventions](https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go)
3. Need to reserve `k8s:enc` prefix formally for encrypted data. Else find a better way to detect transformed data.
Handle failure cases on startup gracefully to avoid causing cascading
errors and poor initialization in other components. Initial errors from
config load cause the initializer to pause and hold requests. Return
typed errors to better communicate failures to clients.
Add code to handle two specific cases - admin wants to bypass
initialization defaulting, and mirror pods (which want to bypass
initialization because the kubelet owns their lifecycle).
Automatic merge from submit-queue (batch tested with PRs 46681, 46786, 46264, 46680, 46805)
Enable Dialer on the Aggregator
Centralize the creation of the dialer during startup.
Have the dialer then passed in to both APIServer and Aggregator.
Aggregator the uses the dialer as its Transport base.
**What this PR does / why we need it**:Enables the Aggregator to use the Dialer/SSHTunneler to connect to the user-apiserver.
**Which issue this PR fixes** : fixes ##46679
**Special notes for your reviewer**:
**Release note**: None
Automatic merge from submit-queue (batch tested with PRs 46239, 46627, 46346, 46388, 46524)
Dynamic webhook admission control plugin
Unit tests pass.
Needs plumbing:
* [ ] service resolver (depends on @wfender PR)
* [x] client cert (depends on ????)
* [ ] hook source (depends on @caesarxuchao PR)
Also at least one thing will need to be renamed after Chao's PR merges.
```release-note
Allow remote admission controllers to be dynamically added and removed by administrators. External admission controllers make an HTTP POST containing details of the requested action which the service can approve or reject.
```
Add location transformer, config for transformers
Location transformer helps choose the most specific transformer for
read/write operations depending on the path of resource being accessed.
Configuration allows use of --experimental-encryption-provider-config
to set up encryption providers. Only AEAD is supported at the moment.
Add new files to BUILD, AEAD => k8s-aes-gcm
Use group resources to select encryption provider
Update tests for configuration parsing
Remove location transformer
Allow specifying providers per resource group in configuration
Add IdentityTransformer configuration option
Fix minor issues with initial AEAD implementation
Unified parsing of all configurations
Parse configuration using a union struct
Run configuration parsing in APIserver, refactor parsing
More gdoc, fix minor bugs
Add test coverage for combined transformers
Use table driven tests for encryptionconfig
Centralize the creation of the dialer during startup.
Have the dialer then passed in to both APIServer and Aggregator.
Aggregator the sets the dialer on its Transport base.
This should allow the SSTunnel to be used but also allow the Aggregation
Auth to work with it.
Depending on Environment InsecureSkipTLSVerify *may* need to be set to
true.
Fixed as few tests to call CreateDialer as part of start-up.
Automatic merge from submit-queue (batch tested with PRs 45327, 46217, 46377, 46428, 46588)
Fix comment typo in kube-apiserver and cachesize
**What this PR does / why we need it**:
Fix comment typo in files cmd/kube-apiserver/app/server.go and pkg/registry/cachesize/cachesize.go
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Not a major issue, just a minor improvement.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
remove init blocks from all admission plugins
**What this PR does / why we need it**:
removes init blocks from all admission plugins
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Allow the /logs handler on the apiserver to be toggled.
Adds a flag to kube-apiserver, and plumbs through en environment variable in configure-helper.sh
ApplyTo adds the admission chain to the server configuration the method lazily initializes a generic plugin
that is appended to the list of pluginInitializers.
apiserver.Config will hold an instance of SharedInformerFactory to ensure we only have once instance.
The field will be initialized in apisever.SecureServingOptions
Automatic merge from submit-queue (batch tested with PRs 42835, 42974)
remove legacy insecure port options from genericapiserver
The insecure port has been a source of problems and it will prevent proper aggregation into a cluster, so the genericapiserver has no need for it. In addition, there's no reason for it to be in the main kube-apiserver flow either. This pull removes it from genericapiserver and removes it from the shared kube-apiserver code. It's still wired up in the command, but its no longer possible for someone to mess up and start using in mainline code.
@kubernetes/sig-api-machinery-misc @ncdc
Automatic merge from submit-queue (batch tested with PRs 42900, 43044, 42896, 43308, 43621)
require codecfactory
The genericapiserver requires a codec to start. Help new comers to the API by forcing them to set it when they create a new config.
Automatic merge from submit-queue
break kube-apiserver start into stages
This is a code shuffle which breaks the kube-apiserver start into
1. set defaults on the options
1. create the generic config from the options
1. create the master config from the generic config and the options
This makes apiserver composition easy/possible later on.
Automatic merge from submit-queue
Fix Multizone pv creation on GCE
When Multizone is enabled static PV creation on GCE
fails because Cloud provider configuration is not
available in admission plugins.
cc @derekwaynecarr @childsb
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Apiserver: wait for Etcd to become available on startup
fixes#37704
Automatic merge from submit-queue
add client-ca to configmap in kube-public
Client CA information is not secret and it's required for any API server trying to terminate a TLS connection. This pull adds the information to configmaps in `kube-public` that look like this:
```yaml
apiVersion: v1
data:
client-ca.crt: |
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
requestheader-allowed-names: '["system:auth-proxy"]'
requestheader-client-ca-file: |
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
requestheader-extra-headers-prefix: '["X-Remote-Extra-"]'
requestheader-group-headers: '["X-Remote-Group"]'
requestheader-username-headers: '["X-Remote-User"]'
kind: ConfigMap
metadata:
creationTimestamp: 2017-02-22T17:54:37Z
name: extension-apiserver-authentication
namespace: kube-system
resourceVersion: "6"
selfLink: /api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication
uid: fa1dd328-f927-11e6-8b0e-28d2447dc82b
```
@kubernetes/sig-auth-api-reviews @liggitt @kubernetes/sig-api-machinery-pr-reviews @lavalamp @sttts
There will need to be a corresponding pull for permissions
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
Add more options to the RecommendedOptions struct.
Builds on https://github.com/kubernetes/kubernetes/pull/41028
Adds `AuditOptions` to the `RecommendedOptions`
@sttts @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
apiserver command line options lead to config
Logically command line options lead to config, not the other way around. We're clean enough now we can actually do the inversion.
WIP because I have some test cycles to fix, but this is all the meat.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
genericapiserver: cut off more dependencies – episode 7
Follow-up of https://github.com/kubernetes/kubernetes/pull/40822
approved based on #40363
Automatic merge from submit-queue
Removed HPA objects from extensions api group
fix#29778
``` release-note
HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead.
```
cc @kubernetes/autoscaling
Automatic merge from submit-queue
Use full package path for definition name in OpenAPI spec
We were using short package name (last part of package name) plus type name for OpenAPI spec definition name. That can result in duplicate names and make the spec invalid. To be sure we will always have unique names, we are going to use full package name as definition name. Also "x-kubernetes-tag" custom field is added to definitions to list Group/Version/Kind for the definitions that has it. This will help clients to discover definitions easier.
Lastly, we've added a reference from old definition names to the new ones to keep backward compatibilities. The list of old definitions will not be updated.
**Release note**:
- Rename OpenAPI definition names to type's full package names to prevent duplicates
- Create OpenAPI extension "x-kubernetes-group-version-kind" for definitions to store Group/Version/Kind
- Deprecate old definition names and create a reference to the new definitions. Old definitions will be removed in the next release.
Automatic merge from submit-queue (batch tested with PRs 38212, 38792, 39641, 36390, 39005)
Generate a kubelet CA and kube-apiserver cert-pair for kubelet auth.
cc @cjcullen
Automatic merge from submit-queue
fix glog message typo about init deserialization cache and watch cache
**What this PR does / why we need it**:
fix typo `Initalizing` to `Initializing`
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue
genericapiserver: unify swagger and openapi in config
- make swagger config customizable
- remove superfluous `Config.Enable*` flags for OpenAPI and Swagger.
This is necessary for downstream projects to tweak the swagger spec.
Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)
Detect long-running requests from parsed request info
Follow up to https://github.com/kubernetes/kubernetes/pull/36064
Uses parsed request info to more tightly match verbs and subresources
Removes regex-based long-running request path matching (which is easily fooled)
```release-note
The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach).
```
Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)
remove unnecessary fields from genericapiserver config
Cleans up some unnecessary fields in the genericapiserver config.
This removes all dependencies on Config during cert generation, only operating
on ServerRunOptions. This way we get rid of the repeated call of Config.Complete
and cleanly stratify the GenericApiServer bootstrapping.
Automatic merge from submit-queue
specify custom ca file to verify the keystone server
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Sometimes the keystone server's certificate is self-signed, mainly used for internal development, testing and etc.
For this kind of ca, we need a way to verify the keystone server.
Otherwise, below error will occur.
> x509: certificate signed by unknown authority
This patch provide a way to pass in a ca file to verify the keystone server when starting `kube-apiserver`.
**Which issue this PR fixes** : fixes#22695, #24984
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
```
Automatic merge from submit-queue
Remove static kubelet client, refactor ConnectionInfoGetter
Follow up to https://github.com/kubernetes/kubernetes/pull/33718
* Collapses the multi-valued return to a `ConnectionInfo` struct
* Removes the "raw" connection info method and interface, since it was only used in a single non-test location (by the "real" connection info method)
* Disentangles the node REST object from being a ConnectionInfoProvider itself by extracting an implementation of ConnectionInfoProvider that takes a node (using a provided NodeGetter) and determines ConnectionInfo
* Plumbs the KubeletClientConfig to the point where we construct the helper object that combines the config and the node lookup. I anticipate adding a preference order for choosing an address type in https://github.com/kubernetes/kubernetes/pull/34259
Automatic merge from submit-queue
Run rbac authorizer from cache
RBAC authorization can be run very effectively out of a cache. The cache is a normal reflector backed cache (shared informer).
I've split this into three parts:
1. slim down the authorizer interfaces
1. boilerplate for adding rbac shared informers and associated listers which conform to the new interfaces
1. wiring
@liggitt @ericchiang @kubernetes/sig-auth
Automatic merge from submit-queue
WantsAuthorizer admission plugin support
The next step of PSP admission is to be able to limit the PSPs used based on user information. To do this the admission plugin would need to make authz checks for the `user.Info` in the request. This code allows a plugin to request the injection of an authorizer to allow it to make the authz checks.
Note: this could be done with a SAR, however since admission is running in the api server using the SAR would incur an extra hop vs using the authorizer directly.
@deads2k @derekwaynecarr
Automatic merge from submit-queue
clean api server cruft
Some cruft has developed over refactors. Remove that cruft.
@liggitt probably last in the chain so far
Automatic merge from submit-queue
Enable service account signing key rotation
fixes#21007
```release-note
The kube-apiserver --service-account-key-file option can be specified multiple times, or can point to a file containing multiple keys, to enable rotation of signing keys.
```
This PR enables the apiserver authenticator to verify service account tokens signed by different private keys. This can be done two different ways:
* including multiple keys in the specified keyfile (e.g. `--service-account-key-file=keys.pem`)
* specifying multiple key files (e.g. `--service-account-key-file current-key.pem --service-account-key-file=old-key.pem`)
This is part of enabling signing key rotation:
1. update apiserver(s) to verify tokens signed with a new public key while still allowing tokens signed with the current public key (which is what this PR enables)
2. give controllermanager the new private key to sign new tokens with
3. remove old service account tokens (determined by verifying signature or by checking creationTimestamp) once they are no longer in use (determined using garbage collection or magic) or some other algorithm (24 hours after rotation, etc). For the deletion to immediately revoke the token, `--service-account-lookup` must be enabled on the apiserver.
4. once all old tokens are gone, update apiservers again, removing the old public key.
Automatic merge from submit-queue
Set deserialization cache size based on target memory usage
**Special notes for your reviewer**:
This is the PR we talked about yesterday.
**Release note**:
```release-note
To reduce memory usage to reasonable levels in smaller clusters, kube-apiserver now sets the deserialization cache size based on the target memory usage.
```
Automatic merge from submit-queue
Allow anonymous API server access, decorate authenticated users with system:authenticated group
When writing authorization policy, it is often necessary to allow certain actions to any authenticated user. For example, creating a service or configmap, and granting read access to all users
It is also frequently necessary to allow actions to any unauthenticated user. For example, fetching discovery APIs might be part of an authentication process, and therefore need to be able to be read without access to authentication credentials.
This PR:
* Adds an option to allow anonymous requests to the secured API port. If enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of `system:anonymous` and a group of `system:unauthenticated`. Note: this should only be used with an `--authorization-mode` other than `AlwaysAllow`
* Decorates user.Info returned from configured authenticators with the group `system:authenticated`.
This is related to defining a default set of roles and bindings for RBAC (https://github.com/kubernetes/features/issues/2). The bootstrap policy should allow all users (anonymous or authenticated) to request the discovery APIs.
```release-note
kube-apiserver learned the '--anonymous-auth' flag, which defaults to true. When enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of 'system:anonymous' and a group of 'system:unauthenticated'.
Authenticated users are decorated with a 'system:authenticated' group.
NOTE: anonymous access is enabled by default. If you rely on authentication alone to authorize access, change to use an authorization mode other than AlwaysAllow, or or set '--anonymous-auth=false'.
```
c.f. https://github.com/kubernetes/kubernetes/issues/29177#issuecomment-244191596
Automatic merge from submit-queue
Allow secure access to apiserver from Admission Controllers
* Allow options.InsecurePort to be set to 0 to switch off insecure access
* In NewSelfClient, Set the TLSClientConfig to the cert and key files
if InsecurePort is switched off
* Mint a bearer token that allows the client(s) created in NewSelfClient
to talk to the api server
* Add a new authenticator that checks for this specific bearer token
Fixes#13598
* Allow options.InsecurePort to be set to 0 to switch off insecure access
* In NewSelfClient, Set the TLSClientConfig to the cert and key files
if InsecurePort is switched off
* Mint a bearer token that allows the client(s) created in NewSelfClient
to talk to the api server
* Add a new authenticator that checks for this specific bearer token
Fixes#13598
Automatic merge from submit-queue
Cleanup non-rest apiserver handlers
- rename MuxHelper -> PathRecorderMux
- move non-rest handlers into routes packages within genericapiserver and `pkg/routes` (those from master)
- move ui and logs handlers out of genericapiserver (they are
not generic)
- make version handler configurable (`config.EnableVersion`)
- rename MuxHelper -> PathRecorderMux
- move non-rest handlers into routes packages within genericapiserver and
`pkg/routes` (those from master)
- move ui and logs handlers out of genericapiserver (they are
not generic)
- make version handler configurable (`config.EnableVersion`)
Automatic merge from submit-queue
pkg/genericapiserver/options: don't import pkg/apiserver
Refactor the authorization options for the API server so
pkg/apiserver isn't directly imported by the options package.
Closes#28544
cc @smarterclayton
@madhusudancs, @nikhiljindal I've updated `federation/cmd/federation-apiserver/app/server.go` to include the RBAC options with this change. I don't know if this was intentionally left out in the first place but would like your feedback.
Automatic merge from submit-queue
make the resource prefix in etcd configurable for cohabitation
This looks big, its not as bad as it seems.
When you have different resources cohabiting, the resource name used for the etcd directory needs to be configurable. HPA in two different groups worked fine before. Now we're looking at something like RC<->RS. They normally store into two different etcd directories. This code allows them to be configured to store into the same location.
To maintain consistency across all resources, I allowed the `StorageFactory` to indicate which `ResourcePrefix` should be used inside `RESTOptions` which already contains storage information.
@lavalamp affects cohabitation.
@smarterclayton @mfojtik prereq for our rc<->rs and d<->dc story.
The codec factory should support two distinct interfaces - negotiating
for a serializer with a client, vs reading or writing data to a storage
form (etcd, disk, etc). Make the EncodeForVersion and DecodeToVersion
methods only take Encoder and Decoder, and slight refactoring elsewhere.
In the storage factory, use a content type to control what serializer to
pick, and use the universal deserializer. This ensures that storage can
read JSON (which might be from older objects) while only writing
protobuf. Add exceptions for those resources that may not be able to
write to protobuf (specifically third party resources, but potentially
others in the future).
