Automatic merge from submit-queue (batch tested with PRs 39152, 39142, 39055)
Add test for json tags on internal and external types
Follow up from https://github.com/kubernetes/kubernetes/pull/38406
- adds static analysis tests preventing internal types from adding new json or protobuf tags
- adds static analysis tests requiring json tags on external types (and enforcing lower-case first letter)
- fixes issues found by the tests
- move rest handler into handlers package
- move errors into handlers/errors package
- move negotation code into handlers/negotation package
- move http response writer helpers into handlers/responsewriter package
- split up pkg/apiserver/apiserver.go
Note: this is only an intermediate step towards a proper genericapiserver
package hierarchy.
Automatic merge from submit-queue (batch tested with PRs 38154, 38502)
Rename "release_1_5" clientset to just "clientset"
We used to keep multiple releases in the main repo. Now that [client-go](https://github.com/kubernetes/client-go) does the versioning, there is no need to keep releases in the main repo. This PR renames the "release_1_5" clientset to just "clientset", clientset development will be done in this directory.
@kubernetes/sig-api-machinery @deads2k
```release-note
The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go
```
Automatic merge from submit-queue (batch tested with PRs 38354, 38371)
Add GetOptions parameter to Get() calls in client library
Ref #37473
This PR is super mechanical - the non trivial commits are:
- Update client generator
- Register GetOptions in batch/v2alpha1 group
Automatic merge from submit-queue
Enable OpenAPI spec validation
Spec validation was failing on Jenkins. I am enabling it in this PR to figure out if we can get it pass. No review is necessary until all test passes.
Mark NodeLegacyHostIP will be deprecated in 1.7;
Let cloudprovider that used to only set NodeLegacyHostIP set the IP as both InternalIP and ExternalIP, to allow dprecation in 1.7
Automatic merge from submit-queue
Remove non-generic options from genericapiserver.Config
Remove non-generic options from genericapiserver.Config. Changes the discovery CIDR/IP information to an interface and then demotes several fields.
I haven't pulled from them genericapiserver.Options, but that's a future option we have. Segregation as as a followup at the very least.
Automatic merge from submit-queue
Switch DisruptionBudget api from bool to int allowed disruptions [only v1beta1]
Continuation of #34546. Apparently it there is some bug that prevents us from having 2 different incompatibile version of API in integration tests. So in this PR v1alpha1 is removed until testing infrastructure is fixed.
Base PR comment:
Currently there is a single bool in disruption budget api that denotes whether 1 pod can be deleted or not. Every time a pod is deleted the apiserver filps the bool to false and the disruptionbudget controller sets it to true if more deletions are allowed. This works but it is far from optimal when the user wants to delete multiple pods (for example, by decreasing replicaset size from 10000 to 8000).
This PR adds a new api version v1beta1 and changes bool to int which contains a number of pods that can be deleted at once.
cc: @davidopp @mml @wojtek-t @fgrzadkowski @caesarxuchao
Automatic merge from submit-queue
promote /healthz and /metrics to genericapiserver
Promotes `/healthz` to genericapiserver with methods to add healthz checks before running.
Promotes `/metrics` to genericapiserver gated by config flag.
@lavalamp adds the healthz checks linked to `postStartHooks` as promised.
Automatic merge from submit-queue
Remove static kubelet client, refactor ConnectionInfoGetter
Follow up to https://github.com/kubernetes/kubernetes/pull/33718
* Collapses the multi-valued return to a `ConnectionInfo` struct
* Removes the "raw" connection info method and interface, since it was only used in a single non-test location (by the "real" connection info method)
* Disentangles the node REST object from being a ConnectionInfoProvider itself by extracting an implementation of ConnectionInfoProvider that takes a node (using a provided NodeGetter) and determines ConnectionInfo
* Plumbs the KubeletClientConfig to the point where we construct the helper object that combines the config and the node lookup. I anticipate adding a preference order for choosing an address type in https://github.com/kubernetes/kubernetes/pull/34259
Automatic merge from submit-queue
convert TPR controller to posthook instead of disable flag
Converts the third party resource controller into a posthook using a loopback client instead going direct to etcd. This let's us eliminate more flags and special-casing during initialization. Also, using a client brings us closer to building this without side-effects for downstream composers.
Automatic merge from submit-queue
Change legacy API resource registration
Updates the legacy API resource registration to emphasize its different-ness and to simplify supporting objects. The option has to remain in the genericapiserverconfig for multiple prefixes to enable cases where composers/extenders had composed additional groupless APIs. See OpenShift as an example.
However this is now transparent to "normal" composers.
@ncdc since sttts is out.
Automatic merge from submit-queue
remove testapi.Default.GroupVersion
I'm going to try to take this as a series of mechanicals. This removes `testapi.Default.GroupVersion()` and replaces it with `registered.GroupOrDie(api.GroupName).GroupVersion`.
@caesarxuchao I'm trying to see how much of `pkg/api/testapi` I can remove.
Automatic merge from submit-queue
fix pod eviction storage
Refactor pod eviction storage to remove the tight order coupling of the storage. This also gets us ready to deal with cases where API groups are not co-located on the same server, though the particular client being used would assume a proxy.
Automatic merge from submit-queue
Prune unused parameters and call only once
Two parameters of `InstallServiceErrorHandler` were not used. Pruning them allows us to move the service error handler installation for each api group to be moved into the constructor.
Automatic merge from submit-queue
Cleanup non-rest apiserver handlers
- rename MuxHelper -> PathRecorderMux
- move non-rest handlers into routes packages within genericapiserver and `pkg/routes` (those from master)
- move ui and logs handlers out of genericapiserver (they are
not generic)
- make version handler configurable (`config.EnableVersion`)
Automatic merge from submit-queue
restrict discoverable versions to those that have resources
Prevents versions with no resources from appearing in discovery.
@soltysh ptal. I think you have some issues for this, mind wiring them up?
```
kubectl api-versions
apps/v1alpha1
authentication.k8s.io/v1beta1
authorization.k8s.io/v1beta1
autoscaling/v1
batch/v1
certificates.k8s.io/v1alpha1
extensions/v1beta1
policy/v1alpha1
rbac.authorization.k8s.io/v1alpha1
storage.k8s.io/v1beta1
v1
```
Fixes #29998.
- rename MuxHelper -> PathRecorderMux
- move non-rest handlers into routes packages within genericapiserver and
`pkg/routes` (those from master)
- move ui and logs handlers out of genericapiserver (they are
not generic)
- make version handler configurable (`config.EnableVersion`)
Automatic merge from submit-queue
Doc API group suffix, add test to catch new groups
Spawned from discussion in https://github.com/kubernetes/kubernetes/pull/31887
Doc and add tests to ensure new API groups are suffixed.
Also changed the doc to reference an API group containing the suffix as a starting point for new API groups.
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Automatic merge from submit-queue
add logging for enabled/disabled API Groups
Adds logging to the apiserver to indicate which API groups are enabled and disabled as go through. This will make it easier to identify what's gone wrong in cases where the API server is down during diagnoses and the config is inaccessible, like in GKE tests. For example https://github.com/kubernetes/kubernetes/issues/32185#issuecomment-245255700 .
@wojtek-t This may have made the problem more obvious.
Automatic merge from submit-queue
privatize, document, and scrub GenericAPIServer
I've gone through more of the `GenericAPIServer` struct, started documenting what the fields do and privatizing ones that aren't used elsewhere or are only used by components that need some refactoring too.
Automatic merge from submit-queue
add selfsubjectaccessreview API
Exposes the REST API for self subject access reviews. This allows a user to see whether or not they can perform a particular action.
@kubernetes/sig-auth
Automatic merge from submit-queue
Enable v3 Client as the default on UTs
Updates the default initialization to use clientv3 interface to etcd3, and fixes the UTs.
This PR includes a cherry-pick of https://github.com/kubernetes/kubernetes/pull/30634 so we can validate the tests, so do not merge until that PR is complete.
Automatic merge from submit-queue
Configure webhook
**What this PR does / why we need it**: this configures the image policy webhook + admission controller for gce/gci.
addresses: #22888
**Release note**:
```Configure image verification admission controller and webhook on gce.
```
Automatic merge from submit-queue
add subjectaccessreviews resource
Adds a subjectaccessreviews endpoint that uses the API server's authorizer to determine if a subject is allowed to perform an action.
Part of kubernetes/features#37
Automatic merge from submit-queue
Refactoring runner resource container linedelimiter to it's own pkg
Continuing my work ref #15634
Anyone is ok to review this fix.
Automatic merge from submit-queue
TLS bootstrap API group (alpha)
This PR only covers the new types and related client/storage code- the vast majority of the line count is codegen. The implementation differs slightly from the current proposal document based on discussions in design thread (#20439). The controller logic and kubelet support mentioned in the proposal are forthcoming in separate requests.
I submit that #18762 ("Creating a new API group is really hard") is, if anything, understating it. I've tried to structure the commits to illustrate the process.
@mikedanese @erictune @smarterclayton @deads2k
```release-note-experimental
An alpha implementation of the the TLS bootstrap API described in docs/proposals/kubelet-tls-bootstrap.md.
```
[]()
Automatic merge from submit-queue
Add EndpointReconcilerConfig to master Config
Add EndpointReconcilerConfig to master Config to allow downstream integrators to customize the reconciler and reconciliation interval when starting a customized master
@kubernetes/sig-api-machinery @deads2k @smarterclayton @liggitt @kubernetes/rh-cluster-infra
Add EndpointReconcilerConfig to master Config to allow downstream integrators to customize the reconciler
and reconciliation interval when starting a customized master.
Automatic merge from submit-queue
Automatically create the kube-system namespace
At the same time we ensure that the `default` namespace is present, it also creates `kube-system` if it doesn't exist.
`kube-system` will now exist from the beginning, and will be recreated every 10s if deleted, in the same manner as the `default` ns
This makes UX much better, no need for `kubectl`ing a `kube-system.yaml` file anymore for a function that is essential to Kubernetes (addons). For instance, this makes dashboard deployment much easier when there's no need to check for the `kube-system` ns first.
A follow up in the future may remove places where logic to manually create the kube-system namespace is present.
Also fixed a small bug where `CreateNamespaceIfNeeded` ignored the `ns` parameter and was hardcoded to `api.NamespaceDefault`.
@davidopp @lavalamp @thockin @mikedanese @bryk @cheld @fgrzadkowski @smarterclayton @wojtek-t @dlorenc @vishh @dchen1107 @bgrant0607 @roberthbailey
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/25196)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Remove myself from a bunch of OWNERS files
For the time being I am too overloaded to do non scheduler/admission related reviews that aren't explicitly assigned to me.
cc/ @brendandburns
The codec factory should support two distinct interfaces - negotiating
for a serializer with a client, vs reading or writing data to a storage
form (etcd, disk, etc). Make the EncodeForVersion and DecodeToVersion
methods only take Encoder and Decoder, and slight refactoring elsewhere.
In the storage factory, use a content type to control what serializer to
pick, and use the universal deserializer. This ensures that storage can
read JSON (which might be from older objects) while only writing
protobuf. Add exceptions for those resources that may not be able to
write to protobuf (specifically third party resources, but potentially
others in the future).
Automatic merge from submit-queue
Provide flags to use etcd3 backed storage
ref: #24405
What's in this PR?
- Add a new flag "storage-backend" to choose "etcd2" or "etcd3". By default (i.e. empty), it's "etcd2".
- Take out etcd config code into a standalone package and let it create etcd2 or etcd3 storage backend given user input.
Automatic merge from submit-queue
stop changing the root path of the root webservice
We shouldn't mutate the root path of the root webservice (see usage). Just write the path we want.
Add tests to watch behavior in both protocols (http and websocket)
against all 3 media types. Adopt the
`application/vnd.kubernetes.protobuf;stream=watch` media type for the
content that comes back from a watch call so that it can be
distinguished from a Status result.
Automatic merge from submit-queue
Use the first version as thirdparty resource preferredVersion
First commit is a one-liner, which implements the server-half of #23985.
The other two commits rearrange the test code, and add back a commented out test of thirdparty resource.
@lavalamp @nikhiljindal
Automatic merge from submit-queue
Move /resetMetrics to DELETE /metrics
Reduces the surface area of the API server slightly and allows
downstream components to have deleteable metrics. After this change
genericapiserver will *not* have metrics unless the caller defines it
(allows different apiserver implementations to make that choice on their
own).
@wojtek-t
Automatic merge from submit-queue
Make etcd cache size configurable
Instead of the prior 50K limit, allow users to specify a more sensible size for their cluster.
I'm not sure what a sensible default is here. I'm still experimenting on my own clusters. 50 gives me a 270MB max footprint. 50K caused my apiserver to run out of memory as it exceeded >2GB. I believe that number is far too large for most people's use cases.
There are some other fundamental issues that I'm not addressing here:
- Old etcd items are cached and potentially never removed (it stores using modifiedIndex, and doesn't remove the old object when it gets updated)
- Cache isn't LRU, so there's no guarantee the cache remains hot. This makes its performance difficult to predict. More of an issue with a smaller cache size.
- 1.2 etcd entries seem to have a larger memory footprint (I never had an issue in 1.1, even though this cache existed there). I suspect that's due to image lists on the node status.
This is provided as a fix for #23323
Automatic merge from submit-queue
the component status health check should check whether the scheme of backend storage url is https or not
fix https://github.com/kubernetes/kubernetes/issues/23897, when querying the component status of etcd (backend storage), the scheme of url is not checked and use `http` always, this commit aims to fix this.
Reduces the surface area of the API server slightly and allows
downstream components to have deleteable metrics. After this change
genericapiserver will *not* have metrics unless the caller defines it
(allows different apiserver implementations to make that choice on their
own).
Automatic merge from submit-queue
Implement a streaming serializer for watch
Changeover watch to use streaming serialization. Properly version the
watch objects. Implement simple framing for JSON and Protobuf (but not
YAML).
@wojtek-t @lavalamp
Automatic merge from submit-queue
Additional go vet fixes
Mostly:
- pass lock by value
- bad syntax for struct tag value
- example functions not formatted properly
Here are a list of changes along with an explanation of how they work:
1. Add a new string field called TargetSelector to the external version of
extensions Scale type (extensions/v1beta1.Scale). This is a serialized
version of either the map-based selector (in case of ReplicationControllers)
or the unversioned.LabelSelector struct (in case of Deployments and
ReplicaSets).
2. Change the selector field in the internal Scale type (extensions.Scale) to
unversioned.LabelSelector.
3. Add conversion functions to convert from two external selector fields to a
single internal selector field. The rules for conversion are as follows:
i. If the target resource that this scale targets supports LabelSelector
(Deployments and ReplicaSets), then serialize the LabelSelector and
store the string in the TargetSelector field in the external version
and leave the map-based Selector field as nil.
ii. If the target resource only supports a map-based selector
(ReplicationControllers), then still serialize that selector and
store the serialized string in the TargetSelector field. Also,
set the the Selector map field in the external Scale type.
iii. When converting from external to internal version, parse the
TargetSelector string into LabelSelector struct if the string isn't
empty. If it is empty, then check if the Selector map is set and just
assign that map to the MatchLabels component of the LabelSelector.
iv. When converting from internal to external version, serialize the
LabelSelector and store it in the TargetSelector field. If only
the MatchLabel component is set, then also copy that value to
the Selector map field in the external version.
4. HPA now just converts the LabelSelector field to a Selector interface
type to list the pods.
5. Scale Get and Update etcd methods for Deployments and ReplicaSets now
return extensions.Scale instead of autoscaling.Scale.
6. Consequently, SubresourceGroupVersion override and is "autoscaling"
enabled check is now removed from pkg/master/master.go
7. Other small changes to labels package, fuzzer and LabelSelector
helpers to piece this all together.
8. Add unit tests to HPA targeting Deployments and ReplicaSets.
9. Add an e2e test to HPA targeting ReplicaSets.
Also register replicationcontrollers/scale subresource. Along with
registering the resource, also specify the cross-group override for the
subresource since Scale belongs belongs to autoscaling/v1 but
ReplicationController belongs to api/v1.
Pass down into the server initialization the necessary interface for
handling client/server content type negotiation. Add integration tests
for the negotiation.
checkEndpointSubsetFormat ensures that,
1. the current master's IP is in the list of addresses
2. the number of IPs in the list exactly matches the master count
This is problematic while masters are in the process of starting
because it causes frequent updates to the kubernetes endpoints until
all masters have started and added themselves to the list.
checkEndpointSubsetFormat should report success if the current
master's IP is found and the count of addresses is less than or equal
to the expected count.
Added status REST storage.
Added validation for Status Updates.
Changed job controller to update status rather than just job
(which ignores status updates).
A lot of packages use StringSet, but they don't use anything else from
the util package. Moving StringSet into another package will shrink
their dependency trees significantly.
A cert from GCE shows:
- IP Address:23.236.49.122
- IP Address:10.0.0.1
- DNS:kubernetes,
- DNS:kubernetes.default
- DNS:kubernetes.default.svc
- DNS:kubernetes.default.svc.cluster.local
- DNS:e2e-test-zml-master
A similarly configured self signed cert shows:
- IP Address:23.236.49.122
- IP Address:10.0.0.1
- DNS:kubernetes
- DNS:kubernetes.default
- DNS:kubernetes.default.svc
So we are missing the fqdn kubernetes.default.svc.cluster.local. The
apiserver does not even know the fqdn! it's defined entirely by the
kubelet! We also do not have the cluster name certificate. This may be
--cluster-name= argument to the apiserver but will take a bit more
research.
A lot of the changes in apiserver could have been represented more
cleanly - this returns the signatures to their older behavior (and
unbreaks OpenShift).
