Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
genericapiserver: cut off more dependencies – episode 7
Follow-up of https://github.com/kubernetes/kubernetes/pull/40822
approved based on #40363
Automatic merge from submit-queue
Removed HPA objects from extensions api group
fix#29778
``` release-note
HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead.
```
cc @kubernetes/autoscaling
Automatic merge from submit-queue (batch tested with PRs 39169, 40719, 38954, 40808, 40689)
genericapiserver: cut off more dependencies – episode 5
Follow-up of https://github.com/kubernetes/kubernetes/pull/40658
Left for episode 6:
- pkg/storage/etc
- pkg/storage/etc3
- pkg/storage/storagebackend/factory
approved based on #40363
Automatic merge from submit-queue
Add websocket support for port forwarding
#32880
**Release note**:
```release-note
Port forwarding can forward over websockets or SPDY.
```
- adjust ports to int32
- CRI flows the websocket ports as query params
- Do not validate ports since the protocol is unknown
SPDY flows the ports as headers and websockets uses query params
- Only flow query params if there is at least one port query param
- split out port forwarding into its own package
Allow multiple port forwarding ports
- Make it easy to determine which port is tied to which channel
- odd channels are for data
- even channels are for errors
- allow comma separated ports to specify multiple ports
Add portfowardtester 1.2 to whitelist
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
move pkg/storage to apiserver
Mechanical move of `pkg/storage` (not sub packages) to `k8s.io/apiserver`.
@sttts
Automatic merge from submit-queue (batch tested with PRs 40111, 40368, 40342, 40274, 39443)
Eliminate "Unknown service type: ExternalName"
When creating an ExternalName service, rest.go still generate the warning message "Unknown service type: ExternalName". This should be eliminated as this type of service is supported now.
Automatic merge from submit-queue (batch tested with PRs 40405, 38601, 40083, 40730)
Remove extensions/v1beta1.Jobs leftover
It looks like I missed this one file when removing `extensions/v1beta1.Jobs` last time (#38614).
@caesarxuchao ptal, since you were reviewing last time
@kubernetes/sig-api-machinery-misc fyi
Automatic merge from submit-queue
genericapiserver: cut off more dependencies – episode 3
Compare commit subjects.
This is the first step to get `apiserver_test.go` and `watch_test.go` in `pkg/genericapiserver/endpoints` cutoff from k8s.io/kubernetes dependencies.
After this we have to sync client-go and then "episode 4" can go in.
approved based on #40363
Automatic merge from submit-queue
Update authn/authz owners and reviewers
Add myself as potential reviewer to several authn/authz-related packages.
Add @deads2k as approver to several packages he wrote.
Automatic merge from submit-queue (batch tested with PRs 38739, 40480, 40495, 40172, 40393)
Rename controller pkg/registry/core/controller to pkg/registry/core/r…
…eplicationcontroller
**What this PR does / why we need it**:
Rename controller pkg/registry/core/controller to pkg/registry/core/replicationcontroller
This will clarify the purpose of the controller since intent is replicationcontroller
Please refer to
https://github.com/kubernetes/kubernetes/issues/17648
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
NONE
**Release note**:
```NONE
```
Automatic merge from submit-queue
promote certificates api to beta
Mostly posting to see what breaks but also this API is ready to be promoted.
```release-note
Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading.
```
@kubernetes/api-approvers @jcbsmpsn @pipejakob
Automatic merge from submit-queue
move pkg/fields to apimachinery
Purely mechanical move of `pkg/fields` to apimachinery.
Discussed with @lavalamp on slack. Moving this an `labels` to apimachinery.
@liggitt any concerns? I think the idea of field selection should become generic and this ends up shared between client and server, so this is a more logical location.
Automatic merge from submit-queue
genericapiserver: cut off pkg/serviceaccount dependency
**Blocked** by pkg/api/validation/genericvalidation to be split up and moved into apimachinery.
Automatic merge from submit-queue (batch tested with PRs 39948, 39997)
Fix ScheduledJob -> CronJob rename leftovers
I found a few leftovers from the rename I did some time ago.
@kubernetes/sig-apps-misc ptal
Automatic merge from submit-queue
run staging client-go update
Chasing to see what real problems we have in staging-client-go.
@sttts you get similar results?
Automatic merge from submit-queue
replace global registry in apimachinery with global registry in k8s.io/kubernetes
We'd like to remove all globals, but our immediate problem is that a shared registry between k8s.io/kubernetes and k8s.io/client-go doesn't work. Since client-go makes a copy, we can actually keep a global registry with other globals in pkg/api for now.
@kubernetes/sig-api-machinery-misc @lavalamp @smarterclayton @sttts
Automatic merge from submit-queue (batch tested with PRs 39684, 39577, 38989, 39534, 39702)
Set PodStatus QOSClass field
This PR continues the work for https://github.com/kubernetes/kubernetes/pull/37968
It converts all local usage of the `qos` package class types to the new API level types (first commit) and sets the pod status QOSClass field in the at pod creation time on the API server in `PrepareForCreate` and in the kubelet in the pod status update path (second commit). This way the pod QOS class is set even if the pod isn't scheduled yet.
Fixes#33255
@ConnorDoyle @derekwaynecarr @vishh
Automatic merge from submit-queue
retry RBAC initialization for up to 30 seconds, kill server on failure
RBAC initialization needs to complete in order to bootstrap a cluster. When the bootstrapping fails (etcd not ready has happened in e2e runs), things fail badly and we don't even kill the API server to force it to retry. This retries for up to 30 seconds and kills the server if it never succeeds.
Fixes https://github.com/kubernetes/kubernetes/issues/39108
Automatic merge from submit-queue (batch tested with PRs 38354, 38371)
Add GetOptions parameter to Get() calls in client library
Ref #37473
This PR is super mechanical - the non trivial commits are:
- Update client generator
- Register GetOptions in batch/v2alpha1 group
Automatic merge from submit-queue (batch tested with PRs 37328, 38102, 37261, 31321, 38146)
Make thirdparty codec able to decode DeleteOptions
Fix#37278.
Without this PR, the gvk sent to the delegated codec will be the thirdparty one, which is not recognized by the delegated codec (usually api.Codecs).
RESTMapping method can now rely on RESTMappings by passing versions parameter and taking the first match found by RESTMappings method. In addition
a UT that test the new method has been added.
The only change in logic to what was before is when calling RESTMapping
we search all defaultGroupVersion as opposed to just one when no mapping was found for provided versions.
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
[etcd] test cleanup: remove unnecessary AddPrefix()
What?
Remove etcdtest.AddPrefix() in tests. They will be automatically prepended in etcd storage.
Why?
ref: #36290#36374
After the change, it will double prepend.
Automatic merge from submit-queue
Reuse fields and labels
This should significantly reduce memory allocations in apiserver in large cluster.
Explanation:
- every kubelet is refreshing watch every 5-10 minutes (this generally is not causing relist - it just renews watch)
- that means, in 5000-node cluster, we are issuing ~10 watches per second
- since we don't have "watch heartbets", the watch is issued from previously received resourceVersion
- to make some assumption, let's assume pods are evenly spread across pods, and writes for them are evenly spread - that means, that a given kubelet is interested in 1 per 5000 pod changes
- with that assumption, each watch, has to process 2500 (on average) previous watch events
- for each of such even, we are currently computing fields.
This PR is fixing this problem.
Automatic merge from submit-queue
switch bootstrap controller to use a client where possible
While looking at https://github.com/kubernetes/kubernetes/issues/37040, I found more places where we can use a normal client instead of a direct to etcd connection.
@wojtek-t you made similar changes in the same controller.
Automatic merge from submit-queue
Fix logic error in graceful deletion
If a resource has the following criteria:
1. deletion timestamp is not nil
2. deletion graceperiod seconds as persisted in storage is 0
the resource could never be deleted as we always returned pending graceful.
Automatic merge from submit-queue
Rename ScheduledJobs to CronJobs
I went with @smarterclayton idea of registering named types in schema. This way we can support both the new (CronJobs) and old (ScheduledJobs) resource name. Fixes#32150.
fyi @erictune @caesarxuchao @janetkuo
Not ready yet, but getting close there...
**Release note**:
```release-note
Rename ScheduledJobs to CronJobs.
```
Automatic merge from submit-queue
Adding more e2e tests for federated namespace cascading deletion and fixing bugs
Ref https://github.com/kubernetes/kubernetes/issues/33612
Adding more e2e tests for testing cascading deletion of federated namespace.
New tests are now verifying that cascading deletion happen when DeletionOptions.OrphanDependents=false and it does not happen when DeleteOptions.OrphanDependents=true.
Also updated deletion helper to always add OrphanFinalizer. generic registry will remove it if DeleteOptions.OrphanDependents=false. Also updated namespace registry to do the same.
We need to add the orphan finalizer to keep the orphan by default behavior. We assume that its dependents are going to be orphaned and hence add that finalizer. If user does not want the orphan behavior, he can do so using DeleteOptions and then the registry will remove that finalizer.
cc @kubernetes/sig-cluster-federation @caesarxuchao @derekwaynecarr
Automatic merge from submit-queue
Handle redirects in apiserver proxy handler
Overview:
1. Peek at the HTTP response from the proxied backend
2. If it is a redirect response (302/3), redo the request to the redirect location
3. If it's not a redirect, forward the response to the client and then set up the proxy as before
This change is required for implementing streaming requests in the Container Runtime Interface (CRI). See [design](https://docs.google.com/document/d/1OE_QoInPlVCK9rMAx9aybRmgFiVjHpJCHI9LrfdNM_s/edit).
For https://github.com/kubernetes/kubernetes/issues/29579
/cc @yujuhong
Automatic merge from submit-queue
Remove non-generic options from genericapiserver.Config
Remove non-generic options from genericapiserver.Config. Changes the discovery CIDR/IP information to an interface and then demotes several fields.
I haven't pulled from them genericapiserver.Options, but that's a future option we have. Segregation as as a followup at the very least.
Alter how runtime.SerializeInfo is represented to simplify negotiation
and reduce the need to allocate during negotiation. Simplify the dynamic
client's logic around negotiating type. Add more tests for media type
handling where necessary.
Automatic merge from submit-queue
Deny service ClusterIP update from `None`
**What this PR does / why we need it**: Headless service should not be transformed into a service with ClusterIP, therefore update of this field if it's set to `None` is disallowed.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#33029
**Release note**:
```release-note
Changing service ClusterIP from `None` is not allowed anymore.
```
Automatic merge from submit-queue
Support resourceVersion in GetToList - unify interface of List and Ge…
This pretty much unifies the interface of List() and GetToList() methods of storage interface.
I'm going to use it in a subsequent PR to improve performance of the whole cluster.
Automatic merge from submit-queue
Optimize label selector
The number of values for a given label is generally pretty small (in huge majority of cases it is exactly one value).
Currently computing selectors is up to 50% of CPU usage in both apiserver and scheduler.
Changing the structure in which those values are stored from map to slice improves the performance of typical usecase for computing selectors.
Early results:
- scheduler throughput it ~15% higher
- apiserver cpu-usage is also lower (seems to be also ~10-15%)
Automatic merge from submit-queue
Loadbalanced client src ip preservation enters beta
Sounds like we're going to try out the proposal (https://github.com/kubernetes/kubernetes/issues/30819#issuecomment-249877334) for annotations -> fields on just one feature in 1.5 (scheduler). Or do we want to just convert to fields right now?
Automatic merge from submit-queue
Remove static kubelet client, refactor ConnectionInfoGetter
Follow up to https://github.com/kubernetes/kubernetes/pull/33718
* Collapses the multi-valued return to a `ConnectionInfo` struct
* Removes the "raw" connection info method and interface, since it was only used in a single non-test location (by the "real" connection info method)
* Disentangles the node REST object from being a ConnectionInfoProvider itself by extracting an implementation of ConnectionInfoProvider that takes a node (using a provided NodeGetter) and determines ConnectionInfo
* Plumbs the KubeletClientConfig to the point where we construct the helper object that combines the config and the node lookup. I anticipate adding a preference order for choosing an address type in https://github.com/kubernetes/kubernetes/pull/34259
Automatic merge from submit-queue
convert TPR controller to posthook instead of disable flag
Converts the third party resource controller into a posthook using a loopback client instead going direct to etcd. This let's us eliminate more flags and special-casing during initialization. Also, using a client brings us closer to building this without side-effects for downstream composers.
Automatic merge from submit-queue
Avoid unnecessary allocation
This is supposed to avoid unnecessary memory allocations.
PodToSelectableFields seems to be the biggest contributor to memory allocations:
```
Showing top 10 nodes out of 247 (cum >= 83166442)
flat flat% sum% cum cum%
1796823715 31.09% 31.09% 1796823715 31.09% k8s.io/kubernetes/pkg/registry/core/pod.PodToSelectableFields
530856268 9.19% 40.28% 530856268 9.19% k8s.io/kubernetes/pkg/storage.NamespaceKeyFunc
241505351 4.18% 44.46% 241505351 4.18% reflect.unsafe_New
...
```
Automatic merge from submit-queue
Run rbac authorizer from cache
RBAC authorization can be run very effectively out of a cache. The cache is a normal reflector backed cache (shared informer).
I've split this into three parts:
1. slim down the authorizer interfaces
1. boilerplate for adding rbac shared informers and associated listers which conform to the new interfaces
1. wiring
@liggitt @ericchiang @kubernetes/sig-auth
Automatic merge from submit-queue
Add default cluster role bindings
Add default cluster roles bindings to rbac bootstrapping. Also adds a case for allowing escalation when you have no authenticator.
@liggitt I expect you may need to make peace with this.
Automatic merge from submit-queue
Use nodeutil.GetHostIP consistently when talking to nodes
Most of our communications from apiserver -> nodes used
nodutil.GetNodeHostIP, but a few places didn't - and this meant that the
node name needed to be resolvable _and_ we needed to populate valid IP
addresses.
```release-note
The apiserver now uses addresses reported by the kubelet in the Node object's status for apiserver->kubelet communications, rather than the name of the Node object. The address type used defaults to `InternalIP`, `ExternalIP`, and `LegacyHostIP` address types, in that order.
```
Automatic merge from submit-queue
remove testapi.Default.GroupVersion
I'm going to try to take this as a series of mechanicals. This removes `testapi.Default.GroupVersion()` and replaces it with `registered.GroupOrDie(api.GroupName).GroupVersion`.
@caesarxuchao I'm trying to see how much of `pkg/api/testapi` I can remove.
Automatic merge from submit-queue
fix pod eviction storage
Refactor pod eviction storage to remove the tight order coupling of the storage. This also gets us ready to deal with cases where API groups are not co-located on the same server, though the particular client being used would assume a proxy.
Automatic merge from submit-queue
Refactor: separate KubeletClient & ConnectionInfoGetter concepts
KubeletClient implements ConnectionInfoGetter, but it is not a complete
implementation: it does not set the kubelet port from the node record,
for example.
By renaming the method so that it does not implement the interface, we
are able to cleanly see where the "raw" GetConnectionInfo is used (it is
correct) and also have go type-checking enforce this for us.
This is related to #25532; I wanted to satisfy myself that what we were doing there was correct, and I wanted also to ensure that the compiler could enforce this going forwards.
This patch adds the option to set a nodeport when creating a NodePort
service. In case of a port allocation error due to a specified port
being out of the valid range, the error now includes the valid
range. If a `--node-port` value is not specified, it defaults to zero, in
which case the allocator will default to its current behavior of
assigning an available port.
This patch also adds a new helper function in `cmd/util/helpers.go` to
retrieve `Int32` cobra flags.
**Example**
```
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=1
The Service "mynodeport" is invalid: spec.ports[0].nodePort: Invalid
value: 1: provided port is not in the valid range. Valid ports range
from 30000-32767
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=30000
service "mynodeport" created
$ oc describe service mynodeport
Name: mynodeport
Namespace: default
Labels: app=mynodeport
Selector: app=mynodeport
Type: NodePort
IP: 172.30.81.254
Port: 8080-7777 8080/TCP
NodePort: 8080-7777 30000/TCP
Endpoints: <none>
Session Affinity: None
No events.
```
Most of our communications from apiserver -> nodes used
nodutil.GetNodeHostIP, but a few places didn't - and this
meant that the node name needed to be resolvable _and_ we needed
to populate valid IP addresses.
Fix the last few places that used the NodeName.
Issue #18525
Issue #9451
Issue #9728
Issue #17643
Issue #11543
Issue #22063
Issue #2462
Issue #22109
Issue #22770
Issue #32286
KubeletClient implements ConnectionInfoGetter, but it is not a complete
implementation: it does not set the kubelet port from the node record,
for example.
By renaming the method so that it does not implement the interface, we
are able to cleanly see where the "raw" GetConnectionInfo is used (it is
correct) and also have go type-checking enforce this for us.
Contination of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
I rolled back some of this from a previous commit because it just got
to big/messy. Will follow up with additional PRs
Signed-off-by: Doug Davis <dug@us.ibm.com>
Automatic merge from submit-queue
Make sure finalizers prevent deletion on storage that supports graceful deletion
Fixing bug:
Non-empty Finalizers fails to prevent a pod from being deleted, if deleteOptions.GracefulPeriod=0. See https://github.com/kubernetes/kubernetes/issues/32157#issuecomment-245778483
We didn't hit any issue with orphan finalizer because all our tests set finalizers on RC or RS, whose storage doesn't support graceful deletion.
cc @thockin @lavalamp
Automatic merge from submit-queue
api storage: Decouple Decorator from Filter
Continue #28249
What?
This PR decouples Decorator from Filter, i.e. remove Decorator in createFilter().
- For List, Decorator is called on returned list object.
- For Watch, we implement a new watcher to pipe through decorator. Error will be returned as a watch event.
Why?
- We want to change filter to SelectionPredicate struct. But Decorator is designed to be coupled with filtering.
- Per the discussion in #28249, decorator shouldn't be coupled to filter and error from Decorator should be returned instead of assuming false filtering.
Automatic merge from submit-queue
add selfsubjectaccessreview API
Exposes the REST API for self subject access reviews. This allows a user to see whether or not they can perform a particular action.
@kubernetes/sig-auth
Automatic merge from submit-queue
Split path validation into a separate library
This PR splits path segment validation into it's own package. This cuts off one of the restclient's dependency paths to some docker packages, and completely eliminates its dependency on go-restful swagger validation.
cc @kubernetes/sig-api-machinery
Automatic merge from submit-queue
Allow services which use same port, different protocol to use the same nodePort for both
fix#20092
@thockin @smarterclayton ptal.
Automatic merge from submit-queue
return destroy func to clean up internal resources of storage
What?
Provide a destroy func to clean up internal resources of storage.
It changes **unit tests** to clean up resources. (Maybe fix integration test in another PR.)
Why?
Although apiserver is designed to be long running, there are some cases that it's not.
See https://github.com/kubernetes/kubernetes/issues/31262#issuecomment-242208771
We need to gracefully shutdown and clean up resources.
- Error from Decorator will be returned instead of assuming false filtering
- For List, Decorator is called on return list object
- For Watch, we implement a new watcher to pipe through decorator
1. When overlapping deployments are discovered, annotate them
2. Expose those overlapping annotations as warnings in kubectl describe
3. Only respect the earliest updated one (skip syncing all other overlapping deployments)
4. Use indexer instead of store for deployment lister
Automatic merge from submit-queue
[GarbageCollector] Allow per-resource default garbage collection behavior
What's the bug:
When deleting an RC with `deleteOptions.OrphanDependents==nil`, garbage collector is supposed to treat it as `deleteOptions.OrphanDependents==true", and orphan the pods created by it. But the apiserver is not doing that.
What's in the pr:
Allow each resource to specify the default garbage collection behavior in the registry. For example, RC registry's default GC behavior is Orphan, and Pod registry's default GC behavior is CascadingDeletion.
Automatic merge from submit-queue
Enable v3 Client as the default on UTs
Updates the default initialization to use clientv3 interface to etcd3, and fixes the UTs.
This PR includes a cherry-pick of https://github.com/kubernetes/kubernetes/pull/30634 so we can validate the tests, so do not merge until that PR is complete.
Automatic merge from submit-queue
Make labels, fields expose selectable requirements
What?
This is to change the labels/fields Selector interface and make them expose selectable requirements. We reuse labels.Requirement struct for label selector and add fields.Requirement for field selector.
Why?
In order to index labels/fields, we need them to tell us three things: index key (a field or a label), operator (greater, less, or equal), and value (string, int, etc.). By getting selectable requirements, we are able to pass them down and use them for indexing in storage layer.
Automatic merge from submit-queue
Basic scaler/reaper for petset
Currently scaling or upgrading a petset is more complicated than it should be. Would be nice if this made code freeze on friday. I'm planning on a follow up change with generation number and e2es post freeze.
Automatic merge from submit-queue
change all PredicateFunc to use SelectionPredicate
What?
- This PR changes all PredicateFunc in registry to return SelectionPredicate instead of Matcher interface.
Why?
- We want to pass SelectionPredicate to storage layer. Matcher interface did not expose enough information for indexing.
Convert single GV and lists of GVs into an interface that can handle
more complex scenarios (everything internal, nothing supported). Pass
the interface down into conversion.
Automatic merge from submit-queue
Change kubectl create to use dynamic client
https://github.com/kubernetes/kubernetes/issues/16764https://github.com/kubernetes/kubernetes/issues/3955
This is a series of changes to allow kubectl create to use discovery-based REST mapping and dynamic clients.
cc @kubernetes/sig-api-machinery
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
kubectl will no longer do client-side defaulting on create and replace.
```
Automatic merge from submit-queue
Move new etcd storage (low level storage) into cacher
In an effort for #29888, we are pushing forward this:
What?
- It changes creating etcd storage.Interface impl into creating config
- In creating cacher storage (StorageWithCacher), it passes config created above and new etcd storage inside.
Why?
- We want to expose the information of (etcd) kv client to cacher. Cacher storage uses this information to talk to remote storage.
Automatic merge from submit-queue
Run goimport for the whole repo
While removing GOMAXPROC and running goimports, I noticed quite a lot of other files also needed a goimport format. Didn't commit `*.generated.go`, `*.deepcopy.go` or files in `vendor`
This is more for testing if it builds.
The only strange thing here is the gopkg.in/gcfg.v1 => github.com/scalingdata/gcfg replace.
cc @jfrazelle @thockin
Automatic merge from submit-queue
add subjectaccessreviews resource
Adds a subjectaccessreviews endpoint that uses the API server's authorizer to determine if a subject is allowed to perform an action.
Part of kubernetes/features#37
The swagger spec for pods/{name}/log does not include
"text/plain" as a possible content-type for the the response.
So we implement ProducesMIMETypes to make sure "text/plain"
gets added to the default list ot content-types.
the v1.json was generated by running:
hack/update-generated-swagger-docs.sh;./hack/update-swagger-spec.sh;
Fixes#14071
Automatic merge from submit-queue
make reousrce prefix consistent with other registries
Storage class registry was added recently in #29694.
In other registries, resource prefix was changed to a more generic way using RESTOptions.ResourcePrefix.
This PR is an attempt to make both consistent.
Automatic merge from submit-queue
make the resource prefix in etcd configurable for cohabitation
This looks big, its not as bad as it seems.
When you have different resources cohabiting, the resource name used for the etcd directory needs to be configurable. HPA in two different groups worked fine before. Now we're looking at something like RC<->RS. They normally store into two different etcd directories. This code allows them to be configured to store into the same location.
To maintain consistency across all resources, I allowed the `StorageFactory` to indicate which `ResourcePrefix` should be used inside `RESTOptions` which already contains storage information.
@lavalamp affects cohabitation.
@smarterclayton @mfojtik prereq for our rc<->rs and d<->dc story.
Automatic merge from submit-queue
cleanup wrong naming: limitrange -> hpa
The code is in `horizontalpodautoscaler/strategy.go`, but the parameter is "limitrange". This is legacy copy-paste issue...
Automatic merge from submit-queue
include metadata in third party resource list serialization
Third party resource listing does not include important metadata such as resourceVersion and apiVersion. This commit includes the missing metadata and also replaces the string templating with an anonymous struct.
With the introduction of batch/v1 and batch/v2alpha1, any
MultiRESTMapper that has per groupversion mappers (as a naive discovery
client would create) would end up being unable to call RESTMapping() on
batch.Jobs. As we finish up discovery we will need to be able to choose
prioritized RESTMappings based on the service discovery doc.
This change implements RESTMappings(groupversion) which returns all
possible RESTMappings for that kind. That allows a higher level call to
prioritize the returned mappings by server or client preferred version.
Automatic merge from submit-queue
ListOptions: add test for ResourceVersion > 0 in List
ref: #28472
Done:
- Add a test for ResourceVersion > 0 in registry (cache) store List()
- Fix the docs.
Automatic merge from submit-queue
TLS bootstrap API group (alpha)
This PR only covers the new types and related client/storage code- the vast majority of the line count is codegen. The implementation differs slightly from the current proposal document based on discussions in design thread (#20439). The controller logic and kubelet support mentioned in the proposal are forthcoming in separate requests.
I submit that #18762 ("Creating a new API group is really hard") is, if anything, understating it. I've tried to structure the commits to illustrate the process.
@mikedanese @erictune @smarterclayton @deads2k
```release-note-experimental
An alpha implementation of the the TLS bootstrap API described in docs/proposals/kubelet-tls-bootstrap.md.
```
[]()
Automatic merge from submit-queue
cacher.go: remove NewCacher func
NewCacher is a wrapper of NewCacherFromConfig. NewCacher understands
how to create a key func from scopeStrategy. However, it is not the
responsibility of cacher. So we should remove this function, and
construct the config in its caller, which should understand scopeStrategy.
Automatic merge from submit-queue
Remove EncodeToStream(..., []unversioned.GroupVersion)
Was not being used. Is a signature change and is necessary for post 1.3 work on Templates and other objects that nest objects.
Extracted from #26044
NewCacher is a wrapper of NewCacherFromConfig. NewCacher understands
how to create a key func from scopeStrategy. However, it is not the
responsibility of cacher. So we should remove this function, and
construct the config in its caller, which should understand scopeStrategy.
Automatic merge from submit-queue
Expose GET and PATCH for status subresource
We can do this for other status subresource. I only updated node/status in this PR to unblock https://github.com/kubernetes/node-problem-detector/issues/9.
cc @Random-Liu @lavalamp
Automatic merge from submit-queue
validate third party resources
addresses validation portion of https://github.com/kubernetes/kubernetes/issues/22768
* ThirdPartyResource: validates name (3 segment DNS subdomain) and version names (single segment DNS label)
* ThirdPartyResourceData: validates objectmeta (name is validated as a DNS label)
* removes ability to use GenerateName with thirdpartyresources (kind and api group should not be randomized, in my opinion)
test improvements:
* updates resttest to clean up after create tests (so the same valid object can be used)
* updates resttest to take a name generator (in case "foo1" isn't a valid name for the object under test)
action required for alpha thirdpartyresource users:
* existing thirdpartyresource objects that do not match these validation rules will need to be removed/updated (after removing thirdpartyresourcedata objects stored under the disallowed versions, kind, or group names)
* existing thirdpartyresourcedata objects that do not match the name validation rule will not be able to be updated, but can be removed
Automatic merge from submit-queue
Add pod condition PodScheduled to detect situation when scheduler tried to schedule a Pod, but failed
Set `PodSchedule` condition to `ConditionFalse` in `scheduleOne()` if scheduling failed and to `ConditionTrue` in `/bind` subresource.
Ref #24404
@mml (as it seems to be related to "why pending" effort)
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24459)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Make ThirdPartyResource a root scoped object
ThirdPartyResource (the registration of a third party type) belongs at the cluster scope. It results in resource handlers installed in every namespace, and the same name in two namespaces collides (namespace is ignored when determining group/kind).
ThirdPartyResourceData (an actual instance of that type) is still namespace-scoped.
This PR moves ThirdPartyResource to be a root scope object. Someone previously using ThirdPartyResource definitions in alpha should be able to move them from namespace to root scope like this:
setup (run on 1.2):
```
kubectl create ns ns1
echo '{"kind":"ThirdPartyResource","apiVersion":"extensions/v1beta1","metadata":{"name":"foo.example.com"},"versions":[{"name":"v8"}]}' | kubectl create -f - --namespace=ns1
echo '{"kind":"Foo","apiVersion":"example.com/v8","metadata":{"name":"MyFoo"},"testkey":"testvalue"}' | kubectl create -f - --namespace=ns1
```
export:
```
kubectl get thirdpartyresource --all-namespaces -o yaml > tprs.yaml
```
remove namespaced kind registrations (this shouldn't remove the data of that type, which is another possible issue):
```
kubectl delete -f tprs.yaml
```
... upgrade ...
re-register the custom types at the root scope:
```
kubectl create -f tprs.yaml
```
Additionally, pre-1.3 clients that expect to read/write ThirdPartyResource at a namespace scope will not be compatible with 1.3+ servers, and 1.3+ clients that expect to read/write ThirdPartyResource at a root scope will not be compatible with pre-1.3 servers.
The codec factory should support two distinct interfaces - negotiating
for a serializer with a client, vs reading or writing data to a storage
form (etcd, disk, etc). Make the EncodeForVersion and DecodeToVersion
methods only take Encoder and Decoder, and slight refactoring elsewhere.
In the storage factory, use a content type to control what serializer to
pick, and use the universal deserializer. This ensures that storage can
read JSON (which might be from older objects) while only writing
protobuf. Add exceptions for those resources that may not be able to
write to protobuf (specifically third party resources, but potentially
others in the future).
Automatic merge from submit-queue
Petset controller
Took longer than I expected. Main parts of this pr are:
1. Identity generation based on petset spec (volumes are mapped per discussion in #18016)
2. Ensure that we create/delete pets in sequence
3. Ensuring that we create, wait for healthy, create; or delete, wait for terminationGrace, delete
4. Controller that watches apiserver and drives actual -> desired
PVCs are not deleted, yet.
Automatic merge from submit-queue
Move internal types of job from pkg/apis/extensions to pkg/apis/batch
This addressed the job part of #23216, this is still WIP. Will notify once finished. I'd like to have it in before starting working on ScheduledJob.
@lavalamp @erictune fyi
Add tests to watch behavior in both protocols (http and websocket)
against all 3 media types. Adopt the
`application/vnd.kubernetes.protobuf;stream=watch` media type for the
content that comes back from a watch call so that it can be
distinguished from a Status result.
Automatic merge from submit-queue
Make etcd cache size configurable
Instead of the prior 50K limit, allow users to specify a more sensible size for their cluster.
I'm not sure what a sensible default is here. I'm still experimenting on my own clusters. 50 gives me a 270MB max footprint. 50K caused my apiserver to run out of memory as it exceeded >2GB. I believe that number is far too large for most people's use cases.
There are some other fundamental issues that I'm not addressing here:
- Old etcd items are cached and potentially never removed (it stores using modifiedIndex, and doesn't remove the old object when it gets updated)
- Cache isn't LRU, so there's no guarantee the cache remains hot. This makes its performance difficult to predict. More of an issue with a smaller cache size.
- 1.2 etcd entries seem to have a larger memory footprint (I never had an issue in 1.1, even though this cache existed there). I suspect that's due to image lists on the node status.
This is provided as a fix for #23323
Automatic merge from submit-queue
Implement a streaming serializer for watch
Changeover watch to use streaming serialization. Properly version the
watch objects. Implement simple framing for JSON and Protobuf (but not
YAML).
@wojtek-t @lavalamp
Automatic merge from submit-queue
Additional go vet fixes
Mostly:
- pass lock by value
- bad syntax for struct tag value
- example functions not formatted properly
Here are a list of changes along with an explanation of how they work:
1. Add a new string field called TargetSelector to the external version of
extensions Scale type (extensions/v1beta1.Scale). This is a serialized
version of either the map-based selector (in case of ReplicationControllers)
or the unversioned.LabelSelector struct (in case of Deployments and
ReplicaSets).
2. Change the selector field in the internal Scale type (extensions.Scale) to
unversioned.LabelSelector.
3. Add conversion functions to convert from two external selector fields to a
single internal selector field. The rules for conversion are as follows:
i. If the target resource that this scale targets supports LabelSelector
(Deployments and ReplicaSets), then serialize the LabelSelector and
store the string in the TargetSelector field in the external version
and leave the map-based Selector field as nil.
ii. If the target resource only supports a map-based selector
(ReplicationControllers), then still serialize that selector and
store the serialized string in the TargetSelector field. Also,
set the the Selector map field in the external Scale type.
iii. When converting from external to internal version, parse the
TargetSelector string into LabelSelector struct if the string isn't
empty. If it is empty, then check if the Selector map is set and just
assign that map to the MatchLabels component of the LabelSelector.
iv. When converting from internal to external version, serialize the
LabelSelector and store it in the TargetSelector field. If only
the MatchLabel component is set, then also copy that value to
the Selector map field in the external version.
4. HPA now just converts the LabelSelector field to a Selector interface
type to list the pods.
5. Scale Get and Update etcd methods for Deployments and ReplicaSets now
return extensions.Scale instead of autoscaling.Scale.
6. Consequently, SubresourceGroupVersion override and is "autoscaling"
enabled check is now removed from pkg/master/master.go
7. Other small changes to labels package, fuzzer and LabelSelector
helpers to piece this all together.
8. Add unit tests to HPA targeting Deployments and ReplicaSets.
9. Add an e2e test to HPA targeting ReplicaSets.
