Automatic merge from submit-queue
oidc client plugin: reduce round trips and fix scopes requested
This PR attempts to simplify the OpenID Connect client plugin to
reduce round trips. The steps taken by the client are now:
* If ID Token isn't expired:
* Do nothing.
* If ID Token is expired:
* Query /.well-known discovery URL to find token_endpoint.
* Use an OAuth2 client and refresh token to request new ID token.
This avoids the previous pattern of always initializing a client,
which would hit the /.well-known endpoint several times.
The client no longer does token validation since the server already
does this. As a result, this code no longer imports
github.com/coreos/go-oidc, instead just using golang.org/x/oauth2
for refreshing.
Overall reduction in tests because we're not verify as many things
on the client side. For example, we're no longer validating the
id_token signature (again, because it's being done on the server
side).
This has been manually tested against dex, and I hope to continue
to test this over the 1.7 release cycle.
cc @mlbiam @frodenas @curtisallen @jsloyer @rithujohn191 @philips @kubernetes/sig-auth-pr-reviews
```release-note
NONE
```
Updates https://github.com/kubernetes/kubernetes/issues/42654
Closes https://github.com/kubernetes/kubernetes/issues/37875
Closes https://github.com/kubernetes/kubernetes/issues/37874
This changes the event aggregation behavior so that, when multiple events are
deduplicated, the aggregated event includes the message of the latest related
event.
This fixes an issue where the original event expires due to TTL, and the
aggregate event doesn't contain any useful message.
All registry.Store objects already set a non-nil DeleteStrategy.
This change ensures that all future objects do so as well.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Automatic merge from submit-queue
Add test for cross namespace watch and list
**What this PR does / why we need it**: Add more integration test for kube-apiextensions-server
**Which issue this PR fixes** : fixes https://github.com/kubernetes/kubernetes/issues/45511
**Special notes for your reviewer**: The client with cluster scope also works, but it seems to be trivial
@deads2k
Automatic merge from submit-queue (batch tested with PRs 42042, 46139, 46126, 46258, 46312)
Append X-Forwarded-For in proxy handler
Append the request sender's IP to the `X-Forwarded-For` header chain when proxying requests. This is important for audit logging (https://github.com/kubernetes/features/issues/22) in order to capture the client IP (specifically in the case of federation or kube-aggregator).
/cc @liggitt @deads2k @ericchiang @ihmccreery @soltysh
Automatic merge from submit-queue (batch tested with PRs 42042, 46139, 46126, 46258, 46312)
Remove kubectl's dependence on pkg/api/helper
**What this PR does / why we need it**:
Remove kubectl's dependence on pkg/api/helper, as part of
broader effort to isolate kubectl from the rest of k8s.
In this case, the code becomes private to kubectl; nobody else uses it.
**Which issue this PR fixes**
Part of a series of PRs to address kubernetes/community#598
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
remove init blocks from all admission plugins
**What this PR does / why we need it**:
removes init blocks from all admission plugins
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46149, 45897, 46293, 46296, 46194)
Use storage instead of REST for the CRD finalizer
**What this PR does / why we need it**:
Switch the custom resource definition finalizer controller to use
storage instead of a REST client, because a client could incorrectly try
to delete ThirdPartyResources whose names happen to collide with the
CustomResource instances.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Before this change, if the config was empty, ConfirmUsable() would
return an "invalid configuration" error instead of examining and
honoring the value of the --namespace flag. This change looks at the
overrides first, and returns the overridden value if it exists before
attempting to check if the config is usable. This is most applicable to
in-cluster clients, where they don't have a kubeconfig but they do have
a token and can use KUBERNETES_SERVICE_HOST/_PORT.
Switch the custom resource definition finalizer controller to use
storage instead of a REST client, because a client could incorrectly try
to delete ThirdPartyResources whose names happen to collide with the
CustomResource instances.
Automatic merge from submit-queue (batch tested with PRs 45587, 46286)
PDB Max Unavailable Field
Completes https://github.com/kubernetes/features/issues/285
```release-note
Adds a MaxUnavailable field to PodDisruptionBudget
```
Individual commits are self-contained; Last commit can be ignored because it is autogenerated code.
cc @kubernetes/sig-apps-api-reviews @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45766, 46223)
Scheduler should use a shared informer, and fix broken watch behavior for cached watches
Can be used either from a true shared informer or a local shared
informer created just for the scheduler.
Fixes a bug in the cache watcher where we were returning the "current" object from a watch event, not the historic event. This means that we broke behavior when introducing the watch cache. This may have API implications for filtering watch consumers - but on the other hand, it prevents clients filtering from seeing objects outside of their watch correctly, which can lead to other subtle bugs.
```release-note
The behavior of some watch calls to the server when filtering on fields was incorrect. If watching objects with a filter, when an update was made that no longer matched the filter a DELETE event was correctly sent. However, the object that was returned by that delete was not the (correct) version before the update, but instead, the newer version. That meant the new object was not matched by the filter. This was a regression from behavior between cached watches on the server side and uncached watches, and thus broke downstream API clients.
```
Automatic merge from submit-queue
tighten and simplify owners in some staging repos
With the move to staging, we can have much cleaner owners across the related packages. This pares down the list of OWNERS to better match for code and activity. It should help get PRs directed to people more active and familiar with the areas for quicker review.
@kubernetes/sig-api-machinery-misc
@lavalamp @smarterclayton ptal.
Automatic merge from submit-queue (batch tested with PRs 46060, 46234)
Speedup generating selflinks for list and watch requests
I've seen profiles, where GenerateSelflink was 8-9% of whole cpu usage of apiserver (profiles over 30s). Most of this where spent in getting RequestInfo from the context and creating the context.
This PR changes the API of the GenerateLink method of the namer which results in computing the context and requestInfo only once per LIST/WATCH request (instead of computing it for every single returned element of LIST/WATCH).
@smarterclayton @deads2k - can one of you please take a look?
Automatic merge from submit-queue
vSphere storage policy support for dynamic volume provisioning
Till now, vSphere cloud provider provides support to configure persistent volume with VSAN storage capabilities - kubernetes#42974. Right now this only works with VSAN.
Also there might be other use cases:
- The user might need a way to configure a policy on other datastores like VMFS, NFS etc.
- Use Storage IO control, VMCrypt policies for a persistent disk.
We can achieve about 2 use cases by using existing storage policies which are already created on vCenter using the Storage Policy Based Management service. The user will specify the SPBM policy ID as part of dynamic provisioning
- resultant persistent volume will have the policy configured with it.
- The persistent volume will be created on the compatible datastore that satisfies the storage policy requirements.
- If there are multiple compatible datastores, the datastore with the max free space would be chosen by default.
- If the user specifies the datastore along with the storage policy ID, the volume will created on this datastore if its compatible. In case if the user specified datastore is incompatible, it would error out the reasons for incompatibility to the user.
- Also, the user will be able to see the associations of persistent volume object with the policy on the vCenter once the volume is attached to the node.
For instance in the below example, the volume will created on a compatible datastore with max free space that satisfies the "Gold" storage policy requirements.
```
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: zeroedthick
storagepolicyName: Gold
```
For instance in the below example, the vSphere CP checks if "VSANDatastore" is compatible with "Gold" storage policy requirements. If yes, volume will be provisioned on "VSANDatastore" else it will error that "VSANDatastore" is not compatible with the exact reason for failure.
```
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: zeroedthick
storagepolicyName: Gold
datastore: VSANDatastore
```
As a part of this change, 4 commits have been added to this PR.
1. Vendor changes for vmware/govmomi
2. Changes to the VsphereVirtualDiskVolumeSource in the Kubernetes API. Added 2 additional fields StoragePolicyName, StoragePolicyID
3. Swagger and Open spec API changes.
4. vSphere Cloud Provider changes to implement the storage policy support.
**Release note**:
```release-note
vSphere cloud provider: vSphere Storage policy Support for dynamic volume provisioning
```
Automatic merge from submit-queue (batch tested with PRs 38990, 45781, 46225, 44899, 43663)
Support parallel scaling on StatefulSets
Fixes#41255
```release-note
StatefulSets now include an alpha scaling feature accessible by setting the `spec.podManagementPolicy` field to `Parallel`. The controller will not wait for pods to be ready before adding the other pods, and will replace deleted pods as needed. Since parallel scaling creates pods out of order, you cannot depend on predictable membership changes within your set.
```
Automatic merge from submit-queue (batch tested with PRs 38990, 45781, 46225, 44899, 43663)
migrate set generation to go_genrule
Depends on https://github.com/kubernetes/release/pull/238
Automatic merge from submit-queue (batch tested with PRs 46133, 46211, 46224, 46205, 45910)
Correct client-go TPR example type registration
**What this PR does / why we need it**:
Eliminate duplicate registration of third-party resource types in the _client-go_ example, precluding any number of readers from copying the mistake into their own applications as they adapt the example to their own needs.
**Special notes for your reviewer**:
See [the preceding discussion](a6c97715ed (commitcomment-22146536)) about a6c97715ed, committed as part of #45463 but only noticed after the merge.
It's possible that we could take a few of the changes proposed here and apply them more broadly throughout the rest of the code, such as not exporting the `AddToScheme` var in favor of an actual function declaration. Given the flux in #44784, I'd prefer that we don't hold up these small touch-ups on a broader unification.
People I expect will want to weigh in: @sttts, @caesarxuchao, and @nilebox.
Automatic merge from submit-queue (batch tested with PRs 46133, 46211, 46224, 46205, 45910)
clientgo/examples/in-cluster: add instructions to run the example
Automatic merge from submit-queue
LocalStorage api
**What this PR does / why we need it**:
API changes to support persistent local volumes, as described [here](https://github.com/kubernetes/community/pull/306)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Part of #43640
**Special notes for your reviewer**:
There were a few items I was concerned about. Will add review comments in those places.
**Release note**:
NONE
Note will be added in subsequent PR with the volume plugin changes
Automatic merge from submit-queue
Moved qos to api.helpers.
**What this PR does / why we need it**:
The `GetPodQoS` is also used by other components, e.g. kube-scheduler and it's not bound to kubelet; moved it to api helpers so client-go.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #N/A
**Release note**:
```release-note-none
```
Created OWNERS_ALIASES called sig-apps-reviewers from the union of reviewers in:
pkg/controller/{cronjob,deployment,daemon,job,replicaset,statefulset}/OWNERS
except removed inactive user bprashanth
Created OWNERS_ALIASES called sig-apps-api-reviewers as the intersection
of sig-apps-reviewers and the approvers from pkg/api/OWNERS.
Used those OWNERS_ALIASES as the reviewers/approvers for the disruption controller,
and API.
The underlying storage has always returned the old object on watch
delete events when filtering. The cache watcher does not, which means a
downsteam caller gets different behavior.
This fixes the cache watcher to be consistent with our long term
behavior for watch. It may result in a behavior change (the filter
becomes more precise) but this was a regression in behavior.
This PR attempts to simplify the OpenID Connect client plugin to
reduce round trips. The steps taken by the client are now:
* If ID Token isn't expired:
* Do nothing.
* If ID Token is expired:
* Query /.well-known discovery URL to find token_endpoint.
* Use an OAuth2 client and refresh token to request new ID token.
This avoids the previous pattern of always initializing a client,
which would hit the /.well-known endpoint several times.
The client no longer does token validation since the server already
does this. As a result, this code no longer imports
github.com/coreos/go-oidc, instead just using golang.org/x/oauth2
for refreshing.
Automatic merge from submit-queue (batch tested with PRs 45864, 46169)
Fix unit tests for autoregister_controller.go reliable
Fixes https://github.com/kubernetes/kubernetes/issues/45538
Still wip, and just have some questions which I left some comments in original issue above
Since NewSchemeBuilder registers the supplied Scheme transformation
functions synchronously, there's no need for a subsequent call
to (*SchemeBuilder).Register against the same instance.
Remove kubectl's dependence on pkg/api/helper, as part of
broader effort to isolate kubectl from the rest of k8s.
In this case, the code becomes private to kubectl; nobody else uses it.
**Which issue this PR fixes**
Part of a series of PRs to address kubernetes/community#598
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
The alpha field podManagementPolicy defines how pods are created,
deleted, and replaced. The new `Parallel` policy will replace pods
as fast as possible, not waiting for the pod to be `Ready` or providing
an order. This allows for advanced clustered software to take advantage
of rapid changes in scale.
Automatic merge from submit-queue
Add protobuf binary version of OpenAPI spec
Fixes#45833
Partially fixes#42841
```release-note
OpenAPI spec is now available in protobuf binary and gzip format (with ETag support)
```
Automatic merge from submit-queue
kube-apiserver: check upgrade header to detect upgrade connections
Without this every connection with "Connection" header but without upgrade request are rejected. A simple
curl will set "Connection", but does not intent to upgrade.
Automatic merge from submit-queue (batch tested with PRs 46014, 46152)
format reads arg 3, have only 2 args, add i
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45346, 45903, 45958, 46042, 45975)
Azure disk api
This is to update the AzureDiskApi and split it from the implementation which is caught in rebase hell...
Once this is merged, we'll get the implementation in.
@smarterclayton suggested this as a way to break the rebase hell logjam. request for a quick review.
Thanks!
This patch adds instructions for how to run the in-cluster client-go example.
To make this example executable, providing a Dockerfile and build steps so
that it can directly be run on minikube.
This is part of the body of work improving the client library samples.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Automatic merge from submit-queue (batch tested with PRs 46075, 46059, 46095, 46097)
Integration test for kube-apiextensions-server: integers
**What this PR does / why we need it**: Check if integers are present after decoding.
Originally an issue for TPRs: #30213
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: for #45511
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
@sttts
Automatic merge from submit-queue (batch tested with PRs 41535, 45985, 45929, 45948, 46056)
add CRD finalizer to remove CRs
Fixes https://github.com/kubernetes/kubernetes/issues/45878
This adds a finalizer for customresourcedefinitions to prevent CRD deletion until all CR instances are gone.
@sdminonne I lost track of your issue, but here's the fix
I'm at a loss for how to test this. It's tested from the outside by ensuring that a CRD delete removes its instances (integration test) and we could add more integration tests, but for unit tests I can't seem to find a mock `dynamic.ClientPool` and its not easily writeable at the moment. I'm thinking about saying we just add more black box tests given the options.
Automatic merge from submit-queue
Remove ObjectNameFunc from custom resources
@deads2k as far as I can tell these `ObjectNameFunc`s are unnecessary.
Signed-off-by: Monis Khan <mkhan@redhat.com>
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Move defaulting logic for ExternalTrafficPolicy into defaults.go
Link #45812.
Per [#45812/comment](https://github.com/kubernetes/kubernetes/issues/45812#issuecomment-301632807), move defaulting logic to defaults.go so that federation will get the same code.
This PR does not have any functional change :)
/assign @freehan @thockin
/cc @madhusudancs @nikhiljindal @shashidharatd
Also /cc @caesarxuchao per the client-go changes.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45852, 45965, 45941)
Update k8s.io/gengo dependency
**What this PR does / why we need it**:
Brings in changes to improve type checker and deep copy
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Move all API related annotations into annotation_key_constants.go
Separate from #45869. See https://github.com/kubernetes/kubernetes/pull/45869#discussion_r116839411 for details.
This PR does nothing but move constants around :)
/assign @caesarxuchao
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42895, 45940)
apiserver: no Status in body for http 204
Fixes http writer errors as with 204 we must not send any body.
Needed downstream for https://github.com/openshift/origin/issues/14213.
Automatic merge from submit-queue (batch tested with PRs 45709, 41939)
Add an AEAD encrypting transformer for storing secrets encrypted at rest
Tweak the ValueTransformer interface slightly to support additional
context information (to allow authenticated data to be generated by the
store and passed to the transformer). Add a prefix transformer that
looks for known matching prefixes and uses them. Add an AES GCM
transformer that performs AEAD on the values coming in and out of the
store.
Implementation of https://docs.google.com/document/d/1lFhPLlvkCo3XFC2xFDPSn0jAGpqKcCCZaNsBAv8zFdE/edit# and https://github.com/kubernetes/kubernetes/issues/12742
Automatic merge from submit-queue (batch tested with PRs 44520, 45253, 45838, 44685, 45901)
API for server side tabular output
These are the APIs necessary to implement propsoal kubernetes/community#363
They consist of a new meta group (v1alpha1) that indicates these are alpha apis for the server as a whole, a new kind `TableList` which is a simple row + header arranged table capable of returning both object and columnar data, a `TableListOptions` for altering the behavior of the return, and `PartialObjectMetadata` which is an "interface" style API object which allows a client to ask any object for their metadata (without having to know how to parse the object or perform gymnastics).
Extracted from #40848
A few minor tweaks still required.
Automatic merge from submit-queue (batch tested with PRs 44520, 45253, 45838, 44685, 45901)
start serving customresourcedefinition based on status
This exposes the `customresourcedefinition/status` endpoint, wires a controller to drive `NameConflict` conditions, and serves discovery from status, not spec.
Next steps after this include wiring the conditions into handling and reswizzling the handling chain to be cleaner now that we have a custom mux.
Tweak the ValueTransformer interface slightly to support additional
context information (to allow authenticated data to be generated by the
store and passed to the transformer). Add a prefix transformer that
looks for known matching prefixes and uses them. Add an AES GCM
transformer that performs AEAD on the values coming in and out of the
store.
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)
Apiregistration v1alpha1→v1beta1
Promoting apiregistration api from v1alpha1 to v1beta1.
API Registration is responsible for registering an API `Group`/`Version` with
another kubernetes like API server. The `APIService` holds information
about the other API server in `APIServiceSpec` type as well as general
`TypeMeta` and `ObjectMeta`. The `APIServiceSpec` type have the main
configuration needed to do the aggregation. Any request coming for
specified `Group`/`Version` will be directed to the service defined by
`ServiceReference` (on port 443) after validating the target using provided
`CABundle` or skipping validation if development flag `InsecureSkipTLSVerify`
is set. `Priority` is controlling the order of this API group in the overall
discovery document.
The return status is a set of conditions for this aggregation. Currently
there is only one condition named "Available", if true, it means the
api/server requests will be redirected to specified API server.
```release-note
API Registration is now in beta.
```
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
Update 'More info' links on API types and fields
> **Please just review the first commit, the rest is generated files.**
Recent docs website shuffling during 1.6 caused majority of links in the API
types and fields to break. Since we do not have server-side 301 redirects, user
has to click an extra link, and the #target fragment in the URL will be lost. (This is
because GitHub’s redirect_from feature is not ideal.)
For the time being, I have manually gone through all of them to bring them up to date
and add HTTPS to those missing it. This is a docs-only change and impacts generated
code, generated swaggers, API reference docs etc.
cc: @steveperry-53 @devin-donnelly @chenopis fyi, docs links changes (even small title changes) easily breaks links in API reference, Swagger, kubectl explain, and many other places.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Automatic merge from submit-queue
Update instructions for client staging copy
Following the existing instructions results in:
```
% ./staging/copy.sh
Do not run this script directly, but via hack/update-staging-client-go.sh.
```
Also format README.md as markdown.
**What this PR does / why we need it**: Updates docs to match reality
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44326, 45768)
custom resource De-Registration testcase
@deads2k as discussed via IRC
Built on top of #45732
To reproduce #45767
Uncomment code in `TestDeRegistrationAndReRegistration` to reproduce `panic`
Automatic merge from submit-queue (batch tested with PRs 45408, 45355, 45528)
Admission options spits out admission control
**What this PR does / why we need it**:
This PR adds ApplyTo method to AdmissionOptions struct. The method creates and initialises admission control to the server configuration.
**Release note**:
```
NONE
```
This plugin acquires a fresh access token for apiserver from Azure Active
Directory using the device code flow. The access token is saved in the
configuration in order to be reused for upcomming accesses to appiserver.
In additon the access token is automatically refreshed when expired.
Automatic merge from submit-queue
add --as-group option to cli
The usecase of this change:
When a super user grant some RBAC permissions to a group, he can use
--as--group to test whether the group get the permissions.
Note that now we support as-groups, as-user-extra in kubeconfig file after this change.
**Release note**:
```NONE
```
@liggitt
Automatic merge from submit-queue
fix missing argument for Errorf(%v): format reads arg 1, have only 0 args
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
**What this PR does / why we need it**:
add the missing code of "len(res.Items)"
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
delete the code after "return"
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
**What this PR does / why we need it**:
After return, the code is unreachable, delete it.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Internal audit API
Forked from https://github.com/kubernetes/kubernetes/pull/45315
I pulled out only the internal audit API types and would like to merge this to unblock the audit implementation work while we figure out the versioned types and code generation.
I will continue to iterate on https://github.com/kubernetes/kubernetes/pull/45315, but lets get this internal type submitted.
/cc @ericchiang @ihmccreery
Automatic merge from submit-queue (batch tested with PRs 44337, 45775, 45832, 45574, 45758)
Tighten validation of mirror pod annotations
Tightens validation for pods with a mirror pod annotation:
1. spec.nodeName must be set
2. makes the mirror pod annotation immutable
3. starts validating pod-specific annotations during pod status update
None of these changes affect usage of the mirror pod annotation by kubelets, which only set it on pod creation (verified this is true back to 1.5.x)
the second commit updates the pod validation tests to look for specific error messages (best reviewed ignoring whitespace changes)
This is the validation portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and https://github.com/kubernetes/features/issues/279
```release-note
Mirror pods must now indicate the nodeName they are bound to on creation. The mirror pod annotation is now treated as immutable and cannot be added to an existing pod, removed from a pod, or modified.
```
Automatic merge from submit-queue (batch tested with PRs 45171, 43947, 45788, 45822, 45808)
rename to CustomResourceDefinition
Related to https://github.com/kubernetes/kubernetes/issues/45277
This does a straight rename of `CustomResource` to `CustomResourceDefinition` in response to https://github.com/kubernetes/kubernetes/issues/45277 to avoid confusion.
It'll be a little painful on tests in progress, but it's easier now than later.
Automatic merge from submit-queue (batch tested with PRs 41331, 45591, 45600, 45176, 45658)
Updating generic registry to return UID of the deleted resource
Ref https://github.com/kubernetes/kubernetes/issues/42594
cc @kubernetes/sig-api-machinery-pr-reviews @smarterclayton
```release-note
Updating apiserver to return UID of the deleted resource. Clients can use this UID to verify that the resource was deleted or waiting for finalizers.
```
Automatic merge from submit-queue (batch tested with PRs 41331, 45591, 45600, 45176, 45658)
Move client/unversioned/remotecommand to client-go
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.
Module remotecommand relies on util/exec module which was copied to client-go/pkg/util
Automatic merge from submit-queue (batch tested with PRs 45070, 45821, 45732, 45494, 45789)
apiextesions-server integration tests: test multiple registration
**What this PR does / why we need it**:
Add integration test for Multiple registrations XREF: #45511
**Special notes for your reviewer**:
@deads2k my first plan was to submit one single PR for both tests but I'm facing one issue with resources deregistration. Get in touch via slack
Automatic merge from submit-queue
Uses container/heap for DelayingQueue
The current implementation of DelayingQueue doesn't perform very well when a large number of items (at random delays) are inserted. The original authors seemed to be aware of this and noted it in a `TODO` comment. This is my attempt at switching the implementation to use a priority queue based on `container/heap`.
Benchmarks from before the change:
```
╰─ go test -bench=. -benchmem | tee /tmp/before.txt
BenchmarkDelayingQueue_AddAfter-8 300000 256824 ns/op 520 B/op 3 allocs/op
PASS
ok k8s.io/kubernetes/staging/src/k8s.io/client-go/util/workqueue 77.237s
```
After:
```
╰─ go test -bench=. -benchmem | tee /tmp/after.txt
BenchmarkDelayingQueue_AddAfter-8 500000 3519 ns/op 406 B/op 4 allocs/op
PASS
ok k8s.io/kubernetes/staging/src/k8s.io/client-go/util/workqueue 2.969s
```
Comparison:
```
╰─ benchcmp /tmp/before.txt /tmp/after.txt
benchmark old ns/op new ns/op delta
BenchmarkDelayingQueue_AddAfter-8 256824 3519 -98.63%
benchmark old allocs new allocs delta
BenchmarkDelayingQueue_AddAfter-8 3 4 +33.33%
benchmark old bytes new bytes delta
BenchmarkDelayingQueue_AddAfter-8 520 406 -21.92%
```
I also find the `container/heap`-based code a bit more easy to understand. The implementation of the PriorityQueue is based on the documentation for `container/heap`.
Feedback definitely welcomed. This is one of my first contributions.
```release-note
NONE
```
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.
Module remotecommand relies on util/exec module which will be copied to client-go/pkg/util
Automatic merge from submit-queue
kube-apiextensions-server: Fix potential SEGV with null delegate handler
**What this PR does / why we need it**:
In the kube-apiextensions-server there is a fallback value for `null` delegate to `http.NotFoundHandler()` in handling group and versions discovery, but no fallback for custom resources endpoint.
It leads to SEGV when running with `genericapiserver.EmptyDelegate`.
ApplyTo adds the admission chain to the server configuration the method lazily initializes a generic plugin
that is appended to the list of pluginInitializers.
apiserver.Config will hold an instance of SharedInformerFactory to ensure we only have once instance.
The field will be initialized in apisever.SecureServingOptions
The usecase of this change:
When a super user grant some RBAC permissions to a group, he can use
--as-group to test whether the group get the permissions.
Note that now we support as-groups, as-user-extra in kubeconfig file
after this change.
Automatic merge from submit-queue (batch tested with PRs 44748, 45692)
Limiting client go packages visibility, round 3
Continue the work in the merged PR https://github.com/kubernetes/kubernetes/pull/45258
These packages in client-go will be gone after #44065 is fixed:
pkg/api/helper, pkg/api/util, internal version of api groups, API install packages.
This PR removes the dependency on these packages and add bazel visibility rules to prevent relapse.
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Promotes Source IP preservation for Virtual IPs from Beta to GA
Fixes#33625. Feature issue: kubernetes/features#27.
Bullet points:
- Declare 2 fields (ExternalTraffic and HealthCheckNodePort) that mirror the ESIPP annotations.
- ESIPP alpha annotations will be ignored.
- Existing ESIPP beta annotations will still be fully supported.
- Allow promoting beta annotations to first class fields or reversely.
- Disallow setting invalid ExternalTraffic and HealthCheckNodePort on services. Default ExternalTraffic field for nodePort or loadBalancer type service to "Global" if not set.
**Release note**:
```release-note
Promotes Source IP preservation for Virtual IPs to GA.
Two api fields are defined correspondingly:
- Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
- Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
```
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Replacing “Pod Injection Policy” with “Pod Preset” in the code documentation
**What this PR does / why we need it**:
Replacing the leftovers of the old term "Pod Injection Policy" with "Pod Preset" in the code documentation.
Automatic merge from submit-queue (batch tested with PRs 45571, 45657, 45638, 45663, 45622)
Added indexer description for the API of NewIndexerInformer.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
plumb stopch to post start hook index since many of them are starting go funcs
Many post-start hooks require a stop channel to properly terminate their go funcs.
@p0lyn0mial I think you need this for https://github.com/kubernetes/kubernetes/pull/45355 ptal.
@ncdc per request
@sttts can you review too since Andy is out?
change import of client-go/api/helper to kubernetes/api/helper
remove unnecessary use of client-go/api.registry
change use of client-go/pkg/util to kubernetes/pkg/util
remove dependency on client-go/pkg/apis/extensions
remove unnecessary invocation of k8s.io/client-go/extension/intsall
change use of k8s.io/client-go/pkg/apis/authentication to v1
Automatic merge from submit-queue
HTML escape apiserver errors to avoid triggering vulnerability scanners.
Simple XSS scans might fetch /<script>alert('vulnerable')</script>, and
fail when the response body includes the script tag verbatim, despite
the headers directing the browser to interpret the response as text.
This isn't a real vulnerability, but it's easier to fix this here than
it is to fix the scanners.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Remove mentioning insecure server (which is not supported anymore) from API server docs
**What this PR does / why we need it**:
Remove mentioning insecure serving from the docs, since only secure serving is supported now.
Automatic merge from submit-queue
apimachinery: NotRegisteredErr for known kinds not registered in target GV
Fixes the fall back to core v1 for *Options in the parameter encoder of the dynamic client.
The dynamic client uses NotRegisteredErr to fall back to core v1 if ListOptions is not known
in the given GV. This commit fixes the case that ListOptions is known in some group, but not
in the given one.
The dynamic client uses NotRegisteredErr to fall back to core v1 if ListOptions is not known
in the given GV. This commit fixes the case that ListOptions is known in some group, but not
in the given one.
Automatic merge from submit-queue (batch tested with PRs 45556, 45561, 45256)
add defaulting for customresources
This adds the promised defaulting for customresources. Namespaced by default, listkind=kind+List, singular=toLower(kind).
Automatic merge from submit-queue
add validation for customresourcedefintions
Add basic validation for customresource definitions.
@adohe if you had review bandwidth, this is a relatively small one.
Automatic merge from submit-queue (batch tested with PRs 45382, 45384, 44781, 45333, 45543)
Do roundtrip testing with external kinds in client-go TPR example
This tests that our serialization machinery works for TPR types, i.e. without internal counterpart and without generated code.
/cc @nilebox
Automatic merge from submit-queue (batch tested with PRs 45382, 45384, 44781, 45333, 45543)
Copy internal types to metrics
Supersedes #45306.
#45306 removed the internal types and suggested whoever needs the internal types should define their own copy, and use the code-gen tools to generated the conversion functions. Per offline discussion with @DirectXMan12, we wanted to go that direction but it's not clear where to put the internal types yet. Hence, as a temporary solution, we decided copy the referred client-go/pkg/api types into metrics api to avoid the dependency.
The commit "remove need of registry from custom_metrics/client.go" is similar to what I did to the fake client in an earlier PR. Let me know if you want to put the commit in another PR.
Automatic merge from submit-queue
Add myself to client-go OWNERS
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
This commit turns on client-gen for k8s.io/metrics. Clients are
generated for `k8s.io/metrics/pkg/apis/metrics` (both internal and
v1alpha1). `k8s.io/metrics/pkg/apis/custom_metrics` uses a bespoke
client due to the unique nature of its paths.
Automatic merge from submit-queue (batch tested with PRs 45453, 45307, 44987)
Init cache with assigned non-terminated pods before scheduling
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45220
**Release note**:
```release-note
The fix makes scheduling go routine waiting for cache (e.g. Pod) to be synced.
```
Simple XSS scans might fetch /<script>alert('vulnerable')</script>, and
fail when the response body includes the script tag verbatim, despite
the headers directing the browser to interpret the response as text.
This isn't a real vulnerability, but it's easier to fix this here than
it is to fix the scanners.
Automatic merge from submit-queue
oidc auth plugin not to override the Auth header if it's already exits
**What this PR does / why we need it**:
oidc auth client plugin should not override the `Authorization` header if it's already exits.
**Which issue this PR fixes** :
fix oidc auth plugin override the` Authorization` header
**Special notes for your reviewer**:
**Release note**:
Automatic merge from submit-queue (batch tested with PRs 44798, 45537, 45448, 45432)
use apiservice.status to break apart controller and handling concerns
Still needs tests.
This starts breaking the handler and controller aspects of the aggregator by making use of status and conditions instead of actually running a specific check on demand.
@kubernetes/sig-api-machinery-pr-reviews
@luxas since you've been asking
Automatic merge from submit-queue
apiserver: injectable default watch cache size
This makes it possible to override the default watch capacity in the REST options getter. Before this PR the default is written into the storage struct explicitly, and if it is the default, the REST options getter didn't know. With this the PR the default is applied late and can be injected from the outside.
Automatic merge from submit-queue (batch tested with PRs 45481, 45463)
ThirdPartyResource example: added watcher example, code cleanup
**NOTE**: This is a cleaned and updated version of PR https://github.com/kubernetes/kubernetes/pull/43027
**What this PR does / why we need it**:
An example of using go-client for watching on ThirdPartyResource events (create/update/delete).
Automatic merge from submit-queue (batch tested with PRs 45508, 44258, 44126, 45441, 45320)
cloud initialize node in external cloud controller
@thockin This PR adds support in the `cloud-controller-manager` to initialize nodes (instead of kubelet, which did it previously)
This also adds support in the kubelet to skip node cloud initialization when `--cloud-provider=external`
Specifically,
Kubelet
1. The kubelet has a new flag called `--provider-id` which uniquely identifies a node in an external DB
2. The kubelet sets a node taint - called "ExternalCloudProvider=true:NoSchedule" if cloudprovider == "external"
Cloud-Controller-Manager
1. The cloud-controller-manager listens on "AddNode" events, and then processes nodes that starts with that above taint. It performs the cloud node initialization steps that were previously being done by the kubelet.
2. On addition of node, it figures out the zone, region, instance-type, removes the above taint and updates the node.
3. Then periodically queries the cloudprovider for node addresses (which was previously done by the kubelet) and updates the node if there are new addresses
```release-note
NONE
```
Following the existing instructions results in:
```
% ./staging/copy.sh
Do not run this script directly, but via hack/update-staging-client-go.sh.
```
Also format README.md as markdown.
Automatic merge from submit-queue
refactor names for the apiserver handling chain
The names and structure around the handling chain got a bit confused. This simplifies it back out into a single struct with three parts: overall handler, gorestful handler, pathrecording mux and makes the delegate wiring simpler
Automatic merge from submit-queue
remove bearer token from headers after we consume it
Updates the bearer token authenticator to remove the bearer token from the request headers after it is consumed. Nothing else in the stack should try to use it and we don't want to accidentally leak it somewhere.
@liggitt @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue
Add OWNERS for staging and api
Part of #44420
* api/ is a copy of pkg/api (same reasoning)
* staging/ is the set of people who should be allowing new top level nested packages + the set of people who can change the staging machinery code
Open to changes to staging/ - very rarely changed. Added owners for the other items
Automatic merge from submit-queue (batch tested with PRs 45182, 45429)
CustomResources in separate API server
Builds on https://github.com/kubernetes/kubernetes/pull/45115.
This adds a basic handler for custom resources. No status handling, no finalizers, no controllers, but basic CRUD runs to allow @enisoc and others to start considering migration.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 43006, 45305, 45390, 45412, 45392)
update the doc of function NewPathRecorderMux
The doc of function NewPathRecorderMux() is out of date. Update it.
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 43006, 45305, 45390, 45412, 45392)
Update go-restful dependency
This is required by #44787. But because both this and the changes in 44787 need constant rebase, I am trying to get this one in separately to make less rebases.
The change is only a dependency update.
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
Use dedicated UnixUserID and UnixGroupID types
**What this PR does / why we need it**:
DRYs up type definitions by using the dedicated types in apimachinery
**Which issue this PR fixes**
#38120
**Release note**:
```release-note
UIDs and GIDs now use apimachinery types
```
Automatic merge from submit-queue
OWNERS: add directxman12 to pkg/apis/autoscaling
Added directxman12 (current SIG lead of SIG-autoscaling) as a reviewer for pkg/apis/autoscaling.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Fix yet another bug in OpenAPI extension generation
A comment extension for a type is ignored if the type already has extensions (e.g. GVK).
Automatic merge from submit-queue (batch tested with PRs 45362, 45159, 45321, 45238)
Remove redundent GetObjectKind() defined on types
Embedding TypeMeta is enough.
Automatic merge from submit-queue
use our own serve mux that directs how we want
alternative to https://github.com/kubernetes/kubernetes/pull/44405
I really wanted to avoid writing my own, but the gorilla mux works via redirect, which would be a change. This does exact pattern matches only unless someone explicitly requests a prefix match.
@liggitt happier?
Automatic merge from submit-queue
bump(golang.org/x/oauth2): a6bd8cefa1811bd24b86f8902872e4e8225f74c4
As I tackle https://github.com/kubernetes/kubernetes/issues/42654 kubectl's OpenID Connect plugin will start using golang.org/x/oauth2 for refreshing, instead of go-oidc's own hand rolled oauth2 implementation. In preparation, update golang.org/x/oauth2 to include 7374b3f1ec which fixes refreshing with Okta.
We also somehow removed the dependency on `google.golang.org/appengine`. Maybe 8cf58155e4?
cc @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue
Remove imports of k8s.io/client-go/pkg/api
k8s.io/client-go/pkg/api will be eliminated by #44784, so we need to remove all imports of it.
This PR is large, but mostly simple changes, like:
* using constants defined in the v1 package instead of those in the internal package
* using k8s.io/client-go/kubernetes/scheme.Codecs/Schemes instead of those in k8s.io/client-go/pkg/api
* using `v1.SchemeGroupVersion` instead of `api.Registry.GroupOrDie(api.GroupName).GroupVersion` in tests
* importing `k8s.io/kubernetes/pkg/api` instead of `k8s.io/client-go/pkg/api` if possible
With this PR https://github.com/kubernetes/kubernetes/pull/45159, the only remaining imports of `k8s.io/client-go/pkg/api` are from k8s.io/metrics.
``` bash
$ git grep "\"k8s.io/client-go/pkg/api\"" | grep -v "staging/src/k8s.io/client-go/pkg/api"
staging/src/k8s.io/metrics/pkg/apis/custom_metrics/install/install.go:25: "k8s.io/client-go/pkg/api"
Binary file staging/src/k8s.io/metrics/pkg/apis/custom_metrics/types.generated.go matches
staging/src/k8s.io/metrics/pkg/apis/custom_metrics/types.go:22: "k8s.io/client-go/pkg/api"
staging/src/k8s.io/metrics/pkg/apis/metrics/install/install.go:26: "k8s.io/client-go/pkg/api"
staging/src/k8s.io/metrics/pkg/apis/metrics/types.go:21: "k8s.io/client-go/pkg/api"
Binary file staging/src/k8s.io/metrics/pkg/apis/metrics/v1alpha1/zz_generated.conversion.go matches
Binary file staging/src/k8s.io/metrics/pkg/apis/metrics/zz_generated.deepcopy.go matches
staging/src/k8s.io/metrics/pkg/client/clientset_generated/clientset/fake/clientset_generated.go:24: "k8s.io/client-go/pkg/api"
staging/src/k8s.io/metrics/pkg/client/custom_metrics/client.go:26: "k8s.io/client-go/pkg/api"
staging/src/k8s.io/metrics/pkg/client/custom_metrics/fake/fake_client.go:24: "k8s.io/client-go/pkg/api"
```
Automatic merge from submit-queue (batch tested with PRs 45272, 45115)
initial types for TPRs
This pull starts creating the types described by https://github.com/kubernetes/community/blob/master/contributors/design-proposals/thirdpartyresources.md . In the initial pull different names were suggested. I've started this pull with `CustomResource.apiextensions.k8s.io`.
The structure begins as a separate API server to facilitate rapid prototyping and experimentation, but the end result will be added to the end of the `kube-apiserver` chain as described in https://github.com/kubernetes/community/blob/master/sig-api-machinery/api-extensions-position-statement.md .
Because it is separate to start (not included in any default server), I don't think we need a perfect name, but I'd like to be close.
@kubernetes/sig-api-machinery-misc @enisoc @smarterclayton @erictune
Automatic merge from submit-queue
Fixes broken link in PodSpec
The PodSpec reference to NodeSelector misses
the `.md` extension in the file.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45227, 43003, 45231)
separate discovery from the apiserver
This decouples the API discovery handlers from the core API server code. It separates the code into a new package and clarifies interfaces with existing TPR code.
@kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45100, 45152, 42513, 44796, 45222)
apiserver: split endpoint tests from client-go
Use the example apigroup and create its own scheme.
Fixes https://github.com/kubernetes/kubernetes/issues/45209.
Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)
Move k8s.io/metrics to staging/
This is to break the cyclic dependency in our code base: kubernetes depends on k8s.io/metrics, which depends on kubernetes/staging/client-go.
@DirectXMan12 i actually moved it to staging because we will need the flexibility to update metrics code directly to do many planned refactors, so the copy of metrics in kubernetes has to be the source of truth.
client-gen is not enabled for the code yet, we can enable it after you port your changes to client-gen.
`make generated_files` is enabled for metrics.
Automatic merge from submit-queue
Delete "hard-coded" default value in flags usage.
**What this PR does / why we need it**:
Some flags of kubernetes components have "hard-coded" default values in their usage info. In fact, [pflag pkg](https://github.com/kubernetes/kubernetes/blob/master/vendor/github.com/spf13/pflag/flag.go#L602-L608) has already added a string `(default value)` automatically in the usage info if the flag is initialized. Then we don't need to hard-code the default value in usage info. After this PR, if we want to update the default value of a flag, we only need to update the flag where it is initialized. `pflag` will update the usage info for us. This will avoid inconsistency.
For example:
Before
```
kubelet -h
...
--node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. Default: 10s (default 10s)
...
```
After
```
kubelet -h
...
--node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. (default 10s)
...
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
This PR doesn't delete some "hard-coded" default values because they are not explicitly initialized. We still need to hard-code them to give users friendly info.
```
--allow-privileged If true, allow containers to request privileged mode. [default=false]
```
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 44044, 44766, 44930, 45109, 43301)
add APIService conditions
Adds conditions to the APIServiceStatus struct and fixes up generators that appear to have slipped.
The first condition is "ServiceAvailable" which will provide the status currently derived in the discovery handler that decides about whether to expose the version in discovery.
@kubernetes/sig-api-machinery-pr-reviews @liggitt @ncdc
Automatic merge from submit-queue
kubectl binary plugins
**What this PR does / why we need it**:
Introduces the ability to extend `kubectl` by adding third-party plugins that will be exposed through `kubectl`.
Plugins are executable commands written in any language. To be included as a plugin, a binary or script file has to
1. be located under one of the supported plugin path locations:
1.1 `~/.kubectl/plugins` dir
1.2. one or more directory set in the `KUBECTL_PLUGINS_PATH` env var
1.3. the `kubectl/plugins` dir under one or more directory set in the `XDG_DATA_DIRS` env var, which defaults to `/usr/local/share:/usr/share`
2. in any of the plugin path above, have a subfolder with the plugin file(s)
3. in the subfolder, contain at least a `plugin.yaml` file that describes the plugin
Example:
```
$ cat ~/.kube/plugins/myplugin/plugin.yaml
name: "myplugin"
shortDesc: "My plugin's short description"
command: "echo Hello plugins!"
$ kubectl myplugin
Hello plugins!
```
~~In case the plugin declares `tunnel: true`, the plugin engine will pass the `KUBECTL_PLUGIN_API_HOST` env var when calling the plugin binary. Plugins can then access the Kube REST API in "http://$KUBECTL_PLUGIN_API_HOST/api" using the same context currently in use by `kubectl`.~~
Test plugins are provided in `pkg/kubectl/plugins/examples`. Just copy (or symlink) the files to `~/.kube/plugins` to test.
**Which issue this PR fixes**:
Related to the discussions in the proposal document: https://github.com/kubernetes/kubernetes/pull/30086 and https://github.com/kubernetes/community/pull/122.
**Release note**:
```release-note
Introduces the ability to extend kubectl by adding third-party plugins. Developer preview, please refer to the documentation for instructions about how to use it.
```
Automatic merge from submit-queue (batch tested with PRs 44868, 44350)
build external watch event so simple encoders can encode
`kube-apiserver` clients require a specific serialization of `watch.Event` to function properly. There is no reason to allow flexibility of serialization at this point since no client would able to understand a different encoding.
I found this which trying to use a simple, unstructured json encoder and the clients kept choking on watches because it serialized without the proper json tags.
@kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue
prevent corrupted spdy stream after hijacking connection
This PR fixes corner case in spdy stream code where some bytes would never arrive at the server.
Reading directly from a hijacked connection isn't safe because some data may have already been read by the server before `Hijack` was called. To ensure all data will be received it's safer to read from the returned `bufio.Reader`. This problem seem to happen more frequently when using Go 1.8.
This is described in https://golang.org/pkg/net/http/#Hijacker:
> // The returned bufio.Reader may contain unprocessed buffered
// data from the client.
I came across this while debugging a flaky test that used code from the `k8s.io/apimachinery/pkg/util/httpstream/spdy` package. After filling the code with debug logs and long hours running the tests in loop in the hope of catching the error I finally caught something weird.
The first word on the first spdy frame [read by the server here](b625085230/vendor/github.com/docker/spdystream/spdy/read.go (L148)) had the value `0x03000100`. See, the first frame to arrive on the server was supposed to be a control frame indicating the creation of a new stream, but all control frames need the high-order bit set to 1, which was not the case here, so the saver mistakenly assumed this was a data frame and the stream would never be created. The correct value for the first word of a SYN_STREAM frame was supposed to be `0x80030001` and this lead me on the path of finding who had consumed the first 1 byte prior to the frame reader being called and finally finding the problem with the Hijack call.
I added a new test to try stressing this condition and ensuring that this bug doesn't happen anymore. However, it's quite ugly as it loops 1000 times creating streams on servers to increase the chances of this bug happening. So, I'm not sure whether it's worth it to keep this test or if I should remove it from the PR. Please let me know what you guys think and I'll be happy to update this.
Fixes#45093#45089#45078#45075#45072#45066#45023
Automatic merge from submit-queue
Prepare for move zz_generated_deepcopy.go to k8s.io/api
This is in preparation to move deep copies to with the types to the types repo (see https://github.com/kubernetes/gengo/pull/47#issuecomment-296855818). The init() function is referring the `SchemeBuilder` defined in the register.go in the same packge, so we need to revert the dependency.
This PR depends on https://github.com/kubernetes/gengo/pull/49, otherwise verification will fail.
Automatic merge from submit-queue (batch tested with PRs 45052, 44983, 41254)
Non-controversial part of #44523
For easier review of #44523, i extracted the non-controversial part out to this PR.
Automatic merge from submit-queue (batch tested with PRs 44940, 44974, 44935)
apimachinery/pkg/util/wait: Fix potential goroutine leak in pollInternal().
**What this PR does / why we need it**:
Without the change, the wait function wouldn't exit until the timeout
happens, so if the timeout is set to a big value and the Poll() is run
inside a loop, then the total goroutines will increase indefinitely.
This PR fixes the issue by closing the stop channel to tell the wait function
to exit immediately if condition is true or any error happens.
Automatic merge from submit-queue (batch tested with PRs 44940, 44974, 44935)
Remove import of internal api package in generated external-versioned listers
Follow up of https://github.com/kubernetes/kubernetes/pull/44523
One line change in cmd/libs/go2idl/lister-gen/generators/lister.go, and simple changes in pkg/apis/autoscaling/v2alpha1/register.go, other changes are generated.
The internal api package will be eliminated from client-go, so these imports should be removed. Also, it's more correct to report the versioned resource in the error.
Automatic merge from submit-queue
Fix mergepatch.HasConflicts().
**What this PR does / why we need it**:
This fixes some false negatives:
* If a map had multiple entries, only the first was checked.
* If a list had multiple entries, only the first was checked.
**Which issue this PR fixes**:
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
OpenAPI support for kubectl
Support for openapi spec in kubectl.
Includes:
- downloading and caching openapi spec to a local file
- parsing openapi spec into binary serializable datastructures (10x faster load times 600ms -> 40ms)
- caching parsed openapi spec in memory for each command
```release-note
NONE
```
Add support for following redirects to the SpdyRoundTripper. This is
necessary for clients using it directly (e.g. the apiserver talking
directly to the kubelet) because the CRI streaming server issues a
redirect for streaming requests.
Also extract common logic for following redirects.
Automatic merge from submit-queue (batch tested with PRs 41287, 41636, 44881, 44826)
Bump to go1.8 and remove the edge GOROOT
**What this PR does / why we need it**:
Bumps to go1.8; we get:
- performance improvements
- build time improvements
- the possibility to remove the hacky edge-GOROOT for arm and ppc64le that must use go1.8
- all other awesome features that are included in go1.8: https://golang.org/doc/go1.8
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/38228
**Special notes for your reviewer**:
@ixdy Please push the image ASAP so we can see if this passes all tests
**Release note**:
```release-note
Upgrade go version to v1.8
```
cc @ixdy @bradfitz @jessfraz @wojtek-t @timothysc @spxtr @thockin @smarterclayton @bprashanth @gmarek
Automatic merge from submit-queue (batch tested with PRs 41287, 41636, 44881, 44826)
apiserver: add pkg/util/webhook tests
This commit adds tests for pkg/util/webhooks. The purpose of this was not only for better code coverage but also to alleviate the need for consumers to write their own tests for core functionality.
**Reasoning**
While working on the generic webhook admission plugin for https://github.com/kubernetes/community/pull/132, I found out that there are no tests for `staging/k8s.io/apiserver/pkg/util/webhook` and I also noticed that various places in the k8s sources were also replicating the same core tests that could/should be handled in `staging/k8s.io/apiserver/pkg/util/webhook`:
* `staging/src/k8s.io/apiserver/plugin/pkg/authorizer/webhook/webhook_test.go`
* `staging/src/k8s.io/apiserver/plugin/pkg/authenticator/token/webhook/webhook_test.go`
* `plugin/pkg/admission/imagepolicy/admission_test.go`
* _possibly more..._
That's what this PR does, adds tests for `staging/k8s.io/apiserver/pkg/util/webhook` so that consumers no longer have to worry about testing client configuration issues, TLS issues, etc. just to use `staging/k8s.io/apiserver/pkg/util/webhook`.
cc @cjcullen
Without the change, the wait function wouldn't exit until the timeout
happens, so if the timeout is set to a big value and the Poll() is run
inside a loop, then the total goroutines will increase indefinitely.
This PR fix the issue by closing the stop channel to tell the wait function
to exit immediately if condition is true or any error happens.
Automatic merge from submit-queue
bump bazel build to go1.8.1 and remove invalid unit tests
part of https://github.com/kubernetes/kubernetes/issues/38228
I firmly believe that unit tests that check error strings are incorrect unit tests. If we care about what type of error is returned, we need to use public error types. Anywhere we are using generic errors, we don't care other then that we saw an error.
Automatic merge from submit-queue (batch tested with PRs 44601, 44842, 44893, 44491, 44588)
Define const annotation variable once
We do not need to define the const annotation var twice in pkg/volume and pkg/volume/validation
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42477, 44462)
Use storage.v1 instead of v1beta1
storage.v1beta1 was used to work around GKE which did not expose v1. Now that GKE is updated, we can switch everything to v1.
This is simple sed v1beta1 -> v1 + enabled a new test + changed preference of exposed interfaces in `storage/install/install.go`.
@msau42, PTAL and let me know when GKE is updated with storage v1 API and this PR can be actually merged.
@kubernetes/sig-storage-pr-reviews
```release-note
NONE
```
Automatic merge from submit-queue
Stop treating in-cluster-config namespace as an override
Fixes#44835
The namespace of an in-cluster config should behave like the namespace specified in a kubeconfig file... it should be used as the default namespace, but be able to be overridden by namespaces specified in yaml files passed to `kubectl create -f`.
```release-note
Restored the ability of kubectl running inside a pod to consume resource files specifying a different namespace than the one the pod is running in.
```
This commit adds tests for pkg/util/webhooks. The purpose of this was
not only for better code coverage but also to alleviate the need for
consumers to write their own tests for core functionality.
Automatic merge from submit-queue
PATCH: Fix erroneous meaningful conflict for numeric values.
The wrong json package was used, resulting in patches being unmarshaled with numbers as float64 rather than int64. This in turn confused `HasConflicts()` which expects numeric types to match.
The end result was false positives of meaningful conflicts, such as:
```
there is a meaningful conflict (firstResourceVersion: "8517", currentResourceVersion: "8519"):
diff1={"metadata":{"resourceVersion":"8519"},"spec":{"replicas":0},"status":"conditions":null,"fullyLabeledReplicas":null,"replicas":0}}
, diff2={"spec":{"replicas":0}}
```
This is branched from a discussion on https://github.com/kubernetes/kubernetes/pull/43469.
```release-note
Fix false positive "meaningful conflict" detection for strategic merge patch with integer values.
```
Automatic merge from submit-queue (batch tested with PRs 44837, 44779, 44492)
remove duplicate code from restful namer
The deleted codes have the same logic with function ObjectName.
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 44837, 44779, 44492)
Default ObjectNameFunc for all REST Stores
All `Store`s in Kubernetes follow the same logic for determining the name of an object. This change makes it so that `CompleteWithOptions` defaults the `ObjectNameFunc` if it is not specified. Thus a user does not need to remember to use `ObjectMeta.Name`. Using the wrong field as the name can lead to an object which has a name that bypasses normal object name validation.
Signed-off-by: Monis Khan <mkhan@redhat.com>
cc @liggitt @soltysh for review
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Fix Content-Type error of apis
**What this PR does / why we need it**:
Response's Content-type of `/apis` is `text/plain`, not `application/json`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44646
**Special notes for your reviewer**:
We could use `responsewriters.WriteObjectNegotiated()` to write Content-type correctly.
**Release note**:
```release-note
```
Automatic merge from submit-queue
etcd component status check should include credentials
- [x] Add TLS credentials into `pkg/genericapiserver.Backend`.
- [x] Add TLS credentials into `pkg/registry/core/componentstatus.Server`.
- [x] `pkg/probe/http.httpProber` should accept the TLS credentials.
Now it is working.
```console
$ kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health": "true"}
```
Fixes https://github.com/kubernetes/kubernetes/issues/27343.
The wrong json package was used, resulting in patches being unmarshaled
with numbers as float64 rather than int64.
This in turn confused HasConflicts() which expects numeric types to match.
The end result was false positives of meaningful conflicts, such as:
```
there is a meaningful conflict (firstResourceVersion: "8517",
currentResourceVersion: "8519"):
diff1={"metadata":{"resourceVersion":"8519"},"spec":{"replicas":0},"status":{"conditions":null,"fullyLabeledReplicas":null,"replicas":0}}
, diff2={"spec":{"replicas":0}}
```
This fixes some false negatives:
* If a map had multiple entries, only the first was checked.
* If a list had multiple entries, only the first was checked.
Automatic merge from submit-queue
Explicit namespace from kubeconfig should override in-cluster config
Fixes#43662
If an explicitly specified namespace is read from a kubeconfig file, we should not fall back to in-cluster config
```release-note
kubectl commands run inside a pod using a kubeconfig file now use the namespace specified in the kubeconfig file, instead of using the pod namespace. If no kubeconfig file is used, or the kubeconfig does not specify a namespace, the pod namespace is still used as a fallback.
```
All Stores in Kubernetes follow the same logic for determining the name
of an object. This change makes it so that CompleteWithOptions defaults
the ObjectNameFunc if it is not specified. Thus a user does not need to
remember to use ObjectMeta.Name. Using the wrong field as the name can
lead to an object which has a name that bypasses normal object name
validation.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 42202, 40784, 44642, 44623, 44761)
Split out AdmissionOptions
**What this PR does / why we need it**: The changes in this PR split out AdmissionOptions from ServerRunOptions. In the long term AdmissionOptions will accepts various dependencies
and spit out AdmissionControl
**Release note**:
NONE
Automatic merge from submit-queue
Include system:authenticated group when impersonating
Fixes#43227
An authorized impersonation request solely for a specific username previously resulted in a `user.Info` that did not include either the `system:authenticated` or `system:unauthenticated` groups. That meant that permissions intended to be granted to all users, like discovery, would be denied the impersonated user.
This allows `kubectl get pods --as=<username>` to work as expected
```release-note
API requests using impersonation now include the `system:authenticated` group in the impersonated user automatically.
```
Automatic merge from submit-queue
client-go PortForwarder: don't log an error if net.Listen fails
**What this PR does / why we need it**:
If ipv6 is disabled the client-go PortForwarder logs an error:
```
E0406 17:26:06.983609 29625 portforward.go:209] Unable to create listener: Error listen tcp6 [::1]:30143: socket: address family not supported by protocol
```
This pull request removes this error log. Reasons:
* Tunnel creation might still succeed on ipv4 and the error just confuses the user (see https://github.com/kubernetes/helm/issues/2223).
* If the tunnel can't be created the error gets also loggen in `listenOnPort`.
* runtime.HandleError should only be used on errors "_when a non-user facing piece of code cannot return an error"_ which isn't the case here.
**Which issue this PR fixes**
Fixes https://github.com/kubernetes/helm/issues/2223
Automatic merge from submit-queue
Minor cleanups
Minor improvements:
- `ValidateNoNewFinalizers`: remove unused const
- Mention that mutation of `spec.initContainers[*].image` field is allowed
- Improve godoc comments
Automatic merge from submit-queue (batch tested with PRs 44594, 44651)
remove strings.compare(), use string native operation
I notice we use strings.Compare() in some code, we can remove it and use native operation.
