Commit Graph

10808 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
455e9fff09 Merge pull request #46176 from vmware/vSphereStoragePolicySupport
Automatic merge from submit-queue

vSphere storage policy support for dynamic volume provisioning

Till now, vSphere cloud provider provides support to configure persistent volume with VSAN storage capabilities - kubernetes#42974. Right now this only works with VSAN.

Also there might be other use cases:

- The user might need a way to configure a policy on other datastores like VMFS, NFS etc.
- Use Storage IO control, VMCrypt policies for a persistent disk.

We can achieve about 2 use cases by using existing storage policies which are already created on vCenter using the Storage Policy Based Management service. The user will specify the SPBM policy ID as part of dynamic provisioning 

- resultant persistent volume will have the policy configured with it. 
- The persistent volume will be created on the compatible datastore that satisfies the storage policy requirements. 
- If there are multiple compatible datastores, the datastore with the max free space would be chosen by default.
- If the user specifies the datastore along with the storage policy ID, the volume will created on this datastore if its compatible. In case if the user specified datastore is incompatible, it would error out the reasons for incompatibility to the user.
- Also, the user will be able to see the associations of persistent volume object with the policy on the vCenter once the volume is attached to the node.

For instance in the below example, the volume will created on a compatible datastore with max free space that satisfies the "Gold" storage policy requirements.

```
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
       name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
      diskformat: zeroedthick
      storagepolicyName: Gold
```

For instance in the below example, the vSphere CP checks if "VSANDatastore" is compatible with "Gold" storage policy requirements. If yes, volume will be provisioned on "VSANDatastore" else it will error that "VSANDatastore" is not compatible with the exact reason for failure.

```
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
       name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
      diskformat: zeroedthick
      storagepolicyName: Gold
      datastore: VSANDatastore
```

As a part of this change, 4 commits have been added to this PR.

1. Vendor changes for vmware/govmomi
2. Changes to the VsphereVirtualDiskVolumeSource in the Kubernetes API. Added 2 additional fields StoragePolicyName, StoragePolicyID
3. Swagger and Open spec API changes.
4. vSphere Cloud Provider changes to implement the storage policy support.

**Release note**:


```release-note
vSphere cloud provider: vSphere Storage policy Support for dynamic volume provisioning
```
2017-05-22 23:41:10 -07:00
Kubernetes Submit Queue
bb56937b92 Merge pull request #46055 from deads2k/crd-01-embed
Automatic merge from submit-queue (batch tested with PRs 46022, 46055, 45308, 46209, 43590)

embed kube-apiextensions inside of kube-apiserver

To reduce operation complexity, we decided to include the kube-apiextensions-server inside of kube-apiserver (https://github.com/kubernetes/community/blob/master/sig-api-machinery/api-extensions-position-statement.md#q-should-kube-aggregator-be-a-separate-binaryprocess-than-kube-apiserver).  With the API reasonably well established and a finalizer about merge, I think its time to add ourselves.

This pull wires kube-apiextensions-server ahead of the TPRs so that one will replace the other if both are added by accident (CRDs should have priority) and wires a controller for automatic aggregation.

WIP because I still need tests: unit test for controller, test-cmd test to mirror the TPR test.


```release-note
Adds the `CustomResourceDefinition` (crd) types to the `kube-apiserver`.  These are the successors to `ThirdPartyResource`.  See https://github.com/kubernetes/community/blob/master/contributors/design-proposals/thirdpartyresources.md for more details.
```
2017-05-22 19:59:57 -07:00
System Administrator
83520a7470 Kubernetes core API changes for vSphere 2017-05-22 19:43:29 -07:00
Kubernetes Submit Queue
c2c5051adf Merge pull request #44899 from smarterclayton/burst
Automatic merge from submit-queue (batch tested with PRs 38990, 45781, 46225, 44899, 43663)

Support parallel scaling on StatefulSets

Fixes #41255

```release-note
StatefulSets now include an alpha scaling feature accessible by setting the `spec.podManagementPolicy` field to `Parallel`.  The controller will not wait for pods to be ready before adding the other pods, and will replace deleted pods as needed.  Since parallel scaling creates pods out of order, you cannot depend on predictable membership changes within your set.
```
2017-05-22 19:07:09 -07:00
Kubernetes Submit Queue
503a8421fb Merge pull request #38990 from mikedanese/go-genrule-sets
Automatic merge from submit-queue (batch tested with PRs 38990, 45781, 46225, 44899, 43663)

migrate set generation to go_genrule

Depends on https://github.com/kubernetes/release/pull/238
2017-05-22 19:06:58 -07:00
Kubernetes Submit Queue
8e98f1dfec Merge pull request #45910 from seh/correct-tpr-example-registration
Automatic merge from submit-queue (batch tested with PRs 46133, 46211, 46224, 46205, 45910)

Correct client-go TPR example type registration

**What this PR does / why we need it**:

Eliminate duplicate registration of third-party resource types in the _client-go_ example, precluding any number of readers from copying the mistake into their own applications as they adapt the example to their own needs.

**Special notes for your reviewer**:

See [the preceding discussion](a6c97715ed (commitcomment-22146536)) about a6c97715ed, committed as part of #45463 but only noticed after the merge.

It's possible that we could take a few of the changes proposed here and apply them more broadly throughout the rest of the code, such as not exporting the `AddToScheme` var in favor of an actual function declaration. Given the flux in #44784, I'd prefer that we don't hold up these small touch-ups on a broader unification.

People I expect will want to weigh in: @sttts, @caesarxuchao, and @nilebox.
2017-05-22 15:50:07 -07:00
Kubernetes Submit Queue
2cb0cda3e0 Merge pull request #46133 from ahmetb/clientgo-examples/in-cluster
Automatic merge from submit-queue (batch tested with PRs 46133, 46211, 46224, 46205, 45910)

clientgo/examples/in-cluster: add instructions to run the example
2017-05-22 15:49:58 -07:00
Michelle Au
5fcb82dde9 Fix storage node affinity helpers 2017-05-22 15:30:27 -07:00
Kubernetes Submit Queue
f6b3d083b7 Merge pull request #44640 from msau42/local-pv-api
Automatic merge from submit-queue

LocalStorage api

**What this PR does / why we need it**:
API changes to support persistent local volumes, as described [here](https://github.com/kubernetes/community/pull/306)

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Part of #43640

**Special notes for your reviewer**:
There were a few items I was concerned about.  Will add review comments in those places.

**Release note**:

NONE

Note will be added in subsequent PR with the volume plugin changes
2017-05-22 14:39:05 -07:00
Eric Tune
12fbd82529 Merge pull request #45301 from erictune/disrupt-approve
Move PDB controller and type ownership to SIG-Apps
2017-05-22 14:21:41 -07:00
Kubernetes Submit Queue
91adb3eacf Merge pull request #44906 from k82cn/moved_qos_to_v1helper
Automatic merge from submit-queue

Moved qos to api.helpers.

**What this PR does / why we need it**:
The `GetPodQoS` is also used by other components, e.g. kube-scheduler and it's not bound to kubelet; moved it to api helpers so client-go.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #N/A

**Release note**:

```release-note-none
```
2017-05-22 13:30:06 -07:00
Eric Tune
b17e3c14eb Move PDB controller and type ownership to SIG-Apps
Created OWNERS_ALIASES called sig-apps-reviewers from the union of reviewers in:
 pkg/controller/{cronjob,deployment,daemon,job,replicaset,statefulset}/OWNERS
except removed inactive user bprashanth

Created OWNERS_ALIASES called sig-apps-api-reviewers as the intersection
of sig-apps-reviewers and the approvers from pkg/api/OWNERS.

Used those OWNERS_ALIASES as the reviewers/approvers for the disruption controller,
and API.
2017-05-22 12:55:28 -07:00
Kubernetes Submit Queue
4d6ef25f64 Merge pull request #46057 from mengqiy/improvePatchTypeError
Automatic merge from submit-queue

improve type assertion error

Per discussion https://github.com/kubernetes/kubernetes/pull/45980#issuecomment-302524829.

```release-note
NONE
```
2017-05-22 11:51:10 -07:00
Wojciech Tyczynski
b4018f7da1 Pass RequestInfo to GenerateLink 2017-05-22 20:45:43 +02:00
Clayton Coleman
e9e69356e4 Cache watch returns incorrect object on DELETED events
The underlying storage has always returned the old object on watch
delete events when filtering. The cache watcher does not, which means a
downsteam caller gets different behavior.

This fixes the cache watcher to be consistent with our long term
behavior for watch. It may result in a behavior change (the filter
becomes more precise) but this was a regression in behavior.
2017-05-22 13:50:14 -04:00
Eric Chiang
6915f85757 oidc client plugin: reduce round trips and switch to golang.org/x/oauth2
This PR attempts to simplify the OpenID Connect client plugin to
reduce round trips. The steps taken by the client are now:

* If ID Token isn't expired:
   * Do nothing.
* If ID Token is expired:
   * Query /.well-known discovery URL to find token_endpoint.
   * Use an OAuth2 client and refresh token to request new ID token.

This avoids the previous pattern of always initializing a client,
which would hit the /.well-known endpoint several times.

The client no longer does token validation since the server already
does this. As a result, this code no longer imports
github.com/coreos/go-oidc, instead just using golang.org/x/oauth2
for refreshing.
2017-05-22 10:31:38 -07:00
Wojciech Tyczynski
25f0fe1adb Pass Context to GenerateLink 2017-05-22 19:26:11 +02:00
Kubernetes Submit Queue
83b49b5a92 Merge pull request #45864 from shiywang/autoregister_controller
Automatic merge from submit-queue (batch tested with PRs 45864, 46169)

Fix unit tests for autoregister_controller.go reliable

Fixes https://github.com/kubernetes/kubernetes/issues/45538
Still wip, and just have some questions which I left some comments in original issue above
2017-05-22 08:08:54 -07:00
Steven E. Harris
3a8e44bda8 Remove redundant third-party type registration
Since NewSchemeBuilder registers the supplied Scheme transformation
functions synchronously, there's no need for a subsequent call
to (*SchemeBuilder).Register against the same instance.
2017-05-22 09:29:51 -04:00
Steven E. Harris
fe3ca9a6eb Copyedit third-party resource registration example 2017-05-22 09:29:51 -04:00
deads2k
446e959bf7 make CRD apiservice controller 2017-05-22 08:54:14 -04:00
deads2k
a637c49c8d embed apiextensions server into kube-apiserver 2017-05-22 08:53:07 -04:00
deads2k
e7871dbab2 tighten and simplify owners in some staging repos 2017-05-22 08:19:19 -04:00
Jeffrey Regan
5a9b06b717 **What this PR does / why we need it**:
Remove kubectl's dependence on pkg/api/helper, as part of
broader effort to isolate kubectl from the rest of k8s.
In this case, the code becomes private to kubectl; nobody else uses it.

**Which issue this PR fixes**

Part of a series of PRs to address kubernetes/community#598

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-05-21 20:44:21 -07:00
Michelle Au
936269a2eb Generated code for local persistent volumes 2017-05-21 12:20:23 -07:00
shiywang
8fe3db7030 Fix unit tests for autoregister_controller.go reliable 2017-05-21 23:57:52 +08:00
Clayton Coleman
2861ae5eb9 Support burst in stateful set scale up and down
The alpha field podManagementPolicy defines how pods are created,
deleted, and replaced. The new `Parallel` policy will replace pods
as fast as possible, not waiting for the pod to be `Ready` or providing
an order. This allows for advanced clustered software to take advantage
of rapid changes in scale.
2017-05-21 01:14:26 -04:00
Clayton Coleman
ad720cc651 generated: bazel 2017-05-20 21:58:38 -04:00
Clayton Coleman
5ac3214c42 Mutation cache should support retrieving items from ByIndex()
Allows tokens controller to observe updates
2017-05-20 14:19:49 -04:00
Clayton Coleman
5439cfd245 Add a filtering resource handler for informers
Allows an informer consumer to easily filter a set of changes out,
possibly to maintain a smaller cache or to only operate on a known set
of objects.
2017-05-20 14:19:48 -04:00
Clayton Coleman
3e095d12b4 Refactor move of client-go/util/clock to apimachinery 2017-05-20 14:19:48 -04:00
Clayton Coleman
8013212db5 Move client-go/util/clock to apimachinery/pkg/util/clock
For reuse
2017-05-20 14:19:47 -04:00
Clayton Coleman
8e1639a71b Change LRUExpireCache to use hashicorp cache to expose Keys()
Removes the spawning of goroutines in the cache (which could be a
hotspot for anything in the critical path) as well.
2017-05-20 14:19:47 -04:00
Clayton Coleman
529e627c8a Move pkg/util/cache to apimachinery
Will be used by client-go as well
2017-05-20 14:19:46 -04:00
Kubernetes Submit Queue
a8bff0ed9a Merge pull request #45836 from mbohlool/openapi_pb
Automatic merge from submit-queue

Add protobuf binary version of OpenAPI spec

Fixes #45833
Partially fixes #42841

```release-note
OpenAPI spec is now available in protobuf binary and gzip format (with ETag support)
```
2017-05-20 11:01:04 -07:00
Klaus Ma
fd0190fd68 generated client-go. 2017-05-20 07:52:32 -04:00
Kubernetes Submit Queue
2c2b5f7379 Merge pull request #45085 from sttts/sttts-aggregator-upgrade
Automatic merge from submit-queue

kube-apiserver: check upgrade header to detect upgrade connections

Without this every connection with "Connection" header but without upgrade request are rejected. A simple
curl will set "Connection", but does not intent to upgrade.
2017-05-20 02:08:00 -07:00
Kubernetes Submit Queue
bdeac66adc Merge pull request #46014 from YuPengZTE/devFinishRequest
Automatic merge from submit-queue (batch tested with PRs 46014, 46152)

format reads arg 3, have only 2 args, add i

Signed-off-by: yupengzte <yu.peng36@zte.com.cn>



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-05-20 00:59:26 -07:00
Kubernetes Submit Queue
113cf85612 Merge pull request #45903 from brendandburns/azure-disk-api
Automatic merge from submit-queue (batch tested with PRs 45346, 45903, 45958, 46042, 45975)

Azure disk api

This is to update the AzureDiskApi and split it from the implementation which is caught in rebase hell...

Once this is merged, we'll get the implementation in.

@smarterclayton suggested this as a way to break the rebase hell logjam. request for a quick review.

Thanks!
2017-05-19 22:29:30 -07:00
ymqytw
dd80b915e0 improve type assertion error 2017-05-19 18:07:59 -07:00
mbohlool
4d4abf3ba6 Update bazel 2017-05-19 15:39:08 -07:00
mbohlool
4b0fbfe1ee bugfix: form parameters should have type in OpenAPI spec 2017-05-19 15:39:08 -07:00
mbohlool
161b480107 Add protobuf binary version of openapi spec 2017-05-19 15:39:08 -07:00
mbohlool
67025046a5 Add gnostic to Godep 2017-05-19 15:39:08 -07:00
Ahmet Alp Balkan
8604ed6d99 clientgo/examples/in-cluster: add instructions to run the example
This patch adds instructions for how to run the in-cluster client-go example.
To make this example executable, providing a Dockerfile and build steps so
that it can directly be run on minikube.

This is part of the body of work improving the client library samples.

Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
2017-05-19 14:55:10 -07:00
Jordan Liggitt
da8ae29620 Return MethodNotSupported when accessing unwatcheable resource with ?watch=true 2017-05-19 15:52:38 -04:00
Kubernetes Submit Queue
ff4607342f Merge pull request #46059 from nikhita/test-int-preserve
Automatic merge from submit-queue (batch tested with PRs 46075, 46059, 46095, 46097)

Integration test for kube-apiextensions-server: integers

**What this PR does / why we need it**: Check if integers are present after decoding.
Originally an issue for TPRs: #30213

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: for #45511 

**Special notes for your reviewer**:

**Release note**:

```
NONE
```
@sttts
2017-05-19 08:35:08 -07:00
Ma Shimiao
c0feba4f9f remove unreachable code
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
2017-05-19 16:47:15 +08:00
Kubernetes Submit Queue
d9005df6cb Merge pull request #46061 from shyamjvs/master-metrics-fix
Automatic merge from submit-queue

Copy static variable 'verb' before instrumenting APIserver call to prevent overwriting

Fixes https://github.com/kubernetes/kubernetes/issues/45566

cc @wojtek-t @gmarek
2017-05-19 00:03:51 -07:00
Brendan Burns
97f6c52ffd more stuff. 2017-05-18 20:31:10 -07:00