Automatic merge from submit-queue
bazel: stamp multiple packages by using x_defs instead of linkstamp in go_binary rules
**What this PR does / why we need it**: Fixes regression introduced sometime in the last few months that prevented bazel-built clusters from identifying version properly.
It does so by updating the bazelbuild/rules_go and kubernetes/repo-infra dependencies to support using stamp values in `go_binary` `x_defs`, and then changing our `go_binary` rules to use `x_defs` instead of `linkstamp`.
This whole charade is necessary because we need to stamp version information in multiple packages.
This pretty much only affects the bazel build, so it should be low risk.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45298
**Special notes for your reviewer**: depends on https://github.com/kubernetes/repo-infra/pull/18; should not be merged before it.
**Release note**:
```release-note
NONE
```
/assign @spxtr @mikedanese
Automatic merge from submit-queue
StorageOS Volume Plugin
**What this PR does / why we need it**:
This PR adds a new volume plugin for StorageOS volumes. StorageOS runs as a container on Kubelet nodes, aggregating local or attached storage and making its capacity available to all nodes within the cluster. More information at http://storageos.com.
The StorageOS plugin supports:
1. Dynamic Provisioning using Storage Classes
2. Persistent Volumes and Persistent Volume Claims.
**Which issue this PR fixes**
A feature request has been created:
https://github.com/kubernetes/features/issues/190
This isn't on the schedule for 1.6 as I wasn't sure when it would be ready. We intend to make the StorageOS container openly available within the 1.6 lifetime.
**Special notes for your reviewer**:
Separate commits for feature and godep changes.
**Release note**:
```release-note
StorageOS Volume Driver
[StorageOS](http://www.storageos.com) can be used as a storage provider for Kubernetes. With StorageOS, capacity from local or attached storage is pooled across the cluster, providing converged infrastructure for cloud-native applications.
```
Automatic merge from submit-queue (batch tested with PRs 45057, 47259)
Allow conversion-gen to take types that are not in the output path
Part of fixing https://github.com/kubernetes/kubernetes/issues/44065.
Partially address https://github.com/kubernetes/features/issues/282.
The first commit is the changes to the conversion-gen (and vendor).
The second commit moves the api/v1 types to staging, and the third commit shows the resulted changes in the generated conversions.
Automatic merge from submit-queue
kubeadm: Enable the Node Authorizer/Admission plugin in v1.7
**What this PR does / why we need it**:
This is similar to https://github.com/kubernetes/kubernetes/pull/46796, but for kubeadm.
Basically it was a part of https://github.com/kubernetes/kubernetes/pull/46796, but there were some other upgradability and compability concerns for kubeadm I took care of while working today.
Example:
```console
$ kubeadm init --kubernetes-version v1.7.0-beta.0
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.7.0-beta.0
[init] Using Authorization mode: [RBAC Node]
...
$ sudo kubectl --kubeconfig=/etc/kubernetes/kubelet.conf get secret foo
Error from server (Forbidden): User "system:node:thegopher" cannot get secrets in the namespace "default".: "no path found to object" (get secrets foo)
$ echo '{"apiVersion":"v1","kind":"Node","metadata":{"name":"foo"}}' | sudo kubectl create -f - --kubeconfig=/etc/kubernetes/kubelet.conf
Error from server (Forbidden): error when creating "STDIN": nodes "foo" is forbidden: node thegopher cannot modify node foo
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Depends on https://github.com/kubernetes/kubernetes/pull/46864 (uses that PR as a base, will rebase once it's merged)
Please only review the second commit. Will also fix tests in a minute.
**Release note**:
```release-note
kubeadm: Enable the Node Authorizer/Admission plugin in v1.7
```
@mikedanese @liggitt @pipejakob @roberthbailey @jbeda @timothysc
Automatic merge from submit-queue (batch tested with PRs 47024, 47050, 47086, 47081, 47013)
kubeadm: Make the creation of the RBAC rules phase idempotent
**What this PR does / why we need it**:
Bugfix: Currently kubeadm fails with a non-zero code if resources it's trying to create already exist. This PR fixes that by making kubeadm try to Update resources that already exist.
After this PR, https://github.com/kubernetes/kubernetes/pull/46879 and a beta.1 release, kubeadm will be fully upgradeable from v1.6 to v1.7 using only kubeadm init.
Last piece of https://github.com/kubernetes/kubeadm/issues/288
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes: https://github.com/kubernetes/kubeadm/issues/288
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: Modifications to cluster-internal resources installed by kubeadm will be overwritten when upgrading from v1.6 to v1.7.
```
@pipejakob @mikedanese @timothysc
Automatic merge from submit-queue (batch tested with PRs 43005, 46660, 46385, 46991, 47103)
Consolidate sysctl commands for kubelet
**What this PR does / why we need it**:
These commands are important enough to be in the Kubelet itself.
By default, Ubuntu 14.04 and Debian Jessie have these set to 200 and
20000. Without this setting, nodes are limited in the number of
containers that they can start.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#26005
**Special notes for your reviewer**:
I had a difficult time writing tests for this. It is trivial to create a fake sysctl for testing, but the Kubelet does not have any tests for the prior settings.
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46235, 44786, 46833, 46756, 46669)
implements StatefulSet update
**What this PR does / why we need it**:
1. Implements rolling update for StatefulSets
2. Implements controller history for StatefulSets.
3. Makes StatefulSet status reporting consistent with DaemonSet and ReplicaSet.
https://github.com/kubernetes/features/issues/188
**Special notes for your reviewer**:
**Release note**:
```release-note
Implements rolling update for StatefulSets. Updates can be performed using the RollingUpdate, Paritioned, or OnDelete strategies. OnDelete implements the manual behavior from 1.6. status now tracks
replicas, readyReplicas, currentReplicas, and updatedReplicas. The semantics of replicas is now consistent with DaemonSet and ReplicaSet, and readyReplicas has the semantics that replicas did prior to this release.
```
Automatic merge from submit-queue
kubeadm: improve quickstart instructions
**What this PR does / why we need it**:
Improves instructional output following setup of a kubernetes master with kubeadm.
This helps prevent unnecessary support overhead such as: https://github.com/kubernetes/kubeadm/issues/212#issuecomment-292015402
**Example current output**:
```
To start using your cluster, you need to run (as a regular user):
sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
```
**Example new output**:
```
To start using your cluster, you need to run (as a regular user):
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46718, 46828, 46988)
Simply changed the names of packages of some admission plugins.
**What this PR does / why we need it**:
This PR simply changes the names of packages of some admission plugins so that they are not in conflict with admission pkg.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46718, 46828, 46988)
Update docs/ links to point to main site
**What this PR does / why we need it**:
This updates various links to either point to kubernetes.io or to the kubernetes/community repo instead of the legacy docs/ tree in k/k
Pre-requisite for #46813
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-docs-maintainers @chenopis @ahmetb @thockin
Automatic merge from submit-queue
Rename {kube- ->}apiextensions-{-> api}server
Completely mechanical renaming, as preparation of the repo sync. This repo is generic, the prefix does not make sense.
Part of issue https://github.com/kubernetes/kubernetes/issues/46702
TODOs
- [ ] Waiting for the submit queue to settle, then rebase.
Automatic merge from submit-queue (batch tested with PRs 46897, 46899, 46864, 46854, 46875)
kubeadm: Idempotent service account creation.
**What this PR does / why we need it**:
During `kubeadm init`, ignore errors for duplicates when creating service accounts.
https://github.com/kubernetes/kubeadm/issues/278
Fixes: https://github.com/kubernetes/kubeadm/issues/288
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46897, 46899, 46864, 46854, 46875)
kubeadm: Make kubeadm use the right CSR approver for the right version
**What this PR does / why we need it**:
fixes regression caused in: https://github.com/kubernetes/kubernetes/pull/45619
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes: https://github.com/kubernetes/kubeadm/issues/289
**Special notes for your reviewer**:
cc @pipejakob our e2e CI should probably go green after this change
**Release note**:
```release-note
NONE
```
@mikedanese @pipejakob @timothysc @liggitt
Automatic merge from submit-queue (batch tested with PRs 46787, 46876, 46621, 46907, 46819)
kubeadm: Only create bootstrap configmap if not exists.
**What this PR does / why we need it**:
The fact that this method was not idempotent was breaking kubeadm upgrades.
https://github.com/kubernetes/kubeadm/issues/278
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
kubeadm: don't duplicate master taint if it already exists.
**What this PR does / why we need it**:
This helps enable a graceful upgrade/downgrade process between 1.6.x and 1.7.x kubeadm clusters (although no guarantees outside of that range) by doing:
$ kubeadm init --kubernetes-version <version> --skip-preflight-checks
Without this change, the command fails with an error that the node taint is duplicated.
This is part of https://github.com/kubernetes/kubeadm/issues/278
**Release note**:
```release-note
NONE
```
Fixes: https://github.com/kubernetes/kubeadm/issues/288
Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)
Generate protobuf for the audit API
Add protobuf generation & annotations for the advanced auditing API. We're not using them directly now, but settling on the field numbering will help with some 3rd party integrations.
For kubernetes/features#22
/cc @ihmccreery
Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)
While deleting a namespace, the TPR instances under this ns should be…
… deleted.
While deleting a namespace, the TPR instances under this ns should be deleted.
Fixed#46736
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)
promote tls-bootstrap to beta
last commit of this PR.
Towards https://github.com/kubernetes/kubernetes/issues/46999
```release-note
Promote kubelet tls bootstrap to beta. Add a non-experimental flag to use it and deprecate the old flag.
```
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
Add configuration for encryption providers
## Additions
Allows providing a configuration file (using flag `--experimental-encryption-provider-config`) to use the existing AEAD transformer (with multiple keys) by composing mutable transformer, prefix transformer (for parsing providerId), another prefix transformer (for parsing keyId), and AES-GCM transformers (one for each key). Multiple providers can be configured using the configuration file.
Example configuration:
```
kind: EncryptionConfig
apiVersion: v1
resources:
- resources:
- namespaces
providers:
- aes:
keys:
- name: key1
secret: c2vjcmv0iglzihnly3vyzq==
- name: key2
secret: dghpcybpcybwyxnzd29yza==
- identity: {}
```
Need for configuration discussed in:
#41939
[Encryption](3418b4e4c6/contributors/design-proposals/encryption.md)
**Pathway of a read/write request**:
1. MutableTransformer
2. PrefixTransformer reads the provider-id, and passes the request further if that matches.
3. PrefixTransformer reads the key-id, and passes the request further if that matches.
4. GCMTransformer tries decrypting and authenticating the cipher text in case of reads. Similarly for writes.
## Caveats
1. To keep the command line parameter parsing independent of the individual transformer's configuration, we need to convert the configuration to an `interface{}` and manually parse it in the transformer. Suggestions on better ways to do this are welcome.
2. Flags `--encryption-provider` and `--encrypt-resource` (both mentioned in [this document](3418b4e4c6/contributors/design-proposals/encryption.md) ) are not supported in this because they do not allow more than one provider, and the current format for the configuration file possibly supersedes their functionality.
3. Currently, it can be tested by adding `--experimental-encryption-provider-config=config.yml` to `hack/local-up-cluster.sh` on line 511, and placing the above configuration in `config.yml` in the root project directory.
Previous discussion on these changes:
https://github.com/sakshamsharma/kubernetes/pull/1
@jcbsmpsn @destijl @smarterclayton
## TODO
1. Investigate if we need to store keys on disk (per [encryption.md](3418b4e4c6/contributors/design-proposals/encryption.md (option-1-simple-list-of-keys-on-disk)))
2. Look at [alpha flag conventions](https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go)
3. Need to reserve `k8s:enc` prefix formally for encrypted data. Else find a better way to detect transformed data.
Handle failure cases on startup gracefully to avoid causing cascading
errors and poor initialization in other components. Initial errors from
config load cause the initializer to pause and hold requests. Return
typed errors to better communicate failures to clients.
Add code to handle two specific cases - admin wants to bypass
initialization defaulting, and mirror pods (which want to bypass
initialization because the kubelet owns their lifecycle).
This helps enable a graceful upgrade/downgrade process between 1.6.x and
1.7.x kubeadm clusters (although no guarantees outside of that range) by
doing:
$ kubeadm init --kubernetes-version <version> --skip-preflight-checks
Without this change, the command fails with an error that the node taint
is duplicated.
This is part of https://github.com/kubernetes/kubeadm/issues/278
Automatic merge from submit-queue (batch tested with PRs 46681, 46786, 46264, 46680, 46805)
Enable Dialer on the Aggregator
Centralize the creation of the dialer during startup.
Have the dialer then passed in to both APIServer and Aggregator.
Aggregator the uses the dialer as its Transport base.
**What this PR does / why we need it**:Enables the Aggregator to use the Dialer/SSHTunneler to connect to the user-apiserver.
**Which issue this PR fixes** : fixes ##46679
**Special notes for your reviewer**:
**Release note**: None
Automatic merge from submit-queue (batch tested with PRs 40760, 46706, 46783, 46742, 46751)
complete the controller context for init funcs
This completes the conversion to initFuncs for the controller initialization to make easier and more manageable to add them.
Automatic merge from submit-queue
Implement Daemonset history
~Depends on #45867 (the 1st commit, ignore it when reviewing)~ (already merged)
Ref https://github.com/kubernetes/community/pull/527/ and https://github.com/kubernetes/community/pull/594
@kubernetes/sig-apps-api-reviews @kubernetes/sig-apps-pr-reviews @erictune @kow3ns @lukaszo @kargakis
---
TODOs:
- [x] API changes
- [x] (maybe) Remove rollback subresource if we decide to do client-side rollback
- [x] deployment controller
- [x] controller revision
- [x] owner ref (claim & adoption)
- [x] history reconstruct (put revision number, hash collision avoidance)
- [x] de-dup history and relabel pods
- [x] compare ds template with history
- [x] hash labels (put it in controller revision, pods, and maybe deployment)
- [x] clean up old history
- [x] Rename status.uniquifier when we reach consensus in #44774
- [x] e2e tests
- [x] unit tests
- [x] daemoncontroller_test.go
- [x] update_test.go
- [x] ~(maybe) storage_test.go // if we do server side rollback~
kubectl part is in #46144
---
**Release note**:
```release-note
```
