Automatic merge from submit-queue
AWS: check validity of KSM key before creating a new encrypted disk.
AWS CreateVolume call does not check if referenced encryption key actually exists and returns a valid new AWS EBS volume even though an invalid key was specified. Later on it removes the EBS silently when its encryption fails.
To work around this buggy behavior we manually check that the key exists before calling CreateVolume.
Fixes#48438
/sig aws
Please review carefully. Can we safely assume that Kubernetes controller-manager can read encryption keys?
```release-note
aws: Kubernetes now checks existence of provided KSM (Key Management Service) key before creating an encrypted AWS EBS.
```
Kubelet makes sure that /var/lib/kubelet is rshared when it starts.
If not, it bind-mounts it with rshared propagation to containers
that mount volumes to /var/lib/kubelet can benefit from mount propagation.
Automatic merge from submit-queue
e2e: Add tests for network tiers in GCE
This test depends on #51301, which adds the new feature. Only the `e2e: Add tests for network tiers in GCE` commit is new.
#51301 should pass this new test.
WaitForAttach failed consistently with this error:
Heuristic determination of mount point failed:stat /var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/10.128.0.3:3260-iqn.2003-01.org.linux-iscsi.f21.x8664:sn.4b0aae584f7c-lun-0: no such file or directory
We should ignore "no such file or directory" eror, the directory is created
just few lines below.
Automatic merge from submit-queue (batch tested with PRs 51439, 51361, 51140, 51539, 51585)
Enable alpha GCE disk API
This PR builds on top of #50467 to allow the GCE disk API to use either the alpha or stable APIs.
CC @freehan
Automatic merge from submit-queue (batch tested with PRs 51439, 51361, 51140, 51539, 51585)
Iscsi handle failed mount
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes#50556
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
`golint` emits various warnings about missing comments for exported
funcitons and types.
This PR adds missing documentation strings to functions and types. Also
adds punctuation to current documentation strings in line with Go coding
standards.
Automatic merge from submit-queue
Add storageClass.mountOptions and use it in all applicable plugins
split off from https://github.com/kubernetes/kubernetes/pull/50919 and still dependent on it. cc @gnufied
issue: https://github.com/kubernetes/features/issues/168
```release-note
Add mount options field to StorageClass. The options listed there are automatically added to PVs provisioned using the class.
```
For pod volumes that reference a PVC, add a PVCRef to the corresponding
volume stat. This allows metrics to be indexed/queried by PVC name
which is more user-friendly than Pod reference
Automatic merge from submit-queue (batch tested with PRs 51377, 46580, 50998, 51466, 49749)
feat(#21648 )Add kubectl set env command.
**What this PR does / why we need it**:
#21648
Moved from OpenShift to Kubenetes.
@kargakis @smarterclayton
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51377, 46580, 50998, 51466, 49749)
Implement IPVS-based in-cluster service load balancing
**What this PR does / why we need it**:
Implement IPVS-based in-cluster service load balancing. It can provide some performance enhancement and some other benefits to kube-proxy while comparing iptables and userspace mode. Besides, it also support more sophisticated load balancing algorithms than iptables (least conns, weighted, hash and so on).
**Which issue this PR fixes**
#17470#44063
**Special notes for your reviewer**:
* Since the PR is a bit large, I splitted it and move the commits related to ipvs util pkg to PR #48994. Hopefully can make it easier to review.
@thockin @quinton-hoole @kevin-wangzefeng @deepak-vij @haibinxie @dhilipkumars @fisherxu
**Release note**:
```release-note
Implement IPVS-based in-cluster service load balancing
```
Automatic merge from submit-queue
Implement stop function in streaming server.
Implement streaming server stop, so that we could properly stop streaming server.
We need this to properly stop cri-containerd.
Automatic merge from submit-queue (batch tested with PRs 49961, 50005, 50738, 51045, 49927)
adding validations on kubelet starting configurations
**What this PR does / why we need it**:
I found some validations of kubelet starting options were missing when I was creating a custom cluster from scratch. The kubelet does not check invalid configurations on `--cadvisor-port`, `--event-burst`, `--image-gc-high-threshold`, etc. I have added some validations in kubelet like validations in `cmd/kube-apiserver/app/options/validation.go`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Adds additional validation for kubelet in `pkg/kubelet/apis/kubeletconfig/validation`.
```
Automatic merge from submit-queue (batch tested with PRs 49961, 50005, 50738, 51045, 49927)
Add --append-hash flag to kubectl create configmap/secret
**What this PR does / why we need it**:
Specifying this new flag will automatically hash the configmap/secret
contents with sha256 and append the first 40 hex-encoded bits of the
hash to the name of the configmap/secret. This is especially useful for
workflows that generate configmaps/secrets from files (e.g.
--from-file).
See this Google doc for more background:
https://docs.google.com/document/d/1x1fJ3pGRx20ujR-Y89HUAw8glUL8-ygaztLkkmQeCdU/edit
**Release note**:
```release-note
Adds --append-hash flag to kubectl create configmap/secret, which will append a short hash of the configmap/secret contents to the name during creation.
```
Automatic merge from submit-queue
Add Google cloud KMS service for envelope encryption transformer
This adds the required pieces which will allow addition of KMS based encryption providers (envelope transformer).
For now, we will be implementing it using Google Cloud KMS, but the code should make it easy to add support for any other such provider which can expose Decrypt and Encrypt calls.
Writing tests for Google Cloud KMS Service may cause a significant overhead to the testing framework. It has been tested locally and on GKE though.
Upcoming after this PR:
* Complete implementation of the envelope transformer, which uses LRU cache to maintain decrypted DEKs in memory.
* Track key version to assist in data re-encryption after a KEK rotation.
Development branch containing the changes described above: https://github.com/sakshamsharma/kubernetes/pull/4
Envelope transformer used by this PR was merged in #49350
Concerns #48522
Planned configuration:
```
kind: EncryptionConfig
apiVersion: v1
resources:
- resources:
- secrets
providers:
- kms:
cachesize: 100
configfile: gcp-cloudkms.conf
name: gcp-cloudkms
- identity: {}
```
gcp-cloudkms.conf:
```
[GoogleCloudKMS]
kms-location: global
kms-keyring: google-container-engine
kms-cryptokey: example-key
```
Automatic merge from submit-queue (batch tested with PRs 51298, 51510, 51511)
GCE: Add a fake forwarding rule service
Also add more methods to the address service. These
will be used for testing soon.
Automatic merge from submit-queue (batch tested with PRs 44719, 48454)
check job ActiveDeadlineSeconds
**What this PR does / why we need it**:
enqueue a sync task after ActiveDeadlineSeconds
**Which issue this PR fixes** *:
fixes#32149
**Special notes for your reviewer**:
**Release note**:
```release-note
enqueue a sync task to wake up jobcontroller to check job ActiveDeadlineSeconds in time
```
Automatic merge from submit-queue (batch tested with PRs 44719, 48454)
Fix handling of APIserver errors when saving provisioned PVs.
When API server crashes *after* saving a provisioned PV and before sending
200 OK, the controller tries to save the PV again. In this case, it gets
AlreadyExists error, which should be interpreted as success and not as error.
Especially, a volume that corresponds to the PV should not be deleted in the
underlying storage.
Fixes#44372
```release-note
NONE
```
@kubernetes/sig-storage-pr-reviews
When a node IP is set and a cloud provider returns the same address with
several types, on the first address was accepted. With the changes made
in PR #45201, the vSphere cloud provider returned the ExternalIP first,
which led to a node without any InternalIP.
The behaviour is modified to return all the address types for the
specified node IP.
Issue #48760
