Automatic merge from submit-queue (batch tested with PRs 57868, 58284, 56370, 58400, 58439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor service controller to common controller pattern
**What this PR does / why we need it**:
The service controller currently uses a non-standard controller pattern that adds unneeded complexity. This PR moves the service controller to use common tools like the rate limited queue with exponential backoff versus tracking retry backoff in the cachedService object.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#55528
**Special notes for your reviewer**:
This does change the retry backoff from a linear backoff to an exponential backoff. The min and max retry delay stays the same.
There was only one case where we didn't want to retry, which was when we updated the service status since that handled retries itself. In that case I moved to the common pattern of using `runtime.HandleError`.
**Release note**:
```release-note
NONE
```
/cc @luxas @wlan0 @andrewsykim @thockin
@prydie
/sig cluster-lifecycle
/sig networking
/kind cleanup
Automatic merge from submit-queue (batch tested with PRs 57868, 58284, 56370, 58400, 58439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Show findmnt command output in case of error
**What this PR does / why we need it**:
We don't print the error output from the `findmnt` command, which makes it pretty hard to debug.
The current implementation just print:
`
I0115 11:59:36.078285 18230 nsenter_mount.go:168] Failed findmnt command for path /var/lib/origin/openshift.local.volumes/pods/8d723ec8-f9e9-11e7-bc52-025000000001/volumes/kubernetes.io~secret/pvinstaller-token-bxh42: exit status 1
`
With this patch, we can actually see the error:
`
I0115 12:04:30.908164 23301 nsenter_mount.go:168] Failed findmnt command for path /var/lib/origin/openshift.local.volumes/pods/8ff8bb98-f9e9-11e7-bc52-025000000001/volumes/kubernetes.io~secret/deployer-token-g4sg9: nsenter: failed to execute /findmnt: No such file or directory
exit status 1
`
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix ipvs proxy mode kubeadm usage
**What this PR does / why we need it**:
Fix ipvs proxy mode usage of kubeadm in ipvs README file.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add FSType for CSI volume source
**What this PR does / why we need it**:
Add FSType for CSI volume source to specify filesystems (alpha defaults to `ext4`)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58183
**Special notes for your reviewer**:
**Release note**:
```release-note
Add FSType for CSI volume source to specify filesystems
```
/assign @saad-ali
cc @vladimirvivien
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
openstack: remove orphaned routes from terminated instances
**What this PR does / why we need it**:
At the moment the openstack cloudprovider only returns routes where the `NextHop` address points to an existing openstack instance. This is a problem when an instance is terminated before the corresponding node is removed from k8s. The existing route is not returned by the cloudprovider anymore and therefore never considered for deletion by the route controller. When the route's `DestinationCIDR` is reassigned to a new node the router ends up with two routes pointing to a different `NextHop` leading to broken networking.
This PR removes skipping routes pointing to unknown next hops when listing routes. This should cause [this conditional](93dc3763b0/pkg/controller/route/route_controller.go (L208)) in the route controller to succeed and have the route removed if the route controller [feels responsible](93dc3763b0/pkg/controller/route/route_controller.go (L206)).
```release-note
OpenStack cloudprovider: Ensure orphaned routes are removed.
```
When we convert the OpenStack cloud provider to run in an external
process, we should be able to use kubernetes Secrets capability to
inject the OS_* variables. This way we can specify the cloud
configuration as a configmap, specify secrets for the userid/password
information. The configmap can be mounted as a file. the secrets can
be made available as environment variables. the external controller
itself can run as a pod/daemonset.
For backward compat, we preload all the OS_* variables, if anything
is in the config file, then that overrides the environment variables.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Rework method of updating atomic-updated data volumes
**What this PR does / why we need it**:
This change affects the way that secret, configmap, downwardAPI and projected volumes (which all use the same underlying code) implement their data update functionality.
* Instead of creating a subdirectory hierarchy that will contain symlinks to each actual data file, create only symlinks to items in the root of the volume, whether they be files or directories.
* Rather than comparing the user-visible data directory to see if an update is needed, compare with the current version of the data directory.
* Fix data dir timestamp format year
* Create `..data` symlink even when a data volume has no data so consumers can have simplified update watch logic.
**Which issue(s) this PR fixes**:
Fixes#57421
**Release note**:
```release-note
Correct issues that arise when volumes are mounted beneath another secret, configmap, downwardAPI or projected volume
```
Automatic merge from submit-queue (batch tested with PRs 53631, 56960). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove unused code in UT files in pkg/
**What this PR does / why we need it**:
Remove unused code in UT files in pkg/ .
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update vendor spf13/cobra to enforce required flags
**What this PR does / why we need it**:
spf13/cobra#502 has enforced checking flags that marked as required, an error will be raised if unset.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:fixes #54855
xref #48400fixeskubernetes/kubectl#121
**Special notes for your reviewer**:
/assign @liggitt @eparis
**Release note**:
```release-note
kubectl now enforces required flags at a more fundamental level
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support for custom tls cipher suites in api server and kubelet
**What this PR does / why we need it**:
This pull request aims to solve the problem of users not able to set custom cipher suites in the api server.
Several users have requested this given that some default ciphers are vulnerable.
There is a discussion in #41038 of how to implement this. The options are:
- Setting a fixed list of ciphers, but users will have different requirements so a fixed list would be problematic.
- Letting the user set them by parameter, this requires adding a new parameter that could be pretty long with the list of all the ciphers.
I implemented the second option, if the ciphers are not passed by parameter, the Go default ones will be used (same behavior as now).
**Which issue this PR fixes**
fixes#41038
**Special notes for your reviewer**:
The ciphers in Go tls config are constants and the ones passed by parameters are a comma-separated list. I needed to create the `type CipherSuitesFlag` to support that conversion/mapping, because i couldn't find any way to do this type of reflection in Go.
If you think there is another way to implement this, let me know.
If you want to test it out, this is a ciphers combination i tested without the weak ones:
```
TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
```
If this is merged i will implement the same for the Kubelet.
**Release note**:
```release-note
kube-apiserver and kubelet now support customizing TLS ciphers via a `--tls-cipher-suites` flag
```
Automatic merge from submit-queue (batch tested with PRs 58411, 58407, 52863). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
low hanging fruit for using cobra commands
This makes the simple updates to use cobra commands instead of individual ones
/assign liggitt
/assign ncdc
/assign sttts
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixed TearDown of NFS with root squash.
NFS plugin should not use `IsLikelyNotMountPoint()`, as it uses `lstat()` / `stat()` to determine if the NFS volume is still mounted - NFS server may use root_squash and kubelet may not be allowed to do `lstat()` / `stat()` there.
It must use slower `IsNotMountPoint()` instead, including in `TearDown()` function.
**Release note**:
```release-note
NONE
```
/assign @gnufied @rootfs
Automatic merge from submit-queue (batch tested with PRs 58375, 58306, 58386, 57420, 58035). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix windows UT for proxy mode
**What this PR does / why we need it**:
/kind bug
Fix windows UT for proxy mode.
Run UTs for `kubeproxyconfig/validation` in windows,
```
=== RUN TestValidateKubeProxyConfiguration
--- FAIL: TestValidateKubeProxyConfiguration (0.00s)
validation_test.go:83: expected success: [KubeProxyConfiguration.Mode.ProxyMode: Invalid value: "ipvs": must be kernelspace,userspace or blank (blank means the most-available proxy [currently userspace])]
```
That's because proxy mode "IPVS" is not valid in windows.
**Which issue(s) this PR fixes**:
Fixes#58037
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58375, 58306, 58386, 57420, 58035). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Track run status explicitly rather than non-nil check on stopCh
Fixes#57044
GC and quota controllers use a non-nil stop channel as a signal Run() has been called, so ensure that condition holds even when a nil stop channel is passed in
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add jsafrane as util/mount approver.
I've written large parts of the mounter and I am fixing it regularly.
**Release note**:
```release-note
NONE
```
This change affects the way that secret, configmap, downwardAPI and projected
volumes (which all use the same underlying code) implement their data update
functionality.
* Instead of creating a subdirectory hierarchy that itself
will contain symlinks to each actual data file, create only
symlinks to items in the root of the volume, whether they
be files or directories.
* Rather than comparing the user-visible data directory
to see if an update is needed, compare with the current
version of the data directory.
* Fix data dir timestamp format year
* Create ..data symlink even when a data volume has no data so
consumers can have simplified update watch logic.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
pkg/securitycontext/util_test.go(TestAddNoNewPrivileges): update tests
**What this PR does / why we need it**:
This PR improves existing test in the following ways:
- remove irrelevant test cases
- add test case for `AllowPrivilegeEscalation: nil`
- explicitly specify input and expected outcome
This is addressed to the following review comment: https://github.com/kubernetes/kubernetes/pull/47019#discussion_r135808264
**Release note**:
```release-note
NONE
```
PTAL @jessfraz @kubernetes/sig-auth-pr-reviews
CC @simo5
This PR add comments for the background why plugin gets loopback
device and removes loopback device even if operation_generator has
same functionality.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
set fsGroup by securityContext.fsGroup in azure file
**What this PR does / why we need it**:
set fsGroup by securityContext.fsGroup in azure file
This PR respects `securityContext.fsGroup`, while if user both sets `gid=xxx` in mountOptions in azure storage class and `securityContext.fsGroup`, `gid=xxx` setting in mountOptions will win.
```
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
fsGroup: 2000
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: gcr.io/google-samples/node-hello:1.0
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#44085
**Special notes for your reviewer**:
**Release note**:
```
set fsGroup by securityContext.fsGroup in azure file
```
/assign @karataliu
cc @rootfs @avaranovich
Automatic merge from submit-queue (batch tested with PRs 58319, 58345). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
azure disk: if the disk is not found, immediately detach it. This prevents azure keeps the bad request and stops issuing new request
**What this PR does / why we need it**:
Detach and clear bad disk URI
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58344
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58319, 58345). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Call Dial in blocking mode
**What this PR does / why we need it**:
# Tests which uncover underlying problem
On the current master code:
1. comment out stub plugin server start [here](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/cm/deviceplugin/endpoint_test.go#L127-L128)
2. run `go test -v k8s.io/kubernetes/pkg/kubelet/cm/deviceplugin --run TestNewEndpoint --count 1`. Test succeeds.
3. run `go test -v k8s.io/kubernetes/pkg/kubelet/cm/deviceplugin --run TestRun --count 1`. Test hangs with:
> listAndWatch ended unexpectedly for device plugin mock with error rpc error: code = Unavailable desc = grpc: the connection is unavailable
`2` does not fail even though it invokes Dial and no listening server is running. It is because currently Dial is not waiting till the connection turns to Ready state. And this case does not invokes any RPC call over the `conn` returned by `Dial`
`3` hangs because this tests involves actual usage of `conn`(client) and there it does not find listening server because we dint start and deliberately stopped in `1`.
`Dial` should be using `WithBlock` option which ensures that `conn` is returned only when connection is in `Ready` state.
After using `WithBlock` in the `Dial`, in this PR, if `1`, `2` and `3` are repeated. Both, `2` and `3`, fails at `Dial`, which is expected behavior. By `fail`, I meant Dial blocks forever if `WithTimeout` is not used or otherwise, a timeout failure.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58281
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
/area hw-accelerators
/sig node
/cc @jiayingz @RenaudWasTaken @vishh @ScorpioCPH @sjenning @derekwaynecarr @jeremyeder @lichuqiang @tengqm
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fail earlier on discovery failures
When discovery fails, the object mapper and typer are incomplete. This change reflects that and returns errors instead of nils.
@juanvallejo
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[quota controller] remove extra queue.Add()
requeue immediately after an error may end-up with hot-loop
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add hostPorts to pod describer
**What this PR does / why we need it**:
Missing `HostPorts` when describing pods
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/assign @mengqiy @shiywang
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
feat(fakeclient): push event on watched channel on add/update/delete
**What this PR does / why we need it**:
This PR enables watch function for kubernetes [fakeclient](1bcf0b0a22/staging/src/k8s.io/client-go/kubernetes/fake/clientset_generated.go (L88)).
This fake client add watchReactorFunction by wrapping [watch.NewFake](1bcf0b0a22/staging/src/k8s.io/client-go/kubernetes/fake/clientset_generated.go (L98)) which is a `chan Event` but actually nothing pushes objects into this channel. So all watch function called by fake client will never return or never receive any object.
This PR intercepts ReactionFunc of `Create / Update / DeleteActionImpl` and will push the requested object to channel.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes#54075
**Special notes for your reviewer**:
**Release note**:
```dev-release-note
enable watch function for fake client
```
Automatic merge from submit-queue (batch tested with PRs 58260, 58326). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[azure] fix TestGetInstanceIDByNodeName data race
**What this PR does / why we need it**:
fix ut case data race
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#58325
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
