Automatic merge from submit-queue (batch tested with PRs 67396, 67097, 67395, 67365, 67099). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Ignore EIO error in unmount path
**What this PR does / why we need it**:
This PR ignores EIO in unmount path. XFS shuts down filesystem when the target is down and it returns EIO for the stat calls used in unmount path.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66868
**Special notes for your reviewer**:
We already handle ESTALE & ENOTCONN errors in isCorruptedMnt Call. Adding EIO to that list covers the XFS shutdown case.
Also Flexvolume doesn't check for these errors in its current form. Updated Flexvolume code to handle it.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66177, 66185, 67136, 67157, 65065). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: reduce logging for backoff situations
xref https://bugzilla.redhat.com/show_bug.cgi?id=1555057#c6
Pods that are in `ImagePullBackOff` or `CrashLoopBackOff` currently generate a lot of logging at the `glog.Info()` level. This PR moves some of that logging to `V(3)` and avoids logging in situations where the `SyncPod` only fails because pod are in a BackOff error condition.
@derekwaynecarr @liggitt
Automatic merge from submit-queue (batch tested with PRs 67071, 66906, 66722, 67276, 67039). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix for duplicate revisions created by StatefulSet
**What this PR does / why we need it**: This PR replaces PR #65038 as a fix to issue #55159. The statefulset controller can in some situations create more controller revisions than necessary and this change makes sure the controller checks with the API server and only create new revision if the raw data is different.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55159
**Special notes for your reviewer**:
**Release note**:
```release-note
Avoid creating new controller revisions for statefulsets when cache is stale
```
Automatic merge from submit-queue (batch tested with PRs 67071, 66906, 66722, 67276, 67039). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move dependency within kubernetes core to staging
**What this PR does / why we need it**:
This PR moves a dependency from with Kubernetes core (k8s.io/kubernetes/pkg/apis/core) to staging
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Helps Fix: Remove Kubectl dependencies on kubernetes/pkg/api and kubernetes/pkg/apis
https://github.com/kubernetes/kubectl/issues/83
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 67071, 66906, 66722, 67276, 67039). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
#50102 Task 1: Move apimachinery/pkg/watch.Until into client-go/tools/watch.UntilWithoutRetry
**What this PR does / why we need it**:
This is a split off from https://github.com/kubernetes/kubernetes/pull/50102 to go in smaller pieces.
Moves `apimachinery/pkg/watch.Until` into `client-go/tools/watch.UntilWithoutRetry` and adds context so it is cancelable.
**Release note**:
```release-note
NONE
```
**Dev release note**:
```dev-release-note
`apimachinery/pkg/watch.Until` has been moved to `client-go/tools/watch.UntilWithoutRetry`.
While switching please consider using the new `client-go/tools/watch.UntilWithSync` or `client-go/tools/watch.Until`.
```
/cc @smarterclayton @kubernetes/sig-api-machinery-pr-reviews
/milestone v1.12
/priority important-soon
/kind bug
(bug after the main PR which is this split from)
Automatic merge from submit-queue (batch tested with PRs 66491, 66587, 66856, 66657, 66923). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
PVC Protection: Wait for Pod delete
Currently, the PVC protection controller will remove its finalizer when
all Pods using a PVC reach at least a Terminating state. However,
certain volumes cannot be guaranteed to be umounted until a Pod is
deleted. Only Pods not in the current pods list can be considered
deleted, so we're removing the exception to not check Terminating Pods.
```release-note
NONE
```
Resolves: #65552
Signed-off-by: Jose A. Rivera <jarrpa@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 66491, 66587, 66856, 66657, 66923). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
add space for output
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66491, 66587, 66856, 66657, 66923). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert #63905: Setup dns servers and search domains for Windows Pods
**What this PR does / why we need it**:
From https://github.com/kubernetes/kubernetes/pull/63905#issuecomment-396709775:
> I don't think this change does anything on Windows. On windows, the network endpoint configuration is taken care of completely by CNI. If you would like to pass on the custom dns polices from the pod spec, it should be dynamically going to the cni configuration that gets passed to CNI. From there, it would be passed down to platform and would be taken care of appropriately by HNS.
> etc\resolve.conf is very specific to linux and that should remain linux speicfic implementation. We should be trying to move away from platform specific code in Kubelet.
Docker is not managing the networking here for windows. So it doens't really care about any network settings. So passing it to docker shim's hostconfig also doens;t make sense here.
DNS for Windows containers will be set by CNI plugins. And this change also introduced two endpoints for sandbox container. So this PR reverts #63905 .
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
The PR should also be cherry-picked to release-1.11.
Also, https://github.com/kubernetes/kubernetes/issues/66588 is opened to track the process of pushing this to CNI.
**Release note**:
```release-note
Revert #63905: Setup dns servers and search domains for Windows Pods. DNS for Windows containers will be set by CNI plugins.
```
/sig windows
/sig node
/kind bug
Automatic merge from submit-queue (batch tested with PRs 66491, 66587, 66856, 66657, 66923). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
optimize ipvs get nodeIP
**What this PR does / why we need it**:
Optimize ipvs get nodeIP.
The original ipvs `NodeIPs` need first get all local type address to set1, then get address of dummy device `kube-ipvs0` to set2, then do diff of set1 and set2 to get local addresses we need.
This work gonna result in unnecessary resource consumption, especially for large cluster, will have lots address in dummy device `kube-ipvs0`.
This pr optimized the workaround.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
attachdetach controller: attach volumes immediately when Pod's PVCs are bound
**What this PR does / why we need it**:
Let attachdetach controller to attach volumes immediately when Pod's PVCs are bound.
Current attachdetach controller calls `util.ProcessPodVolume` to add pod volumes into `desiredStateOfWorld` on these events:
- podAdd event
- podUpdate event
- podDelete event
- periodical `desiredStateOfWorldPopulator.findAndAddActivePod`
But if a pod is created with PVCs not bound, no volumes will be added into `desiredStateOfWorld` [because PVCs not bound](https://github.com/kubernetes/kubernetes/blob/v1.12.0-alpha.0/pkg/controller/volume/attachdetach/util/util.go#L99). When pv controller binds PVCs successfully, attachdetach controller will not add pod volumes immediately because it does not watch on PVC events.
It will wait until a pod update event is triggered (normally will not happen because no new status will be reported by kubelet) or `desiredStateOfWorldPopulator.findAndAddActivePod` is called (maybe 0~3 minutes later, see [timer configs](https://github.com/kubernetes/kubernetes/blob/v1.12.0-alpha.0/pkg/controller/volume/attachdetach/attach_detach_controller.go)).
In bad case, pod start time will be very long (~3 minutes + ~2 minutes (kubelet max exponential backoff)), for example: https://github.com/kubernetes/kubernetes/issues/64549#issuecomment-409440546.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64549
**Special notes for your reviewer**:
**Release note**:
```release-note
attachdetach controller attaches volumes immediately when Pod's PVCs are bound
```
Automatic merge from submit-queue (batch tested with PRs 66780, 67330). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Changed admission controller to allow volume expansion for all volume plugins
**What this PR does / why we need it**:
There are two motivations for this change:
1. CSI plugins are soon going to support volume expansion. For such plugins, admission controller doesn't know whether the plugins are capabale of supporting volume expansion or not.
2. Currently, admission controller rejects PVC updates for in-tree plugins that don't support volume expansion (e.g., NFS, iSCSI). This change allows external controllers to expand volumes similar to how external provisioners are accommodated.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
This PR mimics the behavior of the PV controller when PVs are provisioned externally by logging and setting a new event for PVs that are being expanded externally. As SIG Storage is planning new types of operations on PVs, it may make more sense to a have a single event for all actions taken by external controllers.
**Release note**:
```release-note
The check for unsupported plugins during volume resize has been moved from the admission controller to the two controllers that handle volume resize.
```
/sig storage
/assign @gnufied @jsafrane @wongma7
Currently, the PVC protection controller will remove its finalizer when
all Pods using a PVC reach at least a Terminating state. However,
certain volumes cannot be guaranteed to be umounted until a Pod is
deleted. Only Pods not in the current pods list can be considered
deleted, so we're removing the exception to not check Terminating Pods.
Signed-off-by: Jose A. Rivera <jarrpa@redhat.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
vsphere: enable TestInvalidCaCert
This test can be enabled now with the govmomi vendor update in 5c44fd871f
**What this PR does / why we need it**:
Enables a test that had been skipped due to vendor dependency.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
ipvs: add addrtype match for nodeport
**What this PR does / why we need it**:
before this PR:
```
-A KUBE-SERVICES -m comment --comment "Kubernetes nodeport TCP port for masquerade purpose" -m set --match-set KUBE-NODE-PORT-TCP dst -j KUBE-NODE-PORT
-A KUBE-SERVICES -m comment --comment "Kubernetes service cluster ip + port for masquerade purpose" -m set --match-set KUBE-CLUSTER-IP dst,dst -j KUBE-MARK-MASQ
-A KUBE-SERVICES -m set --match-set KUBE-CLUSTER-IP dst,dst -j ACCEPT
-A KUBE-NODE-PORT -p tcp -m comment --comment "Kubernetes nodeport TCP port with externalTrafficPolicy=local" -m set --match-set KUBE-NODE-PORT-LOCAL-TCP dst -j RETURN
-A KUBE-NODE-PORT -j KUBE-MARK-MASQ
```
after this PR:
```
-A KUBE-NODE-PORT -p tcp -m comment --comment "Kubernetes nodeport TCP port with externalTrafficPolicy=local" -m set --match-set KUBE-NODE-PORT-LOCAL-TCP dst -j RETURN
-A KUBE-NODE-PORT -p tcp -m comment --comment "Kubernetes nodeport TCP port for masquerade purpose" -m set --match-set KUBE-NODE-PORT-TCP dst -j KUBE-MARK-MASQ
-A KUBE-SERVICES -m comment --comment "Kubernetes service cluster ip + port for masquerade purpose" -m set --match-set KUBE-CLUSTER-IP dst,dst -j KUBE-MARK-MASQ
-A KUBE-SERVICES -m set --match-set KUBE-CLUSTER-IP dst,dst -j ACCEPT
-A KUBE-SERVICES -m addrtype --dst-type LOCAL -j KUBE-NODE-PORT
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65459
**Special notes for your reviewer**:
manually tested cases:
- ClusterIP distributed to pod on same node
- ClusterIP distributed to pod on other node
- NodePort distributed to pod on same node
- NodePort distributed to pod on other node
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 67274, 67285). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: plumb context for log requests
This allows kubelets to stop the necessary work when the context has
been canceled (e.g., connection closed), and not leaking a goroutine
and inotify watcher waiting indefinitely.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Partially fixes#64315
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix kubelet to not leak goroutines/intofiy watchers on an inactive connection if it's closed
```
Automatic merge from submit-queue (batch tested with PRs 67274, 67285). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move dependency from k/k/pkg/apis/core to staging
**What this PR does / why we need it**:
This PR moves a dependency from with Kubernetes core (k8s.io/kubernetes/pkg/apis/core) to staging
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Helps Fix: Remove Kubectl dependencies on kubernetes/pkg/api and kubernetes/pkg/apis
https://github.com/kubernetes/kubectl/issues/83
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66984, 67236, 67216, 62721, 67106). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix azure disk create failure due to sdk upgrade
**What this PR does / why we need it**:
fix azure storage account creation failure, this happens only on unmanaged k8s cluster on Azure.
This bug is due to azure-sdk-for-go API change introduced in v1.11:
fbe7db0e3f/services/storage/mgmt/2017-10-01/storage/models.go (L381-L382)
there is a new field `Kind` which is required, so any sdk upgrade from and old version would break the storage account creation since old code won't use `Kind`. I have filed an issue to azure-sdk-for-go: https://github.com/Azure/azure-sdk-for-go/issues/2182
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#67234
**Special notes for your reviewer**:
**Release note**:
```
fix azure disk create failure due to sdk upgrade
```
/kind bug
/sig azure
/assign @feiskyer
FYI @khenidak @brendandburns
This allows kubelets to stop the necessary work when the context has
been canceled (e.g., connection closed), and not leaking a goroutine
and inotify watcher waiting indefinitely.
Automatic merge from submit-queue (batch tested with PRs 67017, 67190, 67110, 67140, 66873). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add wait loop for multipath devices to appear
It takes a variable amount of time for the multipath daemon
to create /dev/dm-XX in response to new LUNs being discovered.
The old iscsi_util code only discovered the multipath device
if it was created quickly enough, but in a significant number
of cases, kubelet would grab one of the individual paths and
put a filesystem it on before multipathd could construct a
multipath device.
This change waits for the multipath device to get created for
up to 10 seconds, but only if the PV actually had more than
one portal.
fixes#60894
```release-note
Dynamic provisions that create iSCSI PVs can ensure that multipath is used by specifying 2 or more target portals in the PV, which will cause kubelet to wait up to 10 seconds for the multipath device. PVs with just one portal continue to work as before, with kubelet not waiting for the multipath device and just using the first disk it finds.
```
Automatic merge from submit-queue (batch tested with PRs 67017, 67190, 67110, 67140, 66873). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CSI plugin now calls NodeGetInfo() to get driver's node ID
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#67040
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig storage
@sbezverk @vladimirvivien @saad-ali
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use OWNERS-based labeling for kind/api-change
**What this PR does / why we need it**:
This replaces the following path-label munger config, except
we're using kind/api-change for everything instead of two
different kind/ labels
```
^pkg/api/([^/]+/)?types.go$ kind/api-change
^pkg/api/([^/]+/)?register.go$ kind/new-api
^pkg/apis/[^/]+/([^/]+/)?types.go$ kind/api-change
^pkg/apis/[^/]+/([^/]+/)?register.go$ kind/new-api
```
This fixes https://github.com/kubernetes/test-infra/issues/9007
**Special notes for your reviewer**:
This can be merged independently of us turning path-label off of
mungegithub. While both labeling options are active, they won't
clash, just race.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66602, 67178, 67207, 67125, 66332). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add UT to RBD volume test of TestGetAccessModes and TestRequiresRemount
**What this PR does / why we need it**:
Add UT to RBD volume test of TestGetAccessModes and TestRequiresRemount
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 66602, 67178, 67207, 67125, 66332). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubectl: recreating resources for immutable fields when force is applied
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#66390
**Special notes for your reviewer**:
/assign soltysh juanvallejo
/cc @kubernetes/sig-cli-bugs
**Release note**:
```release-note
kubectl: recreating resources for immutable fields when force is applied
```
This replaces the following path-label munger config, except
we're using kind/api-change for everything instead of two
different kind/ labels
```
^pkg/api/([^/]+/)?types.go$ kind/api-change
^pkg/api/([^/]+/)?register.go$ kind/new-api
^pkg/apis/[^/]+/([^/]+/)?types.go$ kind/api-change
^pkg/apis/[^/]+/([^/]+/)?register.go$ kind/new-api
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move validation dependency from pkg/apis/core/validation to staging
**What this PR does / why we need it**:
Moves a dependency in metricsutil from `pkg/apis/core/validation` to the staging dependency.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Helps Fix: **Remove Kubectl dependencies on kubernetes/pkg/api and kubernetes/pkg/apis**
https://github.com/kubernetes/kubectl/issues/83
**Special notes for your reviewer**:
```release-note
NONE
```
