Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
dockershim/network: add dcbw to OWNERS as an approver
I've been involved with the kubelet network code, including most
of this code, for a couple years and contributed a good number
of PRs for these directories. I've also been a SIG Network
co-lead for couple years.
I've also been on the CNI maintainers team for a couple years.
```release-note
NONE
```
@freehan @thockin @kubernetes/sig-network-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 64974, 65009, 65018). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
daemon: add custom node indexer
**What this PR does / why we need it**:
<img width="863" alt="screen shot 2018-06-11 at 20 54 03" src="https://user-images.githubusercontent.com/44136/41279030-ad842020-6e2b-11e8-80d4-0a71ee415d30.png">
Based on this CPU profile, it looks like a lot of CPU cycles/cores are spend by retrieving a list of **all** pods in the cluster. On large clusters with multiple daemonset this might lead to locking the shared pod informer List() of every other controller that might need it or use it.
The indexer in the PR will index the pods based on nodeName assigned for these pods. That means the amount of pods returned from the ByIndex() function is fairly small and the call should be fast.
Additionally we can also use this index to check whether a node already run the pod without listing all pods in the cluster again.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert #64189: Fix Windows CNI for the sandbox case
**What this PR does / why we need it**:
This reverts PR #64189, which breaks DNS for Windows containers.
Refer https://github.com/kubernetes/kubernetes/pull/64189#issuecomment-395248704
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64861
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc @madhanrm @PatrickLang @alinbalutoiu @dineshgovindasamy
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding scale error retries
**What this PR does / why we need it**:
ScaleWithRetries will retry all retryable errors, not only conflict error.
ref #63030
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
volume: decrease memory allocations for debugging messages
**What this PR does / why we need it**:
<img width="1769" alt="screen shot 2018-06-11 at 13 15 31" src="https://user-images.githubusercontent.com/44136/41230128-ebf7233c-6d7e-11e8-899d-6251a5fde236.png">
On large clusters, where the glog is not running on V(5) using the format as: `glog.V(5).Infof(fmt.Sprintf(....))` will cause the code inside `Infof()` to be ran and generate a tons of memory allocations even if the output of those messages are not returned to the console...
This patch should reduce those calls and also the string allocations done by message generation.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64503, 64903, 64643, 64987). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use unix.EpollWait to determine when memcg events are available to be Read
**What this PR does / why we need it**:
This fixes a file descriptor leak introduced in https://github.com/kubernetes/kubernetes/pull/60531 when the `--experimental-kernel-memcg-notification` kubelet flag is enabled. The root of the issue is that `unix.Read` blocks indefinitely when reading from an event file descriptor and there is nothing to read. Since we refresh the memcg notifications, these reads accumulate until the memcg threshold is crossed, at which time all reads complete. However, if the node never comes under memory pressure, the node can run out of file descriptors.
This PR changes the eviction manager to use `unix.EpollWait` to wait, with a 10 second timeout, for events to be available on the eventfd. We only read from the eventfd when there is an event available to be read, preventing an accumulation of `unix.Read` threads, and allowing the event file descriptors to be reclaimed by the kernel.
This PR also breaks the creation, and updating of the memcg threshold into separate portions, and performs creation before starting the periodic synchronize calls. It also moves the logic of configuring memory thresholds into memory_threshold_notifier into a separate file.
This also reverts https://github.com/kubernetes/kubernetes/pull/64582, as the underlying leak that caused us to disable it for testing is fixed here.
Fixes#62808
**Release note**:
```release-note
NONE
```
/sig node
/kind bug
/priority critical-urgent
Automatic merge from submit-queue (batch tested with PRs 64503, 64903, 64643, 64987). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix a bug of wrong parameters which could cause token projection failure
**What this PR does / why we need it**:
a parameter wrong order bug
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
improve memory footprint of daemonset simulate
**What this PR does / why we need it**:
This is an alternative for https://github.com/kubernetes/kubernetes/pull/64915 (it might be not needed if that PR will merge)
During memory profiling of OpenShift, we noticed a significant amount of object allocations done by `IsControlledBy()`. @sttts found that the `GetObjectReferences()` method is doing a deep-copy of the object references.
This PR simplify the `IsControlledBy()` to just iterate over the ownerRefs, without copying them.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix bug excludeCIDRs was not assign in func NewProxier
**What this PR does / why we need it**:
fix bug excludeCIDRs was not assign in func NewProxier
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
I've been involved with the kubelet network code, including most
of this code, for a couple years and contributed a good number
of PRs for these directories. I've also been a SIG Network
co-lead for couple years.
I've also been on the CNI maintainers team for a couple years.
Automatic merge from submit-queue (batch tested with PRs 63905, 64855). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Setup dns servers and search domains for Windows Pods
**What this PR does / why we need it**:
Kubelet is depending on docker container's ResolvConfPath (e.g. /var/lib/docker/containers/439efe31d70fc17485fb6810730679404bb5a6d721b10035c3784157966c7e17/resolv.conf) to setup dns servers and search domains. While this is ok for Linux containers, ResolvConfPath is always an empty string for windows containers. So that the DNS setting for windows containers is always not set.
This PR setups DNS for Windows sandboxes. In this way, Windows Pods could also use kubernetes dns policies.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61579
**Special notes for your reviewer**:
Requires Docker EE version >= 17.10.0.
**Release note**:
```release-note
Setup dns servers and search domains for Windows Pods in dockershim. Docker EE version >= 17.10.0 is required for propagating DNS to containers.
```
/cc @PatrickLang @taylorb-microsoft @michmike @JiangtianLi
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add Vertical Pod Autoscaler to autoscaling/v2beta1
**What this PR does / why we need it**:
Adds Vertical Pod Autoscaler (https://github.com/kubernetes/community/blob/master/contributors/design-proposals/autoscaling/vertical-pod-autoscaler.md) to the autoscaling API (which currently has the Horizontal Pod Autoscaler).
This is needed for the Vertical Pod Autoscaler beta.
**Special notes for your reviewer**:
/cc @thockin @mwielgus @DirectXMan12
FYI. changes that add pkg/registry/autoscaling/verticalpodautoscaler/... will follow.
**Release note**:
```Add Vertical Pod Autoscaler to autoscaling/v2beta1.```
Automatic merge from submit-queue (batch tested with PRs 63386, 64624, 62297, 64847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Keep kubectl delete hack for DaemonSet until 1.12
**What this PR does / why we need it**:
As pointed out here: https://github.com/kubernetes/kubernetes/pull/64797#issuecomment-395152965, the server side fix is in 1.11. We're keeping kubectl hack until 1.12 to support 1.11 kubectl + 1.10.0-4 server.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63386, 64624, 62297, 64847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactoring GCE Disk APIs to use generated client
**What this PR does / why we need it**: Improves maintainability and testing of GCE disks code.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62296
Note that `gen.go` and `gen_test.go` are generated files.
I'm planning to do a more extensive refactor that takes advantage of the generated cloud provider mocks, but that'll be in a separate PR and will be a larger change.
/cc @davidz627
/assign @saad-ali
/release-note-none
/sig storage
Automatic merge from submit-queue (batch tested with PRs 63386, 64624, 62297, 64847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Final kubeadm-kubelet integration refactor PR
**What this PR does / why we need it**:
Note: Work in progress
This PR:
- [x] Updates the debs/rpms to do the "right thing" with the new integration flow
- Broken out into https://github.com/kubernetes/kubernetes/pull/64780
- [x] Uploads the `CRISocket` information to the Node object as an annotation
- Broken out into: https://github.com/kubernetes/kubernetes/pull/64792
- [x] Makes the `kubeadm init` / `kubeadm join` flow to be preflight, stop kubelet, write config/env files, daemon-reload, start kubelet
- [x] Renames `.NodeRegistration.ExtraArgs` to `.NodeRegistration.KubeletExtraArgs` as discussed in the SIG meeting
- [x] Adds a `kubeadm upgrade node config` command for fetching the latest configuration and writing it down to the node before upgrading the kubelet
- [x] Makes dynamic kubelet config actually get enabled when the feature gate in kubeadm is specifically opted into by the user
- [x] Fixes misc. minor bugs
- [x] Makes sure `kubeadm init --dry-run` works, so the dry-run functionality works for the kubelet integration as well
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
kubeadm: Add a new `kubeadm upgrade node config` command
```
@kubernetes/sig-cluster-lifecycle-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 63386, 64624, 62297, 64847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Let the garbage collector use json merge patch when SMP is not supported
**What this PR does / why we need it**:
Let garbage collector fallback to use json merge patch when strategic merge patch returns 415. This enables orphan delete on custom resources.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56348
**Special notes for your reviewer**:
This PR is developed based on https://github.com/kubernetes/kubernetes/pull/56595. Ref https://github.com/kubernetes/kubernetes/pull/56606 for more information.
**Release note**:
```release-note
Orphan delete is now supported for custom resources
```
/sig api-machinery
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CSI implementation of raw block volume support
**What this PR does / why we need it**:
This PR implements support for block volumes feature.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64722
**Special notes for your reviewer**:
**Release note**:
```release-note
Provides API support for external CSI storage drivers to support block volumes.
```
Automatic merge from submit-queue (batch tested with PRs 64749, 64797). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Handle deleted DaemonSet properly
**What this PR does / why we need it**:
After kubectl reapers are removed (#63979) and foreground deletion are used, DaemonSet controller may race with garbage collector when it tries to update DaemonSet status of the DaemonSet being deleted.
Here's what happened:
1. Someone/something performs a foreground deletion on a DaemonSet
1. DaemonSet finalizer and DeletionTimestamp are both set
1. DaemonSet history objects (ControllerRevisions) and pods are being deleted by garbage collector; meanwhile, DaemonSet controller tries to update DaemonSet status.
* Updating DaemonSet status requires constructing DaemonSet history objects, to figure out current revision and which pods do/don't belong to current revision
1. When updating DaemonSet status, DaemonSet controller tries to create a DaemonSet history object that matches current DaemonSet spec
1. Garbage collector then tries to delete that DaemonSet history object. And repeat.
Because we can't make DaemonSet pods be deleted before DaemonSet history objects (DaemonSet history objects don't own DaemonSet pods!), we cannot reliably calculate DaemonSet status without history objects anyways. Therefore, we don't update DaemonSet status for DaemonSet being deleted.
Note that the reason why the kubectl delete hack works is because it forces DaemonSet pods to be removed before history objects.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64313
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix kubectl -o
Fix kubectl -o error message:
Before this change:
```
kubectl get pods -o foo
error: unable to match a printer suitable for the output format "" and the options specified: &get.PrintFlags{JSONYamlPrintFlags:(*genericclioptions.JSONYamlPrintFlags)(0x23aa610), NamePrintFlags:(*genericclioptions.NamePrintFlags)(0xc42058b4e0), TemplateFlags:(*printers.KubeTemplatePrintFlags)(0xc4206765e0), CustomColumnsFlags:(*printers.CustomColumnsPrintFlags)(0xc420676620), HumanReadableFlags:(*get.HumanPrintFlags)(0xc4204eb180), NoHeaders:(*bool)(0xc4206fefbc), OutputFormat:(*string)(0xc42058b4d0)}
```
After this change:
```
Kubectl get pods -o foo
error: unable to match a printer suitable for the output format "foo", allowed formats are: json,yaml,name,template,go-template,go-template-file,templatefile,jsonpath,jsonpath-file,custom-columns-file,custom-columns,wide
```
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
kubectl will list all allowed print formats when an invalid format is passed.
```
Automatic merge from submit-queue (batch tested with PRs 63717, 64646, 64792, 64784, 64800). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update cadvisor godeps to v0.30.0
**What this PR does / why we need it**:
cAdvisor godep update corresponding to 1.11
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63204
**Release note**:
```release-note
Use IONice to reduce IO priority of du and find
cAdvisor ContainerReference no longer contains Labels. Use ContainerSpec instead.
Fix a bug where cadvisor failed to discover a sub-cgroup that was created soon after the parent cgroup.
```
/sig node
/kind bug
/priority critical-urgent
/assign @dchen1107
Automatic merge from submit-queue (batch tested with PRs 63717, 64646, 64792, 64784, 64800). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reconcile extended resource capacity after kubelet restart.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/64632
**Special notes for your reviewer**:
**Release note**:
```release-note
Kubelet will set extended resource capacity to zero after it restarts. If the extended resource is exported by a device plugin, its capacity will change to a valid value after the device plugin re-connects with the Kubelet. If the extended resource is exported by an external component through direct node status capacity patching, the component should repatch the field after kubelet becomes ready again. During the time gap, pods previously assigned with such resources may fail kubelet admission but their controller should create new pods in response to such failures.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Promote sysctl annotations to fields
#
**What this PR does / why we need it**:
Promoting experimental sysctl feature from annotations to API fields.
**Special notes for your reviewer**:
Following sysctl KEP: https://github.com/kubernetes/community/pull/2093
**Release note**:
```release-note
The Sysctls experimental feature has been promoted to beta (enabled by default via the `Sysctls` feature flag). PodSecurityPolicy and Pod objects now have fields for specifying and controlling sysctls. Alpha sysctl annotations will be ignored by 1.11+ kubelets. All alpha sysctl annotations in existing deployments must be converted to API fields to be effective.
```
**TODO**:
* [x] - Promote sysctl annotation in Pod spec
* [x] - Promote sysctl annotation in PodSecuritySpec spec
* [x] - Feature gate the sysctl
* [x] - Promote from alpha to beta
* [x] - docs PR - https://github.com/kubernetes/website/pull/8804
Automatic merge from submit-queue (batch tested with PRs 64009, 64780, 64354, 64727, 63650). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Never clean backoff in job controller
**What this PR does / why we need it**:
In https://github.com/kubernetes/kubernetes/pull/60985 I've added a mechanism which allows immediate job status update, unfortunately that broke the backoff logic seriously. I'm sorry for that. I've changed the `immediate` mechanism so that it NEVER cleans the backoff, but for the cases when we want fast status update it uses a zero backoff.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#62382
**Special notes for your reviewer**:
/assign @janetkuo
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 64009, 64780, 64354, 64727, 63650). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: Update cloud provider to use TPU v1 API
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
GCE: Update cloud provider to use TPU v1 API
```
/assign @vishh
/assign @cheftako
