Automatic merge from submit-queue (batch tested with PRs 58216, 58193, 53033, 58219, 55921). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix device plugin endpoint UT
**What this PR does / why we need it**:
Fix some issues in device plugin endpoint UT.
**Which issue(s) this PR fixes**:
Fixes#55920
**Special notes for your reviewer**:
@jiayingz @RenaudWasTaken @lichuqiang PTAL.
/sig node
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
wires the generic scaler to kubectl
**What this PR does / why we need it**:
the changes introduced in this commit plumbs in the generic scaler to kubectl.
note that we don't change the behaviour of kubectl.
For example it won't scale new resources. That's the end goal.
The first step is to retrofit existing code to use the generic scaler.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
The lbaas.opts.SubnetId should be set by subnet id.
Fix#58145
The getSubnetIDForLB() should return subnet id rather than net id.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improved readability for messages being logged
**What this PR does / why we need it**:
This improves the readability for messages seen by end-user. /cc @jwforres @bsalamat - For UX
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57152
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Allow kubectl drain to continue w ds-managed pods with local storage
**Release note**:
```release-note
NONE
```
Prevents oadm drain from failing if it encounters DaemonSet-managed pods
that have local storage, when the option to ignore DaemonSet-managed
pods has been specified.
Will add a test
cc @kubernetes/sig-cli-misc @deads2k @fabianofranz @dustymabe
Automatic merge from submit-queue (batch tested with PRs 57266, 58187, 58186, 46245, 56509). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Added metrics for predicate and priority evaluation
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45972
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57266, 58187, 58186, 46245, 56509). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Avoid panic in Cloud CIDR Allocator
**What this PR does / why we need it**:
I suspect a race exists where we attempt to look up the CIDR for a terminating node. By the time `updateCIDRAllocation` is called the node has disappeared. We determine it does not have a cloud CIDR (i.e. Alias IP Range) and attempt to record a `CIDRNotAvailable` node status. Unfortunately we reference `node.Name` while `node` is still nil.
By getting the node before looking up the cloud CIDR we avoid the nil pointer dereference, and potentially fail fast in the case the node has disappeared.
**Which issue(s) this PR fixes**:
Fixes#58181
**Release note**:
```release-note
Avoid panic when failing to allocate a Cloud CIDR (aka GCE Alias IP Range).
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Handle Unhealthy devices
Update node capacity with sum of both healthy and unhealthy devices.
Node allocatable reflect only healthy devices.
**What this PR does / why we need it**:
Currently node capacity only reflects healthy devices. Unhealthy devices are ignored totally while updating node status. This PR handles unhealthy devices while updating node status.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57241
**Special notes for your reviewer**:
**Release note**:
<!-- Write your release note:
Handle Unhealthy devices
```release-note
Handle Unhealthy devices
```
/cc @tengqm @ConnorDoyle @jiayingz @vishh @jeremyeder @sjenning @resouer @ScorpioCPH @lichuqiang @RenaudWasTaken @balajismaniam
/sig node
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add `cloud` for the generated GCE interfaces, support structs
Note: this does not wire the generated code.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Comment typo fix in StatefulSet controller
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[FC Plugin] Create proper volumeSpec during ConstructVolumeSpec
**What this PR does / why we need it**:
Currently, FC plugin returns volume name and empty FCVolumeSource during ConstrutVolumeSpec of filesystem volume's reconstruction.
In this fix, ConstructVolumeSpec retrieves global mount path, and analyzes volume parameters such as
WWN, LUN, WWID from the path.
**Which issue(s) this PR fixes** :
Fixes#58085
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc @rootfs @jsafrane
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enable on-demand collection of node metrics
**What this PR does / why we need it**:
This PR enables collecting node-level metrics on-demand. This is useful because it allows the kubelet to respond to resource pressure more quickly.
**Which issue(s) this PR fixes**:
Ref: #51745
**Release note**:
```release-note
NONE
```
/sig node
/priority important-soon
/kind bug
/assign @vishh @derekwaynecarr
cc @tallclair
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
use shared informers for BootstrapSigner controller
**What this PR does / why we need it**:
fix TODO: Switch to shared informers
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fixed the some typo in eviction_manager
**What this PR does / why we need it**:
fixed some wrong typo in `eviction_manager.go`
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove duplicate function
remove duplicate function
**What this PR does / why we need it**:
remove duplicate function `GetDeploymentConditionInternal`
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add custom volume name based on SC parameter.
At present glusterfs dynamic PVs are created with
random names. However an admin would like to have some
handle on the volume names created dynamically for
various purposes. One example would be having a filter
for sorting out PVs created for a particular storage class.
This patch enables the functionality by having a custom
volume name as a prefix to dynamic PVs. This is an optional
parameter in SC and if set, the dynamic volumes are created
in below format where `_` is the field seperator/delimiter:
customvolumeprefix_PVCname_randomUUID
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add stub device plugin for conformance e2e test
**What this PR does / why we need it**:
Add stub device plugin for conformance e2e test
- extend [device_plugin_stub](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/cm/deviceplugin/device_plugin_stub.go) to support e2e test
- add test suite with this device-plugin-stub
- simulate more use cases by deploying some pods to request these resources
**Which issue this PR fixes**:
fixes#52861
**Special notes for your reviewer**:
@vishh @jiayingz PTAL.
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
client-go: remove open-api import from types
This builds off of https://github.com/kubernetes/kube-openapi/pull/21 and removes the following imports from `k8s.io/client-go/kubernetes`:
```
github.com/PuerkitoBio/purell
github.com/PuerkitoBio/urlesc
github.com/emicklei/go-restful
github.com/emicklei/go-restful/log
github.com/go-openapi/jsonpointer
github.com/go-openapi/jsonreference
github.com/go-openapi/spec
github.com/go-openapi/swag
github.com/mailru/easyjson/buffer
github.com/mailru/easyjson/jlexer
github.com/mailru/easyjson/jwriter
golang.org/x/text/cases
golang.org/x/text/internal
golang.org/x/text/internal/tag
golang.org/x/text/language
golang.org/x/text/runes
golang.org/x/text/secure/precis
golang.org/x/text/width
k8s.io/kube-openapi/pkg/common
```
/assign @sttts
/assign @mbohlool
cc @kubernetes/sig-api-machinery-pr-reviews
```release-note
NONE
```
note that we don't change the behaviour of kubectl.
For example it won't scale new resources. That's the end goal.
The first step is to retrofit existing code to use the generic scaler.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove dead code in pkg/api
**What this PR does / why we need it**:
remove dead code
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
git: Use VolumeHost.GetExec() to execute stuff in volume plugins
This prepares volume plugins to run things in containers instead of running
them on the host.
**Special notes for your reviewer**:
* instead of `cmd.SetDir(<dir>); cmd.Exec("git <command>")`, we do `cmd.Exec("git -C <dir> <command>")` - mounter.Exec does not have SetDir()
* there are lot of changes in the tests because a different exec interface is used.
@kubernetes/sig-storage-pr-reviews
```release-note
gitRepo volumes in pods require git 1.8.5 or later
```
/assign @rootfs
Currently, FC plugin returns volume name and empty
FCVolumeSource during ConstrutVolumeSpec during
filesystem volume's reconstruction.
In this fix, ConstructVolumeSpec retrieves global
mount path, analyzes volume parameters such as
WWN, LUN, WWID from the path.
Fixes#58085
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert "no need delete endpoint explicitly in endpoint controller"
Reverts kubernetes/kubernetes#57747
For:
1. `checkLeftoverEndpoints` does not work any more.
2. service delete auto trigger endpoint delete in apiserver, if service is successfully deleted but fail on endpoint delete. Then this endpoint will be garbage resource.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix TestCadvisorListPodStats failure under mac/darwin
**What this PR does / why we need it**:
GetPodCgroupNameSuffix is not really implemented under darwin
(or windows for that matter). So let's just skip over the check
for CPU and Memory if that is not set.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57636
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix bug:Kubelet failure to umount mount points
What this PR does / why we need it:
Fix bug#41141:Kubelet failure to umount glusterfs mount points
kubelet failure to umount mount moints in these cases:
1. glusterfs volume : transport endpoint is not connected
2. nfs volume : stale NFS file handle
this PR will fix it
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
fixes#41141
Special notes for your reviewer:
NONE
Release note:
NONE
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Wrap azure client calls
**What this PR does / why we need it**:
This is a clean up for azure client calls. It adds wrappers over azure clients and moves verbose logs and rate limiter inside.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/cc @cosmincojocar @andyzhangx
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix cinder detach problems
**What this PR does / why we need it**: We have currently huge problems in cinder volume detach. This PR tries to fix these issues.
**Which issue(s) this PR fixes**:
Fixes#50004Fixes#57497
**Special notes for your reviewer**:
**Release note**:
```release-note
openstack cinder detach problem is fixed if nova is shutdowned
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
client-go: remove import of github.com/gregjones/httpcache
Moves NewCacheRoundTripper from `k8s.io/client-go/transport` to its own package. This prevents Kubernetes clients from requiring its dependencies.
This change removes the following transitive imports from `k8s.io/client-go/kubernetes`
```
github.com/google/btree
github.com/gregjones/httpcache
github.com/gregjones/httpcache/diskcache
github.com/peterbourgon/diskv
```
```release-note
NONE
```
At present glusterfs dynamic PVs are created with
random names. However an admin would like to have some
handle on the volume names created dynamically for
various purposes. One example would be having a filter
for sorting out PVs created for a particular storage class.
This patch enables the functionality by having a custom
volume name as a prefix to dynamic PVs. This is an optional
parameter in SC and if set, the dynamic volumes are created
in below format where `_` is the field seperator/delimiter:
customvolumeprefix_PVCname_randomUUID
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix quota controller worker deadlock
The resource quota controller worker pool can deadlock when:
* Worker goroutines are idle waiting for work from queues
* The Sync() method detects discovery updates to apply
The problem is workers acquire a read lock while idle, making write lock
acquisition dependent upon the presence of work in the queues.
The Sync() method blocks on a pending write lock acquisition and won't unblock
until every existing worker processes one item from their queue and releases
their read lock. While the Sync() method's lock is pending, all new read lock
acquisitions will block; if a worker does process work and release its lock, it
will then become blocked on a read lock acquisition; they become blocked on
Sync(). This can easily deadlock all the workers processing from one queue while
any workers on the other queue remain blocked waiting for work.
Fix the deadlock by refactoring workers to acquire a read lock *after* work is
popped from the queue. This allows writers to get locks while workers are idle,
while preserving the worker pause semantics necessary to allow safe sync.
```release-note
Fixes an infrequent problem causing the resource quota controller to become stuck in clusters with low ResourceQuota churn, potentially preventing quota from being recalculated until the controller is restarted or until bursts of diverse quota activity unstick the controller.
```
/cc @kubernetes/sig-api-machinery-bugs
The resource quota controller worker pool can deadlock when:
* Worker goroutines are idle waiting for work from queues
* The Sync() method detects discovery updates to apply
The problem is workers acquire a read lock while idle, making write lock
acquisition dependent upon the presence of work in the queues.
The Sync() method blocks on a pending write lock acquisition and won't unblock
until every existing worker processes one item from their queue and releases
their read lock. While the Sync() method's lock is pending, all new read lock
acquisitions will block; if a worker does process work and release its lock, it
will then become blocked on a read lock acquisition; they become blocked on
Sync(). This can easily deadlock all the workers processing from one queue while
any workers on the other queue remain blocked waiting for work.
Fix the deadlock by refactoring workers to acquire a read lock *after* work is
popped from the queue. This allows writers to get locks while workers are idle,
while preserving the worker pause semantics necessary to allow safe sync.
Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Block volumes Support: iSCSI plugin update
**What this PR does / why we need it**:
Add interface changes to iSCSI volume plugin to enable block volumes support feature.
**Which issue this PR fixes**:
Based on this proposal (kubernetes/community#805 & kubernetes/community#1265) and this feature issue: kubernetes/features#351
**Special notes for your reviewer**:
This PR temporarily includes following changes except iSCSI plugin change for reviewing purpose.
These changes will be removed from the PR once they are merged.
- (#50457) API Change
- (#51494) Container runtime interface change, volumemanager changes, operationexecutor changes
There are another PRs related to this functionality.
(#50457) API Change
(#53385) VolumeMode PV-PVC Binding change
(#51494) Container runtime interface change, volumemanager changes, operationexecutor changes
(#55112) Block volume: Command line printer update
Plugins
(#51493) Block volumes Support: FC plugin update
(#54752) Block volumes Support: iSCSI plugin update
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: Use existing subnetwork of ILB forwarding rule
Fixes#57860
**Test Cases**:
Clusters using auto network with existence of a manual subnet in same region.
- [x] Upgrade 1.7 cluster with existing ILBs to latest. Confirm existing ILBs still are synced.
Version 1.7 does not attempt to fill in the subnetwork, so the forwarding rule was created with the correct subnetwork.
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using wrong subnet) still are synced.
- [x] Latest version creates ILBs using the correct subnet.
Clusters with manual subnets have always and will continue to use the subnet specified in gce.conf.
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using manual subnet) still are synced.
Clusters with legacy networks have always and will continue to use an empty subnet.
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using legacy network) still are synced.
**Release note**:
```release-note
GCE: Allows existing internal load balancers to continue using an outdated subnetwork
```
Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a feature flag for sharing process namespace between containers in a pod
**What this PR does / why we need it**:
This adds a feature flag for the feature described in kubernetes/features#495.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/assign @dchen1107
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
ignore images in used by running containers when GC
**What this PR does / why we need it**:
Let kubelet not attempt to remove images being used by running containers.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57006
**Special notes for your reviewer**:
@kubernetes/sig-node-pr-reviews
**Release note**:
```release-note
ignore images in used by running containers when GC
```
Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Mark kubelet PID namespace flag as deprecated
**What this PR does / why we need it**:
The `--docker-disable-shared-pid` flag will be removed once per-pod configurable process namespace sharing becomes available. Mark it deprecated to notify cluster admins of the upcoming change.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
WIP #41938
**Special notes for your reviewer**:
/assign @yujuhong
**Release note**:
```release-note
The kubelet flag to run docker containers with a process namespace that is shared between all containers in a pod is now deprecated and will be replaced by a new field in `v1.Pod` that configures this behavior.
```
