Automatic merge from submit-queue (batch tested with PRs 51335, 51364, 51130, 48075, 50920)
Graduate custom metrics API to v1beta1
This graduates custom-metrics.metrics.k8s.io/v1alpha1 to custom-metrics.metrics.k8s.io/v1beta1. The move is more-or-less just a straightforward rename.
Part of kubernetes/features#117 and kubernetes/features#118
```release-note
the custom metrics API (custom-metrics.metrics.k8s.io) has moved from v1alpha1 to v1beta1
```
Automatic merge from submit-queue (batch tested with PRs 51335, 51364, 51130, 48075, 50920)
[API] Feature/job failure policy
**What this PR does / why we need it**: Implements the Backoff policy and failed pod limit defined in https://github.com/kubernetes/community/pull/583
**Which issue this PR fixes**:
fixes#27997, fixes#30243
**Special notes for your reviewer**:
This is a WIP PR, I updated the api batchv1.JobSpec in order to prepare the backoff policy implementation in the JobController.
**Release note**:
```release-note
Add backoff policy and failed pod limit for a job
```
Automatic merge from submit-queue (batch tested with PRs 51335, 51364, 51130, 48075, 50920)
Add the possibility to set return values for the FakeDiscovery implementation
**What this PR does / why we need it**:
As an user of the fake clientset (with the fake discovery), I want to be able to set the fake server's version on demand like this for example:
```go
func TestFakingServerVersion(t *testing.T) {
client := fakeclientset.NewSimpleClientset()
fakeDiscovery, ok := client.Discovery().(*fakediscovery.FakeDiscovery)
if !ok {
t.Fatalf("couldn't convert Discovery() to *FakeDiscovery")
}
testGitCommit := "v1.0.0"
fakeDiscovery.FakedServerVersion = &version.Info{
GitCommit: testGitCommit,
}
sv, err := client.Discovery().ServerVersion()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if sv.GitCommit != testGitCommit {
t.Fatalf("unexpected faked discovery return value: %q", sv.GitCommit)
}
}
```
This PR makes that possible, in wait for a more sophisticated FakeDiscovery implementation generally.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 51335, 51364, 51130, 48075, 50920)
Change default validation to openapi
**What this PR does / why we need it**:
And also update the test to match the new error string.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubectl/issues/49
**Special notes for your reviewer**:
**Release note**:
```release-note
Performs validation (when applying for example) against OpenAPI schema rather than Swagger 1.0.
```
Automatic merge from submit-queue
update deprecated interface and fix bug not return when list pod failed in cronjob_controller.go
**What this PR does / why we need it**:
remove some unused redundant code, and fix bug: when list pod failed,
job still deleted but pod may still exist in func `deleteJob`
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
kubectl: Clean up documentation for rollout_status.go
**What this PR does / why we need it**:
`golint` emits various warnings about missing comments for exported funcitons and types.
This PR adds missing documentation strings to functions and types. Also adds punctuation to current documentation strings in line with Go coding standards.
**Release note**:
```release-note
NONE
```
/sig cli
/kind cleanup
/kind documentation
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
Mount propagation in kubelet
Together with #45724 it implements mount propagation as proposed in https://github.com/kubernetes/community/pull/659
There is:
- New alpha annotation that allows user to explicitly set propagation mode for each `VolumeMount` in pod containers (to be replaced with real `VolumeMount.Propagation` field during beta) + validation + tests. "Private" is the default one (= no change to existing pods).
I know about proposal for real API fields for alpha feature in https://docs.google.com/document/d/1wuoSqHkeT51mQQ7dIFhUKrdi3-1wbKrNWeIL4cKb9zU/edit, but it seems it's not implemented yet. It would save me quite lot of code and ugly annotation.
- Updated CRI API to transport chosen propagation to Docker.
- New `kubelet --experimental-mount-propagation` option to enable the previous bullet without modifying types.go (worked around with changing `KubeletDeps`... not nice, but it's better than adding a parameter to `NewMainKubelet` and removing it in the next release...)
```release-note
kubelet has alpha support for mount propagation. It is disabled by default and it is there for testing only. This feature may be redesigned or even removed in a future release.
```
@derekwaynecarr @dchen1107 @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
Add selfsubjectrulesreview in authorization
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes#47834#31292
**Special notes for your reviewer**:
**Release note**:
```release-note
Add selfsubjectrulesreview API for allowing users to query which permissions they have in a given namespace.
```
/cc @deads2k @liggitt
Automatic merge from submit-queue
Make /var/lib/kubelet as shared during startup
This is part of ~~https://github.com/kubernetes/community/pull/589~~https://github.com/kubernetes/community/pull/659
We'd like kubelet to be able to consume mounts from containers in the future, therefore kubelet should make sure that `/var/lib/kubelet` has shared mount propagation to be able to see these mounts.
On most distros, root directory is already mounted with shared mount propagation and this code will not do anything. On older distros such as Debian Wheezy, this code detects that `/var/lib/kubelet` is a directory on `/` which has private mount propagation and kubelet bind-mounts `/var/lib/kubelet` as rshared.
Both "regular" linux mounter and `NsenterMounter` are updated here.
@kubernetes/sig-storage-pr-reviews @kubernetes/sig-node-pr-reviews
@vishh
Release note:
```release-note
Kubelet re-binds /var/lib/kubelet directory with rshared mount propagation during startup if it is not shared yet.
```
Automatic merge from submit-queue (batch tested with PRs 51590, 48217, 51209, 51575, 48627)
Skip system container cgroup stats if undefined
**What this PR does / why we need it**:
the kubelet /stats/summary endpoint tried to look up cgroup stats for containers that are not required. this polluted logs with messages about not finding stats for "" container. this pr skips cgroup stats if the cgroup name is not specified (they are optional anyway)
**Special notes for your reviewer**:
i think this was a regression from recent refactor.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51590, 48217, 51209, 51575, 48627)
Deviceplugin jiayingz
**What this PR does / why we need it**:
This PR implements the kubelet Device Plugin Manager.
It includes four commits implemented by @RenaudWasTaken and a commit that supports allocation.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Design document: kubernetes/community#695
PR tracking: kubernetes/features#368
**Special notes for your reviewer**:
**Release note**:
Extending Kubelet to support device plugin
```release-note
```
Automatic merge from submit-queue
Fix InstanceTypeByProviderID for Azure
**What this PR does / why we need it**:
Fix change in #46940, should return InstanceType in function InstanceTypeByProviderID
Otherwise:
```
I0830 05:01:08.497989 15347 node_controller.go:328] Adding node label from cloud provider: beta.kubernetes.io/instance-type=/subscriptions/{id}/resourceGroups/{id}/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
@brendandburns @realfake
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
add apps/v1beta2 conversion tests
**What this PR does / why we need it**:
add apps/v1beta2 conversion test
Depend on ~~#49751~~(Merged), ~~#49719~~(Merged)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
/cc @janetkuo
**Release note**:
```release-note
add apps/v1beta2 conversion test
```
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
Remove deprecated and experimental fields from KubeletConfiguration
As we work towards providing a stable (v1) kubeletconfig API,
we cannot afford to have deprecated or "experimental" (alpha) fields
living in the KubeletConfiguration struct. This removes all existing
experimental or deprecated fields, and places them in KubeletFlags
instead.
I'm going to send another PR after this one that organizes the remaining
fields into substructures for readability. Then, we should try to move
to v1 ASAP (maybe not v1 in 1.8, given how close we are, but definitely in 1.9).
It makes far more sense to focus on a clean API in kubeletconfig v2,
than to try and further clean up the existing "API" that everyone
already depends on.
fixes: #51657
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
Bugfix: Use local JSON log buffer in parseDockerJSONLog.
**What this PR does / why we need it**:
The issue described in #47800 is due to a race condition in `ReadLogs`: Because the JSON log buffer (`dockerJSONLog`) is package-scoped, any two goroutines modifying the buffer could race and overwrite the other's changes. In particular, one goroutine could unmarshal a JSON log line into the buffer, then another goroutine could `Reset()` the buffer, and the resulting `Stream` would be empty (`""`). This empty `Stream` is caught in a `case` block and raises an `unexpected stream type` error.
This PR creates a new buffer for each execution of `parseDockerJSONLog`, so each goroutine is guaranteed to have a local instance of the buffer.
**Which issue this PR fixes**: fixes#47800
**Release note**:
```release-note
Fixed an issue (#47800) where `kubectl logs -f` failed with `unexpected stream type ""`.
```
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
GCE: Reserve address for ILBs during sync
**What this PR does / why we need it**:
This PR adds the ability for the service controller to hold the ILB's IP during sync which may delete/recreate the forwarding rule.
Fixes: #47531
**Release note**:
```release-note
GCE: Internal load balancer IPs are now reserved during service sync to prevent losing the address to another service.
```
Automatic merge from submit-queue
Use json-iterator instead of ugorji for JSON.
@smarterclayton @wojtek-t
Fixes#36120
xref #18762
```release-note
Switch JSON marshal/unmarshal to json-iterator library. Performance should be close to previous with no generated code.
```
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
Separate feature gates for dynamic kubelet config vs loading from a file
This makes it so these two features can be turned on independently, rather than bundling both under dynamic kubelet config.
fixes: #51664
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
kubectl: Remove ending punctuation from error strings
**What this PR does / why we need it**:
`golint` emits 2 warnings of type:
`error strings should not end with punctuation`
Remove punctuation from end of error strings.
**Release note**:
```release-note
NONE
```
/sig cli
/kind cleanup
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
CPU manager wiring and `none` policy
Blocker for CPU manager #49186 (4 of 6)
* Previous PR in this series: #51140
* Next PR in this series: #51180
cc @balajismaniam @derekwaynecarr @sjenning
**Release note**:
```release-note
NONE
```
TODO:
- [X] In-memory CPU manager state
- [x] Kubelet config value
- [x] Feature gate
- [X] None policy
- [X] Unit tests
- [X] CPU manager instantiation
- [x] Calls into CPU manager from Kubelet container runtime
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Ensure that DaemonSet respects termination
**What this PR does / why we need it**:
#43077 correctly prevents the DaemonSet controller from adopting deleted Pods, but, as pointed out in #50477, the controller now has no sensitivity to the termination lifecycle (i.e TerminationGracePeriodSeconds) of the Pods it creates. This PR attempts to balance the two. DaemonSet controller will now consider deleted Pods owned by a DaemonSet during creation, but it will not consider deleted Pods as targets for adoption.
fixes#50477
```release-note
#43077 introduced a condition where DaemonSet controller did not respect the TerminationGracePeriodSeconds of the Pods it created. This is now corrected.
```
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Fix pod local ephemeral storage usage calculation
We use podDiskUsage to calculate pod local ephemeral storage which is not correct, because podDiskUsage also contains HostPath volume which is considered as persistent storage
This pr fixes it
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51489
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @jingxu97 @vishh
cc @ddysher
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Fix printISCSIVolumeSource to show kubectl describe properly
**What this PR does / why we need it**:
After merging #51189, 'kubectl describe' doesn't show persistent volume properly if the volume is iSCSI.
This PR fixes the problem.
**Which issue this PR fixes** : Fixes#51635
**Special notes for your reviewer**:
Result without InitiatorName.
```
% k describe pv pv0001
Name: pv0001
Labels: <none>
Annotations: volume.beta.kubernetes.io/storage-class=slow
StorageClass: slow
Status: Available
Claim:
Reclaim Policy: Recycle
Access Modes: RWO
Capacity: 1Gi
Message:
Source:
Type: ISCSI (an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod)
TargetPortal: 192.168.122.85:3260
IQN: iqn.2017-05.com.example:rhel7
Lun: 0
ISCSIInterface default
FSType: ext4
ReadOnly: true
Portals: [192.168.122.7:3260]
DiscoveryCHAPAuth: false
SessionCHAPAuth: false
SecretRef: <nil>
InitiatorName: <none>
Events: <none>
```
Result with InitiatorName.
```
% k describe pv pv0001
Name: pv0001
Labels: <none>
Annotations: volume.beta.kubernetes.io/storage-class=slow
StorageClass: slow
Status: Available
Claim:
Reclaim Policy: Recycle
Access Modes: RWO
Capacity: 1Gi
Message:
Source:
Type: ISCSI (an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod)
TargetPortal: 192.168.122.85:3260
IQN: iqn.2017-05.com.example:rhel7
Lun: 0
ISCSIInterface default
FSType: ext4
ReadOnly: true
Portals: [192.168.122.7:3260]
DiscoveryCHAPAuth: false
SessionCHAPAuth: false
SecretRef: <nil>
InitiatorName: iqn.1994-05.com.redhat:185ce16b55ad
Events: <none>
```
@rootfs @humblec @jsafrane
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Fix iSCSI WaitForAttach not mounting a volume
WaitForAttach failed consistently with this error:
```
Heuristic determination of mount point failed:stat /var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/10.128.0.3:3260-iqn.2003-01.org.linux-iscsi.f21.x8664:sn.4b0aae584f7c-lun-0: no such file or directory
```
We should ignore "no such file or directory" eror, the directory is created just few lines below.
Fixes: #51625
**Release note**:
```release-note
NONE
```
/sig storage
/assign @mtanino
Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)
Add a persistent volume label controller to the cloud-controller-manager
Part of https://github.com/kubernetes/features/issues/88
Outstanding concerns needing input:
- [x] Why 5 threads for controller processing?
- [x] Remove direct linkage to aws/gce cloud providers [#51629]
- [x] Modify shared informers to allow added event handlers ability to include uninitialized objects/using unshared informer #48893
- [x] Use cache.MetaNamespaceKeyFunc in event handler?
I'm willing to work on addressing the removal of the direct linkage to aws/gce after this PR gets in.
Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)
Task 1: Tainted node by condition.
**What this PR does / why we need it**:
Tainted node by condition for MemoryPressure, OutOfDisk and so on.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: part of #42001
**Release note**:
```release-note
Tainted nodes by conditions as following:
* 'node.kubernetes.io/network-unavailable=:NoSchedule' if NetworkUnavailable is true
* 'node.kubernetes.io/disk-pressure=:NoSchedule' if DiskPressure is true
* 'node.kubernetes.io/memory-pressure=:NoSchedule' if MemoryPressure is true
* 'node.kubernetes.io/out-of-disk=:NoSchedule' if OutOfDisk is true
```
