Automatic merge from submit-queue (batch tested with PRs 55050, 53464, 54936, 55028, 54928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: dockershim: remove orphaned checkpoint files
Fixes https://github.com/kubernetes/kubernetes/issues/55070
Currently, `ListPodSandbox()` returns a combined list of sandboxes populated from both the runtime and the dockershim checkpoint files. However the sandboxes in the checkpoint files might not exist anymore.
The kubelet sees the sandbox returned by `ListPodSandbox()` and determines it shouldn't be running and calls `StopPodSandbox()` on it. This generates an error when `StopContainer()` is called as the container does not exist. However the checkpoint file is not cleaned up. This leads to subsequent calls to `StopPodSandbox()` that fail in the same way each time.
This PR removes the checkpoint file if StopContainer fails due to container not found.
The only other place `RemoveCheckpoint()` is called, except if it is corrupt, is from `RemoveSandbox()`. If the container does not exist, what `RemoveSandbox()` would have done has been effectively been done already. So this is just clean up.
@derekwaynecarr @eparis @freehan @dcbw
Automatic merge from submit-queue (batch tested with PRs 55050, 53464, 54936, 55028, 54928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
output empty creationTimestamp as null
**Release note**
```release-note
NONE
```
Updates the value of the `creationTimestamp` field to be `null`
when empty, to keep parity between it and `deletionTimestamp`.
Adds a round-trip test to ensure that unstructured objects containing
empty metadata fields are able to be re-converted back into internal
or external objects. Prior to the proposed patch in this PR, an
unstructured object whose `.metadata.creationTimestamp` value had
been set through the metadata accessor to an empty value
(`metav1.Time{}` in this case), was unable to be re-converted to an
internal or external type using the runtime decoder. Conversion would
fail with the error:
```
unstructured_test.go:177: FromUnstructured failed: parsing time "" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "2006"
```
cc @liggitt @fabianofranz
Automatic merge from submit-queue (batch tested with PRs 51401, 54056, 54977, 55017, 55052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
extensions: remove TPR remnants
The extensions group still had the TPR types + generated client. Having this in the codebase doesn't create any problems but would be good to clean up, especially since TPR access has been removed in 1.8.
**Release note**:
```release-note
NONE
```
/assign @sttts @deads2k
Automatic merge from submit-queue (batch tested with PRs 51401, 54056, 54977, 55017, 55052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix example and comments for kubectl convert
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>
**What this PR does / why we need it**:
There are some erros in example and comments in convert
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51401, 54056, 54977, 55017, 55052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[cli] exit when Validate() returns an error
cmdutil.UsageErrorf() does nothing but returna a string. When Validate()
returns an error, we should call exit() function.
fix: https://github.com/kubernetes/kubectl/issues/119
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55063, 54523, 55053). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
unit tests for syncRolloutStatus in pkg/controller/deployment
**What this PR does / why we need it**:
Adds unit tests for syncRolloutStatus func in controller/deployment package.
**Which issue this PR fixes**:
fixes#53663
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix exported method run's comment
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kube-proxy IPVS: Fix IPVS availability check
**What this PR does / why we need it**:
The current implementation of `CanUseIPVSProxier()` reads `/proc/modules`
to check whether IPVS related kernel modules can be loaded.
You might get a false-negative when the kernel modules are installed to
the system but haven't been loaded into the kernel.
This patch firstly try to run `modprobe` to load specified kernel
modules, then just log warnings if error occured.
Secondly, it will check loaded kernel modules by reading
`/proc/modules`, return an error if any required module is missing.
This change will not break the compatability of existing implementation.
Running kube-proxy in a container without mounting `/lib/modules` will
cause `modprobe` warnings, but not raise an error if all required modules are
present in the host kernel.
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix IPVS availability check
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove dependency on drv_cfg binary for querying ScaleIO devices
**What this PR does / why we need it**:
This PR fixes the issue where a ScleIO kubernetes plugin required additional binary `drv_cfg` to be present on the local node to work properly, making it harder for the kubelet binary to be containerized without providing access to local paths inside the container.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54954
**Special notes for your reviewer**:
**Release note**:
```release-note
ScaleIO driver completely removes dependency on drv_cfg binary so a Kubernetes cluster can easily run a containerized kubelet.
```
Automatic merge from submit-queue (batch tested with PRs 54906, 54120, 54934, 54915, 54848). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve unit test coverage for network policy validation.
**What this PR does / why we need it**:
`ok k8s.io/kubernetes/pkg/apis/networking/validation 0.074s coverage: 87.8% of statements`
to
`ok k8s.io/kubernetes/pkg/apis/networking/validation 0.082s coverage: 100.0% of statements`
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
clean up legacy ipv4/32 in ipvs proxy
**What this PR does / why we need it**:
clean up legacy ipv4/32 in ipvs proxy
**Which issue(s) this PR fixes**: closes#51866
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area ipv6
/assign @thockin
cc @danehans
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add kubeadm way in ipvs proxy README
**What this PR does / why we need it**:
As #53962 which support specify proxy mode for kubeadm is already in, we should add ipvs proxy kubeadm way in README.md.
**Which issue(s) this PR fixes**:
Fixes#54978
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
StopPodSandbox should not log when container is already removed
**What this PR does / why we need it**:
StopPodSandbox should not log when a container is already gone. It should only log if it could not stop and the container was still present.
Fixes https://github.com/kubernetes/kubernetes/issues/55021
**Special notes for your reviewer**:
This was seen in our production logs, need to eliminate spam.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add admission handler for device resources allocation
**What this PR does / why we need it**:
Add admission handler for device resources allocation to fail fast during pod creation
**Which issue this PR fixes**
fixes#51592
**Special notes for your reviewer**:
@jiayingz Sorry, there is something wrong with my branch in #51895. And I think the existing comments in the PR might be too long for others to view. So I closed it and opened the new one, as we have basically reach an agreement on the implement :)
I have covered the functionality and unit test part here, and would set about the e2e part ASAP
/cc @jiayingz @vishh @RenaudWasTaken
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
discovery client not depend on pkg/api/legacyscheme
Switch kubectl's cached discovery client to use kubect's scheme (pkg/kubectl/scheme) which only registered with external version types.
The encoding and decoding here uses only `k8s.io/apimachinery/pkg/apis/meta/v1.APIGroupList` and `k8s.io/apimachinery/pkg/apis/meta/v1.APIResourceList` which are not internal version. So it should be safe.
```release-note
NONE
```
/assign @monopole
Automatic merge from submit-queue (batch tested with PRs 55016, 50887). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cleanup resouce test
- Make client-go testing support decoding to external version
- Cleanup bad dependency of `pkg/kubectl/resource` pkg
```release-note
NONE
```
/assign @monopole @caesarxuchao
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a file store utility package in kubelet
More and more components checkpoints (i.e., persist their states) in
kubelet. Refurbish and move the implementation in dockershim to a
utility package to improve code reusability.
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Lift embedded structure out of ManifestURLHeader field
Related: #53833
```release-note
It is now possible to set multiple manifest url headers via the Kubelet's --manifest-url-header flag. Multiple headers for the same key will be added in the order provided. The ManifestURLHeader field in KubeletConfiguration object (kubeletconfig/v1alpha1) is now a map[string][]string, which facilitates writing JSON and YAML files.
```
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support retainkeys strategy for new merge code
- Some prefactoring for retainkeys
- Add retainkeys strategy
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
should check and return err when visit failure
**What this PR does / why we need it**:
should check and return err when visit failure
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55004, 54957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update kube-openapi to use validation
**What this PR does / why we need it**: Moves openapi validation code to kube-openapi, so that we can move the rest of the code to apimachinery repository, so that later we can use it from both the client and the server.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #Nothing
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
admission: wire through validating-only admission
Based on https://github.com/kubernetes/kubernetes/pull/54232.
This is important and required for beta because it affects the shape of the webhook admission plugins that are going to be produced and is needed to make sure that our existing chain continues to properly verify and protect the API objects based on their final state after webhook admission mutators run.
We discussed this in the October 11 API machinery call with @erictune and @caesarxuchao and we agreed to do this as a requirement for beta. See this part of the recording: https://www.youtube.com/watch?v=mrgDPHyr4VY#t=325 .
Automatic merge from submit-queue (batch tested with PRs 54800, 53898, 54812, 54921, 53558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
rename metric reflector_xx_last_resource_version
**What this PR does / why we need it**:
mv reflector name from metric name to metric label
before:
```
reflector_k8s_io_kubernetes_pkg_client_informers_informers_generated_internalversion_factory_go:73_8664_last_resource_version{instance="104.154.20.21:443",job="kubernetes-apiservers"}
```
after
```
reflector_last_resource_version{instance="10.0.2.15:6443",job="kubernetes-apiservers",name="k8s_io_kubernetes_pkg_client_informers_informers_generated_internalversion_factory_go_73_8664"}
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#52121
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubenet: yield lock while executing CNI plugin.
The CNI plugin can take up to 3 seconds to execute. CNI plugins can safely be
executed in parallel, so yield the lock to speed up pod creation.
This caused problems with the pod latency tests - previously, CNI plugins executed
in under 20ms. Now they must wait for DAD to finish and addresses to leave
tentative state.
Fixes: #54651
**What this PR does / why we need it**:
After upgrading CNI plugins to v0.6 in #51250, the pod latency tests began failing. This is because the plugins, in order to support IPv6, need to wait for DAD to finish. Because this
delay is while the kubenet lock is held, it significantly slows down the pod creation rate.
**Special notes for your reviewer**:
The CNI plugins also do locking for their critical paths, so it is safe to run them concurrently.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve deleteOptions doc
**What this PR does / why we need it**:
This PR improves the doc of the 'PropagationPolicy' property of the DeleteOptions struct.
The said field is a string. It is very important a field for people who invoke a delete API.
For example, direct invocation of the deployment-delete API using the python client will have the replicaset and pods left behind. However, this field was not well documented. When people wanted to try different options, they have to dig into k8s source code to find out the acceptable values.
**Which issue this PR fixes**: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54895, 54449). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update the get command to follow more conventions of commands
Pure code movement, builds on top of #54446 and only the last commit is new. Will make refactoring get easier.
Automatic merge from submit-queue (batch tested with PRs 54042, 54185, 54880). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Inventory of kubectl dependency on main repository
**What this PR does / why we need it**:
Add an import-boss file to inventory what kubectl depends on, so
we can prune this list over time and eventually get kubectl out
of the main k/k repository.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Related to https://github.com/kubernetes/kubectl/issues/114
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54042, 54185, 54880). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
refactor ReplicaSet sync call tree
**What this PR does / why we need it**:
This PR refactors ReplicaSet sync call tree by refactoring `manageReplicas` and `syncReplicaSet` functions into smaller functions, and adding unit tests to each of the smaller functions.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: xref #52118
**Release note**:
```release-note
NONE
```
**TODO**:
- `manageReplicas`
- [x] move both outer and inner `batchSize` loops to a helper function named `slowStartBatch`, and test the function
- [x] add a helper function returning a list named `podsToDelete`, test the function, and refactor `DeletePod` loop to use the list
- [x] refactor skipped pod handling such that it happens after `slowStartBatch` returns
- `syncReplicaSet`
- [x] add unit tests for `calculateStatus`
- [x] move `canAdoptFunc` to a helper function
Automatic merge from submit-queue (batch tested with PRs 54894, 54630, 54828, 54926, 54865). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
set leveled logging (v=4) for 'updating container' message
**What this PR does / why we need it**:
Currently cpu_manager.go logs a line for every pod at every reconcilePeriod (10 sec default) when it reconciles and updates the pod's cpuset setting. This creates a lot of logging information that is not very interesting and we should suppress that by default by increasing the logging level.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54804
**Special notes for your reviewer**:
I chose V(4) because that seems to be a popular level for messages at this detail. Happy to follow logging guideline if there is any.
**Release note**:
``` kubelet: cpu_manager logs informative reconcile message at V(4) to reduce clutter ```
