Automatic merge from submit-queue
V2resource fixes
when using kubectl set resources it resets all resource fields that are not being set.
for example
$ kubectl set resources deployments nginx --limits=cpu=100m
followed by
$ kubectl set resources deployments nginx --limits=memory=256Mi
would result in the nginx deployment only limiting memory at 256Mi with the previous
limit placed on the cpu being wiped out. This behavior is corrected so that each invocation
only modifies fields set in that command and changed the testing so that the desired behavior
is checked.
Also a typo:
you must specify an update to requests or limits or (in the form of --requests/--limits)
corrected to
you must specify an update to requests or limits (in the form of --requests/--limits)
Implemented both the dry run and local flags.
Added test cases to show that both flags are operating as intended.
Removed the print statement "running in local mode" as in PR#35112
The original PR associated with these fixes where reverted due to causing a flake in hack/make-rules/test-cmd.sh, I gave the 'kubectl set resources' tests there own deployment and set the terminationGracePeriodSeconds to 0 and have run test-cmd.sh for hours without hitting the flake
Automatic merge from submit-queue
Node Conformance Test: Add system verification
For #30122 and #29081.
This PR introduces system verification test in node e2e and conformance test. It will run before the real test. Once the system verification fails, the test will just fail. The output of the system verification is like this:
```
I0909 23:33:20.622122 2717 validators.go:45] Validating os...
OS: Linux
I0909 23:33:20.623274 2717 validators.go:45] Validating kernel...
I0909 23:33:20.624037 2717 kernel_validator.go:79] Validating kernel version
KERNEL_VERSION: 3.16.0-4-amd64
I0909 23:33:20.624146 2717 kernel_validator.go:93] Validating kernel config
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
I0909 23:33:20.679328 2717 validators.go:45] Validating cgroups...
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
I0909 23:33:20.679454 2717 validators.go:45] Validating docker...
DOCKER_GRAPH_DRIVER: aufs
```
It verifies the system following a predefined `SysSpec`:
``` go
// DefaultSysSpec is the default SysSpec.
var DefaultSysSpec = SysSpec{
OS: "Linux",
KernelVersion: []string{`3\.[1-9][0-9].*`, `4\..*`}, // Requires 3.10+ or 4+
// TODO(random-liu): Add more config
KernelConfig: KernelConfig{
Required: []string{
"NAMESPACES", "NET_NS", "PID_NS", "IPC_NS", "UTS_NS",
"CGROUPS", "CGROUP_CPUACCT", "CGROUP_DEVICE", "CGROUP_FREEZER",
"CGROUP_SCHED", "CPUSETS", "MEMCG",
},
Forbidden: []string{},
},
Cgroups: []string{"cpu", "cpuacct", "cpuset", "devices", "freezer", "memory"},
RuntimeSpec: RuntimeSpec{
DockerSpec: &DockerSpec{
Version: []string{`1\.(9|\d{2,})\..*`}, // Requires 1.9+
GraphDriver: []string{"aufs", "overlay", "devicemapper"},
},
},
}
```
Currently, it only supports:
- Kernel validation: version validation and kernel configuration validation
- Cgroup validation: validating whether required cgroups subsystems are enabled.
- Runtime Validation: currently, only validates docker graph driver.
The validating framework is ready. The specific validation items could be added over time.
@dchen1107
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Rename PetSet to StatefulSet in docs and examples.
**What this PR does / why we need it**: Addresses some of the pre-code-freeze changes for implementing the PetSet --> StatefulSet rename. (#35534)
**Special notes for your reviewer**: This PR only changes docs and examples, as #35731 hasn't been merged yet and I don't want to create merge conflicts. I'll open another PR for any remaining code changes needed after that PR is merged. /cc @erictune @janetkuo @chrislovecnm
Automatic merge from submit-queue
New command: "kubeadm token generate"
As part of #33930, this PR adds a new top-level command to kubeadm to just generate a token for use with the init/join commands. Otherwise, users are left to either figure out how to generate a token on their own, or let `kubeadm init` generate a token, capture and parse the output, and then use that token for `kubeadm join`.
At this point, I was hoping for feedback on the CLI experience, and then I can add tests. I spoke with @mikedanese and he didn't like the original propose of `kubeadm util generate-token`, so here are the runners up:
```
$ kubeadm generate-token # <--- current implementation
$ kubeadm generate token # in case kubeadm might generate other things in the future?
$ kubeadm init --generate-token # possibly as a subcommand of an existing one
```
Currently, the output is simply the token on one line without any padding/formatting:
```
$ kubeadm generate-token
1087fd.722b60cdd39b1a5f
```
CC: @kubernetes/sig-cluster-lifecycle
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
New kubeadm command: generate-token
```
Automatic merge from submit-queue
Add --force to kubectl delete and explain force deletion
--force is required for --grace-period=0. --now is == --grace-period=1.
Improve command help to explain what graceful deletion is and warn about
force deletion.
Part of #34160 & #29033
```release-note
In order to bypass graceful deletion of pods (to immediately remove the pod from the API) the user must now provide the `--force` flag in addition to `--grace-period=0`. This prevents users from accidentally force deleting pods without being aware of the consequences of force deletion. Force deleting pods for resources like StatefulSets can result in multiple pods with the same name having running processes in the cluster, which may lead to data corruption or data inconsistency when using shared storage or common API endpoints.
```
Automatic merge from submit-queue
kubectl: make edit work with lists again
@kubernetes/kubectl this is fixing https://github.com/kubernetes/kubernetes/issues/20519 and slightly changes the behavior of --recursive when the directory that is being edited has files with errors. Previously since `edit` was working on an object basis, bad objects would be skipped and the editor would load the next object. We want to load multiple objects in the same list and it's impossible to load invalid objects in a list so --recursive will not work if there is any error in the directory. I think this is an acceptable trade-off.
Review here: https://github.com/kubernetes/kubernetes/pull/36148/files?w=1
Added test cases to show that both flags are operating as intended.
Removed the print statement "running in local mode" as in PR#35112
The previous attempt at the PR (PR#35050) was reverted for causeing a flake.
I believe that setting the deployments terminationGracePeriodSeconds to 0 should
take care of it. I ran hack/make-rules/test-cmd.sh 50 times in a row without encountering
the flake
--force is required for --grace-period=0. --now is == --grace-period=1.
Improve command help to explain what graceful deletion is and warn about
force deletion.
Automatic merge from submit-queue
SELinux Overhaul
Overhauls handling of SELinux in Kubernetes. TLDR: Kubelet dir no longer has to be labeled `svirt_sandbox_file_t`.
Fixes#33351 and #33510. Implements #33951.
Automatic merge from submit-queue
fixed some issues with kubectl set resources
when using kubectl set resources it resets all resource fields that are not being set.
for example
# kubectl set resources deployments nginx --limits=cpu=100m
followed by
# kubectl set resources deployments nginx --limits=memory=256Mi
would result in the nginx deployment only limiting memory at 256Mi with the previous
limit placed on the cpu being wiped out. This behavior is corrected so that each invocation
only modifies fields set in that command and changed the testing so that the desired behavior
is checked.
Also a typo:
you must specify an update to requests or limits or (in the form of --requests/--limits)
corrected to
you must specify an update to requests or limits (in the form of --requests/--limits)
Implemented both the dry run and local flags.
Added test cases to show that both flags are operating as intended.
Removed the print statement "running in local mode" as in PR#35112
Automatic merge from submit-queue
Verify and update client-go staging area for every PR
We need to keep the staging area up-to-date to prevent PRs from breaking client-go.
It's marked as "WIP" because we need to decide the [versioning strategy](https://github.com/kubernetes/client-go/issues/9) for client-go first. This PR contains breaking changes for client-go.
This is blocking #29934 and potentially #34441
cc @kubernetes/sig-api-machinery
Automatic merge from submit-queue
Make overlapping deployments deletable
@kubernetes/deployment ptal
Fixes https://github.com/kubernetes/kubernetes/issues/34466 by 1) not adding the overlapping annotation in the working deployment, 2) updates observedGeneration for overlapping deployments, and 3) updates the kubectl deployment reaper to do non-cascading deletion for deployments with the overlapping annotation.
Automatic merge from submit-queue
support editing before creating resource
Support `kubectl create -f config.yaml --edit`
Support editing before creating resource from files, urls and stdin.
The behavior is similar to `kubectl edit`
It won't create anything when edit make no change.
partial: #18064
Based on: #33686 and #33973
```release-note
Support editing before creating resource from files, urls and stdin, e.g. `kubectl create -f config.yaml --edit`
It won't create anything when edit make no change.
```
when using kubectl set resources it resets all resource fields that are not being set.
for example
# kubectl set resources deployments nginx --limits=cpu=100m
followed by
# kubectl set resources deployments nginx --limits=memory=256Mi
would result in the nginx deployment only limiting memory at 256Mi with the previous
limit placed on the cpu being wiped out. This behavior is corrected so that each invocation
only modifies fields set in that command and changed the testing so that the desired behavior
is checked.
Also a typo:
you must specify an update to requests or limits or (in the form of --requests/--limits)
corrected to
you must specify an update to requests or limits (in the form of --requests/--limits)
changelog:
- fixed a typo in hack/make-rules/test-cmd.sh "effecting" to "affecting"
Lots of places used sort (or sort -u) but didn't set LC_ALL=C. This
means it could be slightly different on different people's systems. Make
it deterministic.
Automatic merge from submit-queue
add optional mutation checks for shared informer cache
We need to make sure that no one is mutating caches if they're using a shared informer. It is important that whatever is tracking those changes gets the object *before* anyone else possibly could.
This adds the ability to track the original objects in the cache and their current values. Go doesn't have an exit hook or a way to say "wait for non-daemon go-funcs to complete before exit", so this runs a gofunc on a loop that can panic the entire process. It's gated behind an env var.
@derekwaynecarr did I get the right spots to make sure that e2e runs with this flag?
@smarterclayton @kubernetes/rh-cluster-infra
Automatic merge from submit-queue
kubectl: apply prune should fallback to basic delete when a resource has no reaper
Fixes#34790
cc @kubernetes/kubectl @MrHohn
Automatic merge from submit-queue
Add global timeout flag
**Release note**:
```release-note
Add a new global option "--request-timeout" to the `kubectl` client
```
UPSTREAM: https://github.com/kubernetes/client-go/pull/10
This patch adds a global timeout flag (viewable with `kubectl -h`) with
a default value of `0s` (meaning no timeout).
The timeout value is added to the default http client, so that zero
values and default behavior are enforced by the client.
Adding a global timeout ensures that user-made scripts won't hang for an
indefinite amount of time while performing remote calls (right now, remote
calls are re-tried up to 10 times when each attempt fails, however, there is
no option to set a timeout in order to prevent any of these 10 attempts from
hanging indefinitely).
**Example**
```
$ kubectl get pods # no timeout flag set - default to 0s (which means no
timeout)
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$ kubectl get pods --request-timeout=0 # zero means no timeout no timeout flag set
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$kubectl get pods --request-timeout=1ms
Unable to connect to the server: net/http: request canceled while
waiting for connection (Client.Timeout exceeded while awaiting headers)
```
This patch adds a global timeout flag (viewable with `kubectl -h`) with
a default value of `0s` (meaning no timeout).
The timeout value is added to the default http client, so that zero
values and default behavior are enforced by the client.
**Example**
```
$ kubectl get pods # no timeout flag set - default to 0s (which means no
timeout)
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$ kubectl get pods --timeout=0 # zero means no timeout no timeout flag set
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$kubectl get pods --timeout=1ms
Unable to connect to the server: net/http: request canceled while
waiting for connection (Client.Timeout exceeded while awaiting headers)
```
Add a way to set resource limits/requests on running pods
Ref: https://github.com/kubernetes/kubernetes/issues/21648
I squashed the commits to make rebasing easier
Change log:
- fixed a typo that caused the command to be run with kubectl set set instead of the correct kubectl set limit
- added a ResourcesWithPodTemplates to pkg/kubectl/cmd/util/factory.go
instead of hardcoding these resources move there description all in one place
- Fixing some of the flow control in kubectl set limit
- update the help info
- changed the name of ResourcesWithPodTemplates to ResourcesWithPodSpecs to more accuratly describe what it is doing
and changed the variable names to lower case to conform to go's variable naming convention
- changing the name of the command from 'set limit' to 'set resources'
- Adding the new file pkg/kubectl/cmd/set/set_resources.go
- changes to the test cases to reflect the change from 'kubectl set limit' to 'kubectl set resources'
- comment removed
- adding the man page to the git repository attempting to fix Jenkins tests
- adding the user guide
- fixed a few typos
- typo in hack/cmd-test.sh
- implamenting suggestions for command help text
- adding the dry-run flag
- removing the "remove" option in favor of zeroing out request/limits in order to remove them
- changed limits/requests to requests/limit
- changing ResourcesWithPodSpec
- updated generated docs and removed whitespace
- change priint on success message from "resource limits/requests updated" to "resource requirements updated"
- minor rebasing issues - 'hack/test-cmd.sh' now passes
- cmdutil.PrintSuccess added another argument
- fixing mungedocs failure
- removed whitespace from hack/make-rules/test-cmd.sh and an erroneous entry from pkg/cloudprovider/providers/openstack/MAINTAINERS.md
- fixed typo in Short: field of the cobra command
- rebased
- Creating a new factory in the ResourcesWithPodSpecs() so that the testing will pass
- changing ResourcesWithPodSpecs, it doesn't need to be a method of factory
Automatic merge from submit-queue
controller: save older revisions for Deployment's replica sets
@jwforres the only usable way I could find for multiple old revisions for a single replica set is to stuff them as comma-separated values.
@kubernetes/deployment this retains old revisions served by a replica set inside an annotation.
Fixes https://github.com/kubernetes/kubernetes/issues/33844
Automatic merge from submit-queue
Ignore troublesome paths that cause coverage to fail
**What this PR does / why we need it**:
`KUBE_COVER=y make check` currently fails, this patch fixes it.
**Which issue this PR fixes**
fixes#31691
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
This avoids the whole command failing because of errors like the following:
```
# cover k8s.io/kubernetes/pkg/client/restclient
cover: internal error: block 268 overlaps block 270
```
Automatic merge from submit-queue
Update client config invalid option errors to be more specific
This patch adds better error handling for cases where a global option (such as --context or --cluster) causes an invalid config to be returned.
```release-note
release-note-none
```
This patch provides a more relevant error message when a client
configuration option is passed with an invalid or non-existent value.
`$ kubectl get pods --cluster="non-existent"`
```
error: No configuration file found, please login or point to an existing
file
```
`$ kubectl get pods --cluster="non-existent"`
```
error: cluster "non-existent" does not exist
```
Automatic merge from submit-queue
Allow empty annotation values
Related downstream issue: https://github.com/openshift/origin/issues/11175
**Release note**:
```release-note
release-note-none
```
Annotations with empty values can be used, for example, in diagnostics
logging. This patch removes the client-side check for empty values in
an annotation key-value pair.
**Before**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
error: invalid annotation format: node-selector=
```
**After**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
pod "zookeper-1" annotated
```
```
$ kubectl get po/zookeeper-1 --template='{{.metadata.annotations}}'
map[... node-selector: test-label:test]
```
@fabianofranz @liggitt
Annotations with empty values can be used, for example, in diagnostics
logging. This patch removes the client-side check for empty values in
an annotation key-value pair.
**Before**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
error: invalid annotation format: node-selector=
```
**After**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
pod "zookeper-1" annotated
```
```
$ kubectl get po/zookeeper-1 --template='{{.metadata.annotations}}'
map[... node-selector: test-label:test]
```
Automatic merge from submit-queue
Add local option to annotate
**What this PR does / why we need it**:
Add the --local option to be both consistent with other commands and so it can be used with "set selector" and "create service".
**Which issue this PR fixes**
Related: #7296
**Special notes for your reviewer**:
None
**Release note**:
```release-note
Add a new option "--local" to the `kubectl annotate`
```
Automatic merge from submit-queue
Add local option to the label command
**What this PR does / why we need it**:
Add the --local option to be both consistent with other commands and so it can be used with "set selector" and "create service".
**Which issue this PR fixes**
Related: #7296
**Special notes for your reviewer**:
None
**Release note**:
```release-note
Add a new option "--local" to the `kubectl label`
```
Automatic merge from submit-queue
Make node E2E tests more transparent
Add some logging and minor code reorg to make the node E2E tests a little more transparent and understandable.
Automatic merge from submit-queue
Integration tests: Migrate scheduler perf to the integration suite, s…
*Problem*
issues like https://github.com/kubernetes/kubernetes/pull/32384/ happen because we dont compile the integration tests as part of the build
*Solution*
This pr will make it so scheduler_perf is included in the integration tests, but it will run in short mode that only makes sure compilation happens . it also removes the over generalized test/component directory which only has one entry
```
test/integration/replicationcontroller
test/integration/scheduler
test/integration/scheduler
test/integration/scheduler_perf
test/integration/scheduler_perf
test/integration/secrets
test/integration/serviceaccount
test/integration/storageclasses
```
Automatic merge from submit-queue
Fix error message around gcloud calls in node e2e and gubernator
Fixes some janky error messages around gcloud calls.
Automatic merge from submit-queue
Set --alsologtostderr=true in integration test
Without the flag, no glog output are stored in the test results. The logs are useful for debugging flaky tests like https://github.com/kubernetes/kubernetes/issues/30228.
The change also reveals a lot of messages like `W0912 14:19:32.306719 25386 cacher.go:468] Terminating all watchers from cacher *api.LimitRange`, which doesn't seem right.
Automatic merge from submit-queue
Rollout dry run
**What this PR does / why we need it**: rollout undo add dry-run implementation
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#28219
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
rollout undo add dry-run option
```
Automatic merge from submit-queue
add group prefix matching for kubectl usage
Adds prefix matching for groups so that `kubectl get storageclass.storage` matches `storageclass.storage.k8s.io`.
@kubernetes/kubectl
Automatic merge from submit-queue
choose a particular directory test-integration
Enables `make test-integration WHAT=auth` or whatever particular integration test director you want to run.
This avoids the whole command failing because of errors like the following:
```
# cover k8s.io/kubernetes/pkg/client/restclient
cover: internal error: block 268 overlaps block 270
```
For vendor code, igone
For local code replace the switch with an if statement
Automatic merge from submit-queue
Namespace certificates API group
New API groups should follow best-practices for naming, including using DNS names within the k8s.io namespace
```release-note
The certificates API group has been renamed to certificates.k8s.io
```
Running tests rebuilds almost everything. Most unit tests do not need to be
rebuilt. This installs test artifacts and makes subsequent test runs as much
as 6x faster. I saw pkg/apiserver tests drop from 30+ seconds to 5 seconds.
The 'gcloud compute' is called everytime even if it is not needed.
When runnin node e2e tests on RHEL, the test-e2e-node.sh script is run
the way in which it takes execution path without gcloud invocation.
With the current code, gcloud is called everytime.
Thus introducing additional runtime dependency.
