Automatic merge from submit-queue
Make real proxier in hollow-proxy optional (default=true)
Ref https://github.com/kubernetes/kubernetes/pull/45622
This allows using real proxier for hollow proxy, but we use the fake one by default.
cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
Add explicit image tag to cockroachdb example and test
@gyliu513
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
Add integration test for deployment
We have no integration test for Deployment currently. In this PR, add an integration test which covers an e2e test (create a new RollingUpdate deployment), add more replicas to the Deployment, and set minReadySeconds so that we can test maxUnavailable.
Plan to add more integration tests that cover e2e tests after this initial PR is merged.
@kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)
[Federation] Add integration testing for cluster addition
This PR adds integration testing of the sync controller for cluster addition. This ensures coverage equivalency between the integration tests and the old controller unit tests, so those tests are removed by this PR.
Resolves#45257
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
Remove the deprecated --babysit-daemons kubelet flag
```release-note
Removes the deprecated kubelet flag --babysit-daemons
```
This flag has been deprecated for over a year (git blame says marked deprecated on March 1, 2016).
Relatively easy removal - nothing in the Kubelet relies on it anymore.
There was still some stuff in the provisioning scripts. It was easy to rip out, but in general we probably need to be more disciplined about updating the provisioning scripts at the same time that we initially mark things deprecated.
Automatic merge from submit-queue
Move all API related annotations into annotation_key_constants.go
Separate from #45869. See https://github.com/kubernetes/kubernetes/pull/45869#discussion_r116839411 for details.
This PR does nothing but move constants around :)
/assign @caesarxuchao
**Release note**:
```release-note
NONE
```
Federation of daemonset and secret types is now implemented by the
sync controller, and e2e testing for each type is provided via crud
lifecycle e2e tests. This renders the legacy e2e tests for these types
redundant, and this commit removes those tests.
Automatic merge from submit-queue (batch tested with PRs 44520, 45253, 45838, 44685, 45901)
Ensure ordering of using dynamic kubelet config and setting up tests.
This PR simply places the body of the eviction test within its own context. This ensures that the kubelet config is set before the pods are created, and that the kubelet config is reverted only after the pods are deleted.
Automatic merge from submit-queue (batch tested with PRs 45247, 45810, 45034, 45898, 45899)
Apiregistration v1alpha1→v1beta1
Promoting apiregistration api from v1alpha1 to v1beta1.
API Registration is responsible for registering an API `Group`/`Version` with
another kubernetes like API server. The `APIService` holds information
about the other API server in `APIServiceSpec` type as well as general
`TypeMeta` and `ObjectMeta`. The `APIServiceSpec` type have the main
configuration needed to do the aggregation. Any request coming for
specified `Group`/`Version` will be directed to the service defined by
`ServiceReference` (on port 443) after validating the target using provided
`CABundle` or skipping validation if development flag `InsecureSkipTLSVerify`
is set. `Priority` is controlling the order of this API group in the overall
discovery document.
The return status is a set of conditions for this aggregation. Currently
there is only one condition named "Available", if true, it means the
api/server requests will be redirected to specified API server.
```release-note
API Registration is now in beta.
```
Automatic merge from submit-queue (batch tested with PRs 45374, 44537, 45739, 44474, 45888)
fix typo
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45860, 45119, 44525, 45625, 44403)
[Federation] Move annotations and related parsing code as common code
This PR moves some code, which was duplicate, around as common code.
Changes the names of structures used for annotations to common names.
s/FederatedReplicaSetPreferences/ReplicaAllocationPreferences/
s/ClusterReplicaSetPreferences/PerClusterPreferences/
This can be reused in job controller and hpa controller code.
**Special notes for your reviewer**:
@kubernetes/sig-federation-misc
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45408, 45355, 45528)
Admission options spits out admission control
**What this PR does / why we need it**:
This PR adds ApplyTo method to AdmissionOptions struct. The method creates and initialises admission control to the server configuration.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41331, 45591, 45600, 45176, 45658)
Move client/unversioned/remotecommand to client-go
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.
Module remotecommand relies on util/exec module which was copied to client-go/pkg/util
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.
Module remotecommand relies on util/exec module which will be copied to client-go/pkg/util
ApplyTo adds the admission chain to the server configuration the method lazily initializes a generic plugin
that is appended to the list of pluginInitializers.
apiserver.Config will hold an instance of SharedInformerFactory to ensure we only have once instance.
The field will be initialized in apisever.SecureServingOptions
Automatic merge from submit-queue
Reorganize kubelet tree so apis can be independently versioned
@yujuhong @lavalamp @thockin @bgrant0607
This is an example of how we might reorganize `pkg/kubelet` so the apis it exposes can be independently versioned. This would also provide a logical place to put the `KubeletConfiguration` type, which currently lives in `pkg/apis/componentconfig`; it could live in e.g. `pkg/kubelet/apis/config` instead.
Take a look when you have a chance and let me know what you think. The most significant change in this PR is reorganizing `pkg/kubelet/api` to `pkg/kubelet/apis`, the rest is pretty much updating import paths and `BUILD` files.
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Promotes Source IP preservation for Virtual IPs from Beta to GA
Fixes#33625. Feature issue: kubernetes/features#27.
Bullet points:
- Declare 2 fields (ExternalTraffic and HealthCheckNodePort) that mirror the ESIPP annotations.
- ESIPP alpha annotations will be ignored.
- Existing ESIPP beta annotations will still be fully supported.
- Allow promoting beta annotations to first class fields or reversely.
- Disallow setting invalid ExternalTraffic and HealthCheckNodePort on services. Default ExternalTraffic field for nodePort or loadBalancer type service to "Global" if not set.
**Release note**:
```release-note
Promotes Source IP preservation for Virtual IPs to GA.
Two api fields are defined correspondingly:
- Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
- Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
```
Automatic merge from submit-queue (batch tested with PRs 45623, 45241, 45460, 41162)
Revert #45101 Mark PersistentVolumes as [Feature:Volumes]
**What this PR does / why we need it**:
Reverts #45101
`Feature` tag should only be used when a test/suite has dependencies not met by core CI. That is not the case for NFS backed PV tests.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45685, 45572, 45624, 45723, 45733)
Remove a test utility function that is redundant and kinda broken
Framework.WaitForAnEndpoint() has no timeout, so if something goes wrong and the endpoint doesn't get created, the test will hang forever. (This is happening for some reason sometimes in OpenShift right now, and when the CI system eventually times out and kills the VM, it loses the logs that would explain what failed.)
There's already another nearly-identical WaitForEndpoint() method that *does* take a timeout, so people can just use that instead.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45653, 45719, 45729, 45730, 44250)
Remove kubemark.sh as we don't use pod IP from it anymore
This has been pending for sometime now. We no longer seem to actually depend on the downwarp api for the pod IP (hollow-proxy for example now gets it using api call).
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45653, 45719, 45729, 45730, 44250)
Print pod startup latency metric as perfdata
Follows #45657
This should print pod startup latency in same format as api calls latencies.
cc @wojtek-t @gmarek
Automatic merge from submit-queue
Fix flag formatting errors in the node tests
There were three problems:
- Lack of a trailing space after prepending flags.
- Passing multiple flags in a string to --kubelet-flags seems to confuse
the flag parser; it stops parsing ALL flags as soon as it sees the
second kubelet flag. Fortunately, all instances of --kubelet-flags are
combined together, so we can just pass two of those.
- --feature-gates should be passed to the test framework, which then
forwards it to the kubelet, instead of using --kubelet-flags.
This hopefully fixes the dynamic config test failures on COS, which
started after #45602. (See: https://k8s-testgrid.appspot.com/google-node#kubelet-serial-gce-e2e)
Automatic merge from submit-queue
move from daemon_restart.go to framework/util.go
**What this PR does / why we need it**:
Moves the func `nodeExec` from daemon_restart.go to framework/util.go. This is the correct file for this func and is a more intuitive pkg for other callers to use. This is a small step of the larger effort of restructuring e2e tests to be more logically structured and easier for newcomers to understand.
```release-note
NONE
```
cc @timothysc @copejon
Automatic merge from submit-queue (batch tested with PRs 45691, 45667, 45698, 45715)
Make update_owners.py also emit a JSON sig-owners list.
This should experience less churn in general!
I'm going to make the triage page use this list to let sigs have individualized dashboards.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
detach the volume when pod is terminated
When pods are terminated we should detach the volume.
Fixes https://github.com/kubernetes/kubernetes/issues/45191
**Release note**:
```
Detach the volume when pods are terminated.
```
Automatic merge from submit-queue
Add properties file for cos-docker-validation test job
**What this PR does / why we need it**:
This is forked from test/e2e_node/jenkins/docker_validation/jenkins-validation.properties. It is used for COS docker validation test.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue
plumb stopch to post start hook index since many of them are starting go funcs
Many post-start hooks require a stop channel to properly terminate their go funcs.
@p0lyn0mial I think you need this for https://github.com/kubernetes/kubernetes/pull/45355 ptal.
@ncdc per request
@sttts can you review too since Andy is out?
There were three problems:
- Lack of a trailing space after prepending flags.
- Passing multiple flags in a string to --kubelet-flags seems to confuse
the flag parser; it stops parsing ALL flags as soon as it sees the
second kubelet flag. Fortunately, all instances of --kubelet-flags are
combined together, so we can just pass two of those.
- --feature-gates should be passed to the test framework, which then
forwards it to the kubelet, instead of using --kubelet-flags.
This hopefully fixes the dynamic config test failures on COS, which
started after #45602.
Automatic merge from submit-queue
small change to view more test info
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
small change to view more test info, think you very much
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45634, 45480)
Fix BY() format
**What this PR does / why we need it**:
i read other by(), just format, think you
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45569, 45602, 45604, 45478, 45550)
Minor bug fix in start-kubemark-master script
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45569, 45602, 45604, 45478, 45550)
Enable kernel memcg notification for node and cluster GCI/COS testing.
Sets --experimental-kernel-memcg-notification=true when running on the GCI/COS image. It sets this for master and nodes for cluster e2e tests, and for the node in node e2e tests.
Issue #42676
cc @dchen1107 @Random-Liu
Automatic merge from submit-queue
util.go: format for
**What this PR does / why we need it**:
format for.
delete redundant para.
make code clean.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43067, 45586, 45590, 38636, 45599)
Make SchedulerPredicates test more resiliant to recent Node restarts
cc @kubernetes/sig-scheduling-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43067, 45586, 45590, 38636, 45599)
Move rest of performance data gathered by tests to Summaries
cc @shyamjvs
Automatic merge from submit-queue (batch tested with PRs 43067, 45586, 45590, 38636, 45599)
Fix bug in hollow-node deletion in stop-kubemark script
Just noticed.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45453, 45307, 44987)
Migrate the docker client code from dockertools to dockershim
Move docker client code from dockertools to dockershim/libdocker. This includes
DockerInterface (renamed to Interface), FakeDockerClient, etc.
This is part of #43234
Automatic merge from submit-queue (batch tested with PRs 45304, 45006, 45527)
Fixed indent for some statefulset for e2e test.
**What this PR does / why we need it**:
Make sure the e2e test passed for statefulset.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45526
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45304, 45006, 45527)
Add node e2e tests for hostIPC
**What this PR does / why we need it**:
Add node e2e tests for hostIPC.
**Which issue this PR fixes**
Part of #44118.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @Random-Liu @yujuhong
Automatic merge from submit-queue (batch tested with PRs 44798, 45537, 45448, 45432)
Stream output of run-gcloud-compute-with-retries to stdout in realtime
Ref https://github.com/kubernetes/kubernetes/issues/40139#issuecomment-299894222 (3rd point)
This should help us get more info about timeouts during start-kubemark-master.sh.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45481, 45463)
ThirdPartyResource example: added watcher example, code cleanup
**NOTE**: This is a cleaned and updated version of PR https://github.com/kubernetes/kubernetes/pull/43027
**What this PR does / why we need it**:
An example of using go-client for watching on ThirdPartyResource events (create/update/delete).
Automatic merge from submit-queue (batch tested with PRs 45200, 45203)
Add .svc.cluster.local for service host
**What this PR does / why we need it**:
Running into DNS resolve problems, testing to see if this works better
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45508, 44258, 44126, 45441, 45320)
Removed 'default' row from test_owners.csv and Updated update_owners.py
**What this PR does / why we need it**:
Removes the 'default' row from test_owners.csv and the validation/update logic associated with it in update_owners.py.
The 'default' row is being removed because it results in too many issues being assigned to the default test owners when issues are automatically generated.
**Release note**:
```release-note
NONE
```
/assign
Automatic merge from submit-queue (batch tested with PRs 41903, 45311, 45474, 45472, 45501)
Removed old scheduler constructor.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A
**Release note**:
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)
Fix gcloud retries cmd to rightly capture return code
Fixes https://github.com/kubernetes/kubernetes/issues/45489
Seems like the issue was in doing this:
```
local var=$(cmd)
local ret_code=$? #This actually captures return code of "local" above and not "cmd"
```
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)
Separate healthz server from metrics server in kube-proxy
From #14661, proposal is on kubernetes/community#552.
Couple bullet points as in commit:
- /healthz will be served on 0.0.0.0:10256 by default.
- /metrics and /proxyMode will be served on port 10249 as before.
- Healthz handler will verify timestamp in iptables mode.
/assign @nicksardo @bowei @thockin
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
refactor names for the apiserver handling chain
The names and structure around the handling chain got a bit confused. This simplifies it back out into a single struct with three parts: overall handler, gorestful handler, pathrecording mux and makes the delegate wiring simpler
Automatic merge from submit-queue
Clean up petset
**What this PR does / why we need it**:
Rename legacy petset to statefulset.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
Use dedicated UnixUserID and UnixGroupID types
**What this PR does / why we need it**:
DRYs up type definitions by using the dedicated types in apimachinery
**Which issue this PR fixes**
#38120
**Release note**:
```release-note
UIDs and GIDs now use apimachinery types
```
Automatic merge from submit-queue (batch tested with PRs 44590, 44969, 45325, 45208, 44714)
remove mvPodUidDir test
**What this PR does / why we need it**:
Removes a [HostCleanup] test which deleted the pod's uid directory and checked to see that the host was cleaned up. This test consistently failed on gci (COS) due to OS differences related to handling mounting (containerized on COS) and rm'ing files being accessed. See #44592 for more details.
```release-note
NONE
```
cc. @msau42 @copejon
Automatic merge from submit-queue (batch tested with PRs 45364, 45369, 45323)
Fix namespace deletion log in e2e test framework.
**What this PR does / why we need it**:
Current log message for namespace deletion will show under wrong condition.
**Which issue this PR fixes**
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 45316, 45341)
Move hollow-node's initContainer from annotation to field
In 1.6, we have moved init containers spec from beta annotation to field. Updating this in kubemark hollow-node.
This should solve issues with setting of default fields (like ImagePullPolicy) in the container spec for older versions of apiservers too (as annotation->field change has been cherry picked for older releases too). We noticed this issue in https://github.com/kubernetes/kubernetes/issues/45290 where an old apiserver apparently wasn't setting default ImagePullPolicy for container spec inside annotations.
cc @wojtek-t @gmarek @gamename
Automatic merge from submit-queue
Deployment e2e fixes
* don't expect availability on Deployment creations
* switch to framework.Logf everywhere for consistency
* better use of Expect
Automatic merge from submit-queue
Fix golint verify error
I don't know why CI pass the hack/verify-golint.sh test.
But in my environment I get this:
> staging/src/k8s.io/client-go/util/workqueue/queue_test.go is in package workqueue_test, not workqueue
Errors from golint:
test/integration/apiserver/apiserver_test.go:102:13: should omit type string from declaration of var cascDel; it will be inferred from the right-hand side
Please fix the above errors. You can test via "golint" and commit the result.
!!! Error in hack/verify-golint.sh:98
Error in hack/verify-golint.sh:98. 'false' exited with status 1
Call stack:
1: hack/verify-golint.sh:98 main(...)
Exiting with status 1
This change fix this err in my environment.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 45120, 45243)
skip integration test when run make bazel-test
we should opt for a seperate target for integration tests. This is breaking @deads2k who is trying to add an integration test in staging.
Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)
Move federated ingress e2e tests that don't pass to serial suite to avoid running it in presubmits.
This is due to a bug described here: https://github.com/kubernetes/ingress/issues/645.
Federated Ingress e2es fail when there is a high load which is pretty common in the presubmit environment when the PR activity is high. So we are disabling these tests to unblock federation presubmits.
**Release note**:
```release-note
NONE
```
/assign @csbell
cc @kubernetes/sig-federation-pr-reviews @nicksardo
Automatic merge from submit-queue
Add e2e network policy tests.
**What this PR does / why we need it**:
Add set of e2e tests for Network Policy. This has succesfully run againast a deployment using Calico as the network policy provider.
Specifically, adds a new e2e test file (/test/e2e/network_policy.go) which tests TCP connectivity between pods with isolation.
See #25638 for PR that added NetworkPolicy resource.
This PR is a replacement for #27447 .
Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)
Increase pv test ns deletion timeout to the framework's default.
**What this PR does / why we need it**:
Namespace deletion occasionally exceeds the test's timeout of 3 minutes. The test framework uses a default timeout of 5 minutes, so just making it match here.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44790
**Special notes for your reviewer**:
**Release note**:
NONE
Automatic merge from submit-queue
adds Ubuntu node e2e test
**What this PR does / why we need it**:
Add node e2e test for Ubuntu image.
**Special notes for your reviewer**:
**Release note**:
```NONE```
Automatic merge from submit-queue (batch tested with PRs 44062, 45165)
Run SSH e2e test only if SSH keys are present
**What this PR does / why we need it**:
Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
don't use build tags to mark integration tests
This is a bad pattern that leads to checked in code that isn't check for compilation. We should avoid this if it doesn't provide value, which it seems like it doesn't.
Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.
Automatic merge from submit-queue
Add google.com to e2e test only under gce/gke
We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.
Automatic merge from submit-queue (batch tested with PRs 41583, 45117, 45123)
Allow `make test-integration` to pass on OSX
**What this PR does / why we need it**: `make test-integration` isn't passing on my OSX setup (10.11.6, go1.8.1, 17.05.0-ce-rc1). Tests that startup an api server fail because the default `cert-dir` of `/var/run/kubernetes` isn't world-writable. Use a tempdir instead.
**Release note**:
```release-note
NONE
```
ref: #41595
/cc @kubernetes/sig-testing-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 44044, 44766, 44930, 45109, 43301)
e2e test: test azure disk volume
**What this PR does / why we need it**:
E2E test Azure disk volume
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
@kubernetes/sig-testing-pr-reviews
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
e2e: Remove special serviceaccount permission requirement
**What this PR does / why we need it**:
This removes the requirement for having a `default` serviceaccount with permission `create` `pod` in order to run the prestop e2e conformance test.
The `create` `pod` permissions are not needed for this particular test, and having it effectively means that as cluster must have a `default` serviceaccount with `create` `pod` permissions in order to be conformant, which I don't think is desired.
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
`/var/run` is not world-writable on my OSX 10.11.x setup, so tests that
standup a secure apiserver fail with the default cert dir. Use a
tempdir instead.
Automatic merge from submit-queue (batch tested with PRs 45033, 44961, 45021, 45097, 44938)
Disable the kubelet part of metrics collection in kubemark
Fixes https://github.com/kubernetes/kubernetes/issues/45038
This should fix it, as we are just interested in getting the apiserver metrics from kubemark master.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45033, 44961, 45021, 45097, 44938)
Add request count to APICall metric
Ref https://github.com/kubernetes/kubernetes/issues/44701
This should add beside the API call latencies, the count of the requests.
cc @wojtek-t @gmarek
Automatic merge from submit-queue
kubectl binary plugins
**What this PR does / why we need it**:
Introduces the ability to extend `kubectl` by adding third-party plugins that will be exposed through `kubectl`.
Plugins are executable commands written in any language. To be included as a plugin, a binary or script file has to
1. be located under one of the supported plugin path locations:
1.1 `~/.kubectl/plugins` dir
1.2. one or more directory set in the `KUBECTL_PLUGINS_PATH` env var
1.3. the `kubectl/plugins` dir under one or more directory set in the `XDG_DATA_DIRS` env var, which defaults to `/usr/local/share:/usr/share`
2. in any of the plugin path above, have a subfolder with the plugin file(s)
3. in the subfolder, contain at least a `plugin.yaml` file that describes the plugin
Example:
```
$ cat ~/.kube/plugins/myplugin/plugin.yaml
name: "myplugin"
shortDesc: "My plugin's short description"
command: "echo Hello plugins!"
$ kubectl myplugin
Hello plugins!
```
~~In case the plugin declares `tunnel: true`, the plugin engine will pass the `KUBECTL_PLUGIN_API_HOST` env var when calling the plugin binary. Plugins can then access the Kube REST API in "http://$KUBECTL_PLUGIN_API_HOST/api" using the same context currently in use by `kubectl`.~~
Test plugins are provided in `pkg/kubectl/plugins/examples`. Just copy (or symlink) the files to `~/.kube/plugins` to test.
**Which issue this PR fixes**:
Related to the discussions in the proposal document: https://github.com/kubernetes/kubernetes/pull/30086 and https://github.com/kubernetes/community/pull/122.
**Release note**:
```release-note
Introduces the ability to extend kubectl by adding third-party plugins. Developer preview, please refer to the documentation for instructions about how to use it.
```
* don't expect availability on Deployment creations
* switch to framework.Logf everywhere for consistency
* better use of Expect
Signed-off-by: Michail Kargakis <mkargaki@redhat.com>
Just so that we know that we need a cloud provider that
supports volumes to run this test. This is similar to
the change in 63bc42c872.
Ran into this when i was trying to run e2e tests with
local-up-cluster locally and figured out this test will
not work since we don't support local storage persistent
volumes.
Automatic merge from submit-queue (batch tested with PRs 45052, 44983, 41254)
[Federation][e2e] Add 2 new testcases to federation service e2e
**What this PR does / why we need it**:
Add 2 new test cases for federation services
- Federation service updation should update clustered service
- Federation service controller should recreate service shard in cluster, if it gets deleted.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#27623, #35827
Handles one of the tasks discussed in #41253
**Special notes for your reviewer**:
**Release note**:
`NONE`
cc @kubernetes/sig-federation-bugs, @nikhiljindal @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 45052, 44983, 41254)
Non-controversial part of #44523
For easier review of #44523, i extracted the non-controversial part out to this PR.
Automatic merge from submit-queue
Fix cockroachdb statefulset test read/write commands
Explicitly specifying `--insecure` is required on insecure clusters,
which started being enforced in a very recent release. In 2 weeks
we'll have a stable image version that we can reliably pin the
relevant statefulset yaml file to in order to avoid stupid failures
like this. I'm really sorry for the flakes!
**What this PR does / why we need it**:
It fixes the currently broken statefulset test suite - https://storage.googleapis.com/k8s-gubernator/triage/index.html?job=gci-gce-statefulset&test=CockroachDB
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
N/A
**Special notes for your reviewer**:
N/A
**Release note**:
```release-note
NONE
```
@kow3ns
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)
[Federation] Convert Daemonset to use the generic sync controller
To be rebased on master when @perotinus's configmaps PR merges.
Tested integration and e2e.
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)
Cleanup some of the tarball producing code for e2e node tests
This is some e2e node cleanup work I found sitting in a local branch while deleting old local git branches. It looks like it's still useful.
Automatic merge from submit-queue (batch tested with PRs 41106, 44346, 44929, 44979, 45027)
Log error before failing in autoscaling e2e
The gcloud alpha command in e2e fails, but no useful information (error message) is logged.
Automatic merge from submit-queue (batch tested with PRs 41106, 44346, 44929, 44979, 45027)
Make metrics filenames for e2e tests indicate the test better
Currently the names of the json files with metrics for e2e tests are named by appending the `SummaryKind` with a timestamp of the test. It took me some time to figure out which file corresponds to which e2e test due to this. Changing it to have the testname instead of the timestamp.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 44591, 44549)
[Federation][e2e] Fix a failing federation e2e testcase in gce-serial
This is to fix the failing test case in federation [gce-serial](https://k8s-testgrid.appspot.com/cluster-federation#gce-serial) tests. The test case has been failing consistently since we registered the clusters in suite-init instead of doing it in every test case.
Instead of registering and then unregistering, we will be now unregistering and then registering the cluster to federation. this test will be run in serial and will not affect other test cases too.
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Increase timeout for Stackdriver Logging e2e tests
They're failing in CI, because Stackdriver Logging's List method is too slow for this purpose. Quick fix, should be gone completely when reading is implemented properly
/cc @piosz
Automatic merge from submit-queue
e2e tests for VSAN policy support in Kubernetes for vSphere
Following e2e test cases have been implemented for VSAN policy support in Kubernetes for vSphere. These e2e tests are for PR #42974 which out for review.
A total of 8 test cases for below mentioned 5 different scenarios are implemented.
Test cases:
1. Validation of VSAN capabilities.
- hostFailuresToTolerate : Minimum 1 and Max 3. Should be integer.
- stripeWidth: Minimum is 1 and Maximum is 12. Should be integer.
- proportionalCapacity: Expressed in percentage. Should be between 0 and 100. Should be integer.
- iopsLimit: Should be greater than 0. Should be integer.
2. Use a VSAN testbed setup. Try valid VSAN capabilities which are supported by VSAN testbed. Make sure the disk is created with policy configured with it.
- Ex: Using hostFailuresToTolerate=0 and cacheReservation=12
Ex: diskStripes=1 and objectSpaceReservation=30
3. Use a VSAN testbed setup. Try valid VSAN capabilities which are not supported by VSAN testbed. Make sure that the disk creation fails and PVC status is pending.
4. Try using VSAN capabilities on a non-VSAN datastore. PVC status will be pending and it errors to the user saying to VSAN capabilities are not supported on a non-VSAN testbed.
5. Try all 1 to 5 with custom datastore specified by the user.
@jeffvance @divyenpatel
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 43395, 44960)
e2e-node: refactor lifecycle test to avoid selinux issues
Fixes#42905
Previously, the exec hook tests mounted a HostPath volume from /tmp and touched a file as a indicator that the hook had run. This is prohibited by selinux policy on Fedora/RHEL/Centos.
This PR refactors the test to avoid filesystem indication and use the same indication that the HTTP hooks use; a GET to a http endpoint. The exec hooks run `curl` to hit this endpoint and trigger the indication. This simplifies this test quite a bit as well, removing over 85 lines of code.
REVIEWER NOTE: The diff is a mess on this one. Probably better to just review the new version of the file.
@derekwaynecarr
Explicitly specifying `--insecure` is required on insecure clusters,
which started being enforced in a very recent release. In 2 weeks
we'll have a stable image version that we can reliably pin the
relevant statefulset yaml file to in order to avoid stupid failures
like this. I'm really sorry for the flakes!
Automatic merge from submit-queue
Cleanup: replace some hardcoded vars and remove unused functions
Replace some hardcoded vars and remove unused functions
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42477, 44462)
Use storage.v1 instead of v1beta1
storage.v1beta1 was used to work around GKE which did not expose v1. Now that GKE is updated, we can switch everything to v1.
This is simple sed v1beta1 -> v1 + enabled a new test + changed preference of exposed interfaces in `storage/install/install.go`.
@msau42, PTAL and let me know when GKE is updated with storage v1 API and this PR can be actually merged.
@kubernetes/sig-storage-pr-reviews
```release-note
NONE
```
Automatic merge from submit-queue
Move remain ESIPP tests to the slow suite
Continue PR of #38149.
It moves the remain ESIPP tests to the slow suite to help capture breaking changes.
/assign @thockin @freehan
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Stop treating in-cluster-config namespace as an override
Fixes#44835
The namespace of an in-cluster config should behave like the namespace specified in a kubeconfig file... it should be used as the default namespace, but be able to be overridden by namespaces specified in yaml files passed to `kubectl create -f`.
```release-note
Restored the ability of kubectl running inside a pod to consume resource files specifying a different namespace than the one the pod is running in.
```
The docker image is nowhere to be found, so lets remove it.
There have been a request for the Dockerfile here [1], but nobody
seems to care.
redis-proxy is replaced with redis-master in test-cmd-util.sh, to
ensure that the tests still works.
The redis-proxy pod in test/fixtures/doc-yaml/user-guide/multi-pod.yaml
is replaced with valid-pod from test/fixtures/doc-yaml/admin/limitrange/valid-pod.yaml,
so redis-proxy is removed every where.
[1] https://github.com/kubernetes/kubernetes/issues/4914#issuecomment-77209779
Automatic merge from submit-queue
Scheduler configurator looks for a specific key in ConfigMap.Data
**What this PR does / why we need it**: Changes scheduler configurator to look for a specific key in ConfigMap.Data instead of the old logic which expected only one entry to exist in the map. The key is a constant whose value is "policy.cfg".
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Explicit namespace from kubeconfig should override in-cluster config
Fixes#43662
If an explicitly specified namespace is read from a kubeconfig file, we should not fall back to in-cluster config
```release-note
kubectl commands run inside a pod using a kubeconfig file now use the namespace specified in the kubeconfig file, instead of using the pod namespace. If no kubeconfig file is used, or the kubeconfig does not specify a namespace, the pod namespace is still used as a fallback.
```
Automatic merge from submit-queue
Add TestEtcdStoragePath integration test
**What this PR does / why we need it**:
`TestEtcdStoragePath` prevents the accidental movement of objects stored in etcd. It creates a stub of each object and then checks the expected location in etcd. Inadvertent `GroupVersionKind` changes are prevented.
cc @liggitt @soltysh for review.
cc @ncdc because I know you love this 😄
Signed-off-by: Monis Khan <mkhan@redhat.com>
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Mark Stackdriver Logging e2e tests with a feature
Makes Stackdriver Logging e2e tests, except for the most basic one, run in the separate tests suites, prepared by https://github.com/kubernetes/test-infra/pull/2542
Automatic merge from submit-queue
De-Flake Volume E2E: force GCEPD detach to prevent timeout
**What this PR does / why we need it**:
Fix flake`[k8s.io] Volumes [Volume] [k8s.io] PD should be mountable [Flaky] 5m38s.
Flake occurs as a result of an automated detach taking longer than 5 minutes, which exceeds the timeout limit of the delete function.
This PR adds explicit detach and wait func calls before the deletion. By forcing the detach and giving GCE an appropriate timeout limit, this should squash the timeout flake. This also significantly shortens cleanup time.
This PR does not remove the [Flaky] tag. Once this PR is merged, I'll keep an eye on the test grid for ~1 week. If no flakes surface, I'll submit a PR to pull the tag off.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43977
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42177, 42176, 44721)
Job: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings Job into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that Job does not fight with other controllers over control of Pods.
Ref: #24433
**Special notes for your reviewer**:
**Release note**:
```release-note
Job controller now respects ControllerRef to avoid fighting over Pods.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42177, 42176, 44721)
CronJob: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings CronJob into compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that other controllers do not fight over control of objects that a CronJob owns.
**Special notes for your reviewer**:
**Release note**:
```release-note
CronJob controller now respects ControllerRef to avoid fighting with other controllers.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue
adding test for volume fstype validation
**What this PR does / why we need it**:
This PR is adding a test for volume fstype validation. Test verifies fstype specified in storage-class is being honored after volume creation.
Steps:
1. Create StorageClass with fstype set to valid type (default case included).
2. Create PVC which uses the StorageClass created in step 1.
3. Wait for PV to be provisioned.
4. Wait for PVC's status to become Bound.
5. Create pod using PVC on specific node.
6. Wait for Disk to be attached to the node.
7. Execute command in the pod to get fstype.
8. Delete pod and Wait for Volume Disk to be detached from the Node.
9. Delete PVC, PV and Storage Class.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
cc: @jeffvance @tusharnt
Automatic merge from submit-queue (batch tested with PRs 42272, 44696)
e2e test fix: Wait longer when first creating ELB
On any cloud (GCE or AWS), a lag between creating the LoadBalancer and
having it actually start serving traffic is expected. On AWS the lag is
larger, and we weren't correctly using the longer wait on our first
request.
Use a longer wait period on our first request.
Fix#44695
```release-note
NONE
```
Automatic merge from submit-queue
e2e: Prefer kubeconfig host to default
Previously it was necessary to pass ``-host`` to ``e2e.test`` even if ``-kubeconfig`` was specified since otherwise a localhost default would be used. This change ensures that the default is only used when kubeconfig is not set.
cc: @jayunit100
On any cloud (GCE or AWS), a lag between creating the LoadBalancer and
having it actually start serving traffic is expected. On AWS the lag is
larger, and we weren't correctly using the longer wait on our first
request.
Use a longer wait period on our first request.
Fix#44695
Automatic merge from submit-queue (batch tested with PRs 43500, 44073)
[Federation] Add option to retrieve e2e cluster config from secrets
Previously the federation e2e setup was reading member cluster configuration from the test run's kubeconfig. This change removes that dependency in favor of reading member cluster configuration from secrets in the hosting cluster, and caches the configuration to avoid having to read it separately for each test.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue
[Federation] Add simple upgrade test
This PR adds a simple upgrade test that targets all registered federated types.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue (batch tested with PRs 44222, 44614, 44292, 44638)
Update dummy certificate used in porter image
**What this PR does / why we need it**: the dummy certificate used for the porter image is incompatible with go1.8+ since it is missing NULL parameters for the RSA public key.
I haven't pushed the new porter image yet.
I verified that this appears to fix the issue. Based on https://github.com/kubernetes/kubernetes/issues/38228#issuecomment-280213590, for the old cert:
```console
$ openssl x509 -in localhost.crt -outform der | openssl asn1parse -inform der -i
...
105:d=4 hl=2 l= 9 prim: OBJECT :rsaEncryption
116:d=3 hl=2 l= 75 prim: BIT STRING
...
```
and the new cert:
```console
$ openssl x509 -in localhost.crt -outform der | openssl asn1parse -inform der -i
...
127:d=4 hl=2 l= 9 prim: OBJECT :rsaEncryption
138:d=4 hl=2 l= 0 prim: NULL
140:d=3 hl=4 l= 271 prim: BIT STRING
...
```
**Release note**:
```release-note
NONE
```
cc @liggitt @lavalamp @luxas
Automatic merge from submit-queue
namespace deletion and kubelet restart tests for vSphere cloud provider
**What this PR does / why we need it**:
Following tests for vSphere cloud provider are added as part of this PR
**Test to verifiy that a volume mounted to a pod that is deleted while the kubelet is down, unmounts volume when the kubelet returns**
**Steps:**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node. and verify volume is mounted on the node.
6. Stop kubelet.
7. Delete pod.
8. Start kubelet.
9. Verify that volume mount not to be found on the node.
**Test to verify that deleting the Namespace of a PVC and Pod causes the successful detach of Persistent Disk.**
**Steps:**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node.
6. Delete Namespace.
7. Wait for namespace to get deleted. (Namespace deletion should trigger deletion of belonging pods)
8. Verify volume should be detached from the node.
Moved test from https://github.com/kubernetes/kubernetes/pull/41976
**Test to verify that a volume mounted to a pod remains mounted after a kubelet restarts.**
1. Create volume (vmdk)
2. Create PV with volume path for the vmdk.
3. Create PVC to bind with PV.
4. Create a pod using the PVC.
5. Verify disk is attached to the node.
6. Write to the volume
7. Restart kubelet on the node on which pod is provisioned.
8. Verify that written file is accessible after kubelet restart.
9. Delete pod.
10. Delete PV, PVC
11. Wait for VMDK to detach from node.
12. Delete VMDK
**Which issue this PR fixes**
fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc: @abrarshivani @BaluDontu @tusharnt @pdhamdhere
Automatic merge from submit-queue (batch tested with PRs 44500, 44457, 44553, 44267, 44617)
nits in service e2e test logging
Saw some weird logs from CI [gce-alpha-features](https://k8s-testgrid.appspot.com/google-gce#gce-alpha-features&width=5).
```
...
Health checking bootstrap-e2e-minion-group-3tm9, http://35.188.44.206:32733//healthz, expectedSuccess true
...
Expected %!!(MISSING)s(int=2) failures on bootstrap-e2e-minion-group-8h0q//healthz, got 0, err
...
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43000, 44500, 44457, 44553, 44267)
Updates e2e_node test to allow both kubenet and cni to be specified f…
…or the network plugin.
This adds a simple CNI configuration which is added to the node during test setup.
This also modifies the default flags in services/kubelet.go to specify the "cni-bin-dir"
and the "cni-conf-dir" and removes the "network-plugin-dir" flag. This leaves the default
network plugin to kubenet.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
TestEtcdStoragePath prevents the accidental movement of objects stored
in etcd. It creates a stub of each object and then checks the expected
location in etcd. Inadvertent GroupVersionKind changes are prevented.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Add the option to configure e2e access to member clusters from the
same secrets in the host cluster used by the federation control plane.
The default behavior will continue to be sourcing this configuration
from the e2e kubeconfig. The optional behavior can be enabled by
passing --federation-config-from-cluster=true as an argument to
ginkgo.
Automatic merge from submit-queue (batch tested with PRs 44469, 44566, 44467, 44526)
WaitForCacheSync before running attachdetach controller
@gnufied you wrote the test and @ncdc the TODO comment. Let's just run the pv and pvc informers, we do not care about them in this test. But we want to be able to stop the pod Informer at will, hence not just using informers.Start, is my understanding.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)
improve error handling in e2e helpers
**What this PR does / why we need it**:
Changes most of the volume related helper funcs to return error rather than calling `Expect`. This is a better programming practice, is consistent with Go and Kubernetes, and allows helper funcs that create multiple resources to perform cleanup.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)
Change the default CLUSTER_IP_RANGE used by e2e
The existing choice intersects with the range reserved for auto
subnets and cannot be used with some GCP features.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44569, 44398)
Move v1/refs and v1/resource
This PR moves pkg/api/v1/ref.go and pkg/api/v1/resource_helper.go to their own sub packages, it's very similar to 44299 and 44302.
The PR is mostly mechanical, except that
* i moved some utility function from resource.go to pkg/api/v1/pod and pkg/api/v1/node, as they are more appropriate
* i updated the staging/copy.sh to copy the new subpackages, so that helper functions are copied. We can get rid of this copy after client-go stops copying API types.
Automatic merge from submit-queue (batch tested with PRs 44519, 43194, 44513)
[Federation] Add type-agnostic e2e crud test
This PR proposes an e2e test that reuses the type-agnostic crudtester already used for integration testing to validate crud against a deployed cluster. It is intended to to eventually replace the existing e2e tests for simple types like secrets, but for now will run in addition to the existing testing to gain confidence in the coverage it provides.
The deletion corner cases - when orphanDependents is nil or true - do not involve operations in the member clusters and are already well-tested in integration testing and so are not reimplemented here.
Where it can be applied, this approach of abstracting a test from its execution environment - making the test 'retargetable' - can reduce the cost of test development since the bulk of the work can be iterated on as an integration test. It can also serve as a check on assumptions made in the integration test about how a deployed environment will behave.
cc: @kubernetes/sig-federation-pr-reviews @kubernetes/sig-testing-misc @smarterclayton @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 44364, 44361, 42498)
Move v1 helpers
The first 3 commits are other PRs.
This PR move pkg/api/v1/helpers.go to a subpackage, which is almost symmetric to #44296, where pkg/api/helpers.go was moved.
This PR is mostly mechanic, except that
1. moved the 3 methods of Taint and Toleration to pkg/api/methods.go
2. moved constants and types defined in v1/helpers.go to pkg/api/v1/annotataion_key_constants.go and nonstandard_types.go
3. updated staging/copy.sh to copy pkg/api/helpers to client-go, it's otherwise removed from client-go because no other code in client-go depends on the package. Some test code in pkg/controller imports client-go/pkg/api/helpers. After moving api types to its own repo, we can remove these copies of utility function from client-go and ask users to use the ones in the main repo.
(This PR breaks a cyclic import problem i met when I tried to move global variables pkg/api/Scheme and Registry to a subpackage)
Automatic merge from submit-queue (batch tested with PRs 44362, 44421, 44468, 43878, 44480)
use sudo in mv ssh cmd
**What this PR does / why we need it**:
Fixes _HostCleanup_ kubelet e2e test where `sudo` was missing from a ssh `mv` command.
The exact test is:
`host cleanup with volume mounts [Volume][HostCleanup][Flaky] Host cleanup after disrupting NFS volume [NFS] move NFS client pod's UID directory then delete pod` This test has been consistently red in the gce-flaky suite.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44440, 44038, 44302, 44316, 43876)
Make resource gatherer get data about etcd resource usage in kubemark…
… setup
Automatic merge from submit-queue (batch tested with PRs 44424, 44026, 43939, 44386, 42914)
Try in cluster config when input KubeConfig is empty
**What this PR does / why we need it**:
Allows for downstream providers to run e2es "incluster" sans kubeconfig
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
/cc @kubernetes/sig-testing-pr-reviews @marun
Automatic merge from submit-queue
Segregate storage e2es for easier assignment and dep isolation.
**What this PR does / why we need it**:
Follow on from slack conversations to isolate storage deps and have easier assignment of e2es.
As of today there is general consensus that the e2es need to be broken apart, this is 1st iteration on the sig-storage test splitting. Follow on work is needed here.
**Release note**:
```
NONE
```
/cc @saad-ali @kubernetes/sig-storage-pr-reviews @kubernetes/sig-testing-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)
Enable service account token lookup by default
Fixes#24167
```release-note
kube-apiserver: --service-account-lookup now defaults to true, requiring the Secret API object containing the token to exist in order for a service account token to be valid. This enables service account tokens to be revoked by deleting the Secret object containing the token.
```
Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)
Decouple remotecommand
Refactored unversioned/remotecommand to decouple it from undesirable dependencies:
- term package now is not required, and functionality required to resize terminal size can be plugged in directly in kubectl
- in order to remove dependency on kubelet package - constants from kubelet/server/remotecommand were moved to separate util package (pkg/util/remotecommand)
- remotecommand_test.go moved to pkg/client/tests module
Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)
Clean up pre-ControllerRef compatibility logic
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#43323
**Special notes for your reviewer**:
No
**Release note**:
```
NONE
```
This adds a simple CNI configuration which is added to the node during test setup.
This also modifies the default flags in services/kubelet.go to specify the "cni-bin-dir"
and the "cni-conf-dir" and removes the "network-plugin-dir" flag. This leaves the default
network plugin to kubenet.
Automatic merge from submit-queue
add kubelet tests to verify host clean up
**What this PR does / why we need it**:
Increasingly we're seeing more failures in persistent volume e2e tests where pv tests are run in parallel with disruptive tests. The quick solution is to tag the pv tests as Flaky. This pr addresses one cause of the flakiness and adds a disruptive kubelet test.
Once this pr is shown to not produce flakes the [Flaky] tag for the "HostCleanup" tests will be removed in a separate pr.
+ Adds volume tests to _kubelet.go_ motivated by issues [31272](https://github.com/kubernetes/kubernetes/issues/31272) and [37657](https://github.com/kubernetes/kubernetes/issues/37657)
+ Addresses reverted pr [41178](https://github.com/kubernetes/kubernetes/pull/41178) and negates the need for pr [41229](https://github.com/kubernetes/kubernetes/pull/41229)
**Which issue this PR fixes**
Adds regression tests to cover issues: #31272 and #37657
**Special notes for your reviewer**:
It's possible that one of the new tests, which relies on the existence of _/usr/sbin/rpc.nfsd_ in the nfs-server pod, will not work in the GCI container env. If this turns out to be true then I will add a `SkipIfProviderIs("gke")` to the `It` block.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
[e2e] Bump up pod deletion time for source pod IP test
From #44225.
Source pod IP e2e test is pretty flaky lately, and most of the failures seem to be timeout waiting for pod "kube-proxy-mode-detector" to disappear. Didn't found any other thing suspicious.
This PR bumps pod deletion timeout to DefaultPodDeletionTimeout, which is 3 minutes. Hopefully it will mitigate the flakes.
/assign @freehan
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
local up dns defaults/Privileged defaults so that [Conformance] sets mostly pass on local clusters.
Fixes#43651 So that only 4 tests fail out of the box.
Automatic merge from submit-queue
Adding load balancer src cidrs to GCE cloudprovider
**What this PR does / why we need it**:
As of January 31st, 2018, GCP will be sending health checks and l7 traffic from two CIDRs and legacy health checks from three CIDS. This PR moves them into the cloudprovider package and provides a flag for override.
Another PR will need to be address firewall rule creation for external L4 network loadbalancing #40778
**Which issue this PR fixes**
Step one of #40778
Step one of https://github.com/kubernetes/ingress/issues/197
**Release note**:
```release-note
Add flags to GCE cloud provider to override known L4/L7 proxy & health check source cidrs
```
Automatic merge from submit-queue
In 'kubectl describe', find controllers with ControllerRef, instead of showing the original creator
@enisoc @kargakis @kubernetes/sig-apps-pr-reviews @kubernetes/sig-cli-pr-reviews
```release-note
In 'kubectl describe', find controllers with ControllerRef, instead of showing the original creator.
```
Automatic merge from submit-queue
Exit from NewController() for PersistentVolumeController when InitPlugins() failed
Exit from NewController() for PersistentVolumeController when InitPlugins() failed just like NewAttachDetachController() does
**Release note**:
```release-note
NONE
```
@jsafrane @saad-ali PTAL. Thanks in advance
Automatic merge from submit-queue
Move api helpers.go to a subpackage
Part of https://github.com/kubernetes/kubernetes/issues/44065.
This PR moves the pkg/api/helpers.go to its own subpackage. It's mostly a mechanic move, except that
* I removed ConversionError in helpers.go, it's not used by anyone
* I moved the 3 methods of Taint and Toleration to pkg/api/methods.go, and left a TODO saying refactoring these methods to functions.
I'll send a few more PRs to make the k8s.io/kubernetes/pkg/api package only contains the code we want in the k8s.io/api repo, then we can run a [script](a0015fd1be (diff-7a2fbb4371972350ee414c6b88aee1c8)) to cut the new repo.
Automatic merge from submit-queue
Remove alphaProvisioner in PVController and AlphaStorageClassAnnotation
remove alpha annotation and alphaProvisioner
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Remove [Flaky] from presistent volume NFS tests.
**What this PR does / why we need it**:
PV e2e test flaked because of a lack of isolation of test objects (PVs, claims). The common symptoms being volume-mounting pods timing out or an unexpected number of unbound claims being detected.
PR #43645 Introduced the selector labels to test objects that restricted binds to within each "testspace". To accomplish this the test namespace was set as a label for all PVs and selector for all Claims. This allows each test to act as if it's isolated while still creating and binding PVs in parallel w/ other tests.
This has been tested in parallel on both 3-node gci and debian clusters as
`--ginkgo.focus="\[Volume\]" --ginkgo.skip="\[Disruptive\]|\[Feature.*\]|\[Serial\]"`
with out signs of flakiness.
cc @jeffvance
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43304, 41427, 43490, 44352)
Update etcd-client godep to 3.1.5
This transitively level sets the godeps to yank in the 3.1.5 client.
Currently WIP, b/c it required some regen and I had some weird local permissions issue.
xref: #41143
/cc @xiang90 @mml
Automatic merge from submit-queue (batch tested with PRs 43304, 41427, 43490, 44352)
Node failure tests for cluster autoscaler
E2e tests checking whether CA is still working with a single broken node.
cc: @MaciekPytel @jszczepkowski @fgrzadkowski
Automatic merge from submit-queue (batch tested with PRs 43545, 44293, 44221, 43888)
make unstructured items correspond to other items for storage
"normal" `Items` elements include the struct itself, not a pointer to the struct. Some of the deeper bits of storage rely on this behavior in reflective paths.
This updates the `UnstructuredList` to be "normal".
@kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43844, 44284)
Add a retry to cluster-autoscaler e2e
This should fix https://github.com/kubernetes/kubernetes/issues/44268.
The flake was caused by following sequence of events:
1. Cluster was at minimum size (3), some node was unneeded for a while.
2. Setup for some test (scale-down, failure) would increase node group size (to 5) and wait for new nodes to come up.
3. As soon as new node come up (cluster size 4) CA would scale-down the old unneeded node (setting node group size to 4).
4. Node group would not reach size 5 (as the target was now 4) and the test would timeout and fail.
This PR makes the setup monitor re-set the target node group size if the above scenario happens.
Automatic merge from submit-queue (batch tested with PRs 43866, 42748)
hack/cluster: download cfssl if not present
hack/local-up-cluster.sh uses cfssl to generate certificates and
will exit it cfssl is not already installed. But other cluster-up
mechanisms (GCE) that generate certs just download cfssl if not
present. Make local-up-cluster.sh do that too so users don't have
to bother installing it from somewhere.
Automatic merge from submit-queue (batch tested with PRs 41758, 44137)
Removed hostname/subdomain annotation.
fixes#44135
```release-note
Remove `pod.beta.kubernetes.io/hostname` and `pod.beta.kubernetes.io/subdomain` annotations.
Users should use `pod.spec.hostname` and `pod.spec.subdomain` instead.
```
Automatic merge from submit-queue
test/e2e*: add/update README.md files
**What this PR does / why we need it**:
This PR is adding `README.md` files with a link to the documentation to all E2E tests.
Automatic merge from submit-queue (batch tested with PRs 44019, 42225)
federation: Fixing runtime-config support for federation-apiserver
Fixes https://github.com/kubernetes/kubernetes/issues/42587
Ref https://github.com/kubernetes/kubernetes/issues/38593
Fixing the broken `--runtime-config` flag support in federation-apiserver. Fixing the bugs and using it to disable batch and autoscaling groups. Users can enable them by passing `--runtime-config=apis/all=true` to federation-apiserver.
~This also includes a bug fix to kube-apiserver registry that allows users to disable api/v1 resources~
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
Scheduler can recieve its policy configuration from a ConfigMap
**What this PR does / why we need it**: This PR adds the ability to scheduler to receive its policy configuration from a ConfigMap. Before this, scheduler could receive its policy config only from a file. The logic to watch the ConfigMap object will be added in a subsequent PR.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```Add the ability to the default scheduler to receive its policy configuration from a ConfigMap object.
```
Automatic merge from submit-queue
Add node e2e tests for hostPid
For #44118. Add node e2e tests for hostPid.
cc @yujuhong @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue
Add test for provisioning with storage class
This PR re-introduces e2e test for dynamic provisioning with storage classes.
It adds the same test as it was merged in PR #32485 with an extra patch adding region to AWS calls. It works well on my AWS setup, however I'm using shared company account and I can't run kube-up.sh and run the tests in the "official" way.
@zmerlynn, can you please try to run tests that led to #34961?
@justinsb, you're my AWS guru, would there be a way how to introduce fully initialized AWS cloud provider into e2e test framework? It would simplify everything. GCE has it there, but it's easier to initialize, I guess. See https://github.com/kubernetes/kubernetes/blob/master/test/e2e/pd.go#L486 for example - IMO tests should not talk to AWS directly.
Automatic merge from submit-queue (batch tested with PRs 43373, 41780, 44141, 43914, 44180)
move PD delete to AfterEach
**What this PR does / why we need it**:
Fixes a bug in PersistentVolume E2E that causes GCEPDs to be left un-deleted after a test ends.
```release-note
None
```
Automatic merge from submit-queue
fix wrong error return
Signed-off-by: Crazykev <crazykev@zju.edu.cn>
**What this PR does / why we need it**: The err return here is wrong, correct it.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 44191, 44117, 44072)
[Federation] Cleanup e2e framework
This PR is intended to simplify maintenance of the federation e2e tests:
- refactor cluster helpers into framework so they can be reused by upgrade tests
- remove unused functions
- move backend pod management to the tests that require it rather than relying on the common cluster object
- use a slice instead of a map to manage cluster objects to make it simpler for tests that need it to find the primary cluster themselves
Commit ``fed: Refactor e2e cluster functions into framework for reuse`` was originally included in #43500, but is included here because it formed the basis for this cleanup.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue (batch tested with PRs 44191, 44117, 44072)
Fix pv upgrade test failure
**What this PR does / why we need it**:
2 changes were made that broke the pv upgrade test:
1. MakePod() was changed so that the volume mount point is at /mnt/volume<n> instead of /mnt. This change updates the path that the pod writes to.
2. The MakePod() call was changed to CreatePod(), which creates the pod and expects a long running pod. But the upgrade test uses a short running pod, and the call to TestContainerOutput() will also create the pod. This changes it back to MakePod()
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44168
**Special notes for your reviewer**:
**Release note**:
NONE
Automatic merge from submit-queue
[Federation] Add integration test for secrets
This PR adds an integration test for secrets that:
- performs create/read/update/delete on federation resources and validates that the changes are propagated to member clusters.
- uses an abstraction layer (fixture and adapter) to minimize the code required to support each federated type
- It should be possible to replace a test-specific adapter with a runtime adapter in the future (as per #41050)
- reuses fixture (federation api and clusters) across different resource types to minimize setup overhead
- on a fast machine, setup takes ~4s, and validating each type takes ~2s
- uses the [Subtest feature added in Go 1.7](https://blog.golang.org/subtests) to allow the test for a specific controller to be run in isolation
- ``make test-integration WHAT="federation -test.run=TestFederationCRUD/secret"``
Once this PR merges the test can be extended to target other federated types.
This PR targets #40705
cc: @kubernetes/sig-federation-pr-reviews @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 41189, 43818)
Reduce deployment replicas in e2e tests
Fixes https://github.com/kubernetes/kubernetes/issues/41063
There are still two tests that run multiple replicas (testScaledRolloutDeployment, testIterativeDeployments), we may want to revisit them at some point.
Automatic merge from submit-queue (batch tested with PRs 43871, 44053)
Fix original object mutation on patch retry
When applying a patch, the original state of the object and patch are captured so that retries can determine if an overlapping update was made to the object, and the patch API call should abort with a conflict.
However, the `strategicPatchObject -> applyPatchToObject -> StrategicMergeMapPatch -> mergeMap` call mutates both the original object map and the patch map, making them unsuitable for reusing for subsequent calls.
We saw this in a [downstream test](https://github.com/openshift/origin/blob/master/test/integration/patch_test.go) that exercises patch conflict retries, where the data being submitted in a patch was showing up in the original object data map.
Since mergeMap *also* mutates the patch map passed in (deletes patch directives as it encounters them), we *also* cannot reuse the patch map in `applyPatchToObject` once it has been used.
This PR:
* Builds `originalObjMap` separate from the initial patch application for later use in conflict detection
* Changes the `originalPatchMap` to a helper that returns a fresh map from original sources
* Adds an integration test that exercises retry of non-overlapping patches with patches containing `$patch` directives
Automatic merge from submit-queue (batch tested with PRs 44143, 44133)
Mark PD test as flaky.
**What this PR does / why we need it**: Marks "PD should be mountable" as flaky. See #43977, which shows that it has flaked at least 7 times in the last 3 days. @kubernetes/sig-storage-test-failures
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Extract e2e utility code into framework
**What this PR does / why we need it**:
There's a growing dependency on Volume e2e utilities related to creating / test against NFS volumes. For this reason, it's useful to relocate the relevant functions to the `framework` pkg. Doing so makes these utility functions available to e2e tests outside the `e2e` package.
This PR only moves code from the `e2e` package to `framework` and handle the relevant changes in calls. It does not change any logic.
```release-note
NONE
```
@jingxu97 I think there's value here in reducing duplicate code in the `common` package, given that these functions have been copied down to it. However, there's been some divergence. Can you PTAL and let me know if there's any reason we can't remove the duplicate `common` code?
cc @jeffvance
Automatic merge from submit-queue
Fix Cluster-Autoscaler e2e on testgrid
Fix an e2e test failing in CI. The failures were caused by maximum nodes in test env being lower than the number of nodes required by the test.
Automatic merge from submit-queue (batch tested with PRs 43963, 43965)
Wait for clean old RSs statuses in the middle of Recreate rollouts
After https://github.com/kubernetes/kubernetes/pull/43508 got merged, we started returning ReplicaSets with no pods but with stale statuses back to the rollout functions. As a consequence, one of our e2e tests that checks if a Recreate Deployment runs pods from different versions, started flakying because the Deployment status may be incorrect. This change simply waits for the statuses to get cleaned up before proceeding with scaling up the new RS.
Fixes https://github.com/kubernetes/kubernetes/issues/43864
@kubernetes/sig-apps-bugs
