Automatic merge from submit-queue
Pick a specific GCI version by default on GCE.
Prior to this change, a K8s branch (master as well as release) was
pinned to a GCI milestone. It would pick up the latest GCI release on
that milestone at the time of cluster creation. The rationale was the
K8s users would automatically get the bug fixes in newer versions of
GCI. However in practice, it makes the runtime environment
non-deterministic, and lack of continuous e2e tests mean we would run
into breakages sooner or later.
With this change, each K8s release will pick a specific version
of GCI by default (similar to how the Debian-based container-vm gets used).
Users can override the default version through KUBE_GCE_MASTER_IMAGE and
KUBE_GCE_NODE_IMAGE environment variables.
We expect the default GCI version will be updated relatively frequently stay
updated with newer GCI releases. We can also automate the process to
automatically bump the hard-coded GCI version in future.
@vishh @adityakali can you please review?
cc @kubernetes/goog-image FYI
Automatic merge from submit-queue
Create a file from data stored in gobindata to fix kubectl-based exam…
Fix#31539
Adding 1.4 milestone as this fixes P0 flake issue (test completely broken by moving to gobindata). @pwittrock
cc @jayunit100
Prior to this change, a K8s branch (master as well as release) was
pinned to a GCI milestone. It would pick up the latest GCI release on
that milestone at the time of cluster creation. The rationale was the
K8s users would automatically get the bug fixes in newer versions of
GCI. However in practice, it makes the runtime environment
non-deterministic, and lack of continuous e2e tests mean we would run
into breakages sooner or later.
With this change, each K8s release will pick a specific version
of GCI by default (similar to how the Debian-based container-vm gets used).
Users can override the default version through KUBE_GCE_MASTER_IMAGE and
KUBE_GCE_NODE_IMAGE environment variables.
We expect the default GCI version will be updated relatively frequently stay
updated with newer GCI releases. We can also automate the process to
automatically bump the hard-coded GCI version in future.
Automatic merge from submit-queue
test Metadata.ClusterName not saved into etcd
integration test that verifies that we are not storing ClusterName in etcd.
#28921
@nikhiljindal @deepak-vij @quinton-hoole
This commit enables the dynamic kubelet configuration feature for the
node e2e Jenkins serial tests, which is where the test for dynamic kubelet
configuration currently runs.
This gives the node e2e test binary a --feature-gates flag that populates a
FeatureGates field on the test context. The value of this field is forwarded
to the kubelet's --feature-gates flag and is also used to populate the global
DefaultFeatureGate object so that statically-linked components see the same
feature gate settings as provided via the flag.
This means that you can set feature gates via the TEST_ARGS environment
variable when running node e2e tests. For example:
TEST_ARGS='--feature-gates=DynamicKubeletConfig=true'
Automatic merge from submit-queue
Improve e2e framework namespace deletion
This PR addresses the following:
1. framework would delete same namespace multiple times in subsequent test if ns failed to delete in previous test. this caused incorrect error reporting on subsequent tests. updated framework to call delete on all namespaces, and then always clear out namespaces to delete.
1. deleteNs was not verifying all content was removed from the namespace, just pods. this made flakes hard to debug in tests that did not create pods and whose namespace didnt delete. updated framework to verify all content is removed from namespace.
1. improved debugging output when namespace did not delete with more detail on what remains.
This should stop the test from flaking while we figure out why there is
a mismatch between the reported pressure condition and the eviction
manager's decision to evict due to memory pressure.
Automatic merge from submit-queue
Make a scheduler predicates test resiliant to race for scheduledCondi…
Fix#31341
@pwittrock - this fixes a P1 flake.
FYI @mwielgus - I don't think that the race that caused this flake can impact cluster autoscaling, but you probably should know about it.
cc @wojtek-t
Automatic merge from submit-queue
Node E2E: Move host info around test result.
Discussed offline with @yujuhong and @dchen1107. Currently, the node e2e result is organized as:
```
================================================================
Success Finished Host tmp-node-e2e-b6c375c7-e2e-node-containervm-v20160321-image Test Suite
{ginkgo-output}
{framework-error}
================================================================
```
This makes it painful to find which image the test is failing on. The `{ginkgo-output}` is usually quite long, so we have to scroll mouse up and down to find the host name.
This PR changes the test result to:
```
================================================================
Start Host tmp-node-e2e-b6c375c7-e2e-node-containervm-v20160321-image Test Suite
{ginkgo-output}
Success Finished Host tmp-node-e2e-b6c375c7-e2e-node-containervm-v20160321-image Test Suite
{framework-error}
================================================================
```
This is not perfect, but much better than before. We can easily find the host name under the ginkgo test result, like this:
```
================================================================
Start Host test-gci-dev-54-8743-3-0 Test Suite
Running Suite: E2eNode Suite
============================
Random Seed: 1472511489 - Will randomize all specs
Will run 0 of 131 specs
Running in parallel across 8 nodes
I0829 22:58:13.727764 1143 e2e_node_suite_test.go:98] Pre-pulling images so that they are cached for the tests.
I0829 22:58:28.562459 1143 e2e_node_suite_test.go:111] Node services started. Running tests...
I0829 22:58:28.562477 1143 e2e_node_suite_test.go:116] Wait for the node to be ready
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
I0829 22:58:29.742596 1143 e2e_node_suite_test.go:136] Stopping node services...
I0829 22:58:29.742650 1143 services.go:673] Killing process 1423 (services) with -TERM
I0829 22:58:29.860893 1143 e2e_node_suite_test.go:141] Tests Finished
Ran 0 of 131 Specs in 16.185 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 131 Skipped
Ginkgo ran 1 suite in 19.939034297s
Test Suite Passed
Success Finished Host test-gci-dev-54-8743-3-0 Test Suite
================================================================
```
In a following PR, I'll print the test result from different images into different files to make it more clear for debugging. Mark v1.4 because this helps us de-flake test.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Explicitly delete pods in node performance tests
This PR explicitly deletes all created pods at the end in node e2e performance related tests.
The large number of pods may cause namespace cleanup times out (in #30878), therefore we explicitly delete all pods for cleaning up.
Automatic merge from submit-queue
Rewrite disruption e2e test to use versioned client.
This currently includes the changes from #31638. I will rebase once that is merged.
Automatic merge from submit-queue
increase latency and resource limit accroding to test results
This PR increases the latency limit of node e2e density test according to previous test results.
Fixed#30878
Automatic merge from submit-queue
e2e: log wget output on CheckConnectivityToHost error
Log output might help to diagnose e2e flakes, whether they are caused by dns issues or connection timeouts.
Might help with flake https://github.com/kubernetes/kubernetes/issues/28188.
Automatic merge from submit-queue
Fix make test-integration under OSX
Just throw in a doc.go so there's something compilable in the
test/integration/metrics directory.
Fixes#31587
Automatic merge from submit-queue
test/node-e2e: Update CoreOS update disabling
Previously in this saga... #25004
This disables update-engine and locksmithd with ignition instead of
cloud-init so that they're really totally 100% disabled. Our ignition guy promises.
Pretty much every way of disabling them with cloud-init is mildly racy.
Fixes#31633
I think @vishh can say "I told you so" after the comment on https://github.com/kubernetes/kubernetes/pull/30023#discussion-diff-73431324 .. he was right, but it turns out "stop" there doesn't really work either because of the mess that is cloud-init. Fortunately, converting our cloud-init to json and calling it "ignition" works quite well 😄
Testing done: I ssh'd in and verified that yes, they're disabled. I didn't wait on the e2e tests to pass, so we'll let this PR check that.
Automatic merge from submit-queue
Add e2e tests for Federated Ingress
This is e2e code for federation ingress controller.
Based util functions, add federation ingress e2e cases(reuse current k8s ones) and add logic to validate the result.
Automatic merge from submit-queue
E2E tests for the Source IP Preservation for LoadBalancers
Breaking out E2E changes from the main PR - these tests require the Alpha feature gate turned on for this feature otherwise they will consistently fail.
This disables update-engine and locksmithd with ignition instead of
cloud-init so that they're really totally 100% disabled.
Pretty much every way of disabling them with cloud-init is mildly racy.
Fixes#31633
Add a loop to retry the request to account for the TLS Timeout and API
credential error responses outlined by the flakes in #29227.
Fixes#29227
Signed-off-by: Jess Frazelle <me@jessfraz.com>
Automatic merge from submit-queue
add retries for add/update/remove taints on node in taints e2e
fixes taint update conflict in taints e2e by adding retries for add/update/remove taints on node.
ref #27655 and #31066
Automatic merge from submit-queue
Get network name via e2e environment.
This should work, right? I plan to pipe it through into the TestContext soon, just not today, and I'd like some test runtime over the weekend. Open to suggestions.
Automatic merge from submit-queue
[AppArmor] Promote AppArmor annotations to beta
Justification for promoting AppArmor to beta:
1. We will provide an upgrade path to GA
2. We don't anticipate any major changes to the design, and will continue to invest in this feature
3. We will thoroughly test it. If any serious issues are uncovered we can reevaluate, and we're committed to fixing them.
4. We plan to provide beta-level support for the feature anyway (responding quickly to issues).
Note that this does not include the yet-to-be-merged status annotation (https://github.com/kubernetes/kubernetes/pull/31382). I'd like to propose keeping that one alpha for now because I'm not sure the PodStatus is the right long-term home for it (I think a separate monitoring channel, e.g. cAdvisor, would be a better solution).
/cc @thockin @matchstick @erictune
Automatic merge from submit-queue
Delete the broken Celery+RabbitMQ example
The celery container used in the example is broken and does not come up
on most distros. The e2e test that was validating this example was not
detecting the fact the celery pod was crash looping.
I attempted to fix the celery container, but it proved to be tedious.
The proposed fix is to update the glibc version to >= 2.23. In this case
it requires updating the python docker image and the celery base image.
https://github.com/kubernetes/kubernetes/issues/31456
has more details.
I'm deleting the example instead of marking it as broken because a user
might overlook the broken warning and it should be trivial to revert
this PR if someone can fix the celery container.
This function does not actually attempt to connect to the docker daemon,
it just creates a client object that can be used to do so later. The old
name was confusing, as it implied that a failure to touch the docker daemon
could cause program termination (rather than just a failure to create the
client).
Automatic merge from submit-queue
Allow services which use same port, different protocol to use the same nodePort for both
fix#20092
@thockin @smarterclayton ptal.
Automatic merge from submit-queue
Add ExternalName kube-dns e2e test
ExternalName allows kubedns to return CNAME records for external
services. No proxying is involved.
Built on top of and includes #30599
See original issue at
https://github.com/kubernetes/kubernetes/issues/13748
Feature tracking at
https://github.com/kubernetes/features/issues/33
The e2e test is at least as comprehensive as the one for headless services (namely, only to some degree)
```release-note
Add ExternalName services as CNAME references to external ones
```
Most of the contents of docs/ has moved to kubernetes.github.io.
Development of the docs and accompanying files has continued there, making
the copies in this repo stale. I've removed everything but the .md files
which remain to redirect old links. The .yaml config files in the docs
were used by some tests, these have been moved to test/fixtures/doc-yaml,
and can remain there to be used by tests or other purposes.
Automatic merge from submit-queue
Node Conformance Test: Refactor node e2e framework
For #30122, #30174.
Based on #30348.
**Please only review the last 3 commits.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30348, it removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.
The PR refactored the node e2e framework. Moving different utilities into different packages under `pkg/`.
We need to do this because:
1) Files like e2e_remote.go and e2e_build.go should only be used by runner, but they were compiled into the test suite because they were placed in the same package. The worst thing is that it will introduce some never used flags in the test suite binary.
2) Make the directory structure more clear. Only test should be placed in `test/e2e_node`, other utilities should be placed in different packages in `pkg/`.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Add sysctl support
Implementation of proposal https://github.com/kubernetes/kubernetes/pull/26057, feature https://github.com/kubernetes/features/issues/34
TODO:
- [x] change types.go
- [x] implement docker and rkt support
- [x] add e2e tests
- [x] decide whether we want apiserver validation
- ~~[ ] add documentation~~: api docs exist. Existing PodSecurityContext docs is very light and links back to the api docs anyway: 6684555ed9/docs/user-guide/security-context.md
- [x] change PodSecurityPolicy in types.go
- [x] write admission controller support for PodSecurityPolicy
- [x] write e2e test for PodSecurityPolicy
- [x] make sure we are compatible in the sense of https://github.com/kubernetes/kubernetes/blob/master/docs/devel/api_changes.md
- [x] test e2e with rkt: it only works with kubenet, not with no-op network plugin. The later has no sysctl support.
- ~~[ ] add RunC implementation~~ (~~if that is already in kube,~~ it isn't)
- [x] update whitelist
- [x] switch PSC fields to annotations
- [x] switch PSP fields to annotations
- [x] decide about `--experimental-whitelist-sysctl` flag to be additive or absolute
- [x] decide whether to add a sysctl node whitelist annotation
### Release notes:
```release-note
The pod annotation `security.alpha.kubernetes.io/sysctls` now allows customization of namespaced and well isolated kernel parameters (sysctls), starting with `kernel.shm_rmid_forced`, `net.ipv4.ip_local_port_range`, `net.ipv4.tcp_max_syn_backlog` and `net.ipv4.tcp_syncookies` for Kubernetes 1.4.
The pod annotation `security.alpha.kubernetes.io/unsafeSysctls` allows customization of namespaced sysctls where isolation is unclear. Unsafe sysctls must be enabled at-your-own-risk on the kubelet with the `--experimental-allowed-unsafe-sysctls` flag. Future versions will improve on resource isolation and more sysctls will be considered safe.
```
Automatic merge from submit-queue
add throughput in perf data and disable --cgroups-per-qos
This PR adds throughput data to printed perf data for benchmark. It also disables --cgrous-per-qos in jenkinds-benchmark.properties.
Automatic merge from submit-queue
Make sure the StatusCode is taken into account in DoRaw()
**What this PR does / why we need it**:
Currently if there is an error (not found) the error printed out
is to do with the inablity to convert an empty body into the expected json.
This patch will fill in the err correctly.
example of before (with NotFound error):
$ kubectl top node
failed to unmarshall heapster response: json: cannot unmarshal object into Go value of type []v1alpha1.NodeMetrics
Now:
$ kubectl top node
the server could not find the requested resource (get services http:heapster:)
**Which issue this PR fixes**
related to bug #30818
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Node Conformance Test: Remove unnecessary binary copy
For #30122, #30174.
This PR removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Previously this code used http.Get and failed to read/close resp.Body, which
prevented network connection reuse, leaking fds. Now we use http.Head
instead, because its response always has a nil Body, so we don't have to
worry about read/close.
Automatic merge from submit-queue
skip benchmark in jenkins serial test
This PR changes jenkins-serial.properties to skip benchmark tests (with tag [Benchmark]) in jenkins serial tests. It also add more comments in run_e2e.go.
Automatic merge from submit-queue
[Kubelet] Optionally consume configuration from <node-name> named config maps
This extends the Kubelet to check the API server for new node-specific config, and exit when it finds said new config.
/cc @kubernetes/sig-node @mikedanese @timstclair @vishh
**Release note**:
```
Extends Kubelet with Alpha Dynamic Kubelet Configuration. Please note that this alpha feature does not currently work with cloud provider auto-detection.
```
Automatic merge from submit-queue
[GarbageCollector] Allow per-resource default garbage collection behavior
What's the bug:
When deleting an RC with `deleteOptions.OrphanDependents==nil`, garbage collector is supposed to treat it as `deleteOptions.OrphanDependents==true", and orphan the pods created by it. But the apiserver is not doing that.
What's in the pr:
Allow each resource to specify the default garbage collection behavior in the registry. For example, RC registry's default GC behavior is Orphan, and Pod registry's default GC behavior is CascadingDeletion.
Automatic merge from submit-queue
federation: Adding support for namespace admission controls in federation-apiserver
Now that we have namespaces in federation apiserver, we can support namespace admission controls.
There are 3 of these:
namespace/autoprovision, namespace/exists and namespace/lifecycle.
namespace/autoprovision, namespace/exists should be deprecated in kubernetes(https://github.com/kubernetes/kubernetes/issues/31195). Adding support for namespace/lifecycle to federation-apiserver.
As in kube-apiserver, enabling namespace/lifecycle by default.
```release-note
Action required: If you have a running federation control plane, you will have to ensure that for all federation resources, the corresponding namespace exists in federation control plane.
federation-apiserver now supports NamespaceLifecycle admission control, which is enabled by default. Set the --admission-control flag on the server to change that.
```
cc @kubernetes/sig-cluster-federation @quinton-hoole
1. /validate service does not exist, so remove the test for it and add some that actually do exist
2. The namespace does not exist so this will always return NotFound
Note: DoRaw() ignores the StatusCode.
This is in preparation for the next commit
Automatic merge from submit-queue
[e2e density test] Fix unnecessary Delete RC requests when not running latency test
As the following code block
https://github.com/kubernetes/kubernetes/blob/master/test/e2e/density.go#L666-L670
shows, after running each density test case, it will attempt to delete "additional replication controllers" even though there is **no additional replication controller**.
When we are not running latency test, API Server will return "404 error code". So, I propose to move the above code block inside thedetermine statementsif `itArg.runLatencyTest{ }` , looks like:
```
if itArg.runLatencyTest {
...
for i := 1; i <= nodeCount; i++ {
name := additionalPodsPrefix + "-" + strconv.Itoa(i)
c.ReplicationControllers(ns).Delete(name, nil)
}
}
```
In this way, removing RC will be executed only if we set `itArg.runLatencyTest` to be `true`. It can avoid post some necessary requests to API Server.
Issuse is #30977
Automatic merge from submit-queue
[e2e test] Fix e2e test pause image hard code
Use `framework.GetPauseImageName(f.Client)` instead of hard code(such as `"gcr.io/google_containers/pause-amd64:3.0"`) to represent pause image name.
Related issus is #30967
Automatic merge from submit-queue
Node E2E: Remove fatal error in e2e_node_suite_test.go
Addresses https://github.com/kubernetes/kubernetes/issues/30779#issuecomment-240532190.
Currently we run node e2e test in parallel, and ginkgo makes sure that we only initialize test framework in the first test node.
However, because we throw out some fatal error during the initialization. Once there is an fatal error, the first test node will die immediately without reporting any error, and the other nodes will exit because the first node is gone with meaningless error.
If kubelet start fails, we'll get something like:
```
------------------------------
Failure [132.485 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
------------------------------
......
------------------------------
Failure [132.465 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
```
This PR replaces these fatal errors with gomega assertion, with this PR, we'll get:
```
Failure [132.482 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
should be able to start node services.
Expected success, but got an error:
<*errors.errorString | 0xc8203351b0>: {
s: "failed to run server start command \"/tmp/ginkgo869068712/e2e_node.test --run-services-mode --server-start-timeout 2m0s --report-dir --node-name lantaol0.mtv.corp.google.com --disable-kubenet=true --cgroups-per-qos=false --manifest-path /tmp/node-e2e-pod221291440 --eviction-hard memory.available<250Mi\": exit status 255",
}
failed to run server start command "/tmp/ginkgo869068712/e2e_node.test --run-services-mode --server-start-timeout 2m0s --report-dir --node-name lantaol0.mtv.corp.google.com --disable-kubenet=true --cgroups-per-qos=false --manifest-path /tmp/node-e2e-pod221291440 --eviction-hard memory.available<250Mi": exit status 255
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:117
------------------------------
Failure [132.485 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
------------------------------
......
------------------------------
Failure [132.465 seconds]
[BeforeSuite] BeforeSuite
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
BeforeSuite on Node 1 failed
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
```
This is much more informative.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Node E2E: Wait for node ready before the node e2e test started.
Fixes https://github.com/kubernetes/kubernetes/issues/30252.
This PR makes node e2e test wait for exactly one node ready before running other test.
@ronnielai @mtaufen
Automatic merge from submit-queue
pkg/genericapiserver/options: don't import pkg/apiserver
Refactor the authorization options for the API server so
pkg/apiserver isn't directly imported by the options package.
Closes#28544
cc @smarterclayton
@madhusudancs, @nikhiljindal I've updated `federation/cmd/federation-apiserver/app/server.go` to include the RBAC options with this change. I don't know if this was intentionally left out in the first place but would like your feedback.
Automatic merge from submit-queue
Add benchmark to jenkins
This PR contains the following changes:
1. Add more tests in density benchmark test;
2. Add the peak value (100%) in latency and CPU usage statistic data;
3. Move the Ginkgo focus flag from e2e_remote.go to run_e2e.go;
4. Support running benchmark in run_e2e.go. The benchmark configuration file is an extension of image configuration. Each item requires additional GCE machine type (e.g. n1-standard-1, default value will be used if empty) and test names (Ginkgo focus regex strings). A test item is regarded as benchmark if the tests field is non-empty.
Automatic merge from submit-queue
Node Conformance Test: Statically link apiserver
For #30122, #30174.
Based on #30116, #30198, #30200.
**Please only review the 4th PR.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30116, which started e2e services in a separate process.
The 2nd commit is from #30198, it statically linked etcd into the node e2e framework.
The 3rd commit is from #30200, it moved namespace controller into e2e services.
The 4th commit is new, it statically linked apiserver into the node e2e framework.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Node Conformance Test: Move namespace controller to services
For #30122, #30174.
Based on #30116, #30198.
**Please only review the 3rd PR.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30116, which started e2e services in a separate process.
The 2nd commit is from #30198, it statically linked etcd into the node e2e framework.
The 3rd commit is new, it moved namespace controller into e2e services.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Scheduledjobs e2e
@janetkuo resubmitted e2e for SJ, I've updated all scripts to consume `KUBE_RUNTIME_CONFIG` properly in 2nd commit, ptal
Automatic merge from submit-queue
Unblock iterative development on pod-level cgroups
In order to allow forward progress on this feature, it takes the commits from #28017#29049 and then it globally disables the flag that allows these features to be exercised in the kubelet. The flag can be re-added to the kubelet when its actually ready.
/cc @vishh @dubstack @kubernetes/rh-cluster-infra
Automatic merge from submit-queue
When running inside docker, activate service account ASAP
Also switching to just use `GOOGLE_APPLICATION_CREDENTIALS`, rather than both.
x-ref https://github.com/kubernetes/test-infra/issues/318
Automatic merge from submit-queue
Wait for memory to be reclaimed after node_e2e MemoryEviction test
This helps prevent interference with other tests that run immediately after the MemoryEviction test.
/cc @Random-Liu @coufon
Automatic merge from submit-queue
Gubernator bug fixes: mv and GCS bucket permissions
Fixed issue where results file was not moved correctly, and also the permissions issue with the GCS bucket.
Will rebase after #30414 is merged
@timstclair
Automatic merge from submit-queue
Add logging time series to benchmark test
This PR adds a new file benchmark_util.go which contains tool functions for benchmark (we can migrate benchmark related functions into it).
The PR logs time series data for density benchmark test.
Automatic merge from submit-queue
OpenAPI / Swagger2 spec generation
This is alpha version of OpenAPI spec generation. Generated "/swagger.json" file (accessible on api server) is a valid OpenAPI spec with some warnings that will be fixed in next versions of spec generation. Currently it is possible to generate a client using this spec though I did not test the clients.
reference: #13414
**Release note**:
```release-note
Alpha support for OpenAPI (aka. Swagger 2.0) specification serves on /swagger.json
```
Automatic merge from submit-queue
extract common function in ingress and federation e2e to uil
Separate refactoring existing e2e code part from #29773
1. extract common constants, structs and functions in federated-service.go to federation-util.go, move existing util functions from federated-service.go to federation-util.go
2. extract common constants, structs and functions in ingress.go to ingress-util.go, move existing util functions from ingress.go to ingress-util.go
@quinton-hoole
Automatic merge from submit-queue
Node E2E: Make readiness check handling process exits with 0 exit code.
As is mentioned by @mtaufen:
"there is a problem with the way service `start` is currently implemented in test/e2e_node/e2e_service.go. If the Kubelet exits with status 0 before the health check completes, cmdErrorChan will be closed and, as a result, nil will be read from that channel, and you will return a nil error from `start`."
This PR changes the logic to:
1) If the err channel returns an error, return the error
2) If the err channel returns a nil, ignore it and continue checking readiness.
3) If the err channel is closed before readiness check succeeds, replace it with `blockCh` and continue checking readiness.
@mtaufen
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Move UTs that block on apiserver to integration tests.
In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail. These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.
This PR simply shuffles those tests to integration tests, which is where they should be.
/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90
Automatic merge from submit-queue
Add GUBERNATOR flag which produces g8r link for node e2e tests
When you run 'make tests-e2e-node REMOTE=true GUBERNATOR=true' outputs a URL to view the test results on Gubernator. ~~Should work after my PR for Gubernator is merged.~~
@timstclair
Convert single GV and lists of GVs into an interface that can handle
more complex scenarios (everything internal, nothing supported). Pass
the interface down into conversion.
Automatic merge from submit-queue
Implement dynamic provisioning (beta) of PersistentVolumes via StorageClass
Implemented according to PR #26908. There are several patches in this PR with one huge code regen inside.
* Please review the API changes (the first patch) carefully, sometimes I don't know what the code is doing...
* `PV.Spec.Class` and `PVC.Spec.Class` is not implemented, use annotation `volume.alpha.kubernetes.io/storage-class`
* See e2e test and integration test changes - Kubernetes won't provision a thing without explicit configuration of at least one `StorageClass` instance!
* Multiple provisioning volume plugins can coexist together, e.g. HostPath and AWS EBS. This is important for Gluster and RBD provisioners in #25026
* Contradicting the proposal, `claim.Selector` and `volume.alpha.kubernetes.io/storage-class` annotation are **not** mutually exclusive. They're both used for matching existing PVs. However, only `volume.alpha.kubernetes.io/storage-class` is used for provisioning, configuration of provisioning with `Selector` is left for (near) future.
* Documentation is missing. Can please someone write some while I am out?
For now, AWS volume plugin accepts classes with these parameters:
```
kind: StorageClass
metadata:
name: slow
provisionerType: kubernetes.io/aws-ebs
provisionerParameters:
type: io1
zone: us-east-1d
iopsPerGB: 10
```
* parameters are case-insensitive
* `type`: `io1`, `gp2`, `sc1`, `st1`. See AWS docs for details
* `iopsPerGB`: only for `io1` volumes. I/O operations per second per GiB. AWS volume plugin multiplies this with size of requested volume to compute IOPS of the volume and caps it at 20 000 IOPS (maximum supported by AWS, see AWS docs).
* of course, the plugin will use some defaults when a parameter is omitted in a `StorageClass` instance (`gp2` in the same zone as in 1.3).
GCE:
```
apiVersion: extensions/v1beta1
kind: StorageClass
metadata:
name: slow
provisionerType: kubernetes.io/gce-pd
provisionerParameters:
type: pd-standard
zone: us-central1-a
```
* `type`: `pd-standard` or `pd-ssd`
* `zone`: GCE zone
* of course, the plugin will use some defaults when a parameter is omitted in a `StorageClass` instance (SSD in the same zone as in 1.3 ?).
No OpenStack/Cinder yet
@kubernetes/sig-storage
Automatic merge from submit-queue
Adding e2e test for federation replicasets
Its a basic test which tests that we can create and delete replicasets. Will enhance it when we write the replicaset controller.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Allow setting permission mode bits on secrets, configmaps and downwardAPI files
cc @thockin @pmorie
Here is the first round to implement: https://github.com/kubernetes/kubernetes/pull/28733.
I made two commits: one with the actual change and the other with the auto-generated code. I think it's easier to review this way, but let me know if you prefer in some other way.
I haven't written any tests yet, I wanted to have a first glance and not write them till this (and the API) are more close to the "LGTM" :)
There are some things:
* I'm not sure where to do the "AND 0777". I'll try to look better in the code base, but suggestions are always welcome :)
* The write permission on group and others is not set when you do an `ls -l` on the running container. It does work with write permissions to the owner. Debugging seems to show that is something happening after this is correctly set on creation. Will look closer.
* The default permission (when the new fields are not specified) are the same that on kubernetes v1.3
* I do realize there are conflicts with master, but I think this is good enough to have a look. The conflicts is with the autog-enerated code, so the actual code is actually the same (and it takes like ~30 minutes to generate it here)
* I didn't generate the docs (`generated-docs` and `generated-swagger-docs` from `hack/update-all.sh`) because my machine runs out of mem. So that's why it isn't in this first PR, will try to investigate and see why it happens.
Other than that, this works fine here with some silly scripts I did to create a secret&configmap&downwardAPI, a pod and check the file permissions. Tested the "defaultMode" and "mode" for all. But of course, will write tests once this is looking fine :)
Thanks a lot again!
Rodrigo
Automatic merge from submit-queue
Continue on #30774: Change podNamespacer API
continue on #30774, credit to @wojtek-t, Ref #30759
I just fixed a test and converted IsActivePod to operate on *Pod.
Automatic merge from submit-queue
Add tag [benchmark] to node-e2e-test where performance limits are not verified
This PR adds a new tag "[benchmark]" to density and resource-usage node e2e test. The performance limits will not be verified at the end of benchmark tests.
Automatic merge from submit-queue
Implement federation API server authentication e2e tests.
This PR depends on #30397. Please review only the last commit here.
Fixes: Issue #28602.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Node Conformance Test: Statically link etcd
For #30122, #30174.
This PR is part of our roadmap to package node conformance test.
It statically linked etcd into the node e2e framework. In the future all e2e services will be linked in, and print log into the same log file `services.log`.
@dchen1107 @vishh
/cc @kubernetes/sig-node
This implements the proposal in:
docs/proposals/secret-configmap-downwarapi-file-mode.md
Fixes: #28317.
The mounttest image is updated so it returns the permissions of the linked file
and not the symlink itself.
Automatic merge from submit-queue
Fix default resource limits (node allocatable) for downward api volumes and env vars
@kubernetes/rh-cluster-infra @pmorie @derekwaynecarr
Automatic merge from submit-queue
two new pv e2e tests
Added two more pv e2e tests: 1) creating a claim before the pv (both not pre-bound), 2) creating a claim before the pv with the claim pre-bound to the PV via Spec.Volumename.
Automatic merge from submit-queue
Run CI Jenkins node e2e tests in project k8s-jkns-ci-node-e2e
Fixes#27648.
If node VMs leak, they should only harm themselves, not the rest of Jenkins.
This also lets us do VM cleanup without worrying that we might accidentally delete important Jenkins VMs.
The `k8s-jkns-ci-node-e2e` should have the right ACLs in place already. The quota is at defaults, but I don't think we'll need to increase it at this point.
Automatic merge from submit-queue
Let load and density e2e tests use GC if it's on
I've run the 100 and 500 nodes tests and they both pass.
The test-infra half of the PR is https://github.com/kubernetes/test-infra/pull/369
cc @lavalamp
Automatic merge from submit-queue
Move new etcd storage (low level storage) into cacher
In an effort for #29888, we are pushing forward this:
What?
- It changes creating etcd storage.Interface impl into creating config
- In creating cacher storage (StorageWithCacher), it passes config created above and new etcd storage inside.
Why?
- We want to expose the information of (etcd) kv client to cacher. Cacher storage uses this information to talk to remote storage.
Automatic merge from submit-queue
Update core etcd references to use 3.0.4
This updates the core references to use 3.0.4.
There are still legacy references in the code base that should be cleaned, or just removed but I'm reluctant to purge.
/cc @kubernetes/sig-scalability
Automatic merge from submit-queue
Extend the wait time to 2*time.Minute for liveness check test.
Fixes https://github.com/kubernetes/kubernetes/issues/30264.
https://github.com/kubernetes/kubernetes/pull/29814 changes the wait time to 1 minute, which is not enough. That's what causes the flake.
The test expected the container to be restarted, and the container was indeed restarted but it took about 1 minute:
```
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:02:35 -0700 PDT - event for liveness-exec: {default-scheduler } Scheduled: Successfully assigned liveness-exec to e2e-gce-agent-pr-38-0-minion-group-bv95
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:02:36 -0700 PDT - event for liveness-exec: {kubelet e2e-gce-agent-pr-38-0-minion-group-bv95} Pulled: Container image "gcr.io/google_containers/busybox:1.24" already present on machine
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:02:36 -0700 PDT - event for liveness-exec: {kubelet e2e-gce-agent-pr-38-0-minion-group-bv95} Created: Created container with docker id cf4e8e60535e
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:02:36 -0700 PDT - event for liveness-exec: {kubelet e2e-gce-agent-pr-38-0-minion-group-bv95} Started: Started container with docker id cf4e8e60535e
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:02:55 -0700 PDT - event for liveness-exec: {kubelet e2e-gce-agent-pr-38-0-minion-group-bv95} Unhealthy: Liveness probe failed:
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:03:26 -0700 PDT - event for liveness-exec: {kubelet e2e-gce-agent-pr-38-0-minion-group-bv95} Killing: Killing container with docker id cf4e8e60535e: pod "liveness-exec_e2e-tests-container-probe-b1wip(a1f856fc-5e07-11e6-a7e1-42010af00002)" container "liveness" is unhealthy, it will be killed and re-created.
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:03:26 -0700 PDT - event for liveness-exec: {kubelet e2e-gce-agent-pr-38-0-minion-group-bv95} Created: Created container with docker id 0b18537dc794
Aug 9 01:03:40.696: INFO: At 2016-08-09 01:03:26 -0700 PDT - event for liveness-exec: {kubelet e2e-gce-agent-pr-38-0-minion-group-bv95} Started: Started container with docker id 0b18537dc794
```
This PR recovers the wait time to the original 2 * time. Mark P0 to match the corresponding issue.
@fejta
Automatic merge from submit-queue
Remove unversioned federation client, clientset and versioned release_1_3 clientset and all their accesses in e2e tests. Switch everything to federation release_1_4 external client.
cc @kubernetes/sig-cluster-federation
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30397)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Add Time Series Data and Labels in Node density test
This pull requests contain:
1. Increase the pod creation latency limit according to test results;
2. Add 'GetResourceSeriesWithLabels' in 'resource_collector.go' to provide resource usage time series data;
3. Modify 'GetBasicCPUStats' in 'resource_collector.go' to make a copy of CPU usage array before sorting (otherwise time series data is disordered);
4. Add 'ResourceUsageToPerfDataWithLabels' and 'CPUUsageToPerfDataWithLabels' to attach labels to 'PerfData' for benchmark dashboard;
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30333)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Fix killing child sudo process in e2e_node tests
Fixes#29211; re-doing #29380 which was reverted due to cross-platform build failure #29669.
The context is we are trying to kill a process started as `sudo kube-apiserver`, but `sudo` ignores signals from the same process group. Applying `Setpgid` means the `sudo kill` process won't be in the same process group, so will not fall foul of this nifty feature.
~~I also took the liberty of removing some code setting `Pdeathsig` because it claims to be doing something in the same area, but actually it doesn't do that at all. The setting is applied to the forked process, i.e. `sudo`, and it means the `sudo` will get killed if we (`e2e_node.test`) die. This (a) isn't what the comment says and (b) doesn't help because sending SIGKILL to the sudo process leaves sudo's child alive.~~
We do need to use the same 'hack' to access `Setpgid` as the `e2e_node.test` program gets built on Windows (although it doesn't run there).
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29685)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Run node e2e pull VMs in a separate GCP project
I haven't increased quota yet, so we shouldn't merge until I've done so. This is mostly just testing to see if the rest of the plumbing works.
Part of #27648.
cc @fejta @spxtr @pwittrock
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29584)
<!-- Reviewable:end -->
Automatic merge from submit-queue
[GarbageCollector] measure latency
First commit is #27600.
In e2e tests, I measure the average time an item spend in the eventQueue(~1.5 ms), dirtyQueue(~13ms), and orphanQueue(~37ms). There is no stress test in e2e yet, so the number may not be useful.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28387)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Use latest GCI image based on a regex in Node e2e
This PR also makes it possible to run node e2e against multiple previous images, sorted by creation time. A regex for the image name can be used to instruct node e2e to identify test images.
Depends on #29577
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29815)
<!-- Reviewable:end -->
It can run tests against multiple existing images that match a regex.
GCI images will be using a regex.
Signed-off-by: Vishnu kannan <vishnuk@google.com>
Automatic merge from submit-queue
Add test for supplemental gid annotation to pv e2e test
Test for feature added in #29119. Adds the annotation to the pv in the PersistentVolumes e2e test. It tests it by checking the output of 'id -G.' Also tests the actual use case of an nfs server by using an nfs server image 'volume-nfs-gid', modified slightly from 'volume-nfs' to have some permissions set.
@pmorie
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30084)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Cut the client repo, staging it in the main repo
Tracking issue: #28559
ref: https://github.com/kubernetes/kubernetes/pull/25978#issuecomment-232710174
This PR implements the plan a few of us came up with last week for cutting client into its own repo:
1. creating "_staging" (name is tentative) directory in the main repo, using a script to copy the client and its dependencies to this directory
2. periodically publishing the contents of this staging client to k8s.io/client-go repo
3. converting k8s components in the main repo to use the staged client. They should import the staged client as if the client were vendored. (i.e., the import line should be `import "k8s.io/client-go/<pacakge name>`). This requirement is to ease step 4.
4. In the future, removing the staging area, and vendoring the real client-go repo.
The advantage of having the staging area is that we can continuously run integration/e2e tests with the latest client repo and the latest main repo, without waiting for the client repo to be vendored back into the main repo. This staging area will exist until our test matrix is vendoring both the client and the server.
In the above plan, the tricky part is step 3. This PR achieves it by creating a symlink under ./vendor, pointing to the staging area, so packages in the main repo can refer to the client repo as if it's vendored. To prevent the godep tool from messing up the staging area, we export the staged client to GOPATH in hack/godep-save.sh so godep will think the client packages are local and won't attempt to manage ./vendor/k8s.io/client-go.
This is a POC. We'll rearrange the directory layout of the client before merge.
@thockin @lavalamp @bgrant0607 @kubernetes/sig-api-machinery
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29147)
<!-- Reviewable:end -->
Automatic merge from submit-queue
E2E & Node E2E: Move pods test into common directory
This is the 4th part of #29494.
For #29081.
Based on #29092, #29806.
The first commit is squash of all dependent commits. Please only review the last 2 commits.
The 2nd commit migrates pods.go to `common/pods.go`.
Notice that the test `should be schedule with cpu and memory limits` is removed because:
* It doesn't make sense at the node level.
* It should have been tested in scheduler_predicates at the cluster level https://github.com/kubernetes/kubernetes/blob/master/test/e2e/scheduler_predicates.go#L264
The 3rd commit splits pods.go into several pods (nothing is changed, only move code):
* **Liveness probe test:** Moved into `container_probe.go`.
* **Init container test:** Moved into a new file `init_container.go`.
* **Others:** Still in pods.go
@vishh @timstclair
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29814)
<!-- Reviewable:end -->
Automatic merge from submit-queue
[GarbageCollector] only store typeMeta and objectMeta in the gc store
GC only needs to know the apiVersion, kind, and objectMeta of an object. This PR makes the stores of GC only save these fields.
cc @kubernetes/sig-api-machinery
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28480)
<!-- Reviewable:end -->
This test creates three pods with QoS of besteffort, burstable, and
guaranteed, respectively, which each contain a container that tries to
consume almost all the available memory at a rate of about 12Mi/10sec.
The expectation is that eviction will be initiated when the hard
memory.available<250Mi threshold is triggered, and that eviction will proceed
in the order of besteffort, then burstable. Since guaranteed pods should
only be evicted if something charged to the host uses more resources
than were reserved for it, we currently end the test when besteffort and
burstable have both been evicted.
Note that this commit also sets --eviction-hard=memory.available<250Mi
to enable eviction during tests.
Automatic merge from submit-queue
Add integration test for volume controller startup.
Tests #28002 with real etcd (unit tests have a fake one with different behavior).
@kubernetes/sig-storage
Automatic merge from submit-queue
e2e flake: real fix of PodAntiAffinity test
The fix in PR https://github.com/kubernetes/kubernetes/pull/30135 was wrong in using a wrong test condition for an already broken test.
### Summary
The test tries to launch a pod with an anti-affinity annotation, waits 10
seconds and then checks that it is still pending.
But the anti-affinity annotation does not forbid to launch that pod on just
another node that does not have the zone label at all.
This commit changes this behavior by labeling two nodes with the zone label
and then forcing the pod to be launched on one of those two nodes.
**I assume here that a non-existing label is considered as a different label value.**
Fixes#30078
Automatic merge from submit-queue
SchedulerExtender: add failedPredicateMap in Filter() returns
Fix#25797. modify extender.Filter for adding extenders information to “failedPredicateMap” in findNodesThatFit.
When all the filtered nodes that passed "predicateFuncs" don’t pass the extenders filter, the failedPredicateMap hasn’t the extenders information, should add it, I think. So when the length of the “filteredNodes.Items” is 0, we can know the integral information. (The length of the “filteredNodes.Items” is 0, may be because the extenders filter failed.)
Automatic merge from submit-queue
Use report-dir in test framework instead.
We already have `report-dir` option in framework test context.
The node e2e framework should use it as well.
/cc @ronnielai
Automatic merge from submit-queue
node_e2e: Use upstream CoreOS image directly
.. and update it to the latest alpha
This will make updating the CoreOS image in the future much simpler since it won't involve project-copying, manual-baking, or so on.
cc @pwittrock @vishh @bboreham @yifan-gu
Automatic merge from submit-queue
E2E & Node E2E: Move configmap, docker_containers, downward_api, expansion and secrets test into common directory.
This is the 3rd part of #29494.
For #29081.
Based on #29092, #29806.
The first commit is squash of all dependent commits. Please only review the second commit.
The second PR added 17 lines.
@vishh @timstclair
Automatic merge from submit-queue
For e2e_node tests tell etcd to listen on ports 2379 and 4001
This is the default for etcd2, but etcd3 only listens on 2379.
Specifying the ports keeps things consistent no matter which version the user has installed.
Fixes#29117
Automatic merge from submit-queue
Bug fix: Use p.Name instead of pod.Name
For example, if you used `pod.GenerateName`, `pod.Name` might be the empty
string while `p.Name` contains the actual name of your pod. Thus passing
`pod.Name` can result in a `resource name may not be empty` error.
For example, if you used pod.GenerateName, pod.Name might be the empty
string while p.Name contains the actual name of your pod. Thus passing
pod.Name can result in a `resource name may not be empty` error.
The test tries to launch a pod with an anti-affinity annotation, waits 10
seconds and then checks that it is still pending.
But the anti-affinity annotation does not forbid to launch that pod on just
another node that does not have the zone label at all.
This commit changes this behavior by labeling two nodes with the zone label
and then forcing the pod to be launched on one of those two nodes.
Automatic merge from submit-queue
Node E2E: Move the node name initialization to first function of SynchronizedBeforeEach
Currently, we start e2e services in the first function of `SynchronizedBeforeEach` to make sure that we only start them once even we are running test in parallel test nodes.
However, e2e services require `NodeName`, but we initialize `NodeName` in the second function.
This PR moved the initialization logic into the first function, and shared the node name with all test nodes via the `SharedContext`.
Automatic merge from submit-queue
Add density (batch pods creation latency and resource) and resource performance tests to `test-e2e-node' built for Linux only
This PR adds `+build linux' to density_test.go, resource_usage.go and resource_collector.go to last PR #29764.
#29764 fails build because it depends on cgroup which can not be built for os other than Linux.
Automatic merge from submit-queue
E2E & NodeE2E: Move host_path, downwardapi_volume and empty_dir into common directory.
This is the second part of #29494.
For #29081.
Based on #29092, #29806.
The first commit is squash of all dependent commits. Please only review the second commit.
The second PR is only 20 lines of change.
@vishh @timstclair
Automatic merge from submit-queue
Node E2E: Change the node e2e junit file name to junit_{image-name}{test-node-number}.xml
Fixes https://github.com/kubernetes/kubernetes/issues/30103.
Reuse the `report-prefix` in e2e test framework. Now the junit file will be like: `junit_{image-name}{test-node-number}.xml`.
Mark P2 to fix the test result.
/cc @rmmh
Automatic merge from submit-queue
federation: Adding secret API
Adding secret API to federation-apiserver and updating the federation client to include secrets
Automatic merge from submit-queue
Added test to density that will run maximum capacity pods on nodes
Added a test to the Density Suite that will load the kubelets with their maximum capacity number of pods
Automatic merge from submit-queue
Install go-bindata in cross-build image
Another follow-up to #25584.
We need `go-bindata` to create `test/e2e/generated`, and downloading it with `go get` at build time is painful for a variety of reasons. We can just include it in the cross-build image and not worry about it, especially as it updates very infrequently.
This fixes `hack/update-generated-protobuf.sh` as well.
cc @jayunit100 @soltysh
Automatic merge from submit-queue
pv e2e refactor and pre-bind test
refactored persistentvolume e2e so that multiple It() tests can be run. Added one test case for pre-binding, but the overall structure of the test should allow additional test cases to be more easily added.
Automatic merge from submit-queue
Resolve docker-daemon cgroup issue for both systemd and non-systemd node for node e2e tests
Fixed https://github.com/kubernetes/kubernetes/issues/29827
cc/ @coufon this should unblock your pr: #29764
I validated both containervm image and coreos image, and works as expected.
This is also required for adding gci image to node e2e test infrastructure.
Automatic merge from submit-queue
Fix deployment e2e test: waitDeploymentStatus should error when entering an invalid state
Follow up #28162
1. We should check that max unavailable and max surge aren't violated at all times in e2e tests (didn't check this in deployment scaled rollout yet, but we should wait for it to become valid and then continue do the check until it finishes)
2. Fix some minor bugs in e2e tests
@kubernetes/deployment
Automatic merge from submit-queue
Remove myself from test ownership.
These are almost certainly not correct, but probably more likely owners than myself.
@rmmh @dchen1107 @timstclair @erictune @mtaufen @caesarxuchao @fgrzadkowski @krousey @lavalamp
Automatic merge from submit-queue
Fix 29992
Fix#29992.
I copied RC test code to the wrong place to the RS test in #29798. I took a look at the failure reports, they were all failed on the RS test, so #29798 itself is correct.
Marked as P2 since it fixes a test flake that will block everyone.
Automatic merge from submit-queue
Revert "Revert "Drop support for --gce-service-account, require activated creds""
Reverts kubernetes/kubernetes#29242
Automatic merge from submit-queue
Limit number of pods spawned in SchedulerPredicates validates resourc…
Fixes https://github.com/kubernetes/kubernetes/issues/29190,
With this patch test should spawn at most 10 pods on the smallest node.
Automatic merge from submit-queue
integration test: Modify PVs/PVCs during binding.
Previous volume binder code was not able to cope with PVs or PVCs getting modified during the binding process. Current one should be resilient to these changes, so let's test it.
It makes the test approximately twice as long as before, from ~2 seconds to ~4-5.
@kubernetes/sig-storage
Marking as 1.3 target, however it does not really matter here, it's just a test.
Add min size of pod and max number of pods for SchedulerPredicates validate resouce limits test
Fix typo in patch for SchedulerPredicates validate resouce limits test
Moving max number of pods and min pod cpu request to constants
Automatic merge from submit-queue
Add density (batch pods creation latency and resource) and resource performance tests to `test-e2e-node'
This PR contains two new tests (migrate from e2e test):
1. Density test: verify startup latency and resource usage when create a batch of pod with throughput control. Throughput control is done by sleep for an interval between firing concurrently create pod operations.
It tests both batch creation and sequential (back-to-back) creation and report the throughputs.
2. Verify resource usage of steady state kubelet.
The test creates a new resource controller for `test-node-e2e' (resource_controller.go) which monitors resource through a standalone Cadvisor pod (port 8090) with 1s housekeeping interval.
Automatic merge from submit-queue
[Garbage Collector] add e2e tests again
#27151 is reverted because gke didn't start correctly after it's merged (https://github.com/kubernetes/kubernetes/pull/27151#issuecomment-233030686).
The possible problem is the `unbound variable`, which is fixed in the second commit of this PR. However, I cannot verify if the PR will fail the gke suite since I don't have the environment to run that suite.
@wojtek-t @lavalamp
Automatic merge from submit-queue
Update test-owners with new tests, add catch-all assignment to test-infra team.
We will triage any additional failures, since they're more likely to be infra related. If they're not, they can always be reassigned (and the owners list can be updated!)
/cc @kubernetes/test-infra-maintainers
Automatic merge from submit-queue
Node E2E: Add serial jenkins job.
This PR added a jenkins job for serial test. It will run all serial test one by one.
This will be useful for https://github.com/kubernetes/kubernetes/pull/29809.
@coufon @yujuhong @dchen1107
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Add support to quota pvc storage requests
Adds support to quota cumulative `PersistentVolumeClaim` storage requests in a namespace.
Per our chat today @markturansky @abhgupta - this is not done (lacks unit testing), but is functional.
This lets quota enforcement for `PersistentVolumeClaim` to occur at creation time. Supporting bind time enforcement would require substantial more work. It's possible this is sufficient for many, so I am opening it up for feedback.
In the future, I suspect we may want to treat local disk in a special manner, but that would have to be a different resource altogether (i.e. `requests.disk`) or something.
Example quota:
```
apiVersion: v1
kind: ResourceQuota
metadata:
name: quota
spec:
hard:
persistentvolumeclaims: "10"
requests.storage: "40Gi"
```
/cc @kubernetes/rh-cluster-infra @deads2k
Automatic merge from submit-queue
E2E & Node E2E: Add exec util in framework
For #29081.
Based on #29092 and #29494.
For first commit is a squashed commit of all old commits.
**The last 2 commits are new.**
This PR added exec util in framework, and moved `privileged.go` and `kubelet_etc_hosts` into `common` directory.
@vishh @timstclair
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Node E2E: Make node e2e parallel
For https://github.com/kubernetes/kubernetes/issues/29081.
Fix https://github.com/kubernetes/kubernetes/issues/26215.
Based on https://github.com/kubernetes/kubernetes/pull/28807, https://github.com/kubernetes/kubernetes/pull/29020, will rebase after they are merged.
**Only the last commit is new.**
We are going to move more tests into the node e2e test. However, currently node e2e test only run sequentially, the test duration will increase quickly when we add more test.
This PR makes the node e2e test run in parallel so as to shorten test duration, so that we can add more test to improve the test coverage.
* If you run the test locally with `make test-e2e-node`, it will use `-p` ginkgo flag, which uses `(cores-1)` parallel test nodes by default.
* If you run the test remotely or in the Jenkin, the parallelism will be controlled by the environment variable `PARALLELISM`. The default value is `8`, which is reasonable for our test node (n1-standard-1).
Before this PR, it took **833.592s** to run all test on my desktop.
With this PR, it only takes **234.058s** to run.
The pull request node e2e run with this PR takes **232.327s**.
The pull request node e2e run for other PRs takes **673.810s**.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Adding GCI to node e2e.
Depends on https://github.com/kubernetes/kubernetes/pull/29486
Adding the dev release as of now since stable and beta run docker v1.9.1
which is incompatible with kubelet.
Automatic merge from submit-queue
Fix 29451
Fix#29451. I've also checked other tests in that file to make sure they don't have similar problems.
The issue is P0 and will block the submit queue, so I marked this PR as P0.
bindata and yaml, Gobindata automation
bindata utils for generating, go generate
match server version
gitignore for dirty, ca, rbase, KUBE_ROOT, buildfix
(rebased jul-25,29)
Automatic merge from submit-queue
Add API for StorageClasses
This is the API objects only required for dynamic provisioning picked apart from the controller logic.
Entire feature is here: https://github.com/kubernetes/kubernetes/pull/29006
Automatic merge from submit-queue
Remove redundant pod deletion in scheduler predicates tests and fix taints-tolerations e2e
~~In scheduler predicates test, some tests won't clean pods they created when exit with failure, which may lead to pod leak. This PR is to fix it.~~
Remove redundant pod deletion in scheduler predicates tests, since framework.AfterEach() already did the cleanup work after every test.
Also fix the test "validates that taints-tolerations is respected if not matching", refer to the change on taint-toleration test in #29003, and https://github.com/kubernetes/kubernetes/pull/24134#discussion_r63794924.
Automatic merge from submit-queue
make the resource prefix in etcd configurable for cohabitation
This looks big, its not as bad as it seems.
When you have different resources cohabiting, the resource name used for the etcd directory needs to be configurable. HPA in two different groups worked fine before. Now we're looking at something like RC<->RS. They normally store into two different etcd directories. This code allows them to be configured to store into the same location.
To maintain consistency across all resources, I allowed the `StorageFactory` to indicate which `ResourcePrefix` should be used inside `RESTOptions` which already contains storage information.
@lavalamp affects cohabitation.
@smarterclayton @mfojtik prereq for our rc<->rs and d<->dc story.
Automatic merge from submit-queue
Fix mount collision timeout issue
Short- or medium-term workaround for #29555. The root issue being fixed here is that the recent attach/detach work in the kubelet uses a unique volume name as a key that tracks the work that has to be done for each volume in a pod to attach/mount/umount/detach. However, the non-attachable volume plugins do not report unique names for themselves, which causes collisions when a single secret or configmap is mounted multiple times in a pod.
This is still a WIP -- I need to add a couple E2E tests that ensure that tests break in the future if there is a regression -- but posting for early review.
cc @kubernetes/sig-storage
Ultimately, I would like to refine this a bit further. A couple things I would like to change:
1. `GetUniqueVolumeName` should be a property ONLY of attachable volumes
2. I would like to see the kubelet apparatus for attach/mount/umount/detach handle non-attachable volumes specifically to avoid things like the `WaitForControllerAttach` call that has to be done for those volume types now
Automatic merge from submit-queue
Fix killing child sudo process in e2e_node tests
Fixes#29211.
The context is we are trying to kill a process started as `sudo kube-apiserver`, but `sudo` ignores signals from the same process group. Applying `Setpgid` means the `sudo kill` process won't be in the same process group, so will not fall foul of this nifty feature.
I also took the liberty of removing some code setting `Pdeathsig` because it claims to be doing something in the same area, but actually it doesn't do that at all. The setting is applied to the forked process, i.e. `sudo`, and it means the `sudo` will get killed if we (`e2e_node.test`) die. This (a) isn't what the comment says and (b) doesn't help because sending SIGKILL to the sudo process leaves sudo's child alive.
I didn't use the "hack for linux-only" approach because I think `Setpgid` is available on all platforms that `e2e_node` builds on.
This is the default for etcd2, but etcd3 only listens on 2379.
Specifying the ports keeps things consistent no matter which
version the user has installed.
Automatic merge from submit-queue
Faster test
<!--
Checklist for submitting a Pull Request
Please remove this comment block before submitting.
1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->
In attempting to troubleshoot flakes with this test case I actually wanted to understand how it worked.
There's some poor comments that need work.
I added some additional output which may or may not help in debugging the flakes.
I doubt this fixes the flake.
My major concern is the 'refactor' I did of the test case to batch up runs by sub-test-case. As it stood there was a 200ms pause between each sub, so they should not have interfered with each other. Now they are just started as fast as possible, but only 20 run at a time before moving on to the next 20. I am not sure if I am violating the ethos of the original test case.
Runs on my computer are down from 2m40s -> 40s.
Getting rid of the arbitrary client limiting brings it down to ~12 seconds. 11 to fetch the image and <1 to actually run the tests against the proxies. I can add a zero to the number of loops if you want to hit it harder. It would result in 10x as much text output though.
[]()
Automatic merge from submit-queue
Add support for kubectl create quota command
Follow-up of https://github.com/kubernetes/kubernetes/pull/19625
```
Create a resourcequota with the specified name, hard limits and optional scopes
Usage:
kubectl create quota NAME [--hard=key1=value1,key2=value2] [--scopes=Scope1,Scope2] [--dry-run=bool] [flags]
Aliases:
quota, q
Examples:
// Create a new resourcequota named my-quota
$ kubectl create quota my-quota --hard=cpu=1,memory=1G,pods=2,services=3,replicationcontrollers=2,resourcequotas=1,secrets=5,persistentvolumeclaims=10
// Create a new resourcequota named best-effort
$ kubectl create quota best-effort --hard=pods=100 --scopes=BestEffort
```
Automatic merge from submit-queue
Rework pod waiting mechanism in e2e tests to accept pod and watch based
This PR re-applies #28212 which was reverted in #29223. The only difference is that the initial PR contained also `PodStartTimeout` shortening (see [here](4b0c0bd924)) which might caused the problems. Let's give it a 2nd try. I've tested all the flakes and they were passing on my machine.
@smarterclayton @apelisse ptal
- what the test is doing
- how the test is set up
- subsections of the test setup
additional output
- print time spent getting ready to run proxy attempts
- number of test cases
- multiple attempts of each test case
- how many total proxying attempts will be made
- fast path output now has numerical identity of attempt like error output
- error output has time taken and http status like fast path output
batching runs
- run groups of test cases vs starting all 34*20=680 proxy attempts at
the same time.
- don't wait between starting proxy attempts anymore.
proxy e2e changes
- disable the client side rate limiter
- use `By` construct of ginkgo for inline `STEP` logging
- move the waitGroup add outside of the loop
Automatic merge from submit-queue
Syncing imaging pulling backoff logic
- Syncing the backoff logic in the parallel image puller and the sequential image puller to prepare for merging the two pullers into one.
- Moving image error definitions under kubelet/images
Automatic merge from submit-queue
Change SETUP_NODE to True for node e2e docker validation test.
The continuous node e2e docker validation test is failing because:
```
W0722 00:48:52.163940 1265 image_list.go:85] Could not pre-pull image gcr.io/google_containers/netexec:1.4 exit status 1 output: Cannot connect to the Docker daemon. Is the docker daemon running on this host?
```
This is because jenkins is not added to docker user group.
For other images tested in node e2e, jenkins is added to docker user group when the images are initially created https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/environment/setup_host.sh#L102.
However, in node e2e docker validation test, we are using GCI image which doesn't do that.
So we should use the `SETUP_NODE` option to add user to docker group before test running b6c87904f6/test/e2e_node/e2e_remote.go (L150-L159).
This is only one line change, could you help me review the PR? @wonderfly
Thanks a lot! :)
Automatic merge from submit-queue
test/e2e: plug time.Ticker resource leak.
This commit ensures that `logPodStartupStatus` does not leak
running `time.Ticker` instances. Upon termination of the consuming
routine, we stop the ticker.
Automatic merge from submit-queue
use regular client instead of kubectl in scheduler predicate tests when checking/setting/cleanning taints/labels
The existing implementation in scheduler predicate tests uses kubectl to check/set/clean taints/labels on node, which makes the test very related to kubectl.
This PR is to use regular client instead.
Automatic merge from submit-queue
Revert "Drop support for --gce-service-account, require activated creds"
Reverts kubernetes/kubernetes#28802
This appears to break the soak tests with "invalid grant" errors -- see the recent batch of errors in #27920.
Automatic merge from submit-queue
Allow for overriding throughput in load test
We seem to be already supporting higher throughput that what the default is.
I'm going to increase the throughput in our tests:
- speed up scalability tests
- ensure that what I'm seeing locally is really the repeatable case
This PR is a short preparation for those experiments.
[Ideally, I would like to have kubemark-500 to be finishing within 30 minutes. And I think this should be doable pretty soon.]
@gmarek
Automatic merge from submit-queue
Change some node e2e test to use the prepull image framework.
Fix https://github.com/kubernetes/kubernetes/issues/28868.
Node e2e test framework pre-pulls all images in [image_list.go](bc2f223f5a/test/e2e_node/image_list.go)
All node e2e test should use image from the "image_list". If a test needs new image, we should update the image_list to include the new image.
/cc @kubernetes/sig-node to notice people to use `image_list` when adding test. :)
Automatic merge from submit-queue
add tokenreviews endpoint to implement webhook
Wires up an API resource under `apis/authentication.k8s.io/v1beta1` to expose the webhook token authentication API as an API resource. This allows one API server to use another for authentication and uses existing policy engines for the "authoritative" API server to controller access to the endpoint.
@cjcullen you wrote the initial type
Automatic merge from submit-queue
Start namespace controller in node e2e
Fix https://github.com/kubernetes/kubernetes/issues/28320.
Based on https://github.com/kubernetes/kubernetes/pull/28807, only the last 2 commits are new.
Before this PR, there was no namespace controller running in node e2e test infrastructure. We can not enable the [`delete-namespace`](f2ddd60eb9/test/e2e/framework/test_context.go (L109)) flag in the test framework.
So after the test running, there will be running pod left on the test node. This seems to be acceptable in our test infrastructure because we create an new instance each time.
However, in 1.4 we may want to provide part of the test as node conformance test to the user, they definitely don't want the test to leave tons of pods on their node after test running.
Currently, there is no easy way to only start namespace controller in kube-controller-manager (confirmed with @mikedanese), so in this PR I started a "uncontainerized" one in the test infrastructure.
This PR:
* Started the namespace controller in the node e2e test infrastructure and enable the automatic namespace deletion.
* Change the privileged test to use framework (@yujuhong), so that all node e2e tests are using the framework and test pods will be cleaned up by namespace controller.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Switched watches in tests require ResourceVersion to be passed
For testing the Watches are not sufficient in that it might miss the event of transitioning a Pod from one state to another which might happen before we start Watching events. To remedy this, I'm proposing to switch to Gets to always read the actual state of a Pod.
@smarterclayton this fixes https://github.com/openshift/origin/issues/9192 and hopefully all `gave up waiting for pod...` flakes
[]()
Automatic merge from submit-queue
Don't repeat the program name in healthCheckCommand.String()
The name is in both `Path` and `Args[0]`, so start printing args at 1.
Also refactor to avoid an extra space character in the output.
I pondered whether `healthCheckCommand.String()` should check if the slice is empty, to avoid a panic, but it didn't check for `Cmd==nil` before.
Fixes#29107
Automatic merge from submit-queue
Change the docker validation node e2e test to use gci-canary-test
This PR changed the continuous docker validation node e2e test to use the image config file introduced in https://github.com/kubernetes/kubernetes/pull/28708. @euank
This PR also changed the gci image family from `gci-preview-test` to `gci-canary-test`. @wonderfly
Automatic merge from submit-queue
Return (bool, error) in Authorizer.Authorize()
Before this change, Authorize() method was just returning an error, regardless of whether the user is unauthorized or whether there is some other unrelated error. Returning boolean with information about user authorization and error (which should be unrelated to the authorization) separately will make it easier to debug.
Fixes#27974
Automatic merge from submit-queue
Node E2E: Make it possible to share test between e2e and node e2e
This PR is part of the plan to improve node e2e test coverage.
* Now to improve test coverage, we have to copy test from e2e to node e2e.
* When adding a new test, we have to decide its destiny at the very beginning - whether it is a node e2e or e2e.
This PR makes it possible to share test between e2e and node e2e.
By leveraging the mechanism of ginkgo, as long as we can import the test package in the test suite, the corresponding `Describe` will be run to initialize the global variable `_`, and the test will be inserted into the test suite. (See https://github.com/onsi/composition-ginkgo-example)
In the future, we just need to use the framework to write the test, and put the test into `test/e2e/node`, then it will be automatically shared by the 2 test suites.
This PR:
1) Refactored the framework to make it automatically differentiate e2e and node e2e (Mainly refactored the `PodClient` and the apiserver client initialization).
2) Created a new directory `test/e2e/node` and make it shared by e2e and node e2e.
3) Moved `container_probe.go` into `test/e2e/node` to verify the change.
@kubernetes/sig-node
[]()
Automatic merge from submit-queue
[flake fix] Wait for the podInformer to observe the pod
Fix#29065
The problem is that the rc manager hasn't observed pod1, so it creates another pod and scales down, pod1 might get deleted. To fix it, wait for the podInformer to observe the pod before running the rc manager.
Marked as P0 as it's fixing a P0 flake.
Automatic merge from submit-queue
Drop support for --gce-service-account, require activated creds
Now that `gcloud auth activate-service-account` is in remove support in the test framework for default service accounts -- testing GCE/GKE now requires prior gcloud activation.
This commit ensures that `logPodStartupStatus` does not leak
running `time.Ticker` instances. Upon termination of the consuming
routine, we stop the ticker.
Before this change, Authorize() method was just returning an error,
regardless of whether the user is unauthorized or whether there
is some other unrelated error. Returning boolean with information
about user authorization and error (which should be unrelated to
the authorization) separately will make it easier to debug.
Fixes#27974
