Automatic merge from submit-queue (batch tested with PRs 38788, 38821, 38829)
Node Conformance Test: Fix node conformance test.
The test suite could build on my desktop. However it is failing on jenkins.
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-node-kubelet-conformance/1
It turns out that `docker save $IMAGE -o $FILE` only works for docker 1.12. (My desktop is 1.12) For older version docker, we should use `docker save -o $FILE $IMAGE instead`. (Jenkins is using 1.9.1)
@timstclair Could you help me review this short PR? :)
Automatic merge from submit-queue (batch tested with PRs 38154, 38502)
Rename "release_1_5" clientset to just "clientset"
We used to keep multiple releases in the main repo. Now that [client-go](https://github.com/kubernetes/client-go) does the versioning, there is no need to keep releases in the main repo. This PR renames the "release_1_5" clientset to just "clientset", clientset development will be done in this directory.
@kubernetes/sig-api-machinery @deads2k
```release-note
The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go
```
Automatic merge from submit-queue
make spellcheck for test/*
**What this PR does / why we need it**: Increase code readability
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: only a slight effort
**Release note**:
```release-note
```
Automatic merge from submit-queue
genericapiserver: unify swagger and openapi in config
- make swagger config customizable
- remove superfluous `Config.Enable*` flags for OpenAPI and Swagger.
This is necessary for downstream projects to tweak the swagger spec.
Automatic merge from submit-queue
Node Conformance: Node Conformance CI
For https://github.com/kubernetes/kubernetes/issues/37252.
The first 2 commits of this PR are from #38150 and #38152. Please review those 2 PRs first, they are both minor cleanup.
This PR:
* Add `TestSuite` interface in `test/e2e_node/remote` to separate test suite logic (packaging, deploy, run test) from VM lifecycle management logic, so that different test suites can share the same VM lifecycle management logic.
* Different test suites such as node e2e, node conformance, node soaking, cri validation etc. should implement different `TestSuite`.
* `test/e2e_node/runner/remote` will initialize and run different test suite based on the subcommand.
* Add `run-kubelet-mode` which only starts and monitors kubelet, similar with `run-services-mode`. The reason we need this:
* Unlike node e2e, node conformance test doesn't start kubelet inside the test suite (in fact, in the future node e2e shouldn't do that either), it assumes kubelet is already running before the test.
* In fact, node e2e should use similar node bootstrap script like cluster e2e, and the bootstrap script should initialize the node with all necessary node software including kubelet. However, it's not the case now.
* The easiest way for now is to reuse the kubelet start logic in the test suite. So in this PR, we added `run-kubelet-mode`, and use the test binary as a kubelet launcher to start kubelet before running the test.
* Implement node e2e `TestSuite`.
* Implement node conformance `TestSuite`. Use `docker save` and `docker load` to create and deploy conformance docker image; Start kubelet by running test binary in `run-kubelet-mode`; Run conformance test with `docker run`.
This PR will make it easy to implement continuous integration node soaking test and cri validation test (https://github.com/kubernetes/kubernetes/pull/35266).
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Inode Eviction Test is Flaky
This Pull Request:
Marks the InodeEviciton test as flaky
Increases the timeout for disk pressure because coreos has nearly 2 million inodes.
Decreases the status polling interval so we can see eviction ordering better.
@Random-Liu
Automatic merge from submit-queue
Add a package for handling version numbers (including non-"Semantic" versions)
As noted in #32401, we are using Semantic Version-parsing libraries to parse version numbers that aren't necessarily "Semantic". Although, contrary to what I'd said there, it turns out that this wasn't actually currently a problem for the iptables code, because the regexp used to extract the version number out of the "iptables --version" output only pulled out three components, so given "iptables v1.4.19.1", it would have extracted just "1.4.19". Still, it could be a problem if they later release "1.5" rather than "1.5.0", or if we eventually need to _compare_ against a 4-digit version number.
Also, as noted in #23854, we were also using two different semver libraries in different parts of the code (plus a wrapper around one of them in pkg/version).
This PR adds pkg/util/version, with code to parse and compare both semver and non-semver version strings, and then updates kubernetes to use it everywhere (including getting rid of a bunch of code duplication in kubelet by making utilversion.Version implement the kubecontainer.Version interface directly).
Ironically, this does not actually allow us to get rid of either of the vendored semver libraries, because we still have other dependencies that depend on each of them. (cadvisor uses blang/semver and etcd uses coreos/go-semver)
fixes#32401, #23854
Automatic merge from submit-queue
Add an option to run Job in Density/Load config
cc @timothysc @jeremyeder
@erictune @soltysh - I run this test and it seems to me that Job has noticeably worse performance than Deployment. I'll create an issue for this, but this PR is for easy repro.
Automatic merge from submit-queue (batch tested with PRs 38609, 38227)
On kubemark master, kubelet now runs as a supervisord process and all master components as pods
This PR fixes issue #37485
On kubemark, previously we had a custom setup that runs master components as top-level processes under supervisord, which is lighter than running kubelet and docker.
This PR makes kubelet run as a process under supervisord and the master components (apiserver, controller-manager, scheduler, etcd) as pods, making testing on kubemark mimic real clusters better.
Also, start-kubemark-master.sh now closely resembles cluster/gce/gci/configure-helper.sh, allowing easy integration in future.
cc @kubernetes/sig-scalability @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 38419, 38457, 38607)
Node E2E: Update CVM version to e2e-node-containervm-v20161208-image.
I built the new node e2e image from e2e-node-containervm-v20161208-image.
@timstclair
/cc @kubernetes/sig-node
Automatic merge from submit-queue
test: cleanup test logs for deployments
@mfojtik @janetkuo this will help with the deployment logs (should make them a bit cleaner) ptal
Automatic merge from submit-queue (batch tested with PRs 34002, 38535, 37330, 38522, 38423)
Node E2E: `make test-e2e-node` runs the same test with pr builder by default.
This PR makes `make test-e2e-node` run non-serial, non-flaky, non-slow test by default.
This will make it easier to use.
/cc @timstclair
Automatic merge from submit-queue (batch tested with PRs 37860, 38429, 38451, 36050, 38463)
[Part 2] Adding s390x cross-compilation support for gcr.io images in this repo
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: This PR enables s390x support to kube-dns , pause, addon-manager, etcd, hyperkube, kube-discovery etc. This PR also includes the changes due to which it can be cross compiled on x86 host architecture.
**Which issue this PR fixes#34328
**Special notes for your reviewer**: In existing file "build-tools/build-image/cross/Dockerfile" the repository mentioned for installing cross build tool chains for supporting architecture does not have a tool chain for s390x hence in my PR I am changing the repository so that it will be cross compiled for s390x.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
Allows cross compilation of Kubernetes on x86 host for s390x also enables s390x support to kube-dns , pause, addon-manager, etcd, hyperkube, kube-discovery etc
```
Automatic merge from submit-queue (batch tested with PRs 38354, 38371)
Add GetOptions parameter to Get() calls in client library
Ref #37473
This PR is super mechanical - the non trivial commits are:
- Update client generator
- Register GetOptions in batch/v2alpha1 group
Automatic merge from submit-queue (batch tested with PRs 38278, 37770)
Refactor REST storage to use generic defaults
This removes the repetition in the REST storage builders by moving the logic to `restoptions.ApplyOptions`. `registry.StorageWithCacher`/`generic.StorageDecorator` no longer assume that they can build the `keyFunc` for arbitrary objects. `restoptions.ApplyOptions` uses the `registry.Store`'s `KeyFunc` for its call to `generic.StorageDecorator`.
```release-note
Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data.
```
Automatic merge from submit-queue (batch tested with PRs 36419, 38330, 37718, 38244, 38375)
Guarantees drop packets commands succeed in reboot test
Fixes the main case in #33405 and #36230.
Previous attempted fix in #38057.
During the reboot test, the iptables command that was supposed to take the node offline failed to exec.
Turned out the xtables lock was holding by other processes led to this failure. Logs as below:
```
I1202 20:00:29.686] Dec 2 20:00:29.685: INFO: ssh jenkins@146.148.111.167:22: stdout:
"+ sleep 10
+ sudo iptables -I INPUT 1 -s 127.0.0.1 -j ACCEPT
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?"
I1202 20:00:29.686] Dec 2 20:00:29.685: INFO: ssh jenkins@146.148.111.167:22: stderr: ""
I1202 20:00:29.686] Dec 2 20:00:29.685: INFO: ssh jenkins@146.148.111.167:22: exit code: 0
```
This reboot test won't pass if any one of these iptables commands fails. This PR put "reboot" commands into while loops to guarantee it retries until succeed.
`sudo iptables -t filter -nL` is removed since it is clear now that the `FILTER` rules won't be clobbered.
(Tests passed on local cluster.)
@bprashanth
Automatic merge from submit-queue (batch tested with PRs 36419, 38330, 37718, 38244, 38375)
adjusted timeouts for inode eviction and garbage collection tests
Inode eviction tests appear to run slower on coreos than the other operating systems I tested on.
I adjusted the timeout for the test from 10 to 30 minutes to compensate.
Garbage collection tests also flake occasionally due to timeouts.
I adjusted the timeout for runtime commands from 2 to 3 minutes, and removed an unused constant.
cc: @Random-Liu
Automatic merge from submit-queue (batch tested with PRs 36071, 32752, 37998, 38350, 38401)
Pass addressable values to DeepCopy
Extracted from https://github.com/kubernetes/kubernetes/pull/35728
These are the places we are currently calling DeepCopy incorrectly, and we need to fix, even if we don't pick up the changes to DeepCopy in #35728:
* creating a new cloner means we have no generated functions registered
* passing non-addressable values doesn't pick up generated deep copy functions, and forces us into reflective mode
Automatic merge from submit-queue (batch tested with PRs 36071, 32752, 37998, 38350, 38401)
Add test for concurrent evictions requests
This is a followup PR after #37668.
Add a test case to make sure concurrent eviction requests can be handled.
@davidopp @lavalamp
Automatic merge from submit-queue (batch tested with PRs 35939, 38381, 37825, 38306, 38110)
Moved start-kubemark-master.sh from test/kubemark/ to test/kubemark/r…
Automatic merge from submit-queue
Fix useless uuid in container log path node e2e
@timstclair pointed out there're nits in original PR, ref: https://github.com/kubernetes/kubernetes/pull/34877
So this patch:
1. removed useless uuid
2. change all those strings to const
Thanks. 🐱
