Automatic merge from submit-queue
Updating version check for etcd
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Currently, if you `make build`, etcd is not properly version checked because `etcd -version` does multi-line output. This output cannot be version compared. This small change fixes that.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Fix caching of go dirs during build
Sometimes when you press `^C` during `make` the subsequent attempt to run `make` hangs due to zero-sized cache file for `ALL_GO_DIRS` var in Makefile.generated_files:
```
vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ KUBE_JUNIT_REPORT_DIR=/tmp/art KUBE_COVER=y make test
# hangs...
^CMakefile:279: recipe for target 'generated_files' failed
make: *** [generated_files] Interrupt
vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ ls -l .make/all_go_dirs.mk
-rw-rw-r-- 1 vagrant vagrant 0 Aug 18 15:03 .make/all_go_dirs.mk
```
Corresponding process subtree looks like the following:
```
└─make test
└─make -f Makefile.generated_files generated_files
└─bash -c grep --color=never -l '+k8s:deepcopy-gen=' | xargs -n1 dirname | sort -u
├─grep --color=never -l +k8s:deepcopy-gen=
├─sort -u
└─xargs -n1 dirname
```
Let's remove the cache file if `cache_go_dirs.sh` gets interrupted.
Automatic merge from submit-queue
e2e-runner: Get GCI builtin k8s version from GCS
**What this PR does / why we need it**:
The GCI release qualification tests use builtin k8s version to run tests. They used to get the builtin version by parsing descriptions of the test images, but that's unreliable because the description format could change. This PR makes it to read a file checked in GCS that has the exact k8s version number.
@adityakali @spxtr Can you review?
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
Remove implicit Prometheus metrics from client
**What this PR does / why we need it**: This PR starts to cut away at dependencies that the client has.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
The implicit registration of Prometheus metrics for request count and latency have been removed, and a plug-able interface was added. If you were using our client libraries in your own binaries and want these metrics, add the following to your imports in the main package: "k8s.io/pkg/client/metrics/prometheus".
```
cc: @kubernetes/sig-api-machinery @kubernetes/sig-instrumentation @fgrzadkowski @wojtek-t
Automatic merge from submit-queue
Kubelet: add --container-runtime-endpoint and --image-service-endpoint
Flag `--container-runtime-endpoint` (overrides `--container-runtime`) is introduced to identify the unix socket file of the remote runtime service. And flag `--image-service-endpoint` is introduced to identify the unix socket file of the image service.
This PR is part of #28789 Milestone 0.
CC @yujuhong @Random-Liu
Automatic merge from submit-queue
fix path handling in hack/lib/init.sh
Jenkinsfile pipeline jobs get cloned into "\<project\> (\<branch\>)". As a result, I can't use certain things in `hack/lib/init.sh`.
This is a small fix for that problem.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Automatic merge from submit-queue
Implement TLS bootstrap for kubelet using `--experimental-bootstrap-kubeconfig` (2nd take)
Ref kubernetes/features#43 (comment)
cc @gtank @philips @mikedanese @aaronlevy @liggitt @deads2k @errordeveloper @justinsb
Continue on the older PR https://github.com/kubernetes/kubernetes/pull/30094 as there are too many comments on that one and it's not loadable now.
Automatic merge from submit-queue
ImagePolicyWebhook Admission Controller
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: This is an implementation of the [image provenance proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/image-provenance.md). It also includes the API definitions by @Q-Lee from https://github.com/kubernetes/kubernetes/pull/30241
**Special notes for your reviewer**:
Please note that this is the first admission controller to make use of the admission controller config file (`--admission-controller-config-file`). I have defined a format for it but we may want to double check it's adequate for future use cases as well.
The format defined is:
```
{
"imagePolicy": {
"kubeConfigFile": "path/to/kubeconfig/for/backend",
"allowTTL": 50, # time in s to cache approval
"denyTTL": 50, # time in s to cache denial
"retryBackoff": 500, # time in ms to wait between retries
"defaultAllow": true # determines behavior if the webhook backend fails
}
}
```
(or yaml)
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Adding ImagePolicyWebhook admission controller.
```
Automatic merge from submit-queue
Basic scaler/reaper for petset
Currently scaling or upgrading a petset is more complicated than it should be. Would be nice if this made code freeze on friday. I'm planning on a follow up change with generation number and e2es post freeze.
Automatic merge from submit-queue
Scheduledjobs e2e
@janetkuo resubmitted e2e for SJ, I've updated all scripts to consume `KUBE_RUNTIME_CONFIG` properly in 2nd commit, ptal
Automatic merge from submit-queue
When running inside docker, activate service account ASAP
Also switching to just use `GOOGLE_APPLICATION_CREDENTIALS`, rather than both.
x-ref https://github.com/kubernetes/test-infra/issues/318
Automatic merge from submit-queue
Gubernator bug fixes: mv and GCS bucket permissions
Fixed issue where results file was not moved correctly, and also the permissions issue with the GCS bucket.
Will rebase after #30414 is merged
@timstclair
Automatic merge from submit-queue
Add a short `-n` for `kubectl --namespace`
fixes#24078
`--namespace` is a very common flag for nearly every `kubectl` command we have. We should claim `-n` for it.
Automatic merge from submit-queue
add --raw for kubectl get
Adds a `--raw` option to `kubectl get` that allow you specify your URI, but use the transport built by `kubectl`. This is especially useful when working with secured environments that require authentication and authorization to hit non-api endpoints. For example, `kubect get --raw /metrics` or if you want to debug a watch with a view at the exact data `kubectl get --raw '/api/v1/namespaces/one/replicationcontrollers?watch=true'`.
@kubernetes/kubectl
@fabianofranz fyi
Automatic merge from submit-queue
Disable linux/ppc64le compilation by default
Work-around for #30384.
I'm still testing this locally to see if it actually works. The build is slow. (PR Jenkins won't tell us whether this fixes ppc.)
cc @Random-Liu @spxtr @david-mcmahon @luxas
New flag --container-runtime-endpoint (overrides --container-runtime)
is introduced to kubelet which identifies the unix socket file of
the remote runtime service. And new flag --image-service-endpoint is
introduced to kubelet which identifies the unix socket file of the
image service.
Automatic merge from submit-queue
Move UTs that block on apiserver to integration tests.
In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail. These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.
This PR simply shuffles those tests to integration tests, which is where they should be.
/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90
Automatic merge from submit-queue
Revert "Revert "syncNetworkUtil in kubelet and fix loadbalancerSourceRange on GCE
Reverts kubernetes/kubernetes#30729
Automatic merge from submit-queue
Add GUBERNATOR flag which produces g8r link for node e2e tests
When you run 'make tests-e2e-node REMOTE=true GUBERNATOR=true' outputs a URL to view the test results on Gubernator. ~~Should work after my PR for Gubernator is merged.~~
@timstclair
Automatic merge from submit-queue
Quobyte Volume plugin
@quofelix and myself developed a volume plugin for [Quobyte](http://www.quobyte.com) which is a software-defined storage solution. This PR allows Kubernetes users to mount a Quobyte Volume inside their containers over Kubernetes.
Here are some further informations about [Quobyte and Storage for containers](http://www.quobyte.com/containers)
The admission controller adds a default class to PVCs that do not require any
specific class. This way, users (=PVC authors) do not need to care about
storage classes, administrator can configure a default one and all these PVCs
that do not care about class will get the default one.
Automatic merge from submit-queue
kubelet eviction on inode exhaustion
Add support for kubelet to monitor for inode exhaustion of either image or rootfs, and in response, attempt to reclaim node level resources and/or evict pods.
Automatic merge from submit-queue
Add annotations to the PodSecurityPolicy Provider interface
@pweil- is this what you were thinking in terms of API changes? I really like to avoid functions with more than 2 return values, but couldn't think of a cleaner approach in this case.
Automatic merge from submit-queue
Expose flags for new NodeEviction logic in NodeController
Fix#28832
Last PR from the NodeController NodeEviction logic series.
cc @davidopp @lavalamp @mml
Automatic merge from submit-queue
Allow a flag that forces kubelet to have a valid kubeconfig
`--require-kubeconfig` forces the kubelet to use the kubeconfig for all
APIserver communication, and exit cleanly. Allows cluster lifecycle to loop waiting for config to be available.
Fixes#30515
A follow up PR will handle the issue discovered where the DefaultCluster rules applied to kubeconfig allow a malicious party who can bind to localhost:8080 to take advantage of an admin misconfiguration.
@lukemarsden @mikedanese
```release-note
The Kubelet now supports the `--force-kubeconfig` option which reads all client config from the provided `--kubeconfig` file and will cause the Kubelet to exit with error code 1 on error. It also forces the Kubelet to use the server URL from the kubeconfig file rather than the `--api-servers` flag. Without this flag set, a failure to read the kubeconfig file would only result in a warning message.
In a future release, the value of this flag will be defaulted to `true`.
```
Automatic merge from submit-queue
Create a kubekins-e2e image with runner and dependencies
Update dockerized-e2e-runner.sh to just run an image with the appropriate e2e environment variables set.
e2e-runner.sh, upload-to-gcs.sh and e2e.go now live inside the container.
Automatic merge from submit-queue
syncNetworkUtil in kubelet and fix loadbalancerSourceRange on GCE
fixes: #29997#29039
@yujuhong Can you take a look at the kubelet part?
@girishkalele KUBE-MARK-DROP is the chain for dropping connections. Marked connection will be drop in INPUT/OUTPUT chain of filter table. Let me know if this is good enough for your use case.
Automatic merge from submit-queue
Fix subtle build breakage
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
NB: this requires a `make clean` to activate.
@lavalamp @jbeda @quinton-hoole @david-mcmahon
Automatic merge from submit-queue
Let load and density e2e tests use GC if it's on
I've run the 100 and 500 nodes tests and they both pass.
The test-infra half of the PR is https://github.com/kubernetes/test-infra/pull/369
cc @lavalamp
Automatic merge from submit-queue
Add CloudStack cloud provider (extended and refactored)
This PR is superseding PR #26165 is which some groundwork for this PR has been done. So this PR now fixes#26165 and fixes#26045.
I've been in contact with @ngtuna about this updated version of his earlier work (which is still in this PR as one squashed commit) and he has given his 👍 for this 😉
This PR adds additional logic for allocating and associating a public IP, if the `—load-balancer-ip` option is not used. It will do proper management of public IP’s that are allocated by this provider ( so IP’s that are no longer needed/used will also be released again).
Additionally the provider can now also work with CloudStack projects and advanced (VPC) networks. And lastly the Zone interface now returns an actual zone (supplied by the cloud config), a few logical errors are fixed and the first few tests are added.
All the functionality is extensively tested against both basic and advanced (VPC) networks and of course all new and existing (integration) tests are all passing.
Automatic merge from submit-queue
Install bc in the kubekins-test image
I finally figured out those
```
/dev/fd/63: line 124: bc: command not found
/dev/fd/63: line 125: bc: command not found
```
lines that keep appearing in build logs.
Image is not yet pushed.
Automatic merge from submit-queue
Update core etcd references to use 3.0.4
This updates the core references to use 3.0.4.
There are still legacy references in the code base that should be cleaned, or just removed but I'm reluctant to purge.
/cc @kubernetes/sig-scalability
This commit adds logic for allocating and associating a public IP, if the `—load-balancer-ip` option is not used. It will do proper management of IP’s that are allocated by this provider, so IP’s that are no longer needed/used will also be released again.
Additionally the provider can now also work with CloudStack projects and advanced (VPC) networks.
Lastly the Zone interface now returns an actual zone (supplied by the cloud config), a few logical errors are fixed and the first few tests are added.
All the functionality is extensively tested against both basic and advanced (VPC) networks.
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
