Automatic merge from submit-queue
kubectl: apply prune should fallback to basic delete when a resource has no reaper
Fixes#34790
cc @kubernetes/kubectl @MrHohn
Automatic merge from submit-queue
Add global timeout flag
**Release note**:
```release-note
Add a new global option "--request-timeout" to the `kubectl` client
```
UPSTREAM: https://github.com/kubernetes/client-go/pull/10
This patch adds a global timeout flag (viewable with `kubectl -h`) with
a default value of `0s` (meaning no timeout).
The timeout value is added to the default http client, so that zero
values and default behavior are enforced by the client.
Adding a global timeout ensures that user-made scripts won't hang for an
indefinite amount of time while performing remote calls (right now, remote
calls are re-tried up to 10 times when each attempt fails, however, there is
no option to set a timeout in order to prevent any of these 10 attempts from
hanging indefinitely).
**Example**
```
$ kubectl get pods # no timeout flag set - default to 0s (which means no
timeout)
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$ kubectl get pods --request-timeout=0 # zero means no timeout no timeout flag set
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$kubectl get pods --request-timeout=1ms
Unable to connect to the server: net/http: request canceled while
waiting for connection (Client.Timeout exceeded while awaiting headers)
```
This patch adds a global timeout flag (viewable with `kubectl -h`) with
a default value of `0s` (meaning no timeout).
The timeout value is added to the default http client, so that zero
values and default behavior are enforced by the client.
**Example**
```
$ kubectl get pods # no timeout flag set - default to 0s (which means no
timeout)
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$ kubectl get pods --timeout=0 # zero means no timeout no timeout flag set
NAME READY STATUS RESTARTS AGE
docker-registry-1-h7etw 1/1 Running 1 2h
router-1-uv0f9 1/1 Running 1 2h
$kubectl get pods --timeout=1ms
Unable to connect to the server: net/http: request canceled while
waiting for connection (Client.Timeout exceeded while awaiting headers)
```
Add a way to set resource limits/requests on running pods
Ref: https://github.com/kubernetes/kubernetes/issues/21648
I squashed the commits to make rebasing easier
Change log:
- fixed a typo that caused the command to be run with kubectl set set instead of the correct kubectl set limit
- added a ResourcesWithPodTemplates to pkg/kubectl/cmd/util/factory.go
instead of hardcoding these resources move there description all in one place
- Fixing some of the flow control in kubectl set limit
- update the help info
- changed the name of ResourcesWithPodTemplates to ResourcesWithPodSpecs to more accuratly describe what it is doing
and changed the variable names to lower case to conform to go's variable naming convention
- changing the name of the command from 'set limit' to 'set resources'
- Adding the new file pkg/kubectl/cmd/set/set_resources.go
- changes to the test cases to reflect the change from 'kubectl set limit' to 'kubectl set resources'
- comment removed
- adding the man page to the git repository attempting to fix Jenkins tests
- adding the user guide
- fixed a few typos
- typo in hack/cmd-test.sh
- implamenting suggestions for command help text
- adding the dry-run flag
- removing the "remove" option in favor of zeroing out request/limits in order to remove them
- changed limits/requests to requests/limit
- changing ResourcesWithPodSpec
- updated generated docs and removed whitespace
- change priint on success message from "resource limits/requests updated" to "resource requirements updated"
- minor rebasing issues - 'hack/test-cmd.sh' now passes
- cmdutil.PrintSuccess added another argument
- fixing mungedocs failure
- removed whitespace from hack/make-rules/test-cmd.sh and an erroneous entry from pkg/cloudprovider/providers/openstack/MAINTAINERS.md
- fixed typo in Short: field of the cobra command
- rebased
- Creating a new factory in the ResourcesWithPodSpecs() so that the testing will pass
- changing ResourcesWithPodSpecs, it doesn't need to be a method of factory
Automatic merge from submit-queue
add UpdateRuntimeConfig interface
Expose UpdateRuntimeConfig interface in RuntimeService for kubelet to pass a set of configurations to runtime. Currently it only takes PodCIDR.
The use case is for kubelet to pass configs to runtime. Kubelet holds some config/information which runtime does not have, such as PodCIDR. I expect some of kubelet configurations will gradually move to runtime, but I believe cases like PodCIDR, which dynamically assigned by k8s master, need to stay for a while.
Automatic merge from submit-queue
Increase build verbosity for verify-generated-{protobuf,runtime}.sh
Print out more details of the build process to help with debugging #34675.
Automatic merge from submit-queue
kubeadm implement preflight checks
Checks that user running kubeamd init and join is root and will only execute
command if user is root. Moved away from using kubectl error handling to
having kubeadm handle its own errors. This should allow kubeadm to have
more meaningful errors, exit codes, and logging for specific kubeadm use
cases.
fixes#33908
Automatic merge from submit-queue
add flag for enabling RBAC on local-up-cluster
Enables cases like `ALLOW_ANY_TOKEN=true ENABLE_RBAC=true hack/local-up-cluster.sh ` to startup an api server so you can test RBAC against it.
@pweil- You can try this using `oc login localhost:6443 --token=username/group1,group2,system:masters`. Then you should have full rights. You can use `oc login --token=username` or `kubectl create -f - --token=otheruser/othergroup` to simulate other users.
Add skip-preflight-checks to known flags.
Fix bug with preflight checks not returning system is-active as errors.
Fix error handling to use correct function.
Automatic merge from submit-queue
kube-dns: Add --dns-bind-address flag
Currently, `kube-dns` always binds to `0.0.0.0` when starting the SkyDNS server. Sometimes it's useful to have it bind to a different address for testing. So add a new `--dns-bind-address` flag that can be used to configure this (default is `0.0.0.0`).
Automatic merge from submit-queue
Generate unique Operation IDs for root OpenAPI spec
This PR adds a customization method GetOperationID to OpenAPI spec generation and then use it to make sure root spec has unique operation IDs by mostly adding GroupVersion to the start of operation ID.
Automatic merge from submit-queue
Add gcl cluster logging test
This PR changes default logging destination for tests to gcp and adds test for cluster logging using google cloud logging
Fix#20760
Automatic merge from submit-queue
Add default cluster role bindings
Add default cluster roles bindings to rbac bootstrapping. Also adds a case for allowing escalation when you have no authenticator.
@liggitt I expect you may need to make peace with this.
Automatic merge from submit-queue
log-dump.sh: Add a LOG_DUMP_USE_KUBECTL mode, refactor
**What this PR does / why we need it**: This refactors log-dump.sh so that it can optionally just use `"kubectl get node"` output, for dumping logs off a cluster that was brought up using something other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably fail to hit the apiserver and all bets are off, but for the 95% case where you're debugging tests, this is good. To cover the remaining 5%, you could actually squirrel away the relevant external IPs after the e2e.go:Up stage (because we better be able to hit the apiserver then) and fetch them from cache later.
**Special notes for your reviewer**: Sorry for so many changes. :)
This refactors log-dump.sh so that it can optionally just use "kubectl
get node" output, for dumping logs off a cluster that was brought up
using sometihng other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably
fail to hit the apiserver and all bets are off, but for the 95% case
where you're debugging tests, this is good. To cover the remaining 5%,
you could actually squirrel away the relevant external IPs after the
e2e.go:Up stage (because we better be able to hit the apiserver then)
and fetch them from cache later.
Automatic merge from submit-queue
hack: update cherry-pick script to show subject when patch is split
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
This is a continuation of #34049, which worked, but then I realized some patch files have more than one subject, see example: https://github.com/kubernetes/kubernetes/pull/34228
This cleans the output so it looks like this:
```
Automated cherry pick of #32593
Cherry pick of #32593 on release-1.4.
#32593: Fix audit_test regex for iso8601 timestamps
```
pretty!!!
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
Signed-off-by: Jess Frazelle <acidburn@google.com>
Automatic merge from submit-queue
hack/e2e.go: listen for Interrupt signal and attempt cleanup if run with --down
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: adds a listener on ^C, so if `hack/e2e.go` is run with `--down` and the user ^C, then it will attempt to cleanup the resources.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Signed-off-by: Jess Frazelle <me@jessfraz.com>
Automatic merge from submit-queue
Generate and store federation OpenAPI spec in source tree
A follow up of #33628 to also add federation spec to source tree.
Automatic merge from submit-queue
controller: save older revisions for Deployment's replica sets
@jwforres the only usable way I could find for multiple old revisions for a single replica set is to stuff them as comma-separated values.
@kubernetes/deployment this retains old revisions served by a replica set inside an annotation.
Fixes https://github.com/kubernetes/kubernetes/issues/33844
Automatic merge from submit-queue
user-parameter support for API_BIND_ADDRESS in local-up-cluster.sh
**What this PR does / why we need it**:
Fixes#34262.
user-parameter for the secure api bind address when running a local dev cluster.
Default bind-address is `0.0.0.0` if not specified by the user. I'm guessing this was the intended behaviour of the original code.
Allow to reach the secure API on different interfaces/IPs when running a dev local-up-cluster.
Automatic merge from submit-queue
Add cgroup-driver and cgroups-per-qos flags to kubelet
Add the flags needed to support pod-level cgroups to kubelet.
/cc @vishh @dchen1107 @dubstack
Automatic merge from submit-queue
add delete-namespace-on-failure flag
I have been doing this for a while.
Setting `--delete-namespace=false --clean-start=true` only works if you have only one e2e test running in a loop.
This PR lets someone to set `delete-namespace-on-failure=false` and run multiple tests in parallel and preserve the crime scene. It makes it easier to reproduce failures.
Let me know if this is worth it or there are some other tricks I am not aware.
Automatic merge from submit-queue
Ignore troublesome paths that cause coverage to fail
**What this PR does / why we need it**:
`KUBE_COVER=y make check` currently fails, this patch fixes it.
**Which issue this PR fixes**
fixes#31691
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
This avoids the whole command failing because of errors like the following:
```
# cover k8s.io/kubernetes/pkg/client/restclient
cover: internal error: block 268 overlaps block 270
```
Automatic merge from submit-queue
Update client config invalid option errors to be more specific
This patch adds better error handling for cases where a global option (such as --context or --cluster) causes an invalid config to be returned.
```release-note
release-note-none
```
This patch provides a more relevant error message when a client
configuration option is passed with an invalid or non-existent value.
`$ kubectl get pods --cluster="non-existent"`
```
error: No configuration file found, please login or point to an existing
file
```
`$ kubectl get pods --cluster="non-existent"`
```
error: cluster "non-existent" does not exist
```
Automatic merge from submit-queue
Fix a typo in openapi spec generation
A typo in util.sh resulted in the wrong spec stored in source tree. The commit should be explanatory.
Automatic merge from submit-queue
Speed up dockerized builds
This PR speeds up dockerized builds. First, we make sure that we are as incremental as possible. The bigger change is that now we use rsync to move sources into the container and get data back out.
To do yet:
* [x] Add a random password to rsync. This is 128bit MD4, but it is better than nothing.
* [x] Lock down rsync to only come from the host.
* [x] Deal with remote docker engines -- this should be necessary for docker-machine on the mac.
* [x] Allow users to specify the port for the rsync daemon. Perhaps randomize this or let docker pick an ephemeral port and detect the port?
* [x] Copy back generated files so that users can check them in. This is done for `zz_generated.*` files generated by `make generated_files`
* [x] This should include generated proto files so that we can remove the hack-o-rama that is `hack/hack/update-*-dockerized.sh`
* [x] Start "versioning" the build container and the data container so that the CI system doesn't have to be manually kicked.
* [x] Get some benchmarks to qualify how much faster.
This replaces #28518 and is related to #30600.
cc @thockin @spxtr @david-mcmahon @MHBauer
Benchmarks by running `make clean ; sync ; time bash -xc 'time build/make-build-image.sh ; time sync ; time build/run.sh make ; time sync; time build/run.sh make'` on a GCE n1-standard-8 with PD-SSD.
| setup | build image | sync | first build | sync | second build | total |
|-------|-------------|----- |----------|------|--------------|------|
| baseline | 0m11.420s | 0m0.812s | 7m2.353s | 0m42.380s | 7m8.381s | 15m5.348s |
| this pr | 0m10.977s | 0m15.168s | 7m31.096s | 1m55.692s | 0m16.514s | 10m9.449s |
Automatic merge from submit-queue
Add support for vpshere cloud provider in kubeup
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
vSphere cloud provider added in 1.3 was not configured when deploying via kubeup
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Add support for vSphere Cloud Provider when deploying via kubeup on vSphere.
```
When deploying on vSphere using kube up add configuration
for vSphere cloud provider.
Automatic merge from submit-queue
Add option to set a service nodeport
**Release note**:
```release-note
Add kubectl --node-port option for specifying the service nodeport
```
This patch adds the option to set a nodeport when creating a NodePort
service. In case of a port allocation error due to a specified port
being out of the valid range, the error now includes the valid
range. If a `--node-port` value is not specified, it defaults to zero, in
which case the allocator will default to its current behavior of
assigning an available port.
This patch also adds a new helper function in `cmd/util/helpers.go` to
retrieve `Int32` cobra flags.
**Example**
```
# create a nodeport service with an invalid port
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=1
The Service "mynodeport" is invalid: spec.ports[0].nodePort: Invalid
value: 1: provided port is not in the valid range. Valid ports range
from 30000-32767
# create a nodeport service with a valid port
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=30000
service "mynodeport" created
# create a nodeport service with a port already in use
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=30000
The Service "mynodeport" is invalid: spec.ports[0].nodePort: Invalid value: 3000: provided port is already allocated
$ kubectl describe service mynodeport
Name: mynodeport
Namespace: default
Labels: app=mynodeport
Selector: app=mynodeport
Type: NodePort
IP: 172.30.81.254
Port: 8080-7777 8080/TCP
NodePort: 8080-7777 30000/TCP
Endpoints: <none>
Session Affinity: None
No events.
```
@fabianofranz
Automatic merge from submit-queue
Daemonset in federated clientset
Only hack/update-codegen.sh was edited manually.
cc: @nikhiljindal @quinton-hoole @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Allow empty annotation values
Related downstream issue: https://github.com/openshift/origin/issues/11175
**Release note**:
```release-note
release-note-none
```
Annotations with empty values can be used, for example, in diagnostics
logging. This patch removes the client-side check for empty values in
an annotation key-value pair.
**Before**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
error: invalid annotation format: node-selector=
```
**After**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
pod "zookeper-1" annotated
```
```
$ kubectl get po/zookeeper-1 --template='{{.metadata.annotations}}'
map[... node-selector: test-label:test]
```
@fabianofranz @liggitt
Annotations with empty values can be used, for example, in diagnostics
logging. This patch removes the client-side check for empty values in
an annotation key-value pair.
**Before**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
error: invalid annotation format: node-selector=
```
**After**
```
$ kubectl annotate pod zookeeper-1 node-selector="" --overwrite
pod "zookeper-1" annotated
```
```
$ kubectl get po/zookeeper-1 --template='{{.metadata.annotations}}'
map[... node-selector: test-label:test]
```
Automatic merge from submit-queue
Customizable genericapiserver handler chain
Rebase of https://github.com/kubernetes/kubernetes/pull/33662. This is @sttts, but I rebased it for him since he's out this week.
Automatic merge from submit-queue
Add local option to annotate
**What this PR does / why we need it**:
Add the --local option to be both consistent with other commands and so it can be used with "set selector" and "create service".
**Which issue this PR fixes**
Related: #7296
**Special notes for your reviewer**:
None
**Release note**:
```release-note
Add a new option "--local" to the `kubectl annotate`
```
Automatic merge from submit-queue
Add local option to the label command
**What this PR does / why we need it**:
Add the --local option to be both consistent with other commands and so it can be used with "set selector" and "create service".
**Which issue this PR fixes**
Related: #7296
**Special notes for your reviewer**:
None
**Release note**:
```release-note
Add a new option "--local" to the `kubectl label`
```
Automatic merge from submit-queue
Add deployment to federation clientset
Only hack/update-codegen.sh was updated manually.
cc: @quinton-hoole @nikhiljindal @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
specify flags to test-integration
Allows a specific test to be run in test-integration: ` hack/test-integration.sh auth -test.run=TestKindAuthorization`
@eparis I don't know how good or bad my bash is.
Automatic merge from submit-queue
CRI: Remove the mount name and port name.
Per discussion on https://github.com/kubernetes/kubernetes/issues/33873.
Currently the mount name is not being used and also involves some
incorrect usage (sometimes it's referencing a mount name, sometimes
it's referecing a volume name), so we decide to remove it from CRI.
The port name is also not used, so remove it as well.
Fix#33873Fix#33526
/cc @kubernetes/sig-node @kubernetes/sig-rktnetes
Automatic merge from submit-queue
hack: Update cherry-pick script to include original PR subject
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Updates the cherrypick script to include the original PR subject.
We need it because I know nothing about the original PR based off a number and this is more expressive.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
This adds a flag to override the version we're deploying with kops for
e2es. (The kops experience is disconnected from the actual source tree
we're running tests from, which is similar to GKE.)
We weren't getting incremental builds because of new test only subpackages. Our
voodoo combo of 'go install' and 'go test -c' didn't cache things like
'test/e2e_node/services'. Add the '-i' flag to 'go test' to install test only
dependencies too.
make-generated-{protobuf,runtime}.sh was doing some really nasty stuff with how
the build container was managed in order to copy results out. Since we have
more flexibility to grab results out of the build container, we can now avoid
all of this. Ideally we wouldn't have `hack` calling `build` at all, but we
aren't there yet.
We also add "version" to all docker images and containers
This version is to be incremented manually when we change the shape of the build
image (like changing the golang version or the set of volumes in the data
container). This will delete all older versions of images and containers when
the version is different.
Automatic merge from submit-queue
add optional env var for allowing version skew in test cmd
If you only make a change to a server component and then re-run `test-cmd` you get a skew error. This is really annoying. I've left the default as it was (force version match), but now also allow `ALLOW_SKEW=true hack/test-cmd.sh` to skip the check.
@eparis worth a quick bash check, but I think I got it.
Automatic merge from submit-queue
Use patched golang1.7.1 for cross-builds targeting darwin
This PR extends #32517 to use the patched go1.7.1 introduced by that PR to build all darwin targets (e.g. kubectl).
This is necessary because binaries built with earlier versions of Go regularly segfault on macOS Sierra (see #32999 and #33070).
This solution is somewhat hacky, but we intend to cherry-pick this to 1.4, and switching all of 1.4 to build with go1.7.1 is very high risk.
I haven't yet pushed the cross build image yet, so this will fail to build. Will test locally and update with results.
First step of fixing #33801.
cc @luxas @pwittrock @david-mcmahon @liggitt @smarterclayton @jfrazelle @Starefossen @gerred
Automatic merge from submit-queue
Add OpenAPI specs to source tree
Similar to swagger spec, adding openapi specs in a separate folder api/openapi-spec
To make sure we generate a consistent spec, parameters need to be sorted first.
Automatic merge from submit-queue
add anytoken authenticator
Adds `--insecure-allow-any-token` as a flag to the API server to create an authenticator that will accept any bearer token and transform it into a user by parsing it out as `username/group1,group2,...`.
This gives an easy way to identify as a user and check permissions:
```bash
ALLOW_ANY_TOKEN=true hack/local-up-cluster.sh
kubectl config set-cluster local-kube --server=https://localhost:6443 --insecure-skip-tls-verify=true
kubectl config set-credentials david --token=david/group1
kubectl config set-context local --cluster=local-kube --user=david
kubectl config use-context local
```
@kubernetes/sig-auth
Automatic merge from submit-queue
Fixing local cluster up
hack: fix local-cluster-up advertise_address default value
Fixes: 9fc1d61
Signed-off-by: André Martins <aanm90@gmail.com>
ping @dims
Automatic merge from submit-queue
Allow anonymous API server access, decorate authenticated users with system:authenticated group
When writing authorization policy, it is often necessary to allow certain actions to any authenticated user. For example, creating a service or configmap, and granting read access to all users
It is also frequently necessary to allow actions to any unauthenticated user. For example, fetching discovery APIs might be part of an authentication process, and therefore need to be able to be read without access to authentication credentials.
This PR:
* Adds an option to allow anonymous requests to the secured API port. If enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of `system:anonymous` and a group of `system:unauthenticated`. Note: this should only be used with an `--authorization-mode` other than `AlwaysAllow`
* Decorates user.Info returned from configured authenticators with the group `system:authenticated`.
This is related to defining a default set of roles and bindings for RBAC (https://github.com/kubernetes/features/issues/2). The bootstrap policy should allow all users (anonymous or authenticated) to request the discovery APIs.
```release-note
kube-apiserver learned the '--anonymous-auth' flag, which defaults to true. When enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of 'system:anonymous' and a group of 'system:unauthenticated'.
Authenticated users are decorated with a 'system:authenticated' group.
NOTE: anonymous access is enabled by default. If you rely on authentication alone to authorize access, change to use an authorization mode other than AlwaysAllow, or or set '--anonymous-auth=false'.
```
c.f. https://github.com/kubernetes/kubernetes/issues/29177#issuecomment-244191596
Automatic merge from submit-queue
Change minion to node
Continuation of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed - just let me know.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Contination of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
I rolled back some of this from a previous commit because it just got
to big/messy. Will follow up with additional PRs
Signed-off-by: Doug Davis <dug@us.ibm.com>
Automatic merge from submit-queue
Add jbeda to OWNERS for build, cluster, hack
I'd like to start getting more PRs to review here. I promise I'll stay on top of them now that I have new and improved mail filters.
This splits off all the bash stuff into an interface, and plumbs
through a separate interface to bring up a cluster using "kops"
instead. Right now it assumes kops == AWS.
Automatic merge from submit-queue
build kube-discovery and kubeadm with release
This will start publishing binaries from our ci builds.
@errordeveloper @dgoodwin
Automatic merge from submit-queue
Add a lower-bound for conntrack (128k default)
Fixes#32435
kube-proxy will use a lower-bound, default 128k, for conntrack records. This
is in concert with the per-core scaling factor.
Automatic merge from submit-queue
local-up-cluster.sh: add SERVICE_CLUSTER_IP_RANGE as option
Allows the user the use an environment variable to specify the SERVICE_CLUSTER_IP_RANGE without modifying the `hack/local-up-cluster.sh` script.
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
Make upload-to-gcs.sh use a different prefix for tests in other repos.
For kubernetes/test-infra#476.
Also, put the repo in the build metadata so Gubernator can link to the
right thing for postsubmit jobs.
For PR #$NUM in a given repo, the logs will go to these places:
kubernetes/kubernetes => /pr-logs/pull/$NUM
kubernetes/test-infra => /pr-logs/pull/test-infra/$NUM
google/cadvisor => /pr-logs/pull/google_cadvisor/$NUM
Tested with a manual harness. Note that this leaves the latest/indirect
untouched as well as postsubmit jobs-- *job names* are still unique, the
main issue was that PR numbers might collide. Disturbing as little
structure as possible reduces how many things will break.
Automatic merge from submit-queue
Un-xmlWrap the Test function.
#33361 is pretty much useless.
We should probably take a look at better integrating the new JUnit results into the munger/testgrid/gubernator, but this should make things better for now.
For kubernetes/test-infra#476.
Also, put the repo in the build metadata so Gubernator can link to the
right thing for postsubmit jobs.
For PR #$NUM in a given repo, the logs will go to these places:
kubernetes/kubernetes => /pr-logs/pull/$NUM
kubernetes/test-infra => /pr-logs/pull/test-infra/$NUM
google/cadvisor => /pr-logs/pull/google_cadvisor/$NUM
Tested with a manual harness. Note that this leaves the latest/indirect
untouched as well as postsubmit jobs-- *job names* are still unique, the
main issue was that PR numbers might collide. Disturbing as little
structure as possible reduces how many things will break.
Automatic merge from submit-queue
kubeadm
**What this PR does / why we need it**:
This PR add alpha version of `kubeadm` tool, which allows user to boostrap a cluster rather quite easily. This is the initial contribution from @kubernetes/sig-cluster-lifecycle members, who's aim is to build easy-to-use tools that help to operate a cluster throughout its lifetime.
**Which issue this PR fixes**: a leap towards kubernetes/features#11
**Special notes for your reviewer**: previously seen by many folks in #31221
**Release note**:
```release-note
`kubeadm` (alpha) provides an easy way to securely bootstrap Kubernetes on Linux, see http://kubernetes.io/docs/kubeadm/
```
Automatic merge from submit-queue
Added e2e framweork and a simple test for HA master.
Added e2e framweork for testing HA master. Added one simple e2e test for HA master that first grows and then shrinks GCE cluster.
Automatic merge from submit-queue
Automatic generation of man pages
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Currently genman supports only kubectl command, so this PR will enable genman to generate man pages for rest of the command like - kube-apiserver, kube-controller-manager, kube-proxy, kube-scheduler, kubelet
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
N/A
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
unify available api group versions in our scripts
There are currently many parallel lists of available group versions with slightly different syntaxes in each one. This collapses them into a single list for us to maintain.
Also caught spots where the lists didn't match before.
@sttts @ncdc
Automatic merge from submit-queue
Make node E2E tests more transparent
Add some logging and minor code reorg to make the node E2E tests a little more transparent and understandable.
Automatic merge from submit-queue
Integration tests: Migrate scheduler perf to the integration suite, s…
*Problem*
issues like https://github.com/kubernetes/kubernetes/pull/32384/ happen because we dont compile the integration tests as part of the build
*Solution*
This pr will make it so scheduler_perf is included in the integration tests, but it will run in short mode that only makes sure compilation happens . it also removes the over generalized test/component directory which only has one entry
```
test/integration/replicationcontroller
test/integration/scheduler
test/integration/scheduler
test/integration/scheduler_perf
test/integration/scheduler_perf
test/integration/secrets
test/integration/serviceaccount
test/integration/storageclasses
```
Automatic merge from submit-queue
Refactor cert utils into one pkg, add funcs from bootkube for kubeadm to use
**What this PR does / why we need it**:
We have ended-up with rather incomplete and fragmented collection of utils for handling certificates. It may be worse to consider using `cfssl` for doing all of these things, but for now there is some functionality that we need in `kubeadm` that we can borrow from bootkube. It makes sense to move the utils from bookube into core, as discussed in #31221.
**Special notes for your reviewer**: I've taken the opportunity to review names of existing funcs and tried to make some improvements in that area (with help from @peterbourgon).
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
move registry packages for all API groups
This continues the pattern of `registry/<group>/resource` for our backing storage. This entire pull is nothing but moves. I'll reswizzle the actual storage next, but these are cargo-culted everywhere, so I want to lay this down early.
@sttts @ncdc
Automatic merge from submit-queue
make verify godeps help you fix your problems instead of taunting you
This should save people so much time
(the changes are pretty minor, but the indentation makes them look major)
Automatic merge from submit-queue
Allow secure access to apiserver from Admission Controllers
* Allow options.InsecurePort to be set to 0 to switch off insecure access
* In NewSelfClient, Set the TLSClientConfig to the cert and key files
if InsecurePort is switched off
* Mint a bearer token that allows the client(s) created in NewSelfClient
to talk to the api server
* Add a new authenticator that checks for this specific bearer token
Fixes#13598
Automatic merge from submit-queue
controller: enhance timeout error message for Recreate deployments
Makes the error message from https://github.com/kubernetes/kubernetes/issues/29197 more obvious
@kubernetes/deployment
Automatic merge from submit-queue
Viper direct bindings to TestContext struct with hierarchichal suppor…
Part of #31453 to support hierarchichal parameters. This one does so for density, paves way for other tests as well.
Automatic merge from submit-queue
Add storage api group to hack/update-codegen.sh
Add it to the script then run it in the second commit. 1.4 needs the same change, I'll do it in my other PR.
Automatic merge from submit-queue
Extend all to more resources
Added more things from the list here:
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/cmd.go#L159
Update the devel/kubectl-conventions.md with the rules mentioned by
a few folks on which resources could be added to the special 'all' alias
Automatic merge from submit-queue
Enable hostpath provisioner for vagrant environment
This flag is required to run e2e tests for certain features (petset), and for manual tests and debugging.
related: https://github.com/kubernetes/kubernetes/issues/32119
The new flag, if specified, and if --container-runtime=docker, switches
kubelet to use the new CRI implementation for testing. This is hidden flag
since the feature is still under heavy development and the flag may be changed
in the near future.
Automatic merge from submit-queue
[kubelet] Fix oom-score-adj policy in kubelet
Fixes#32238
We have been having this regression since v1.3. It is critical for GKE/GCE deployments of k8s because docker daemon has a high likelihood of being OOM killed which will end up nuking all containers.
The reason for moving from mnt to pid is that docker daemon moves itself into a new mnt namespace with systemd based deployments.
Automatic merge from submit-queue
Fix error message around gcloud calls in node e2e and gubernator
Fixes some janky error messages around gcloud calls.
Automatic merge from submit-queue
Pet Set Example for Cassandra
- updating cassandra to 3.7
- added pet set example
- adding pet set for Cassandra e2e tests
- changed service as we do not want a lb service, as we are running C*
- updated docs
cc @bgrant0607
cc @kubernetes/examples
We can probably close a couple of other open PR, since I did some other stuff.
Automatic merge from submit-queue
Add separate build process for node test.
This PR is part of https://github.com/kubernetes/kubernetes/pull/31093. However, because currently node e2e is built on `KUBE_TEST_PLATFORMS`, which includes linux/amd64, darwin/amd64, windows/amd64 and linux/arm, it caused #32251 to fail.
In fact, node e2e is running on the same node with kubelet, and it also has built-in apiserver, etcd and namespace controller. All of them are only built on `KUBE_SERVER_PLATFORMS`, so node e2e should also only be built on those platforms.
```
KUBE_SERVER_PLATFORMS=(
linux/amd64
linux/arm
linux/arm64
)
```
This PR added a separate build process for node e2e to address this.
@vishh Do you need this for v1.4? because this blocks your #32251. /cc @dchen1107
Automatic merge from submit-queue
Use a patched golang version for building linux/arm
Fixes: #29904
Right now, linux/arm is broken because of an internal limitation in Go.
I've filed an issue for it here: https://github.com/golang/go/issues/17028
The affected binaries of this limitation are hyperkube and kube-apiserver, which are the largest binaries.
And when we now have a patched go 1.7.1 version for building "unsupported" but important architectures (ref: https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multi-platform.md), we should also include the patch for ppc64le and start building ppc64le again.
As soon as @laboger has the patch I need up on Github, I'll include ppc64le to this PR and we'll merge it
TODO:
- [ ] ~~Update the PR with patches for ppc64le at the same time @luxas~~
- [x] Push the new kube-cross image @ixdy
- [x] Run a full `make release` before to verify nothing breaks @luxas + @ixdy
- [ ] Cherrypick into the 1.4 branch @luxas + (who?)
@lavalamp @smarterclayton @ixdy @rsc @davecheney @wojtek-t @jfrazelle @bradfitz @david-mcmahon @pwittrock
Automatic merge from submit-queue
Cleanup non-rest apiserver handlers
- rename MuxHelper -> PathRecorderMux
- move non-rest handlers into routes packages within genericapiserver and `pkg/routes` (those from master)
- move ui and logs handlers out of genericapiserver (they are
not generic)
- make version handler configurable (`config.EnableVersion`)
- rename MuxHelper -> PathRecorderMux
- move non-rest handlers into routes packages within genericapiserver and
`pkg/routes` (those from master)
- move ui and logs handlers out of genericapiserver (they are
not generic)
- make version handler configurable (`config.EnableVersion`)
Automatic merge from submit-queue
Set --alsologtostderr=true in integration test
Without the flag, no glog output are stored in the test results. The logs are useful for debugging flaky tests like https://github.com/kubernetes/kubernetes/issues/30228.
The change also reveals a lot of messages like `W0912 14:19:32.306719 25386 cacher.go:468] Terminating all watchers from cacher *api.LimitRange`, which doesn't seem right.
Automatic merge from submit-queue
fix hack/verify-golint.sh on a Mac
Relax the location of the files a bit (Don't assume the
k8s.io/kubernetes always!)
Fixes#31905
Automatic merge from submit-queue
Make it possible to run local kubelet independently of cluster
Makes it possible to start a cluster and kubelet independently (was necessary when debugging issues related to restarting kubelet for existing node).
It is required to run automated tests for certain features (petset),
and for manual tests and debugging.
Change-Id: I9203aab6d67c8ff0cc4574473e8d0af888fe1804
Automatic merge from submit-queue
Create a flag for route reconciliation and deprecate unused node-sync-…
cc @pwittrock to decide if it can go in 1.4. It's a zero risk cleanup, that will reduce confusion about cluster configuration.
Automatic merge from submit-queue
etcd: data rollback tool of v3 -> v2
ref: https://github.com/kubernetes/features/issues/44
ref #20504
What?
This provides a rollback tool for some users to rollback etcd data from v3 to v2.
Automatic merge from submit-queue
Add flag to set CNI bin dir, and use it on gci nodes
**What this PR does / why we need it**:
When using `kube-up` on GCE, following #31023 which moved the workers from debian to gci, CNI just isn't working. The root cause is basically as discussed in #28563: one flag (`--network-plugin-dir`) means two different things, and the `configure-helper` script uses it for the wrong purpose.
This PR adds a new flag `--cni-bin-dir`, then uses it to configure CNI as desired.
As discussed at #28563, I have also added a flag `--cni-conf-dir` so users can be explicit
**Which issue this PR fixes** : fixes#28563
**Special notes for your reviewer**:
I left the old flag largely alone for backwards-compatibility, with the exception that I stop setting the default when CNI is in use. The value of `"/usr/libexec/kubernetes/kubelet-plugins/net/exec/"` is unlikely to be what is wanted there.
**Release note**:
```release-note
Added new kubelet flags `--cni-bin-dir` and `--cni-conf-dir` to specify where CNI files are located.
Fixed CNI configuration on GCI platform when using CNI.
```
Automatic merge from submit-queue
Improvements on OpenAPI spec generation
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Reference: #13414
**Release note**:
```release-note
Generate separate OpenAPI spec for each API GroupVersion on /<Group>/<Version>/swagger.json
```
Automatic merge from submit-queue
Generate 1 5 clientset
Generate the 1.5 clientset. Stop updating 1.4 clientset. Remove 1.2 clientset.
@nikhiljindal @lavalamp
I will rebase #31994 atop of this one.
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Automatic merge from submit-queue
Rollout dry run
**What this PR does / why we need it**: rollout undo add dry-run implementation
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#28219
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
rollout undo add dry-run option
```
Automatic merge from submit-queue
Move CSR helper for nodes out of kubelet
**What this PR does / why we need it**:
Including `cmd/kubelet/app` in kubeadm causes flag leakage.
Namelly, the problem is with `pkg/credentialprovider/gcp`, which
leaks `--google-json-key` and changing the behaviour of `init()`
doesn't sound reasonable, given kubelet is the only one who uses
this packages and obviously the flag is part of the functionality.
The helper is already generic enough, it has already been exported
and works well for kubeadm, so moving it should be fine.
**Special notes for your reviewer**: cc @mikedanese @yifan-gu @gtank
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Set Job's labels from PodTemplateSpec if none are specified
Fixes#31009 and replaces #32033.
@AdoHe I've picked your changes, added the additional ones for 'batch/v2alpha1' and most importantly added tests for job's defaulting. ptal
Automatic merge from submit-queue
Add client-server runtime support to local-up-cluster.sh
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: It provides support for using `--container-runtime-endpoint` and `--image-service-endpoint` arguments for kubelet in `local-up-cluster.sh` script.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: ref #28789
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Provide support for --container-runtime-endpoint and
--image-service-endpoint in kubelet.
Ref #28789
Automatic merge from submit-queue
add group prefix matching for kubectl usage
Adds prefix matching for groups so that `kubectl get storageclass.storage` matches `storageclass.storage.k8s.io`.
@kubernetes/kubectl
Automatic merge from submit-queue
Use etcd 2.3.7
This will switch to etcd 2.3.7 for release 1.4, to resolve issues rolling back from 1.4 to 1.3 (while preventing those same issues rolling back to 1.4.0 from a release including etcd 3.0.x).
Fixes#32253.
See #32253 (comment) for etcd roadmap.
Automatic merge from submit-queue
Convert bool to error, helper func for cd to skew
Switch from various functions returning `False` when something bad happens to returning an `error`
Encapsulate logic to switch to the skew directory inside chdirSkew
Also add a TODO for using `hyphen-flags` instead of `underscore_flags`
"go list -e ./..." returns different results when run from
the usual GOPATH location (returns k8s.io/kubernetes as prefix)
and when run outside of it (returns _ followed by full path of
the file). We need to support both scenarios.
Fixes#31905
Including `cmd/kubelet/app` in kubeadm causes flag leakage.
Namelly, the problem is with `pkg/credentialprovider/gcp`, which
leaks `--google-json-key` and changing the behaviour of `init()`
doesn't sound reasonable, given kubelet is the only one who uses
this packages and obviously the flag is part of the functionality.
The helper is already generic enough, it has already been exported
and works well for kubeadm, so moving it should be fine.
Automatic merge from submit-queue
choose a particular directory test-integration
Enables `make test-integration WHAT=auth` or whatever particular integration test director you want to run.
Automatic merge from submit-queue
Don't git clean in hack/jenkins/build.sh
We go through [all the effort](https://github.com/kubernetes/test-infra/pull/521) of activating a service account but then immediately wipe it away in the build script.
We do a full workspace cleanup anyway, so we don't need the `git clean`. We could probably remove the other cleanup lines in this script, too, but I wanted to keep this change as small as possible for now.
Maybe actually fixes https://github.com/kubernetes/test-infra/issues/470?
- updating java to 3.7
- added pet set example
- adding pet set for Cassandra e2e tests
- changed service as we do not want a lb service, as we are running C*
- updated docs
fixing headers and adding exception for run.sh
adding documentation, thank god for reflog
Did not mean to commit that as the README ... fixing
fixing problems in README
fixing more problems in README
more README tweaks
munge updates
updating examples_test for PetSet in Cassandra examples
updating petset to no use better security context
This avoids the whole command failing because of errors like the following:
```
# cover k8s.io/kubernetes/pkg/client/restclient
cover: internal error: block 268 overlaps block 270
```
For vendor code, igone
For local code replace the switch with an if statement
Automatic merge from submit-queue
Skip gofmt for staging/ directory
Don't run gofmt for example on:
staging/src/k8s.io/client-go/1.4/_vendor
Fixes#31875
Automatic merge from submit-queue
Namespace certificates API group
New API groups should follow best-practices for naming, including using DNS names within the k8s.io namespace
```release-note
The certificates API group has been renamed to certificates.k8s.io
```
Running tests rebuilds almost everything. Most unit tests do not need to be
rebuilt. This installs test artifacts and makes subsequent test runs as much
as 6x faster. I saw pkg/apiserver tests drop from 30+ seconds to 5 seconds.
Automatic merge from submit-queue
Generating API reference docs for federation apiserver
Fixes#30541
Adding a script `update-federation-api-reference-docs.sh` similar to the existing `update-api-reference-docs.sh` for kube-apiserver. Have moved the common parts to `hack/lib/swagger.sh`.
The new script will produce API reference docs for federation-apiserver.
Next step will be to surface these docs at kubernetes.io.
cc @kubernetes/sig-cluster-federation @kubernetes/sig-api-machinery @caesarxuchao
Automatic merge from submit-queue
Rewrite disruption e2e test to use versioned client.
This currently includes the changes from #31638. I will rebase once that is merged.
Automatic merge from submit-queue
Node E2E: Move the image and host related logic to be remote run specific.
This PR addresses #31597, and better fixes#31588.
With this PR:
1) All remote logic is moved into remote mode.
2) All gci related logic is only called when `HOSTS` and `IMAGES` are not specified.
3) Run node e2e against other node e2e images will work again - `make test-e2e-node REMOET=true IMAGES=e2e-node-containervm-v20160321-image`.
4) List images will work again - `make test-e2e-node REMOTE=true LIST_IMAGES=true`.
Mark 1.4, because https://github.com/kubernetes/kubernetes/pull/31588 is marked as 1.4. And this makes the node e2e work as is described in the [doc ](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/e2e-node-tests.md#run-tests-using-different-images).
@yujuhong @vishh
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Fix make test-integration under OSX
Just throw in a doc.go so there's something compilable in the
test/integration/metrics directory.
Fixes#31587
Automatic merge from submit-queue
Separate federation build.sh into development and deployment scripts.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.
Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.
This change also copies deploy/* files into the release distribution.
cc @kubernetes/sig-cluster-federation @colhom
```release-note
Federation can now be deployed using the `federation/deploy/deploy.sh` script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current `federation-up.sh`/`federation-down.sh` scripts. Both the scripts are going to co-exist in this release, but the `federation-up.sh`/`federation-down.sh` scripts might be removed in a future release in favor of `federation/deploy/deploy.sh` script.
```
Automatic merge from submit-queue
node e2e script: call 'gcloud compute' only if necessary
**What this PR does / why we need it**:
The ``gcloud compute`` is called everytime even if it is not needed.
When runnin node e2e tests on RHEL, the test-e2e-node.sh script is run
the way in which it takes execution path without ``gcloud`` invocation.
With the current code, ``gcloud`` is called everytime.
Thus introducing additional runtime dependency.
**Special notes for your reviewer**:
First introduced here: https://github.com/kubernetes/kubernetes/pull/29815
Automatic merge from submit-queue
Don't error out if ${go_pkg_dir} already exists
I have 'make' aliased to 'make -j4' for various reasons, and that's always worked fine for Kubernetes in the past. But it recently broke because EEXIST errors from this symlink creation are fatal. So make them not fatal, because if the path already exists, why try to create it again? It exists because two invocations of setup_env() are run in parallel due to the -j4, and they race to create the symlink and one obviously fails.
ln: failed to create symbolic link ‘/home/dcbw/Development/containers/kubernetes/_output/local/go/src/k8s.io/kubernetes’: File exists
Makefile.generated_files:285: recipe for target '_output/bin/deepcopy-gen' failed
make[1]: *** [_output/bin/deepcopy-gen] Error 1
make[1]: *** Waiting for unfinished jobs....
+++ [0826 10:18:02] Generating bindata:
/home/dcbw/Development/containers/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0826 10:18:02] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0826 10:18:03] Building go targets for linux/amd64:
cmd/libs/go2idl/conversion-gen
Makefile:281: recipe for target 'generated_files' failed
make: *** [generated_files] Error 2
Automatic merge from submit-queue
Get network name via e2e environment.
This should work, right? I plan to pipe it through into the TestContext soon, just not today, and I'd like some test runtime over the weekend. Open to suggestions.
The 'gcloud compute' is called everytime even if it is not needed.
When runnin node e2e tests on RHEL, the test-e2e-node.sh script is run
the way in which it takes execution path without gcloud invocation.
With the current code, gcloud is called everytime.
Thus introducing additional runtime dependency.
Automatic merge from submit-queue
[Federation] Downsize the release binary distribution v2.
Second attempt of PR #29632.
There are two things that this PR does:
1. It removes `federation-apiserver` and `federation-controller-manager` from binaries and docker_wrapped_binaries target lists.
2. Build the docker image for `hyperkube` on-the-fly while pushing the federation images.
```release-note
Federation binaries and their corresponding docker images - `federation-apiserver` and `federation-controller-manager` are now folded in to the `hyperkube` binary. If you were using one of these binaries or docker images, please switch to using the `hyperkube` version. Please refer to the federation manifests - `federation/manifests/federation-apiserver.yaml` and `federation/manifests/federation-controller-manager-deployment.yaml` for examples.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes Issue #28633
Automatic merge from submit-queue
Split the version metric out to its own package
This PR breaks a client dependency on prometheus. Combined with #30638, the client will no longer depend on these packages.
Most of the contents of docs/ has moved to kubernetes.github.io.
Development of the docs and accompanying files has continued there, making
the copies in this repo stale. I've removed everything but the .md files
which remain to redirect old links. The .yaml config files in the docs
were used by some tests, these have been moved to test/fixtures/doc-yaml,
and can remain there to be used by tests or other purposes.
Also build the hyperkube docker image on-the-fly.
This is only a temporary fix until the proposal in issue
https://github.com/kubernetes/kubernetes/issues/28630 is implemented.
Also, the new build/deployment method completely obviates this step.
We use debian image instead of busybox and do not build hyperkube as a
static binary yet. Wait until PR
https://github.com/kubernetes/kubernetes/pull/26028 is merged to build
static hyperkube binaries.
Automatic merge from submit-queue
Fix sort hint in `hack/verify-golint.sh`
The `verify-golint.sh` sorts all items with `LANG=C sort`, but it only hints developers to use `sort`, which causes a little trouble for me.
/cc @jfrazelle @sttts
Automatic merge from submit-queue
Node Conformance Test: Refactor node e2e framework
For #30122, #30174.
Based on #30348.
**Please only review the last 3 commits.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30348, it removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.
The PR refactored the node e2e framework. Moving different utilities into different packages under `pkg/`.
We need to do this because:
1) Files like e2e_remote.go and e2e_build.go should only be used by runner, but they were compiled into the test suite because they were placed in the same package. The worst thing is that it will introduce some never used flags in the test suite binary.
2) Make the directory structure more clear. Only test should be placed in `test/e2e_node`, other utilities should be placed in different packages in `pkg/`.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Keep vendor/ and Godep/ when creating the staging client, add a readme
In copy.sh, instead of removing the vendor/, moving it to _vendor. vendor/ is needed when we publish the staging client to its own repository.
Automatic merge from submit-queue
Explicitly set CLOUDSDK_CONFIG override early in e2e-runner
I'm guessing this is why a number of gci-on-gke builds started failing after https://github.com/kubernetes/test-infra/pull/432 was merged - we're overriding the gcloud configuration location from the default, so the activated service account no longer works. Previously we did this service activation late, but after we'd overridden the value.
I've changed it to explicitly set to a known-good value every time at the very start.
cc @vishh @maisem @spxtr @fejta
Automatic merge from submit-queue
Add admission controller for default storage class.
The admission controller adds a default class to PVCs that do not require any
specific class. This way, users (=PVC authors) do not need to care about
storage classes, administrator can configure a default one and all these PVCs
that do not care about class will get the default one.
The marker of default class is annotation "volume.beta.kubernetes.io/storage-class", which must be set to "true" to work. All other values (or missing annotation) makes the class non-default.
Based on @thockin's code, added tests and made it not to reject a PVC when no class is marked as default.
.
@kubernetes/sig-storage
Automatic merge from submit-queue
Add kubelet --network-plugin-mtu flag for MTU selection
* Add network-plugin-mtu option which lets us pass down a MTU to a network provider (currently processed by kubenet)
* Add a test, and thus make sysctl testable
Automatic merge from submit-queue
Fix path to kubernetes_skew
Previously this was `$(pwd)/kubernetes/../kubernetes_old` this is equivalent to `$(pwd)/kubernetes_old` although we also changed the `old` to `skew` instead.
Fixes https://github.com/kubernetes/kubernetes/issues/31149
Automatic merge from submit-queue
AppArmor PodSecurityPolicy support
Implements the AppArmor PodSecurityPolicy support based on the alpha API proposed [here](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/apparmor.md#pod-security-policy)
This implementation deviates from the original proposal in one way: it adds a separate option for specifying a default profile:
```
apparmor.security.alpha.kubernetes.io/defaultProfileName
```
This has several advantages over the original proposal:
- The default is explicit, rather than implicit on the ordering
- The default can be specified without constraining the allowed profiles
- The allowed profiles can be restricted without specifying a default (requires every pod to explicitly set a profile)
The E2E cluster does not currently enable the PodSecurityPolicy, so I will submit E2E tests in a separate PR.
/cc @dchen1107 @pweil- @sttts @jfrazelle @Amey-D
MTU selection is difficult, and if there is a transport such as IPSEC in
use may be impossible. So we allow specification of the MTU with the
network-plugin-mtu flag, and we pass this down into the network
provider.
Currently implemented by kubenet.
Automatic merge from submit-queue
Use kubekins-test:v20160822 everywhere
A step towards fixing #31148. We also need to update the test-infra repo to use the new e2e image. I'll fold that into https://github.com/kubernetes/test-infra/pull/419.
Both gcr.io/google-containers/kubekins-test:v20160822 and gcr.io/google-containers/e2e-test:v20160822 have already been pushed.
Automatic merge from submit-queue
Updating version check for etcd
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Currently, if you `make build`, etcd is not properly version checked because `etcd -version` does multi-line output. This output cannot be version compared. This small change fixes that.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Fix caching of go dirs during build
Sometimes when you press `^C` during `make` the subsequent attempt to run `make` hangs due to zero-sized cache file for `ALL_GO_DIRS` var in Makefile.generated_files:
```
vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ KUBE_JUNIT_REPORT_DIR=/tmp/art KUBE_COVER=y make test
# hangs...
^CMakefile:279: recipe for target 'generated_files' failed
make: *** [generated_files] Interrupt
vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ ls -l .make/all_go_dirs.mk
-rw-rw-r-- 1 vagrant vagrant 0 Aug 18 15:03 .make/all_go_dirs.mk
```
Corresponding process subtree looks like the following:
```
└─make test
└─make -f Makefile.generated_files generated_files
└─bash -c grep --color=never -l '+k8s:deepcopy-gen=' | xargs -n1 dirname | sort -u
├─grep --color=never -l +k8s:deepcopy-gen=
├─sort -u
└─xargs -n1 dirname
```
Let's remove the cache file if `cache_go_dirs.sh` gets interrupted.
Automatic merge from submit-queue
e2e-runner: Get GCI builtin k8s version from GCS
**What this PR does / why we need it**:
The GCI release qualification tests use builtin k8s version to run tests. They used to get the builtin version by parsing descriptions of the test images, but that's unreliable because the description format could change. This PR makes it to read a file checked in GCS that has the exact k8s version number.
@adityakali @spxtr Can you review?
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
Remove implicit Prometheus metrics from client
**What this PR does / why we need it**: This PR starts to cut away at dependencies that the client has.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
The implicit registration of Prometheus metrics for request count and latency have been removed, and a plug-able interface was added. If you were using our client libraries in your own binaries and want these metrics, add the following to your imports in the main package: "k8s.io/pkg/client/metrics/prometheus".
```
cc: @kubernetes/sig-api-machinery @kubernetes/sig-instrumentation @fgrzadkowski @wojtek-t
Automatic merge from submit-queue
Kubelet: add --container-runtime-endpoint and --image-service-endpoint
Flag `--container-runtime-endpoint` (overrides `--container-runtime`) is introduced to identify the unix socket file of the remote runtime service. And flag `--image-service-endpoint` is introduced to identify the unix socket file of the image service.
This PR is part of #28789 Milestone 0.
CC @yujuhong @Random-Liu
Automatic merge from submit-queue
fix path handling in hack/lib/init.sh
Jenkinsfile pipeline jobs get cloned into "\<project\> (\<branch\>)". As a result, I can't use certain things in `hack/lib/init.sh`.
This is a small fix for that problem.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Automatic merge from submit-queue
Implement TLS bootstrap for kubelet using `--experimental-bootstrap-kubeconfig` (2nd take)
Ref kubernetes/features#43 (comment)
cc @gtank @philips @mikedanese @aaronlevy @liggitt @deads2k @errordeveloper @justinsb
Continue on the older PR https://github.com/kubernetes/kubernetes/pull/30094 as there are too many comments on that one and it's not loadable now.
Automatic merge from submit-queue
ImagePolicyWebhook Admission Controller
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: This is an implementation of the [image provenance proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/image-provenance.md). It also includes the API definitions by @Q-Lee from https://github.com/kubernetes/kubernetes/pull/30241
**Special notes for your reviewer**:
Please note that this is the first admission controller to make use of the admission controller config file (`--admission-controller-config-file`). I have defined a format for it but we may want to double check it's adequate for future use cases as well.
The format defined is:
```
{
"imagePolicy": {
"kubeConfigFile": "path/to/kubeconfig/for/backend",
"allowTTL": 50, # time in s to cache approval
"denyTTL": 50, # time in s to cache denial
"retryBackoff": 500, # time in ms to wait between retries
"defaultAllow": true # determines behavior if the webhook backend fails
}
}
```
(or yaml)
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Adding ImagePolicyWebhook admission controller.
```
Automatic merge from submit-queue
Basic scaler/reaper for petset
Currently scaling or upgrading a petset is more complicated than it should be. Would be nice if this made code freeze on friday. I'm planning on a follow up change with generation number and e2es post freeze.
Automatic merge from submit-queue
Scheduledjobs e2e
@janetkuo resubmitted e2e for SJ, I've updated all scripts to consume `KUBE_RUNTIME_CONFIG` properly in 2nd commit, ptal
Automatic merge from submit-queue
When running inside docker, activate service account ASAP
Also switching to just use `GOOGLE_APPLICATION_CREDENTIALS`, rather than both.
x-ref https://github.com/kubernetes/test-infra/issues/318
Automatic merge from submit-queue
Gubernator bug fixes: mv and GCS bucket permissions
Fixed issue where results file was not moved correctly, and also the permissions issue with the GCS bucket.
Will rebase after #30414 is merged
@timstclair
Automatic merge from submit-queue
Add a short `-n` for `kubectl --namespace`
fixes#24078
`--namespace` is a very common flag for nearly every `kubectl` command we have. We should claim `-n` for it.
Automatic merge from submit-queue
add --raw for kubectl get
Adds a `--raw` option to `kubectl get` that allow you specify your URI, but use the transport built by `kubectl`. This is especially useful when working with secured environments that require authentication and authorization to hit non-api endpoints. For example, `kubect get --raw /metrics` or if you want to debug a watch with a view at the exact data `kubectl get --raw '/api/v1/namespaces/one/replicationcontrollers?watch=true'`.
@kubernetes/kubectl
@fabianofranz fyi
Automatic merge from submit-queue
Disable linux/ppc64le compilation by default
Work-around for #30384.
I'm still testing this locally to see if it actually works. The build is slow. (PR Jenkins won't tell us whether this fixes ppc.)
cc @Random-Liu @spxtr @david-mcmahon @luxas
New flag --container-runtime-endpoint (overrides --container-runtime)
is introduced to kubelet which identifies the unix socket file of
the remote runtime service. And new flag --image-service-endpoint is
introduced to kubelet which identifies the unix socket file of the
image service.
Automatic merge from submit-queue
Move UTs that block on apiserver to integration tests.
In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail. These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.
This PR simply shuffles those tests to integration tests, which is where they should be.
/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90
Automatic merge from submit-queue
Revert "Revert "syncNetworkUtil in kubelet and fix loadbalancerSourceRange on GCE
Reverts kubernetes/kubernetes#30729
Automatic merge from submit-queue
Add GUBERNATOR flag which produces g8r link for node e2e tests
When you run 'make tests-e2e-node REMOTE=true GUBERNATOR=true' outputs a URL to view the test results on Gubernator. ~~Should work after my PR for Gubernator is merged.~~
@timstclair
Automatic merge from submit-queue
Quobyte Volume plugin
@quofelix and myself developed a volume plugin for [Quobyte](http://www.quobyte.com) which is a software-defined storage solution. This PR allows Kubernetes users to mount a Quobyte Volume inside their containers over Kubernetes.
Here are some further informations about [Quobyte and Storage for containers](http://www.quobyte.com/containers)
The admission controller adds a default class to PVCs that do not require any
specific class. This way, users (=PVC authors) do not need to care about
storage classes, administrator can configure a default one and all these PVCs
that do not care about class will get the default one.
Automatic merge from submit-queue
kubelet eviction on inode exhaustion
Add support for kubelet to monitor for inode exhaustion of either image or rootfs, and in response, attempt to reclaim node level resources and/or evict pods.
Automatic merge from submit-queue
Add annotations to the PodSecurityPolicy Provider interface
@pweil- is this what you were thinking in terms of API changes? I really like to avoid functions with more than 2 return values, but couldn't think of a cleaner approach in this case.
Automatic merge from submit-queue
Expose flags for new NodeEviction logic in NodeController
Fix#28832
Last PR from the NodeController NodeEviction logic series.
cc @davidopp @lavalamp @mml
