Automatic merge from submit-queue
Attach Detach Controller Business Logic
This PR adds the meat of the attach/detach controller proposed in #20262.
The PR splits the in-memory cache into a desired and actual state of the world.
Automatic merge from submit-queue
use kubectl-with-retry on pause & resume
attempts to fix https://github.com/kubernetes/kubernetes/issues/25645 by using `kubectl-with-retry` on `rollout {pause,resume}` (`resume` is for safe measures) instead of `kubectl` directly, as is done with other `rollout {pause,resume}` tests in this same script.
Automatic merge from submit-queue
e2e-runner: Drop Trusty support in favor of GCI
Now that GCI (Google Container-VM Image) is out, we will start running e2e tests
with it instead of Ubuntu Trusty. This change updates `e2e-runner.sh` to replace
the Trusty related logic with GCI.
_Note that this change has to go in the same time as https://github.com/kubernetes/test-infra/pull/54_
@spxtr Can you review?
cc/ @andyzheng0831 @kubernetes/goog-image
Split controller cache into actual and desired state of world.
Controller will only operate on volumes scheduled to nodes that
have the "volumes.kubernetes.io/controller-managed-attach" annotation.
- Add junit test reported
- Write etcd.log, kubelet.log and kube-apiserver.log to files instead of stdout
- Scp artifacts to the jenkins WORKSPACE
Fixes#25966
Automatic merge from submit-queue
Renaming federated-apiserver to federation-apiserver
Its not a "federated" apiserver. Its an apiserver for federation.
The corresponding controller manager is also called federation-controller-manager.
If we are going to rename, it should be done in 1.3 before people write scripts referencing the binary name.
Automatic merge from submit-queue
Add a 'kubectl clusterinfo dump' option
Ref: #3500
@bgrant0607 @smarterclayton @jszczepkowski
Usage:
```
# Dump current cluster state to stdout
kubectl clusterinfo dump
# Dump current cluster state to /tmp
kubectl clusterinfo dump --output-directory=/tmp
# Dump all namespaces to stdout
kubectl clusterinfo dump --all-namespaces
# Dump a set of namespaces to /tmp
kubectl clusterinfo dump --namespaces default,kube-system --output-directory=/tmp
```
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/20672)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Improve the speed of do-nothing build.
As @thockin found out here https://github.com/kubernetes/kubernetes/issues/24518, vast majority of the do-nothing build time is spent in rebuilding the test binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them while building packages.
On my workstation, do-nothing hack/build-go.sh time goes from ~20 secs to ~4 secs, of which only ~1 sec is from doing test binary staleness check now (as opposed to ~17 secs it took to build the test binaries before). I did some experiments to bring this time down to <1 sec. I measured using go test -bench, but it was not very useful in this case. I believe, a vast majority of that ~1 second is being spent in fork/exec and piping the results back to the staleness check program along with the ser-deser involved, but it needs to be validated. Not a proof, but to provide some supporting evidence to this claim, running `go list -f format packages` in the shell takes about 600ms irrespective of what's in the format.
Tests are TBD. I am still trying to figure out how to test this, but I would like to get early feedback
cc @mikedanese @mml
Automatic merge from submit-queue
Cache Webhook Authentication responses
Add a simple LRU cache w/ 2 minute TTL to the webhook authenticator.
Kubectl is a little spammy, w/ >= 4 API requests per command. This also prevents a single unauthenticated user from being able to DOS the remote authenticator.
Automatic merge from submit-queue
Updaing QoS policy to be at the pod level
Quality of Service will be derived from an entire Pod Spec, instead of being derived from resource specifications of individual resources per-container.
A Pod is `Guaranteed` iff all its containers have limits == requests for all the first-class resources (cpu, memory as of now).
A Pod is `BestEffort` iff requests & limits are not specified for any resource across all containers.
A Pod is `Burstable` otherwise.
Note: Existing pods might be more susceptible to OOM Kills on the node due to this PR! To protect pods from being OOM killed on the node, set `limits` for all resources across all containers in a pod.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/14943)
<!-- Reviewable:end -->
Automatic merge from submit-queue
add CIDR allocator for NodeController
This PR:
* use pkg/controller/framework to watch nodes and reduce lists when allocate CIDR for node
* decouple the cidr allocation logic from monitoring status logic
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/19242)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Add 'kubectl set image'
```release-note
Add "kubectl set image" for easier updating container images (for pods or resources with pod templates).
```
**Usage:**
```
kubectl set image (-f FILENAME | TYPE NAME) CONTAINER_NAME_1=CONTAINER_IMAGE_1 ... CONTAINER_NAME_N=CONTAINER_IMAGE_N
```
**Example:**
```console
# Set a deployment's nginx container image to 'nginx:1.9.1', and its busybox container image to 'busybox'.
$ kubectl set image deployment/nginx busybox=busybox nginx=nginx:1.9.1
# Update all deployments' nginx container's image to 'nginx:1.9.1'
$ kubectl set image deployments nginx=nginx:1.9.1 --all
# Update image of all containers of daemonset abc to 'nginx:1.9.1'
$ kubectl set image daemonset abc *=nginx:1.9.1
# Print result (in yaml format) of updating nginx container image from local file, without hitting the server
$ kubectl set image -f path/to/file.yaml nginx=nginx:1.9.1 --local -o yaml
```
I abandoned the `--container=xxx --image=xxx` flags in the [deploy proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/deploy.md#kubectl-set) since it's much easier to use with just KEY=VALUE (CONTAINER_NAME=CONTAINER_IMAGE) pairs.
Ref #21648
@kubernetes/kubectl @bgrant0607 @kubernetes/sig-config
[]()
Automatic merge from submit-queue
Fix JENKINS_USE_SKEW_KUBECTL
I got this logic wrong; the first is a NOT comparison, so the second should only be available if that NOT comparison returns true.
Automatic merge from submit-queue
Add a timeout to the node e2e Ginkgo test runner
Also add a few debugging statements to indicate progress.
Should help prevent #25639, since we'll timeout tests before Jenkins times out the build.
Automatic merge from submit-queue
Cassandra examples updating images to v9
- this is a version bump for the C* image
- I also increased the cpu to .5 because .1 is slow like warm death
Who can actually run the build to get the container to the examples repo?
Now that GCI (Google Container-VM Image) is out, we will start running e2e tests
with it instead of Ubuntu Trusty. This change updates `e2e-runner.sh` to replace
the Trusty related logic with GCI.
This patch adds the --exit-on-lock-contention flag, which must be used
in conjunction with the --lock-file flag. When provided, it causes the
kubelet to wait for inotify events for that lock file. When an 'open'
event is received, the kubelet will exit.
Automatic merge from submit-queue
Implement the cluster-controller of kubernetes federation
This PR is the cluster-controller of kubernetes federation.
This controller of federation phase 1 just collect the status of federated clusters.
Automatic merge from submit-queue
Remove call to setup_env from hack/e2e-node-test.sh
[]()
This messes with the GOPATH in a way that confuses ginkgo,
and doesn't appear to be necessary to run the tests
Removes likely unnecessary use of sudo, though it should be noted that
e2e_node_suite_test.go uses sudo to do something with locksmithd, so you
currently end up using it anyway.
Updates the error for not having ginkgo built to recommend the new vendor/
path for ginkgo, and to recommend using make instead of hack/build-go.sh
Should resolve#25682
This messes with the GOPATH in a way that confuses ginkgo,
and doesn't appear to be necessary to run the tests
Removes likely unnecessary use of sudo, though it should be noted that
e2e_node_suite_test.go uses sudo to do something with locksmithd, so you
currently end up using it anyway.
Updates the error for not having ginkgo built to recommend the new vendor/
path for ginkgo, and to recommend using make instead of hack/build-go.sh
Should resolve#25682
Automatic merge from submit-queue
Build hyperkube in hack/local-up-cluster instead of separate binaries
This speeds up the build time a lot.
Also add arm64 as a valid arch
This is quite trivial. Let's get it in asap
@thockin @pmorie @liggitt @eparis @zmerlynn @smarterclayton
Automatic merge from submit-queue
Use --format='value(name)' with gcloud instead of grep/awk/cut
Fixing our fragile parsing of `gcloud` is getting old (#24746, #25159, maybe others?).
Instead, let's just get the proper output out of `gcloud` in the first place.
Automatic merge from submit-queue
Support struct,array,slice types when sorting kubectl output
Fixes https://github.com/kubernetes/kubernetes/issues/24328.
Briefly, `sorting_printer` only take cares of the following type kinds:
* `reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64`
* `reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64`
* `reflect.Float32, reflect.Float64`
* `reflect.String`
* `reflect.Ptr`
This commit aims to add `reflect.Struct, reflect.Slice, reflect.Array`.
/cc @bgrant0607
Automatic merge from submit-queue
Adding Services to federation clientset
Commits:
1. Regenerate the client without any changes to client-gen
2. Update clientgen to add a parameter to specify generating client only for Services v1 object.
3. Regenerate federation_internalclientset
4. Regenerate federation_release_1_3
Second commit is the most important one. Other 3 commits are auto generated by running client-gen.
I have added a command line argument to client-gen that takes in a list of group/version/resource. If a group version is part of this list, then only the resources in this list are included in the client. For other group versions, the existing check of genclient=true in types.go is used.
Other alternatives considered were:
* Update genclient in types.go to mention the clientset name in which it should be included instead of just saying genclient=true (so Services will say genclient=core,federation while all other v1 resources will say genclient=core). This requires a code change in types.go to change a client set.
* Create another types.go which will only include Services and use that to generate federation clientset. This will lead to duplicate Service definition.
cc @caesarxuchao @lavalamp @jianhuiz @mfanjie @kubernetes/sig-cluster-federation
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/25443)
<!-- Reviewable:end -->
As thockin found out here https://github.com/kubernetes/kubernetes/issues/24518,
vast majority of the do-nothing build time is spent in rebuilding the test
binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them
while building packages.
Tests are TBD. I am still trying to figure out how to test this.
Automatic merge from submit-queue
The remaining API changes for PodDisruptionBudget.
It's mostly the boilerplate required for the registry, some extra codegen, and a few tests.
Will squash once we're sure it's good.
Automatic merge from submit-queue
Add eviction-pressure-transitition-period flag to kubelet
This PR does the following:
* add the new flag to control how often a node will go out of memory pressure or disk pressure conditions see: https://github.com/kubernetes/kubernetes/pull/25282
* pass an `eviction.Config` into `kubelet` so we can group config
/cc @vishh
Automatic merge from submit-queue
[e2e] kubectl stdin
Problem: Currently kubectl heavily relies on files which have to be (for lack of a better word :):):)) "written" to the file system. This hinders adoption of something like gobindata, by forcing an intermediary generated-assets directory type thing.
Solution: Lets migrate `kubectl.go` testing over to using standard input streams.
cc @kubernetes/sig-testing @timothysc
Since its hard for an individual to remember that we need codecgen and
ginkgo in godeps but don't actually have that dependancy listed in a way
that godep can automatically find.
Automatic merge from submit-queue
WIP v0 NVIDIA GPU support
```release-note
* Alpha support for scheduling pods on machines with NVIDIA GPUs whose kubelets use the `--experimental-nvidia-gpus` flag, using the alpha.kubernetes.io/nvidia-gpu resource
```
Implements part of #24071 for #23587
I am not familiar with the scheduler enough to know what to do with the scores. Mostly punting for now.
Missing items from the implementation plan: limitranger, rkt support, kubectl
support and docs
cc @erictune @davidopp @dchen1107 @vishh @Hui-Zhi @gopinatht
Automatic merge from submit-queue
pkg/apis/rbac: Add Openshift authorization API types
This PR updates #23396 by adding the Openshift RBAC types to a new API group.
Changes from Openshift:
* Omission of [ResourceGroups](4589987883/pkg/authorization/api/types.go (L32-L104)) as most of these were Openshift specific. Would like to add the concept back in for a later release of the API.
* Omission of IsPersonalSubjectAccessReview as its implementation relied on Openshift capability.
* Omission of SubjectAccessReview and ResourceAccessReview types. These are defined in `authorization.k8s.io`
~~API group is named `rbac.authorization.openshift.com` as we omitted the AccessReview stuff and that seemed to be the lest controversial based on conversations in #23396. Would be happy to change it if there's a dislike for the name.~~ Edit: API groups is named `rbac`, sorry misread the original thread.
As discussed in #18762, creating a new API group is kind difficult right now and the documentation is very out of date. Got a little help from @soltysh but I'm sure I'm missing some things. Also still need to add validation and a RESTStorage registry interface. Hence "WIP".
Any initial comments welcome.
cc @erictune @deads2k @sym3tri @philips
Automatic merge from submit-queue
Updating hack/update-codegen to keep federation_clientset updated
Right now, there is no check for this and hence federation_clientset becomes stale over time.
Updating hack/update-codegen to keep federation_clientset updated.
hack/verify-codegen.sh ensures that it is updated.
cc @caesarxuchao @lavalamp @jianhuiz @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Webhook Token Authenticator
Add a webhook token authenticator plugin to allow a remote service to make authentication decisions.
Implements part of #24071
I am not familiar with the scheduler enough to know what to do with the scores. Punting for now.
Missing items from the implementation plan: limitranger, rkt support, kubectl
support and user docs
Automatic merge from submit-queue
Hack update all remove dollar symbol
When not running ./hack/update in silent mode, the script fails due to undefined ``$Updating`` variable.
Automatic merge from submit-queue
Move godeps to vendor/
This is a first-step towards glide support, maybe we don't want or need to take this, but it was easy to try.
This fails to compile, not sure why:
```
# k8s.io/kubernetes/pkg/apis/extensions/v1beta1
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:2703: undefined: extensions.ClusterAutoscaler
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:2703: undefined: ClusterAutoscaler
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:2719: undefined: extensions.ClusterAutoscaler
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:2719: undefined: ClusterAutoscaler
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:2723: undefined: extensions.ClusterAutoscalerList
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:2723: undefined: ClusterAutoscalerList
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:3468: Convert_extensions_JobSpec_To_v1beta1_JobSpec redeclared in this block
previous declaration at _output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion.go:328
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:3845: Convert_extensions_ScaleStatus_To_v1beta1_ScaleStatus redeclared in this block
previous declaration at _output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion.go:98
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:4737: Convert_v1beta1_JobSpec_To_extensions_JobSpec redeclared in this block
previous declaration at _output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion.go:380
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:5186: Convert_v1beta1_ScaleStatus_To_extensions_ScaleStatus redeclared in this block
previous declaration at _output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion.go:120
_output/local/go/src/k8s.io/kubernetes/pkg/apis/extensions/v1beta1/conversion_generated.go:2723: too many errors
!!! Error in /home/thockin/tmp/godep-vendor/src/k8s.io/kubernetes/hack/lib/golang.sh:417
```
Automatic merge from submit-queue
cluster/images/hyperkube: create symlink for each server
Add a kubelet symlink so that the hyperkube image can appear as a kubelet image. https://github.com/kubernetes/kubernetes/issues/24510
a) it doesn't need it
b) changing CWD to a path with symlinks breaks deep within ginkgo, where it
crafts a relative path to ../../../../../../platforms/amd64/whatever which then
traverses the physical path not the symlinked one, and breaks.
Our `realpath` and `readlink -f` functions (required only because of MacOS,
thanks Steve) were poor substitutes at best. Mostly they were downright
broken. This thoroughly overhauls them and adds a test (in comments, since we
don't seem to have shell tests). For all the interesting cases I could think
of, the fakes act just like the real thing.
Then use those and canonicalize KUBE_ROOT. In order to make recursive calls of
our shell tool not additively grow `pwd` we have to essentially make the
sourcing of init.sh idempotent.
Automatic merge from submit-queue
Introduce events flag for describers
Printing events for a given object is not always needed. Thus, introducing --show-events=false to ``kubectl describe`` to skip events printing.
Fixes: #24239
Automatic merge from submit-queue
Reimplement 'pause' in C - smaller footprint all around
Statically links against musl. Size of amd64 binary is 3560 bytes.
I couldn't test the arm binary since I have no hardware to test it on, though I assume we want it to work on a raspberry pi.
This PR also adds the gcc5/musl cross compiling image used to build the binaries.
@thockin
Automatic merge from submit-queue
Add subPath to mount a child dir or file of a volumeMount
Allow users to specify a subPath in Container.volumeMounts so they can use a single volume for many mounts instead of creating many volumes. For instance, a user can now use a single PersistentVolume to store the Mysql database and the document root of an Apache server of a LAMP stack pod by mapping them to different subPaths in this single volume.
Also solves https://github.com/kubernetes/kubernetes/issues/20466.
Automatic merge from submit-queue
Update e2e-runner.sh so it can fetch multiple types of Trusty images
Trusty beta images now work with k8s 1.2.
@spxtr, @andyzheng0831 Can you review this?
Automatic merge from submit-queue
Kubelet: Cleanup with new engine api
Finish step 2 of #23563
This PR:
1) Cleanup go-dockerclient reference in the code.
2) Bump up the engine-api version.
3) Cleanup the code with new engine-api.
Fixes#24076.
Fixes#23809.
/cc @yujuhong
Introduce DescriberSettings for Describer display options
Introduce --show-events flag and DescriberSettings in Describer methods
Introduce unit-tests
Regenerated kubectl describe docs
Add events flag tests to test-cmd.sh
Signed-off-by: dhodovsk@redhat.com
Signed-off-by: jchaloup@redhat.com
Automatic merge from submit-queue
Jenkins: Clean up even if we can't bring cluster up
We're gathering all the cluster logs, go ahead and clean up the
resources.
Automatic merge from submit-queue
Allow etcd to store protobuf objects
Split storage serialization from client negotiation, and allow API server to take flag controlling serialization.
TODO:
* [x] API server still doesn't start - range allocation object doesn't seem to round trip correctly to etcd
* [ ] Verify that third party resources are ignoring protobuf (add a test)
* [ ] Add integration tests that verify storage is correctly protobuf
* [ ] Add a global default for which storage format to prefer?
Automatic merge from submit-queue
Map secret files into dockerized e2e
Rather than copying the GCE and AWS private keys/credentials to each Jenkins VM, we can put them in credentials and map them through.
This is one half of the change; if the relevant environment variables are set, we'll mount the files in.
cc @fejta @rmmh @apelisse
The codec factory should support two distinct interfaces - negotiating
for a serializer with a client, vs reading or writing data to a storage
form (etcd, disk, etc). Make the EncodeForVersion and DecodeToVersion
methods only take Encoder and Decoder, and slight refactoring elsewhere.
In the storage factory, use a content type to control what serializer to
pick, and use the universal deserializer. This ensures that storage can
read JSON (which might be from older objects) while only writing
protobuf. Add exceptions for those resources that may not be able to
write to protobuf (specifically third party resources, but potentially
others in the future).
Automatic merge from submit-queue
kubectl rolling-update support for same image
Fixes#23497.
Enables `kubectl rolling-update --image` to the same image, adding a `--image-pull-policy` flag to remove ambiguity. This allows rolling-update to behave as an "update and/or restart" (https://github.com/kubernetes/kubernetes/issues/23497#issuecomment-212349730), or as a forced update when the same tag can mean multiple versions (e.g. `:latest`). cc @janetkuo @nikhiljindal
Automatic merge from submit-queue
Add several arguments to boilerplate.py
This commit makes the root directory and boilerplate content directory configurable.
The defaults have remained the same, so no behavior changes should be expected.
cc @eparis
ref https://github.com/kubernetes/minikube/pull/37
Automatic merge from submit-queue
Use HOSTNAME in Docker build image tag hash
Fixes#24661 by including `$HOSTNAME` when generating the build image tag hash.
When running the verification checks under Docker, the `$KUBE_ROOT` will be identical across builds, so tags will collide unless we add additional uniqueness. By default, the hostname inside a Docker container is its ID, which should be unique enough for us.
I also deleted a misleading error message from the same check.
@kubernetes/sig-testing
This commit makes the root directory, boilerplate content directory and
the directories to skip configurable.
The defaults have remained the same, so no behavior changes should be expected.
Automatic merge from submit-queue
Run Kubemark builds inside Docker
Since Docker-in-Docker is tricky to get right (esp. wrt volume mounts), I'm only enabling it when necessary for a build (e.g. for kubemark).
cc @spxtr @fejta @wojtek-t
Automatic merge from submit-queue
allow kubectl subcmds to process multiple resources
~~autoscale, expose & patch~~ Many kubectl subcommands were limited to processing one resource at a time.
This PR allows those subcommands to process multiple resources.
This PR is in reference to https://github.com/kubernetes/kubernetes/pull/23116#issuecomment-202360784 by @deads2k
Automatic merge from submit-queue
Up to go 1.6.2 for build and test.
~~1.6.1 contains some security fixes. 1.6.2 should be out soon.~~ 1.6.2 is out :D
Images aren't pushed yet.
Automatic merge from submit-queue
Framework support for node e2e.
This should let us port existing e2e tests to the node e2e suite, if the tests are node specific.
