Automatic merge from submit-queue
Split the version metric out to its own package
This PR breaks a client dependency on prometheus. Combined with #30638, the client will no longer depend on these packages.
Most of the contents of docs/ has moved to kubernetes.github.io.
Development of the docs and accompanying files has continued there, making
the copies in this repo stale. I've removed everything but the .md files
which remain to redirect old links. The .yaml config files in the docs
were used by some tests, these have been moved to test/fixtures/doc-yaml,
and can remain there to be used by tests or other purposes.
Also build the hyperkube docker image on-the-fly.
This is only a temporary fix until the proposal in issue
https://github.com/kubernetes/kubernetes/issues/28630 is implemented.
Also, the new build/deployment method completely obviates this step.
We use debian image instead of busybox and do not build hyperkube as a
static binary yet. Wait until PR
https://github.com/kubernetes/kubernetes/pull/26028 is merged to build
static hyperkube binaries.
Automatic merge from submit-queue
Fix sort hint in `hack/verify-golint.sh`
The `verify-golint.sh` sorts all items with `LANG=C sort`, but it only hints developers to use `sort`, which causes a little trouble for me.
/cc @jfrazelle @sttts
Automatic merge from submit-queue
Node Conformance Test: Refactor node e2e framework
For #30122, #30174.
Based on #30348.
**Please only review the last 3 commits.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30348, it removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.
The PR refactored the node e2e framework. Moving different utilities into different packages under `pkg/`.
We need to do this because:
1) Files like e2e_remote.go and e2e_build.go should only be used by runner, but they were compiled into the test suite because they were placed in the same package. The worst thing is that it will introduce some never used flags in the test suite binary.
2) Make the directory structure more clear. Only test should be placed in `test/e2e_node`, other utilities should be placed in different packages in `pkg/`.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Keep vendor/ and Godep/ when creating the staging client, add a readme
In copy.sh, instead of removing the vendor/, moving it to _vendor. vendor/ is needed when we publish the staging client to its own repository.
Automatic merge from submit-queue
Explicitly set CLOUDSDK_CONFIG override early in e2e-runner
I'm guessing this is why a number of gci-on-gke builds started failing after https://github.com/kubernetes/test-infra/pull/432 was merged - we're overriding the gcloud configuration location from the default, so the activated service account no longer works. Previously we did this service activation late, but after we'd overridden the value.
I've changed it to explicitly set to a known-good value every time at the very start.
cc @vishh @maisem @spxtr @fejta
Automatic merge from submit-queue
Add admission controller for default storage class.
The admission controller adds a default class to PVCs that do not require any
specific class. This way, users (=PVC authors) do not need to care about
storage classes, administrator can configure a default one and all these PVCs
that do not care about class will get the default one.
The marker of default class is annotation "volume.beta.kubernetes.io/storage-class", which must be set to "true" to work. All other values (or missing annotation) makes the class non-default.
Based on @thockin's code, added tests and made it not to reject a PVC when no class is marked as default.
.
@kubernetes/sig-storage
Automatic merge from submit-queue
Add kubelet --network-plugin-mtu flag for MTU selection
* Add network-plugin-mtu option which lets us pass down a MTU to a network provider (currently processed by kubenet)
* Add a test, and thus make sysctl testable
Automatic merge from submit-queue
Fix path to kubernetes_skew
Previously this was `$(pwd)/kubernetes/../kubernetes_old` this is equivalent to `$(pwd)/kubernetes_old` although we also changed the `old` to `skew` instead.
Fixes https://github.com/kubernetes/kubernetes/issues/31149
Automatic merge from submit-queue
AppArmor PodSecurityPolicy support
Implements the AppArmor PodSecurityPolicy support based on the alpha API proposed [here](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/apparmor.md#pod-security-policy)
This implementation deviates from the original proposal in one way: it adds a separate option for specifying a default profile:
```
apparmor.security.alpha.kubernetes.io/defaultProfileName
```
This has several advantages over the original proposal:
- The default is explicit, rather than implicit on the ordering
- The default can be specified without constraining the allowed profiles
- The allowed profiles can be restricted without specifying a default (requires every pod to explicitly set a profile)
The E2E cluster does not currently enable the PodSecurityPolicy, so I will submit E2E tests in a separate PR.
/cc @dchen1107 @pweil- @sttts @jfrazelle @Amey-D
MTU selection is difficult, and if there is a transport such as IPSEC in
use may be impossible. So we allow specification of the MTU with the
network-plugin-mtu flag, and we pass this down into the network
provider.
Currently implemented by kubenet.
Automatic merge from submit-queue
Use kubekins-test:v20160822 everywhere
A step towards fixing #31148. We also need to update the test-infra repo to use the new e2e image. I'll fold that into https://github.com/kubernetes/test-infra/pull/419.
Both gcr.io/google-containers/kubekins-test:v20160822 and gcr.io/google-containers/e2e-test:v20160822 have already been pushed.
Automatic merge from submit-queue
Updating version check for etcd
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Currently, if you `make build`, etcd is not properly version checked because `etcd -version` does multi-line output. This output cannot be version compared. This small change fixes that.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Fix caching of go dirs during build
Sometimes when you press `^C` during `make` the subsequent attempt to run `make` hangs due to zero-sized cache file for `ALL_GO_DIRS` var in Makefile.generated_files:
```
vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ KUBE_JUNIT_REPORT_DIR=/tmp/art KUBE_COVER=y make test
# hangs...
^CMakefile:279: recipe for target 'generated_files' failed
make: *** [generated_files] Interrupt
vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ ls -l .make/all_go_dirs.mk
-rw-rw-r-- 1 vagrant vagrant 0 Aug 18 15:03 .make/all_go_dirs.mk
```
Corresponding process subtree looks like the following:
```
└─make test
└─make -f Makefile.generated_files generated_files
└─bash -c grep --color=never -l '+k8s:deepcopy-gen=' | xargs -n1 dirname | sort -u
├─grep --color=never -l +k8s:deepcopy-gen=
├─sort -u
└─xargs -n1 dirname
```
Let's remove the cache file if `cache_go_dirs.sh` gets interrupted.
Automatic merge from submit-queue
e2e-runner: Get GCI builtin k8s version from GCS
**What this PR does / why we need it**:
The GCI release qualification tests use builtin k8s version to run tests. They used to get the builtin version by parsing descriptions of the test images, but that's unreliable because the description format could change. This PR makes it to read a file checked in GCS that has the exact k8s version number.
@adityakali @spxtr Can you review?
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
Remove implicit Prometheus metrics from client
**What this PR does / why we need it**: This PR starts to cut away at dependencies that the client has.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
The implicit registration of Prometheus metrics for request count and latency have been removed, and a plug-able interface was added. If you were using our client libraries in your own binaries and want these metrics, add the following to your imports in the main package: "k8s.io/pkg/client/metrics/prometheus".
```
cc: @kubernetes/sig-api-machinery @kubernetes/sig-instrumentation @fgrzadkowski @wojtek-t
Automatic merge from submit-queue
Kubelet: add --container-runtime-endpoint and --image-service-endpoint
Flag `--container-runtime-endpoint` (overrides `--container-runtime`) is introduced to identify the unix socket file of the remote runtime service. And flag `--image-service-endpoint` is introduced to identify the unix socket file of the image service.
This PR is part of #28789 Milestone 0.
CC @yujuhong @Random-Liu
Automatic merge from submit-queue
fix path handling in hack/lib/init.sh
Jenkinsfile pipeline jobs get cloned into "\<project\> (\<branch\>)". As a result, I can't use certain things in `hack/lib/init.sh`.
This is a small fix for that problem.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Automatic merge from submit-queue
Implement TLS bootstrap for kubelet using `--experimental-bootstrap-kubeconfig` (2nd take)
Ref kubernetes/features#43 (comment)
cc @gtank @philips @mikedanese @aaronlevy @liggitt @deads2k @errordeveloper @justinsb
Continue on the older PR https://github.com/kubernetes/kubernetes/pull/30094 as there are too many comments on that one and it's not loadable now.
Automatic merge from submit-queue
ImagePolicyWebhook Admission Controller
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: This is an implementation of the [image provenance proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/image-provenance.md). It also includes the API definitions by @Q-Lee from https://github.com/kubernetes/kubernetes/pull/30241
**Special notes for your reviewer**:
Please note that this is the first admission controller to make use of the admission controller config file (`--admission-controller-config-file`). I have defined a format for it but we may want to double check it's adequate for future use cases as well.
The format defined is:
```
{
"imagePolicy": {
"kubeConfigFile": "path/to/kubeconfig/for/backend",
"allowTTL": 50, # time in s to cache approval
"denyTTL": 50, # time in s to cache denial
"retryBackoff": 500, # time in ms to wait between retries
"defaultAllow": true # determines behavior if the webhook backend fails
}
}
```
(or yaml)
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Adding ImagePolicyWebhook admission controller.
```
Automatic merge from submit-queue
Basic scaler/reaper for petset
Currently scaling or upgrading a petset is more complicated than it should be. Would be nice if this made code freeze on friday. I'm planning on a follow up change with generation number and e2es post freeze.
Automatic merge from submit-queue
Scheduledjobs e2e
@janetkuo resubmitted e2e for SJ, I've updated all scripts to consume `KUBE_RUNTIME_CONFIG` properly in 2nd commit, ptal
Automatic merge from submit-queue
When running inside docker, activate service account ASAP
Also switching to just use `GOOGLE_APPLICATION_CREDENTIALS`, rather than both.
x-ref https://github.com/kubernetes/test-infra/issues/318
Automatic merge from submit-queue
Gubernator bug fixes: mv and GCS bucket permissions
Fixed issue where results file was not moved correctly, and also the permissions issue with the GCS bucket.
Will rebase after #30414 is merged
@timstclair
Automatic merge from submit-queue
Add a short `-n` for `kubectl --namespace`
fixes#24078
`--namespace` is a very common flag for nearly every `kubectl` command we have. We should claim `-n` for it.
Automatic merge from submit-queue
add --raw for kubectl get
Adds a `--raw` option to `kubectl get` that allow you specify your URI, but use the transport built by `kubectl`. This is especially useful when working with secured environments that require authentication and authorization to hit non-api endpoints. For example, `kubect get --raw /metrics` or if you want to debug a watch with a view at the exact data `kubectl get --raw '/api/v1/namespaces/one/replicationcontrollers?watch=true'`.
@kubernetes/kubectl
@fabianofranz fyi
Automatic merge from submit-queue
Disable linux/ppc64le compilation by default
Work-around for #30384.
I'm still testing this locally to see if it actually works. The build is slow. (PR Jenkins won't tell us whether this fixes ppc.)
cc @Random-Liu @spxtr @david-mcmahon @luxas
New flag --container-runtime-endpoint (overrides --container-runtime)
is introduced to kubelet which identifies the unix socket file of
the remote runtime service. And new flag --image-service-endpoint is
introduced to kubelet which identifies the unix socket file of the
image service.
Automatic merge from submit-queue
Move UTs that block on apiserver to integration tests.
In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail. These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.
This PR simply shuffles those tests to integration tests, which is where they should be.
/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90
Automatic merge from submit-queue
Revert "Revert "syncNetworkUtil in kubelet and fix loadbalancerSourceRange on GCE
Reverts kubernetes/kubernetes#30729
Automatic merge from submit-queue
Add GUBERNATOR flag which produces g8r link for node e2e tests
When you run 'make tests-e2e-node REMOTE=true GUBERNATOR=true' outputs a URL to view the test results on Gubernator. ~~Should work after my PR for Gubernator is merged.~~
@timstclair
Automatic merge from submit-queue
Quobyte Volume plugin
@quofelix and myself developed a volume plugin for [Quobyte](http://www.quobyte.com) which is a software-defined storage solution. This PR allows Kubernetes users to mount a Quobyte Volume inside their containers over Kubernetes.
Here are some further informations about [Quobyte and Storage for containers](http://www.quobyte.com/containers)
The admission controller adds a default class to PVCs that do not require any
specific class. This way, users (=PVC authors) do not need to care about
storage classes, administrator can configure a default one and all these PVCs
that do not care about class will get the default one.
Automatic merge from submit-queue
kubelet eviction on inode exhaustion
Add support for kubelet to monitor for inode exhaustion of either image or rootfs, and in response, attempt to reclaim node level resources and/or evict pods.
Automatic merge from submit-queue
Add annotations to the PodSecurityPolicy Provider interface
@pweil- is this what you were thinking in terms of API changes? I really like to avoid functions with more than 2 return values, but couldn't think of a cleaner approach in this case.
Automatic merge from submit-queue
Expose flags for new NodeEviction logic in NodeController
Fix#28832
Last PR from the NodeController NodeEviction logic series.
cc @davidopp @lavalamp @mml
Automatic merge from submit-queue
Allow a flag that forces kubelet to have a valid kubeconfig
`--require-kubeconfig` forces the kubelet to use the kubeconfig for all
APIserver communication, and exit cleanly. Allows cluster lifecycle to loop waiting for config to be available.
Fixes#30515
A follow up PR will handle the issue discovered where the DefaultCluster rules applied to kubeconfig allow a malicious party who can bind to localhost:8080 to take advantage of an admin misconfiguration.
@lukemarsden @mikedanese
```release-note
The Kubelet now supports the `--force-kubeconfig` option which reads all client config from the provided `--kubeconfig` file and will cause the Kubelet to exit with error code 1 on error. It also forces the Kubelet to use the server URL from the kubeconfig file rather than the `--api-servers` flag. Without this flag set, a failure to read the kubeconfig file would only result in a warning message.
In a future release, the value of this flag will be defaulted to `true`.
```
Automatic merge from submit-queue
Create a kubekins-e2e image with runner and dependencies
Update dockerized-e2e-runner.sh to just run an image with the appropriate e2e environment variables set.
e2e-runner.sh, upload-to-gcs.sh and e2e.go now live inside the container.
Automatic merge from submit-queue
syncNetworkUtil in kubelet and fix loadbalancerSourceRange on GCE
fixes: #29997#29039
@yujuhong Can you take a look at the kubelet part?
@girishkalele KUBE-MARK-DROP is the chain for dropping connections. Marked connection will be drop in INPUT/OUTPUT chain of filter table. Let me know if this is good enough for your use case.
Automatic merge from submit-queue
Fix subtle build breakage
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
NB: this requires a `make clean` to activate.
@lavalamp @jbeda @quinton-hoole @david-mcmahon
Automatic merge from submit-queue
Let load and density e2e tests use GC if it's on
I've run the 100 and 500 nodes tests and they both pass.
The test-infra half of the PR is https://github.com/kubernetes/test-infra/pull/369
cc @lavalamp
Automatic merge from submit-queue
Add CloudStack cloud provider (extended and refactored)
This PR is superseding PR #26165 is which some groundwork for this PR has been done. So this PR now fixes#26165 and fixes#26045.
I've been in contact with @ngtuna about this updated version of his earlier work (which is still in this PR as one squashed commit) and he has given his 👍 for this 😉
This PR adds additional logic for allocating and associating a public IP, if the `—load-balancer-ip` option is not used. It will do proper management of public IP’s that are allocated by this provider ( so IP’s that are no longer needed/used will also be released again).
Additionally the provider can now also work with CloudStack projects and advanced (VPC) networks. And lastly the Zone interface now returns an actual zone (supplied by the cloud config), a few logical errors are fixed and the first few tests are added.
All the functionality is extensively tested against both basic and advanced (VPC) networks and of course all new and existing (integration) tests are all passing.
Automatic merge from submit-queue
Install bc in the kubekins-test image
I finally figured out those
```
/dev/fd/63: line 124: bc: command not found
/dev/fd/63: line 125: bc: command not found
```
lines that keep appearing in build logs.
Image is not yet pushed.
Automatic merge from submit-queue
Update core etcd references to use 3.0.4
This updates the core references to use 3.0.4.
There are still legacy references in the code base that should be cleaned, or just removed but I'm reluctant to purge.
/cc @kubernetes/sig-scalability
This commit adds logic for allocating and associating a public IP, if the `—load-balancer-ip` option is not used. It will do proper management of IP’s that are allocated by this provider, so IP’s that are no longer needed/used will also be released again.
Additionally the provider can now also work with CloudStack projects and advanced (VPC) networks.
Lastly the Zone interface now returns an actual zone (supplied by the cloud config), a few logical errors are fixed and the first few tests are added.
All the functionality is extensively tested against both basic and advanced (VPC) networks.
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
Automatic merge from submit-queue
A build and deploy script to manage cluster federation lifecycle and a Makefile wrapper to drive that process.
Also includes a sample config file to describe clusters.
The build script implements the following things:
1. Generates the required configs.
2. Builds the hyperkube binary and the corresponding docker image.
3. Pushes the image to a specified repository.
4. Pulls the federation installer docker images.
5. Builds the Kubernetes clusters described the config.json file.
6. Pushes the federation components to one of the Kubernetes clusters
built in the previous step.
7. Also turns down the federation components and the Kubernetes
clusters.
**NOTE**: Installer images are right now being pulled from my public repository of docker images. I am working on pushing them to our release repository.
```release-note
Cluster Federation components can now be built and deployed using the make command. Please see federation/README.md for details.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes: Issue #26655
Automatic merge from submit-queue
[kubelet] Introduce --protect-kernel-defaults flag to make the tunable behaviour configurable
Let's make the default behaviour of kernel tuning configurable. The default behaviour is kept modify as has been so far.
Automatic merge from submit-queue
return err on `kubectl run --image` with invalid value
When running `kubectl run <configname> --image="Invalid$$%ImageValue%%__"`, a configuration is successfully created with an image name that is not a valid value for an image reference.
This patch validates that the image name is a valid image reference, and returns an error before creating a config if an invalid value is passed.
`$ kubectl run test --image="Invalid__%imagename"`
```
error: Invalid image name "Invalid__%imagename": invalid reference format
```
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30162)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Start verifying golint on a per-package basis as packages are fixed
<!--
Checklist for submitting a Pull Request
Please remove this comment block before submitting.
1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->
```release-note
Added `golint` for pkg/security/podsecuritypolicy/capabilities` along with validation.
```
[]()
This is a POC to start enabling `golint` checks on a per-package basis, we did this on the docker project and it was a great way for new contributors to help and it benefits the project overall. All they have to do is add the package they fixed to the bash array in `hack/verify-golint.sh` and fix all the lint errors.
Eventually when all the packages have been fixed we can change the function to `find_files`. Or something based off which files are changed in a patch set to verify `golint`.
Now I used this specific package as the POC because I wanted to show the downside of this changing the api of the package.
Most of the times this arose in docker/docker we decided that if someone wasn't importing their deps locally then it was their loss, but I'm not sure if you all will agree.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/27911)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Check for CAP_SYS_ADMIN in Kubelet
Comment from nsenter_mount.go header:
The Kubelet process must have CAP_SYS_ADMIN (required by nsenter); at
the present, this effectively means that the kubelet is running in a
privileged container.
Related to #26093
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30176)
<!-- Reviewable:end -->
Automatic merge from submit-queue
[Kubelet] Rename `--config` to `--pod-manifest-path`. `--config` is deprecated.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose. I didn't change the flag name because that
API should remain stable.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29999)
<!-- Reviewable:end -->
Also provide a new --pod-manifest-path flag and deprecate the old
--config one.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose.
Automatic merge from submit-queue
Cut the client repo, staging it in the main repo
Tracking issue: #28559
ref: https://github.com/kubernetes/kubernetes/pull/25978#issuecomment-232710174
This PR implements the plan a few of us came up with last week for cutting client into its own repo:
1. creating "_staging" (name is tentative) directory in the main repo, using a script to copy the client and its dependencies to this directory
2. periodically publishing the contents of this staging client to k8s.io/client-go repo
3. converting k8s components in the main repo to use the staged client. They should import the staged client as if the client were vendored. (i.e., the import line should be `import "k8s.io/client-go/<pacakge name>`). This requirement is to ease step 4.
4. In the future, removing the staging area, and vendoring the real client-go repo.
The advantage of having the staging area is that we can continuously run integration/e2e tests with the latest client repo and the latest main repo, without waiting for the client repo to be vendored back into the main repo. This staging area will exist until our test matrix is vendoring both the client and the server.
In the above plan, the tricky part is step 3. This PR achieves it by creating a symlink under ./vendor, pointing to the staging area, so packages in the main repo can refer to the client repo as if it's vendored. To prevent the godep tool from messing up the staging area, we export the staged client to GOPATH in hack/godep-save.sh so godep will think the client packages are local and won't attempt to manage ./vendor/k8s.io/client-go.
This is a POC. We'll rearrange the directory layout of the client before merge.
@thockin @lavalamp @bgrant0607 @kubernetes/sig-api-machinery
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29147)
<!-- Reviewable:end -->
The Kubelet process must have CAP_SYS_ADMIN, which implies that
the kubelet process must be either running as root or in a privileged
container. Make this check early in the startup sequence and bail out
if necessary.
Related to #26093
Automatic merge from submit-queue
kubectl config set-crentials: add arguments for auth providers
This PR adds `--auth-provider` and `--auth-provider-arg` flags to the
`kubectl config set-credentials` sub-command.
There's currently no way of interacting with the new auth provider framework added in #23066 through kubectl. You have to render a custom kubeconfig to use them. Additionally `kubectl config set` just sort of craps out when attempting to interact with authentication info objects (#29312).
This is a minimal implementation of allowing `kubect config set-credentials` to set fields for client auth providers.
cc @cjcullen @kubernetes/kubectl
Automatic merge from submit-queue
E2E & Node E2E: Move configmap, docker_containers, downward_api, expansion and secrets test into common directory.
This is the 3rd part of #29494.
For #29081.
Based on #29092, #29806.
The first commit is squash of all dependent commits. Please only review the second commit.
The second PR added 17 lines.
@vishh @timstclair
Automatic merge from submit-queue
add subjectaccessreviews resource
Adds a subjectaccessreviews endpoint that uses the API server's authorizer to determine if a subject is allowed to perform an action.
Part of kubernetes/features#37
Automatic merge from submit-queue
enhancements to run local-up-cluster.sh
- fix containerized kubelet: bind mount /dev and cloud credential directories
- add CLOUD_CONFIG env to get openstack credentials
@kubernetes/sig-storage @jayunit100
Automatic merge from submit-queue
Cleanup k8s script noise with a verbosity concept
Fixes https://github.com/kubernetes/kubernetes/issues/30109
The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.
V=2 kube::log::info foo bar
Would only print "foo bar" if $KUBE_VERBOSE >= 2.
Examples:
Default verbosity (1 for make commands):
```
$ make kubelet
+++ [0804 17:23:32] Generating bindata:
/usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0804 17:23:37] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0804 17:23:37] Building go targets for linux/amd64:
cmd/kubelet
# k8s.io/kubernetes/pkg/kubelet
pkg/kubelet/kubelet.go:247: undefined: a
make: *** [kubelet] Error 1
```
Extra verbose (5, comparable to previous levels):
<details>
```
$ make kubelet KUBE_VERBOSE=5
I0804 17:31:05.083395 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/trace.go:151:30: cannot use (traceBufHeader literal) (value of type traceBufHeader) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.083503 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/trace.go:151:7: array length 64 << 10 - unsafe.Sizeof((traceBufHeader literal)) (value of type uintptr) must be constant
I0804 17:31:05.083600 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mgcwork.go:269:37: cannot use (workbufhdr literal) (value of type workbufhdr) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.083654 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mgcwork.go:269:7: array length (_WorkbufSize - unsafe.Sizeof((workbufhdr literal))) / sys.PtrSize (value of type uintptr) must be constant
I0804 17:31:05.084006 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:28: cannot use (hchan literal) (value of type hchan) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.084040 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:66: cannot use (hchan literal) (value of type hchan) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.084076 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:14: unsafe.Sizeof((hchan literal)) + uintptr(-int(unsafe.Sizeof((hchan literal))) & (maxAlign - 1)) (value of type uintptr) is not constant
I0804 17:31:05.085536 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:80:31: cannot use (struct{b bmap; v int64} literal).v (value of type int64) as unsafe.ArbitraryType value in argument to unsafe.Offsetof
I0804 17:31:05.085567 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:80:15: unsafe.Offsetof((struct{b bmap; v int64} literal).v) (value of type uintptr) is not constant
I0804 17:31:05.085788 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:1053:45: cannot convert &zeroinitial (value of type *[1024]byte) to unsafe.Pointer
I0804 17:31:05.086995 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mfinal.go:20:65: cannot use (finalizer literal) (value of type finalizer) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.087031 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mfinal.go:20:11: array length (_FinBlockSize - 2 * sys.PtrSize - 2 * 4) / unsafe.Sizeof((finalizer literal)) (value of type uintptr) must be constant
I0804 17:31:05.087957 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mstats.go:170:39: cannot use memstats.by_size (variable of type [67]struct{size uint32; nmalloc uint64; nfree uint64}) as unsafe.ArbitraryType value in argument to unsafe.Offsetof
I0804 17:31:05.087999 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mstats.go:170:76: cannot use memstats.by_size[0] (variable of type struct{size uint32; nmalloc uint64; nfree uint64}) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.088483 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/panic.go:118:34: cannot use (_defer literal) (value of type _defer) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.088510 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/panic.go:118:20: unsafe.Sizeof((_defer literal)) (value of type uintptr) is not constant
I0804 17:31:05.089812 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/sema.go:42:42: cannot use (semaRoot literal) (value of type semaRoot) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.089845 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/sema.go:42:8: array length sys.CacheLineSize - unsafe.Sizeof((semaRoot literal)) (value of type uintptr) must be constant
I0804 17:31:05.094634 2601 parse.go:307] type checking encountered some errors in "runtime", but ignoring.
I0804 17:31:05.875185 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/davecgh/go-spew/spew/bypass.go:33:26: cannot use (*byte)(nil) (value of type *byte) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.875234 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/davecgh/go-spew/spew/bypass.go:33:12: unsafe.Sizeof((*byte)(nil)) (value of type uintptr) is not constant
I0804 17:31:05.875838 2601 parse.go:307] type checking encountered some errors in "github.com/davecgh/go-spew/spew", but ignoring.
I0804 17:31:05.897216 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/crypto/cipher/xor.go:12:36: cannot use uintptr(0) (constant 0 of type uintptr) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.897261 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/crypto/cipher/xor.go:12:18: int(unsafe.Sizeof(uintptr(0))) (value of type int) is not constant
I0804 17:31:05.897360 2601 parse.go:307] type checking encountered some errors in "crypto/cipher", but ignoring.
I0804 17:31:06.400904 2601 conversion.go:227] considering pkg "k8s.io/kubernetes/federation/apis/core/v1"
I0804 17:31:06.401138 2601 conversion.go:243] tags: ["k8s.io/kubernetes/federation/apis/core"]
I0804 17:31:06.427408 2601 conversion.go:283] no viable conversions, not generating for this package
I0804 17:31:06.427508 2601 main.go:73] Completed successfully.
Go version: go version go1.6.2 linux/amd64
+++ [0804 17:31:06] Generating bindata:
/usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
Generated bindata file : 11536 ../../..//test/e2e/generated/bindata.go lines of lovely automated artifacts
+++ [0804 17:31:12] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0804 17:31:12] Building go targets for linux/amd64:
cmd/kubelet
# k8s.io/kubernetes/pkg/kubelet
pkg/kubelet/kubelet.go:247: undefined: a
!!! Error in /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:506
'go install "${goflags[@]:+${goflags[@]}}" -gcflags "${gogcflags}" -ldflags "${goldflags}" "${nonstatics[@]:+${nonstatics[@]}}"' exited with status 2
Call stack:
1: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:506 kube::golang::build_binaries_for_platform(...)
2: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:692 kube::golang::build_binaries(...)
3: hack/make-rules/build.sh:27 main(...)
Exiting with status 1
!!! Error in /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:596
'( kube::golang::setup_env; echo "Go version: $(go version)"; local host_platform; host_platform=$(kube::golang::host_platform); local goflags goldflags gogcflags; eval "goflags=(${KUBE_GOFLAGS:-})"; goldflags="${KUBE_GOLDFLAGS:-} $(kube::version::ldflags)"; gogcflags="${KUBE_GOGCFLAGS:-}"; local use_go_build; local -a targets=(); local arg; readonly BINDATAS=("${KUBE_ROOT}/test/e2e/framework/gobindata_util.go"); kube::log::status "Generating bindata:" "${BINDATAS[@]}"; for bindata in ${BINDATAS[@]};
do
if [[ -f $bindata ]]; then
go generate "${bindata}";
fi;
done; for arg in "$@";
do
if [[ "${arg}" == "--use_go_build" ]]; then
use_go_build=true;
else
if [[ "${arg}" == -* ]]; then
goflags+=("${arg}");
else
targets+=("${arg}");
fi;
fi;
done; if [[ ${#targets[@]} -eq 0 ]]; then
targets=("${KUBE_ALL_TARGETS[@]}");
fi; local -a platforms=(${KUBE_BUILD_PLATFORMS:-}); if [[ ${#platforms[@]} -eq 0 ]]; then
platforms=("${host_platform}");
fi; local binaries; binaries=($(kube::golang::binaries_from_targets "${targets[@]}")); local parallel=false; if [[ ${#platforms[@]} -gt 1 ]]; then
local gigs; gigs=$(kube::golang::get_physmem); if [[ ${gigs} -ge ${KUBE_PARALLEL_BUILD_MEMORY} ]]; then
kube::log::status "Multiple platforms requested and available ${gigs}G >= threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in parallel"; parallel=true;
else
kube::log::status "Multiple platforms requested, but available ${gigs}G < threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in serial"; parallel=false;
fi;
fi; kube::golang::build_kube_toolchain; if [[ "${parallel}" == "true" ]]; then
kube::log::status "Building go targets for ${platforms[@]} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}";
do
( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: go build started"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-}; kube::log::status "${platform}: go build finished" ) &> "/tmp//${platform//\//_}.build" &
done; local fails=0; for job in $(jobs -p);
do
wait ${job} || let "fails+=1";
done; for platform in "${platforms[@]}";
do
cat "/tmp//${platform//\//_}.build";
done; exit ${fails};
else
for platform in "${platforms[@]}";
do
kube::log::status "Building go targets for ${platform}:" "${targets[@]}"; kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-};
done;
fi )' exited with status 1
Call stack:
1: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:596 kube::golang::build_binaries(...)
2: hack/make-rules/build.sh:27 main(...)
Exiting with status 1
make: *** [kubelet] Error 1
```
</details>
Remaining work: Add a verbosity label to more log messages.
/cc @kubernetes/sig-api-machinery @kubernetes/contributor-experience
Automatic merge from submit-queue
E2E & NodeE2E: Move host_path, downwardapi_volume and empty_dir into common directory.
This is the second part of #29494.
For #29081.
Based on #29092, #29806.
The first commit is squash of all dependent commits. Please only review the second commit.
The second PR is only 20 lines of change.
@vishh @timstclair
The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.
V=2 kube::log::info foo bar
Would only print "foo bar" if $KUBE_VERBOSE >= 2.
Automatic merge from submit-queue
federation: Adding secret API
Adding secret API to federation-apiserver and updating the federation client to include secrets
Automatic merge from submit-queue
Install go-bindata in cross-build image
Another follow-up to #25584.
We need `go-bindata` to create `test/e2e/generated`, and downloading it with `go get` at build time is painful for a variety of reasons. We can just include it in the cross-build image and not worry about it, especially as it updates very infrequently.
This fixes `hack/update-generated-protobuf.sh` as well.
cc @jayunit100 @soltysh
Automatic merge from submit-queue
Make verify-godeps check the results
Verify the results of godep restore/save. When forced to run this finds a handful of real deltas that need to be resolved.
Fixes#29402
@rmmh because you wrote #29571 which is the major culprit
Automatic merge from submit-queue
Remove myself from test ownership.
These are almost certainly not correct, but probably more likely owners than myself.
@rmmh @dchen1107 @timstclair @erictune @mtaufen @caesarxuchao @fgrzadkowski @krousey @lavalamp
Automatic merge from submit-queue
Add dry run capability to cherry_pick_pull.sh
When creating patches for downstream projects like OpenShift, I found it to be helpful to have a dry run mode for the cherry-pick script.
@k8s-oncall
cc @kubernetes/rh-cluster-infra
Automatic merge from submit-queue
e2e-runner: Fix the logic of pulling GCI builtin version of k8s
Apparently setting JENKINS_PUBLISHED_VERSION didn't work because it only worked
with `ci/latest`, `release/latest-1.3` or the like, whereas we wanted to pull
specific versions like `release/v1.3.3`. Added a function
`fetch_gci_version_tars` instead.
@spxtr Can you review? Hopefully this should get our QA tests passing.
Automatic merge from submit-queue
Clone kubernetes/release into something other than /release due to conflict with /release.
It looks like $WORKSPACE is the root of the kubernetes clone. I think ideally we'd move that down a level and allow subdirectories of different repos. I think this may become the norm as we split things up further as well. What do you think and how should we proceed here? @ixdy @spxtr
Automatic merge from submit-queue
[Garbage Collector] add e2e tests again
#27151 is reverted because gke didn't start correctly after it's merged (https://github.com/kubernetes/kubernetes/pull/27151#issuecomment-233030686).
The possible problem is the `unbound variable`, which is fixed in the second commit of this PR. However, I cannot verify if the PR will fail the gke suite since I don't have the environment to run that suite.
@wojtek-t @lavalamp
Automatic merge from submit-queue
Update test-owners with new tests, add catch-all assignment to test-infra team.
We will triage any additional failures, since they're more likely to be infra related. If they're not, they can always be reassigned (and the owners list can be updated!)
/cc @kubernetes/test-infra-maintainers
Automatic merge from submit-queue
Fix make verify
@thockin this fixes `make verify` which was previously failing due to undefined `BRANCH` and `VERBOSE` variables, ptal
Automatic merge from submit-queue
Add a flag for `kubectl expose`to set ClusterIP and allow headless services
- Use `--cluster-ip=None` to create a headless service
- Fixes#10294
Automatic merge from submit-queue
[Federation] Downsize the release binary distribution.
There are two things that this PR implements:
1. It removes `federation-apiserver` and `federation-controller-manager` from binaries and docker_wrapped_binaries target lists.
2. Build the docker image for `hyperkube` on-the-fly while pushing the federation images.
```release-note
Federation binaries and their corresponding docker images - `federation-apiserver` and `federation-controller-manager` are now folded in to the `hyperkube` binary. If you were using one of these binaries or docker images, please switch to using the `hyperkube` version. Please refer to the federation manifests - `federation/manifests/federation-apiserver.yaml` and `federation/manifests/federation-controller-manager-deployment.yaml` for examples.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes Issue #28633
Apparently setting JENKINS_PUBLISHED_VERSION didn't work because it only worked
with `ci/latest`, `release/latest-1.3` or the like, whereas we wanted to pull
specific versions like `release/v1.3.3`. Added a function
`fetch_gci_version_tars` instead.
Automatic merge from submit-queue
e2e-runner: Remove quotes from jq output
Recent GCI e2e test failures suggest that the image name was double quoted when
passed to gcloud. E.g.,
```
09:59:51 - Invalid value for field
'resource.disks[0].initializeParams.sourceImage':
'https://www.googleapis.com/compute/v1/projects/container-vm-image-staging/global/images/"gci-base-53-8530-36-0"'.
The referenced image resource cannot be found.
```
The `-r`, or `--raw-output` option strips quotes from jq's output. With this
change, the output looks like:
```
+ export KUBE_GCE_MASTER_IMAGE=gci-base-54-8650-0-0 # Not quoted
```
@spxtr Can you review this XS PR? All of our e2e jobs are failing due to this.
Also, this was a bug introduced by #29631. I tried several times to duplicate the Kubekins instance locally, but still couldn't get it running. It would be GREAT if Jenkins changes can be tested before getting merged. :)
Automatic merge from submit-queue
change the relative links to definition in operations.html
...to satisfy the new directory layout in the kubernetes.io.
This will make the manual changes in https://github.com/kubernetes/kubernetes.github.io/pull/369 persist.
I still need to bump up the gcr.io/google_containers/gen-swagger-docs image version. I'll do that after I get LGTM here.
cc @bgrant0607 @xiangpengzhao
Automatic merge from submit-queue
Node E2E: Make node e2e parallel
For https://github.com/kubernetes/kubernetes/issues/29081.
Fix https://github.com/kubernetes/kubernetes/issues/26215.
Based on https://github.com/kubernetes/kubernetes/pull/28807, https://github.com/kubernetes/kubernetes/pull/29020, will rebase after they are merged.
**Only the last commit is new.**
We are going to move more tests into the node e2e test. However, currently node e2e test only run sequentially, the test duration will increase quickly when we add more test.
This PR makes the node e2e test run in parallel so as to shorten test duration, so that we can add more test to improve the test coverage.
* If you run the test locally with `make test-e2e-node`, it will use `-p` ginkgo flag, which uses `(cores-1)` parallel test nodes by default.
* If you run the test remotely or in the Jenkin, the parallelism will be controlled by the environment variable `PARALLELISM`. The default value is `8`, which is reasonable for our test node (n1-standard-1).
Before this PR, it took **833.592s** to run all test on my desktop.
With this PR, it only takes **234.058s** to run.
The pull request node e2e run with this PR takes **232.327s**.
The pull request node e2e run for other PRs takes **673.810s**.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
2nd iteration of Gobindata + RepoRoot removals.
Part of my overall life mission to kill reporoot.
Fixes part of #24348
cc @kubernetes/sig-testing .
bindata and yaml, Gobindata automation
bindata utils for generating, go generate
match server version
gitignore for dirty, ca, rbase, KUBE_ROOT, buildfix
(rebased jul-25,29)
Recent GCI e2e test failures suggest that the image name was double quoted when
passed to gcloud. E.g.,
```
09:59:51 - Invalid value for field
'resource.disks[0].initializeParams.sourceImage':
'https://www.googleapis.com/compute/v1/projects/container-vm-image-staging/global/images/"gci-base-53-8530-36-0"'.
The referenced image resource cannot be found.
```
The `-r`, or `--raw-output` option strips quotes from jq's output. With this
change, the output looks like:
```
+ export KUBE_GCE_MASTER_IMAGE=gci-base-54-8650-0-0 # Not quoted
```
Automatic merge from submit-queue
Add support for GCI QA jobs
GCI QA jobs will run tests using GCI daily builds, and the kubernetes built into
the images. All QA jobs will set the `JENKINS_USE_GCI_VERSION` env var.
@Amey-D @spxtr Can you review?
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
Fix ConfigMap related node e2e tests on selinux enabled systems
One selinux enabled systems, it might require to relabel
/var/lib/kubelet, otherwise following tests fail:
Summarizing 7 Failures:
```
[Fail] [k8s.io] ConfigMap [It] updates should be reflected in volume [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e_node/configmap.go:131
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume as non-root with FSGroup [Feature:FSGroup]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume with mappings as non-root [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volumpe [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume with mappings [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume with mappings as non-root with FSGroup [Feature:FSGroup]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume as non-root [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
```
@kubernetes/rh-cluster-infra
GCI QA jobs will run tests using GCI daily builds, and the kubernetes built into
the images. All QA jobs will set the `JENKINS_USE_GCI_VERSION` env var.
Automatic merge from submit-queue
Silence curl output
Removes the following from script output:
curl: (7) Failed to connect to 127.0.0.1 port 8080: Connection refused
Automatic merge from submit-queue
Enable endpoints in kubernetes service started by local-cluster-up.sh
--advertise_address should be set to 127.0.0.1, So let API server pick
the default if necessary.
Fixes#29374
Automatic merge from submit-queue
Add kubelet flag for eviction-minimum-reclaim
This is taken from #27199 as its the most burdensome to rebase and should have little disagreement.
/cc @vishh @ronnielai PTAL
/var/lib/kubelet, otherwise following tests fail:
Summarizing 7 Failures:
[Fail] [k8s.io] ConfigMap [It] updates should be reflected in volume [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e_node/configmap.go:131
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume as non-root with FSGroup [Feature:FSGroup]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume with mappings as non-root [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volumpe [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume with mappings [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume with mappings as non-root with FSGroup [Feature:FSGroup]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume as non-root [Conformance]
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:2115
Automatic merge from submit-queue
Makefile fixes and speed
A few fixes that, together, make the 'make' experience smoother and more seamless.
* `make clean all` works (@soltysh)
* `make clean` does no spend 5 seconds loading deps just to remove them
* deps building is faster
Fixes#28890
Ref #8830
This operation takes 2-5 seconds on every build, but doesn't actually need to
run most of the time. Now we cache it and see if it needs a rebuild (fast)
before actually rebuilding (slow).
Automatic merge from submit-queue
Revert "Drop support for --gce-service-account, require activated creds"
Reverts kubernetes/kubernetes#28802
This appears to break the soak tests with "invalid grant" errors -- see the recent batch of errors in #27920.
Automatic merge from submit-queue
Change some node e2e test to use the prepull image framework.
Fix https://github.com/kubernetes/kubernetes/issues/28868.
Node e2e test framework pre-pulls all images in [image_list.go](bc2f223f5a/test/e2e_node/image_list.go)
All node e2e test should use image from the "image_list". If a test needs new image, we should update the image_list to include the new image.
/cc @kubernetes/sig-node to notice people to use `image_list` when adding test. :)
Automatic merge from submit-queue
add tokenreviews endpoint to implement webhook
Wires up an API resource under `apis/authentication.k8s.io/v1beta1` to expose the webhook token authentication API as an API resource. This allows one API server to use another for authentication and uses existing policy engines for the "authoritative" API server to controller access to the endpoint.
@cjcullen you wrote the initial type
Automatic merge from submit-queue
adds source debug build options
See issue & discussion here: #28227
Enables source debugging the Kubernetes binaries with tools like delve by providing the user with the ability to provide debug build options to the glang compiler.
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
Automatic merge from submit-queue
federation: Adding namespaces API
Adding namespaces API to federation-apiserver and updating the federation client to include namespaces
--------------------------
Original description:
This adds the namespaces API to federation-apiserver.
The first commit is https://github.com/kubernetes/kubernetes/pull/26142.
I spent a whole hour troubleshooting all sorts of auth issues today, until I noticed that it was trying to make a PR on my personal account, while the fork is actually on my company's GH organization account.
Automatic merge from submit-queue
Node E2E: Make it possible to share test between e2e and node e2e
This PR is part of the plan to improve node e2e test coverage.
* Now to improve test coverage, we have to copy test from e2e to node e2e.
* When adding a new test, we have to decide its destiny at the very beginning - whether it is a node e2e or e2e.
This PR makes it possible to share test between e2e and node e2e.
By leveraging the mechanism of ginkgo, as long as we can import the test package in the test suite, the corresponding `Describe` will be run to initialize the global variable `_`, and the test will be inserted into the test suite. (See https://github.com/onsi/composition-ginkgo-example)
In the future, we just need to use the framework to write the test, and put the test into `test/e2e/node`, then it will be automatically shared by the 2 test suites.
This PR:
1) Refactored the framework to make it automatically differentiate e2e and node e2e (Mainly refactored the `PodClient` and the apiserver client initialization).
2) Created a new directory `test/e2e/node` and make it shared by e2e and node e2e.
3) Moved `container_probe.go` into `test/e2e/node` to verify the change.
@kubernetes/sig-node
[]()
Automatic merge from submit-queue
Detect flakes in PR builder e2e runs
Won't be mergable until https://github.com/onsi/ginkgo/pull/261 is agreed upon and merged.
Tossing a PR here to get the e2e test to run on it.
Automatic merge from submit-queue
Drop support for --gce-service-account, require activated creds
Now that `gcloud auth activate-service-account` is in remove support in the test framework for default service accounts -- testing GCE/GKE now requires prior gcloud activation.
Automatic merge from submit-queue
Make sure --record=false is acknowledged when passed to commands
```release-note
Change setting "kubectl --record=false" to stop updating the change-cause when a previous change-cause is found.
```
Ensures that when `--record=false` is explicity set that no `ChangeCauseAnnotation`s are set on the object. Previously, if `--record=true` was used then all following actions triggered a `ChangeCauseAnnotation` even if `--record=false` was set, due to the prior `ChangeCauseAnnotation` existing.
Reference to bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1351127
[]()
Automatic merge from submit-queue
[Kubelet] Improving QOS in kubelet by introducing QoS level Cgroups - `--cgroups-per-qos`
This PR is tied to this upstream issue #27204
Please note that only the last commit is unique to this PR. The first two commits are from previous PR's.
It introduces a new flag in the Kubelet which can be used to specify if the user wants to use the QoS cgroup hierarchy.
cc @kubernetes/sig-node
Automatic merge from submit-queue
Fix build for non-GOPATH users
I want feedback on this. There are 2 commits here, we should probably only do one of them. The release note will depend on which one we choose. Also docs need update.
Fixes#28890Fixes#28987
Automatic merge from submit-queue
[garbage collector] add e2e test
This PR also includes some changes to plumb controller-manager's `--enable_garbage_collector` from the environment variable.
The e2e test will not be run by the core suite because it's marked `[Feature:GarbageCollector]`.
The corresponding jenkins job configuration PR is https://github.com/kubernetes/test-infra/pull/132.
Automatic merge from submit-queue
e2e-runner: Update the image family used for GCI canary test images
@Amey-D @spxtr Can you review?
@Random-Liu Heads up
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
controller-manager support number of garbage collector workers to be configurable
The number of garbage collector workers of controller-manager is a fixed value 5 now, make it configurable should more properly
This mostly takes the previously checked in files and removes them, and moves
the generation to be on-demand instead of manual. Manually verified no change
in generated output.
This allows us to start building real dependencies into Makefile.
Leave old hack/* scripts in place but advise to use 'make'. There are a few
rules that call things like 'go run' or 'build/*' that I left as-is for now.
Automatic merge from submit-queue
node_e2e: configure gce images via config file
This file provides the abiliy to specify image project on a per-image
basis and is more extensible for future changes.
For backwards compatibility and local development convenience, the
existing flags are kept and should work.
The eventual goal is to be able to source some images, such as the CoreOS one (and possibly containervm one) from their upstream projects and do all new configuration changes via a cloud-init key added to the image config.
This PR is a first step there. A following PR will add a config key of `cloud-init` or `user-data` and migrate the CoreOS e2e to use that.
This motivation is driven by the fact that currently the changes needed for the CoreOS image can all be done quickly in cloud-init and this will make it much easier to update the image and ensure that changes are applied consistently.
/cc @timstclair @vishh @yifan-gu @pwittrock
Automatic merge from submit-queue
Node E2E: Prep for continuous Docker validation node e2e test
Based on https://github.com/kubernetes/kubernetes/pull/28516, for https://github.com/kubernetes/kubernetes/issues/25215.
https://github.com/kubernetes/kubernetes/pull/26813 added support to run e2e test on gci preview image and newest docker version.
This PR added the same support to node e2e test.
The main dependencies of node e2e test are `docker`, `kubelet`, `etcd` and `apiserver`.
Currently, node e2e test builds `kubelet` and `apiserver` locally, and copies them into `/tmp` directory in VM instance. GCI also has built-in `docker`. So the only dependency missing is `etcd`.
This PR injected a simple cloud-init script when creating instance to install `etcd` during node startup.
@andyzheng0831 for the cloud init script.
@wonderfly for the gci instance setup.
@pwittrock for the node e2e test change.
/cc @dchen1107
[]()
This file provides the abiliy to specify image project on a per-image
basis and is more extensible for future changes.
For backwards compatibility and local development convenience, the
existing flags are kept and should work.
Search and replace for references to moved examples
Reverted find and replace paths on auto gen docs
Reverting changes to changelog
Fix bugs in test-cmd.sh
Fixed path in examples README
ran update-all successfully
Updated verify-flags exceptions to include renamed files
Automatic merge from submit-queue
Move KUBE_GIT_UPSTREAM out of init.sh and into *-munge-docs.sh.
It is only used in those 2 scripts and this way we can set the value dynamically.
Clean up a bit too (80col, formatting)
This drives conversion generation from file tags like:
// +conversion-gen=k8s.io/my/internal/version
.. rather than hardcoded lists of packages.
The only net change in generated code can be explained as correct. Previously
it didn't know that conversion was available.
Automatic merge from submit-queue
Prep for not checking in generated, part 1/2
This PR is extracted from #25978 - it is just the deep-copy related parts. All the Makefile and conversion stuff is excluded.
@wojtek-t this is literally branched, a bunch of commits deleted, and a very small number of manual fixups applied. If you think this is easier to review (and if it passes CI) you can feel free to go over it again. I will follow this with a conversion-related PR to build on this.
Or if you prefer, just close this and let the mega-PR ride.
@lavalamp
Automatic merge from submit-queue
Node E2E: Disable kubenet for local node e2e test.
After https://github.com/kubernetes/kubernetes/pull/28196, we must manually setup cni and nsenter in local node to run `make test_e2e_node`, which may not be necessary for local development.
I've tried to move cni downloading logic into `BeforeSuite`, however it is still hard to figure out who should install nsenter, manually installed by every developer? in the `setup_host.sh` script? in `BeforeSuite`?
This PR:
* Added a flag to disable kubenet and disabled kubenet in local test.
* Cleaned up the CNI installation logic a bit.
/cc @yujuhong @freehan
[]()
This drives most of the logic of deep-copy generation from tags like:
// +deepcopy-gen=package
..rather than hardcoded lists of packages. This will make it possible to
subsequently generate code ONLY for packages that need it *right now*, rather
than all of them always.
Also remove pkgs that really do not need deep-copies (no symbols used
anywhere).
This re-institutes some of the rolled-back logic from previous commits. It
bounds the scope of what the deepcopy generator is willing to do with regards
to generating and calling generated functions.
Automatic merge from submit-queue
s/positive/negative/ in the error message
I think we meant to imply that "if the error isn't real" then update the
exceptions file - which to me means "false negative", ie. false failure.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Automatic merge from submit-queue
break integration tests into seperate packages so that they run in parallel
If this is broken then our tests aren't threadsafe.
ref #25940
Automatic merge from submit-queue
[client-gen] stop update release_1_3 clientset; create release_1_4
We'll bump the version after we cut 1.3 release. I'll keep this PR assigned to myself in the mean time.
@lavalamp
Automatic merge from submit-queue
Add test/test_owners.csv, for automatic assignment of test failures.
This file will be read by the munger -- see kubernetes/contrib#1264
This also includes a simple script to do minor automatic updates to the CSV.
I'd like to get `update_owners.py` into a more usable state -- right now the CSV is based directly on the Google Sheets data. It has 9 outdated tests and is missing 80 new tests.
I can randomly assign new tests to people on kubernetes-maintainers, but are there any caveats to how the assignment should work? Should they be load balanced? Should some people in the group not receive issues? Etc.
Automatic merge from submit-queue
Implementing a proper master/worker split in the juju cluster code.
```
release-note-none
```
General updates to the cluster/juju Kubernetes provider, to bring it up to date.
Updating the skydns templates to version 11
Updating the etcd container definition to include arch.
Updating the master template to include arch and version for hyperkube container.
Adding dns_domain configuration options.
Adding storage layer options.
[]()
Updating the skydns templates to version 11
Updating the etcd container definition to include arch.
Updating the master template to include arch and version for hyperkube container.
Adding dns_domain configuration options.
Adding storage layer options.
Fixing underscore problem and adding exceptions.
Fixing the underscore flag errors.
Automatic merge from submit-queue
Make generated build tag parameterizable for go2idl
Downstream generators that want to reuse the upstream generated types
need to be able to define a different ignore tag (so that they can see
the already generated types).
@wojtek-t allows third parties to reuse conversions in api/v1 (for instance) because otherwise they are hidden when go2idl parses the dependency packages.
Downstream generators that want to reuse the upstream generated types
need to be able to define a different ignore tag (so that they can see
the already generated types).
Automatic merge from submit-queue
Update "kubectl get all" to display resource type as part of name
fixes#23838
release-note-none
When running "kubectl get all", or printing any output with mixed resource kinds, an additional column is added to the output with each resource's kind:
`kubectl get all --all-namespaces`
```
NAMESPACE NAME DESIRED CURRENT AGE
default rc/docker-registry-1 1 1 23h
testproject rc/node-1 0 0 2d
NAMESPACE NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default svc/docker-registry 172.30.36.42 <none> 5000/TCP 23h
default svc/kubernetes 172.30.0.1 <none> 443/TCP,53/UDP,53/TCP 7d
testproject svc/ruby-ex 172.30.187.128 <none> 8080/TCP 6d
NAMESPACE NAME READY STATUS RESTARTS AGE
default po/docker-registry-1-cpf8o 1/1 Running 1 23h
```
[]()
Automatic merge from submit-queue
kubectl: don't display an empty list when trying to get a single resource that isn't found
Return immediately when attempting to get a singular resource that isn't found, so that we avoid
printing out a List if the output format is something like json or yaml.
Before:
```
$ kubectl get pod/foo -o yaml
apiVersion: v1
items: []
kind: List
metadata: {}
pods "foo" not found
```
After:
```
$ kubectl get pod/foo -o yaml
pods "foo" not found
```
Fixes#28243
@kubernetes/kubectl @kubernetes/rh-ux @smarterclayton @liggitt @deads2k @metral
Automatic merge from submit-queue
Allow specifying secret data using strings
This PR allows specifying non-binary data values in `Secret` objects as `"stringData":{"key":"string value"}`, in addition to the existing base64 []byte serializations in the `data` field.
On write, the keys and values in the `stringData` field are merged to the `data` map, overwriting any values already present in the `data` map. The move is one-way, the `stringData` field is never output when reading from the API.
A Secret could be created like this:
```
{
"kind":"Secret",
"apiVersion":"v1",
"metadata":{"name":"mysecret"},
"data":{
"image":"<base64-encoded-jpg>"
},
"stringData":{
"username": "myuser",
"password": "mypassword"
}
}
```
and when read from the API would look like this:
```
{
"kind":"Secret",
"apiVersion":"v1",
"metadata":{"name":"mysecret",...},
"data":{
"image":"<base64-encoded-jpg>"
"username": "bXl1c2Vy",
"password": "bXlwYXNzd29yZA=="
}
}
```
Automatic merge from submit-queue
Install pip in kubekins test image and install AWS cli when needed
Follow-on to https://github.com/kubernetes/test-infra/pull/66. This fixes AWS hopefully.
I haven't pushed the image yet.
cc @fejta
Automatic merge from submit-queue
invalid arg during applying SELinux label
Tested with Ubuntu 16.04 with SELINUX support. Actually, after `chcon `with long label got: `chcon: failed to set type security context component to ‘system_u:object_r:svirt_sandbox_file_t:s0’: Invalid argument`
I think we meant to imply that "if the error isn't real" then update the
exceptions file- which to me means "false negative", ie. false failure.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Return immediately when attempting to get a singular resource that isn't found, so that we avoid
printing out a List if the output format is something like json or yaml.
Before:
```
$ kubectl get pod/foo -o yaml
apiVersion: v1
items: []
kind: List
metadata: {}
pods "foo" not found
```
After:
```
$ kubectl get pod/foo -o yaml
pods "foo" not found
```
Automatic merge from submit-queue
Ignore cmd/libs/go2idl/generator when running coverage
Temporary fix so we can run coverage (with a PASS).
fixes#24967
Automatic merge from submit-queue
Substitute federation_domain_map parameter with its value in node bootstrap scripts.
This PR also removes the substitution code we added to the build scripts.
**Release Note**
```release-note
If you use one of the kube-dns replication controller manifest in `cluster/saltbase/salt/kube-dns`, i.e. `cluster/saltbase/salt/kube-dns/{skydns-rc.yaml.base,skydns-rc.yaml.in}`, either substitute one of `__PILLAR__FEDERATIONS__DOMAIN__MAP__` or `{{ pillar['federations_domain_map'] }}` with the corresponding federation name to domain name value or remove them if you do not support cluster federation at this time. If you plan to substitute the parameter with its value, here is an example for `{{ pillar['federations_domain_map'] }`
pillar['federations_domain_map'] = "- --federations=myfederation=federation.test"
where `myfederation` is the name of the federation and `federation.test` is the domain name registered for the federation.
```
cc @erictune @kubernetes/sig-cluster-federation @MikeSpreitzer @luxas
[]()
Automatic merge from submit-queue
Include static docker binary in kubekins-test image
Fixes kubernetes/test-infra/issues/47.
I haven't pushed this image yet, so I expect CI to fail for now.
Automatic merge from submit-queue
federation: Upgrading the groupversion to v1beta1
This PR contains 2 commits:
* Removing fields from Cluster API object that we are not using. This includes: Capacity, Allocatable and ClusterMeta.
* Move code and rename groupversion `federation/v1alpha1` to `federation/v1beta1`
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
TLS bootstrap API group (alpha)
This PR only covers the new types and related client/storage code- the vast majority of the line count is codegen. The implementation differs slightly from the current proposal document based on discussions in design thread (#20439). The controller logic and kubelet support mentioned in the proposal are forthcoming in separate requests.
I submit that #18762 ("Creating a new API group is really hard") is, if anything, understating it. I've tried to structure the commits to illustrate the process.
@mikedanese @erictune @smarterclayton @deads2k
```release-note-experimental
An alpha implementation of the the TLS bootstrap API described in docs/proposals/kubelet-tls-bootstrap.md.
```
[]()
Automatic merge from submit-queue
e2e: Allow skipping tests for specific runtimes, skip a few tests under rkt
The main benefit of this is that it gives a developer more useful output (more signal to noise) for things that are known broken on that runtime.
cc @kubernetes/rktnetes-maintainers , @ixdy
I'll run this PR through our jenkins and make sure things look happy and compare to the e2e results for this PR.
Automatic merge from submit-queue
Set network related flags for container gen-swagger-docs
When running `hack/update-api-reference-docs.sh`, I get failure messages as below. This is because:
My env is behind a proxy, I have to input username/password in a page first for authentication before visiting internet. But I have no any way to visit that page in GUI mode in the container gen-swagger-docs. Can we run the container gen-swagger-docs using host network and passing an env variable `https_proxy` to it? I try this way and it works well. This would be helpful for me and other developers who may have this issue.
cc @caesarxuchao Thanks!
```
root@vm:/home/paas/zxp/code/k8s/fork/kubernetes# hack/update-api-reference-docs.sh
Note: This assumes that swagger spec has been updated. Please run hack/update-swagger-spec.sh to ensure that.
Generating api reference docs at /home/paas/zxp/code/k8s/fork/kubernetes/_output/generated_html
Reading swagger spec from: /home/paas/zxp/code/k8s/fork/kubernetes/api/swagger-spec/
--2016-06-09 16:35:26-- https://raw.githubusercontent.com/kubernetes/kubernetes/master/pkg/api/v1/register.go
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Name or service not known.
wget: unable to resolve host address ‘raw.githubusercontent.com’
--2016-06-09 16:36:06-- https://raw.githubusercontent.com/kubernetes/kubernetes/master/pkg/api/v1/register.go
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Name or service not known.
wget: unable to resolve host address ‘raw.githubusercontent.com’
--2016-06-09 16:36:46-- https://raw.githubusercontent.com/kubernetes/kubernetes/master/pkg/api/v1/register.go
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Name or service not known.
wget: unable to resolve host address ‘raw.githubusercontent.com’
!!! Error in hack/update-api-reference-docs.sh:71
'docker run ${user_flags} --rm -v "${TMP_IN_HOST}":/output:z -v "${SWAGGER_PATH}":/swagger-source:z gcr.io/google_containers/gen-swagger-docs:v5 "${SWAGGER_JSON_NAME}" "${REGISTER_FILE_URL}"' exited with status 1
Call stack:
1: hack/update-api-reference-docs.sh:71 main(...)
Exiting with status 1
```
Automatic merge from submit-queue
let patch use --local flag like `kubectl set image`
Adds the concept of a `--local` flag to `kubectl patch`. This flag is similar to `kubectl set image -f --local` because it will use the content of the file as the input to the patch operation instead of using the file content to file resource/name tuples.
This pull lets you run something like `kubectl create deployment --dry-run -o yaml | kubectl set volume --local -f - -o yaml | kubectl patch --local -f - --patch {} | kubectl create -f -`
As proof that it works, you can run against a local file just to mess around with it, but `--local -f -` is the most likely case.
```
$kubectl patch --local -f pkg/api/validation/testdata/v1/validPod.yaml --patch='{"spec": {"restartPolicy":"Never"}}'
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
name: redis-master
name: name
spec:
containers:
- args:
- this
- is
- an
- ok
- command
image: gcr.io/fake_project/fake_image:fake_tag
name: master
resources: {}
restartPolicy: Never
status: {}
```
This is useful for setting rarely used, but immutable fields from `kubectl create` or `kubectl convert` without dropping to an interactive editor.
Some discussion here: https://github.com/kubernetes/kubernetes/issues/21648#issuecomment-218579977
@smarterclayton @kubernetes/kubectl
@eparis @soltysh @stevekuznetsov we've talked about this separately
Automatic merge from submit-queue
Make local-up-cluster.sh IPv6 friendly
Added a new environment variable `API_HOST_IP`. `API_HOST_IP` allows the
user to specify an IPv6 address that is parsable by Golang. `API_HOST` on
the other hand allows the user to specify the IPv6 address to be used in
a URL's format as described in RFC2732.
Example:
```bash
API_HOST_IP="FEDC:BA98:7654:3210:FEDC:BA98:7654:3210"
API_HOST="[FEDC:BA98:7654:3210:FEDC:BA98:7654:3210]
```
or
```bash
API_HOST_IP="FEDC:BA98:7654:3210:FEDC:BA98:7654:3210"
API_HOST="[${API_HOST_IP}]"
```
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
Add -g curl option to hack/lib/util.sh wait_for_url
Add `-g` curl option
```
This option switches off the "URL globbing parser". When you set this option,
you can specify URLs that contain the letters {}[] without having them being
interpreted by curl itself. Note that these letters are not normal legal URL
contents but they should be encoded according to the URI standard.
```
> Why?
So that IPv6 addresses with `[` and `]`, like `[::1]:8080`, work with that script.
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
Check for an empty value in validateField
```release-note
* Fix a panic when args was not supplied with any values.
```
reflect.TypeOf() can take a nil (it then returns a nil), but
Kind() panics on a nil.
Now the user gets the following output:
./kubectl.sh --server=http://localhost:8080 create -f ../../test-files/test-rc.yaml
error validating "../../test-files/test-rc.yaml": error validating data: unexpected nil value for field spec.template.spec.containers[0].args[0]; if you choose to ignore these errors, turn validation off with --validate=false
fixes#20627 and fixes#26927
Automatic merge from submit-queue
Proportionally scale paused and rolling deployments
Enable paused and rolling deployments to be proportionally scaled.
Also have cleanup policy work for paused deployments.
Fixes#20853Fixes#20966Fixes#20754
@bgrant0607 @janetkuo @ironcladlou @nikhiljindal
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/20273)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Build Darwin test binaries on OS X when KUBE_FASTBUILD is set
Without this, on OS X, test binaries would not be built for the local ARCH, and test scripts fail with an obscure error due to not being able to find the correct binary to run.
This makes the behavior on OS X more consistent with Linux.
Tested on OS X and Linux.
Automatic merge from submit-queue
e2e-runner: Improve the logic of detecting Docker releases
Apparently it didn't handle GitHub prereleases well. Also switch from
`JENKINS_GCI_IMAGE_TYPE` to `JENKINS_GCI_IMAGE_FAMILY` following up #27083.
@dchen1107 @fejta Can you review?
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
e2e-runner: Get rid of the uses of JENKINS_GCI_IMAGE_TYPE
It's now `JENKINS_GCI_IMAGE_FAMILY`.
@fejta This is a quick bugfix which should have been done in #27083 . Can you review?
cc/ @kubernetes/goog-image The GCI e2e jobs are failing due to this.
Automatic merge from submit-queue
Allow conformance tests to run on non-GCE providers
fixes https://github.com/kubernetes/kubernetes/issues/26869
Creates a skeleton provider which has all the required function stubs -- but will allow a previously set "skeleton" KUBERNETES_PROVIDER to not be overriden with "gce".
Automatic merge from submit-queue
Get GCI images from image family
The staging images are now created with image families, so we can get rid of the
image indices stored in GCS. Also, update the GCS bucket used for staging
images.
@Amey-D @spxtr Can you review?
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
Fix hack/update-api-reference-docs on master and (future) branches
- update toplevel api object pattern in swagger doc script: the format of the `register.go` file was changed in e3af3451c8. This fixes the bleeding.
- make `hack/update-api-reference-docs.sh` independent from master during **container run**. Container build is still dependent on master, but container rebuilds never happen I guess on a branch.
A rebuild of `gcr.io/google_containers/gen-swagger-docs:v6` is necessary, plus a cherry-pick onto the v1.3 branch.
Fixes the symptoms of #27685 and makes sure future branches do not break again.
The staging images are now created with image families, so we can get rid of the
image indices stored in GCS. Also, get images based on milestone number instead
of "image type".
Automatic merge from submit-queue
Enable all ppc64le builds, except for hyperkube
Partially fixes: #25886
Talked to @Pensu, and all other binaries seem to work fine
@david-mcmahon @ixdy @Pensu @smarterclayton
Automatic merge from submit-queue
Prep for continuous Docker validation test
```release-note
Add a test config variable to specify desired Docker version to run on GCI.
```
We want to continuously validate Docker releases (#25215), on GCI. This change
adds a new test config variable, `KUBE_GCI_DOCKER_VERSION`, through which we can
specify which version of Docker we want to run on the master and nodes. This
change also patches the Jenkins e2e-runner with the ability to fetch the latest
Docker (pre)release, and sets the aforementioned variable accordingly.
Tested on my local Jenkins instance that was able to start a cluster with the latest Docker version (different from installed version) running on both master and nodes.
@dchen1107 Can you review?
cc/ @andyzheng0831 for changes in `cluster/gce/gci/helper.sh`, and @ixdy @spxtr for changes to the Jenkins e2e-runner
cc/ @kubernetes/goog-image
Automatic merge from submit-queue
Revert "Revert "GCI: add support for network plugin""
PR #27027 added the network plugin support in GCI config, but later a bug in the network plugin broke e2e tests (see issue #27118). The bug was fixed by #27141 and we have been repeatedly run the serial e2e tests more than 10 times to verify the fix. Now it should be safe to put the GCI network plugin support back.
We will first merge in the master branch and monitor the Jenkins serial tests for a while and then cherry-pick it into release-1.3 branch.
Automatic merge from submit-queue
Avoid duplicate building in Jenkins unit/integration job
Partially adopts #26392: don't run `hack/build-go.sh` in the unit/integration job, since we do that already for e2e.
We do need to still build kubectl, however, so do that in `hack/test-cmd.sh`.
x-ref #25940
Automatic merge from submit-queue
Fixes and improvements to Photon Controller backend for kube-up
- Improve reliability of network address detection by using MAC
address. VMware has a MAC OUI that reliably distinguishes the VM's
NICs from the other NICs (like the CBR). This doesn't rely on the
unreliable reporting of the portgroup.
- Persist route changes. We configure routes on the master and nodes,
but previously we didn't persist them so they didn't last across
reboots. This persists them in /etc/network/interfaces
- Fix regression that didn't configure auth for kube-apiserver with
Photon Controller.
- Reliably run apt-get update: Not doing this can cause apt to fail.
- Remove unused nginx config in salt
- Improve reliability of network address detection by using MAC
address. VMware has a MAC OUI that reliably distinguishes the VM's
NICs from the other NICs (like the CBR). This doesn't rely on the
unreliable reporting of the portgroup.
- Persist route changes. We configure routes on the master and nodes,
but previously we didn't persist them so they didn't last across
reboots. This persists them in /etc/network/interfaces
- Fix regression that didn't configure auth for kube-apiserver with
Photon Controller.
- Reliably run apt-get update: Not doing this can cause apt to fail.
- Remove unused nginx config in salt
Automatic merge from submit-queue
federation: fix dns provider initialization issues
This PR is based on the integration test with Google DNS API. This is the first time of full integration test.
So multiple issues was found and I combined all of them in this single PR
1. add dns provider initialization and add ensureDns call when removing federation service.
2. add new flags federation-name and zone-name to controller manager, both are used as part of the dns record name
3. fix assertion failure at rrsets.go#L61, which will cause panic
4. change getFederationDNSZoneName to get zoneName from config instead of hard code
5. change logic of ensureDnsRrsets, only add new dns record when endpointReachable(set to true when ready address is catched) is true
6. fix bug in processEndpointUpdate, only call ensuredns when ready address is caught
7. change behavior of syncService, there is cases that endpoint is created before ingress IP assignment, so before there is defect for this case, ensureDns was not called when service being updated, so if Ingress IP is assigned after endpoint ready address is caught, the corresponding A records can not be created
8. add a checking before update federation service
@nikhiljindal , can you help to add 1.3 milestone when @quinton-hoole is on leave?
Thanks.
[]()
Added a new environment variable API_HOST_IP. API_HOST_IP allows the
user to specify an IPv6 address that is parsable by Golang. API_HOST on
the other hand allows the user to specify the IPv6 address to be used in
a URL's format as described in RFC2732.
Example:
API_HOST_IP="FEDC:BA98:7654:3210:FEDC:BA98:7654:3210"
API_HOST="[FEDC:BA98:7654:3210:FEDC:BA98:7654:3210]
or
API_HOST_IP="FEDC:BA98:7654:3210:FEDC:BA98:7654:3210"
API_HOST="[${API_HOST_IP}]"
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
fix recursive & non-recursive kubectl get of generic output format
This PR fixes the issues with `kubectl get` in https://github.com/kubernetes/kubernetes/issues/26466
Changes made:
- fix printing when using the generic output format in both non-recursive & recurvise settings to ensure that errors are being shown
- add tests to check printing generic output in a **non-recursive** setting with non-existent pods
- clean up the **recursive** `kubectl get` tests
/cc @janetkuo
Automatic merge from submit-queue
Stop checking in generated documents
Part of #26205
Auto generated docs are **NO LONGER CHECKED IN**, only placeholders.
To generate them, e.g. before exporting docs, run hack/generate-docs.sh.
hack/verify-generated-docs.sh ensures that generated docs are merely the
placeholder text.
hack/update-generated-docs.sh puts the placeholder text in the proper
places.
The old munge behavior is moved into hack/{update|verify}-munge-docs.sh.
Pull the latest build of the release the server is running, rather
than matching exact version. This allows GKE to pick up test fixes
from branch head, instead of waiting for a patch.
Automatic merge from submit-queue
Add a custom main instead of the standard test main, to reduce stack …
Adds a custom test main handler (see: `TestMain` in https://golang.org/pkg/testing/ for details)
Partial fix for https://github.com/kubernetes/kubernetes/issues/25965
This does the standard timeout, but strips non-kubernetes stacks out of the stack trace (e.g. it filters things like:
```
goroutine 466 [IO wait, 7 minutes]:
net.runtime_pollWait(0x7fd74c4672c0, 0x72, 0xc821614000)
/usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc8215c21b0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8215c21b0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8215c2150, 0xc821614000, 0x1000, 0x1000, 0x0, 0x7fd74c491050, 0xc820014058)
/usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc820a5a090, 0xc821614000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7fd74c465258, 0xc820a5a090, 0xc8215f0068, 0xc821614000, 0x1000, 0x1000, 0x405773, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1687 +0x67
net/http.(*noteEOFReader).Read(0xc8215ae1a0, 0xc821614000, 0x1000, 0x1000, 0xc82159ad1d, 0x0, 0x0)
<autogenerated>:284 +0xd0
bufio.(*Reader).fill(0xc8202a2b40)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202a2b40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8215f0000)
/usr/local/go/src/net/http/transport.go:1073 +0x177
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:857 +0x10a6
```
We may want to get even more aggressive in the future.
@kubernetes/sig-testing
Automatic merge from submit-queue
GCI: add support for network plugin
I had run e2e against a cluster with both master and nodes on GCI a couple of times. The PR auto tests will cover the hybrid cluster with just master on GCI.
cc/ @roberthbailey @fabioy @kubernetes/goog-image
Auto generated docs are **NO LONGER CHECKED IN**, only placeholders.
To generate them, e.g. before exporting docs, run hack/generate-docs.sh.
hack/verify-generated-docs.sh ensures that generated docs are merely the
placeholder text.
hack/update-generated-docs.sh puts the placeholder text in the proper
places.
The old munge behavior is moved into hack/{update|verify}-munge-docs.sh.
Also includes other improvements:
- Makefile rule to run tests against remote instance using existing host or image
- Makefile will reuse an instance created from an image if it was not torn down
- Runner starts gce instances in parallel with building source
- Runner uses instance ip instead of hostname so that it doesn't need to resolve
- Runner supports cleaning up files and processes on an instance without stopping / deleting it
- Runner runs tests using `ginkgo` binary to support running tests in parallel
Automatic merge from submit-queue
Enable support for memory eviction configuration via salt
Added evictions based on memory by default whenever the available memory is < 100Mi.
Updated GCE and GCI.
Automatic merge from submit-queue
Always skew the client when running version-skew tests
As mentioned in the comments:
- for upgrade jobs, we want kubectl to be at the same version as master.
- for client skew tests, we want to use the skewed kubectl (that's what we're testing).
So, remove gate on `JENKINS_USE_SKEW_KUBECTL`, and always use the skewed `kubectl`.
This should go in before https://github.com/kubernetes/test-infra/pull/109.
We want to continuously validate Docker releases (#25215), on GCI. This change
adds a new test config variable, `KUBE_GCI_DOCKER_VERSION`, through which we can
specify which version of Docker we want to run on the master and nodes. This
change also patches the Jenkins e2e-runner with the ability to fetch the latest
Docker (pre)release, and sets the aforementioned variable accordingly.
Automatic merge from submit-queue
Don't allow deps with no discernible license
This updates the few deps we had with no LICENSE file to current versions that do have that file. It also disallows new deps without obvious licenses.
Automatic merge from submit-queue
Attach/Detach Controller Kubelet Changes
This PR contains changes to enable attach/detach controller proposed in #20262.
Specifically it:
* Introduces a new `enable-controller-attach-detach` kubelet flag to enable control by attach/detach controller. Default enabled.
* Removes all references `SafeToDetach` annotation from controller.
* Adds the new `VolumesInUse` field to the Node Status API object.
* Modifies the controller to use `VolumesInUse` instead of `SafeToDetach` annotation to gate detachment.
* Modifies kubelet to set `VolumesInUse` before Mount and after Unmount.
* There is a bug in the `node-problem-detector` binary that causes `VolumesInUse` to get reset to nil every 30 seconds. Issue https://github.com/kubernetes/node-problem-detector/issues/9#issuecomment-221770924 opened to fix that.
* There is a bug here in the mount/unmount code that prevents resetting `VolumeInUse in some cases, this will be fixed by mount/unmount refactor.
* Have controller process detaches before attaches so that volumes referenced by pods that are rescheduled to a different node are detached first.
* Fix misc bugs in controller.
* Modify GCE attacher to: remove retries, remove mutex, and not fail if volume is already attached or already detached.
Fixes#14642, #19953
```release-note
Kubernetes v1.3 introduces a new Attach/Detach Controller. This controller manages attaching and detaching volumes on-behalf of nodes that have the "volumes.kubernetes.io/controller-managed-attach-detach" annotation.
A kubelet flag, "enable-controller-attach-detach" (default true), controls whether a node sets the "controller-managed-attach-detach" or not.
```
Automatic merge from submit-queue
Fixes#26526 - hack/update-api-reference-docs.sh
I opened a Pull request to fix this issue https://github.com/kubernetes/kubernetes/issues/26526
The problem is that the update script ignores white spaces but the verify script doesn't which leads to a strange behaviour -> you use the update script but the verify script tells you that the api docs are not up to date.
This PR contains Kubelet changes to enable attach/detach controller control.
* It introduces a new "enable-controller-attach-detach" kubelet flag to
enable control by controller. Default enabled.
* It removes all references "SafeToDetach" annoation from controller.
* It adds the new VolumesInUse field to the Node Status API object.
* It modifies the controller to use VolumesInUse instead of SafeToDetach
annotation to gate detachment.
* There is a bug in node-problem-detector that causes VolumesInUse to
get reset every 30 seconds. Issue https://github.com/kubernetes/node-problem-detector/issues/9
opened to fix that.
Add it as a special case package root and import the license file. This was
the only UNKNOWN license, prior to the change to not look at upstream repo
state.
This change recovers some of the side effects of
https://github.com/kubernetes/kubernetes/pull/26197, i.e., keeps the defaults of
`NODE_IMAGE` and `NODE_IMAGE_PROJECT` to `MASTER_IMAGE` and
`MASTER_IMAGE_PROJECT`, for backward compatibility. Although it keeps
`OS_DISTRIBUTION` defaulting to `gci`, the default settings of these vars are
moved to `cluster/gce/util.sh` and conditioned on `OS_DISTRIBUTION==gci`.
Automatic merge from submit-queue
Don't set the env var CC when not cross-compiling
I noticed that this script was trying to use `arm-linux-gnueabi-gcc` also when running natively on arm.
When running natively, `CC` should always be `gcc` (which also is the default)
Also added `federation-controller-manager` to the static list, I think someone forgot to do that.
@ixdy @david-mcmahon @spiffxp @spxtr
Automatic merge from submit-queue
Get install-etcd.sh to work on OSX
Check the platform before we download the etcd binary. On OSX, save
the zip file, unzip it and link the directory with the correct
name.
Fixes#26041
Automatic merge from submit-queue
Add -addr option to hack/lib/etcd.h
Added -addr to etcd hack script so it listens on the same address specified in the `host` variable.
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
Support per-test-environment ginkgo flags for node e2e tests to facilitate skipping miss behaving tests in PR builder
We had an issue today where some node e2e tests were timing out in the pr builder. We want to be able to skip tests in the pr builder and leave them running in the CI if this happens again.
[]()
Automatic merge from submit-queue
Adds --network-plugin-dir argument to hack/local-up-cluster.sh
CNI Network Plugin developers who want to use hack/local-up-cluster.sh need to specify the --network-plugin-dir option to kubelet.
Automatic merge from submit-queue
Add more global options to local-up-cluster.sh
I added a couple more global variables to the `local-up-cluster.sh` script.
The `KUBELET_HOST` environment name make sense to me. If it isn't the best name I will change it.
The `ETCD` ones make sense since they are being used under `hack/lib/etcd.sh`
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
kube-controller-manager: Add configure-cloud-routes option
This allows kube-controller-manager to allocate CIDRs to nodes (with
allocate-node-cidrs=true), but will not try to configure them on the
cloud provider, even if the cloud provider supports Routes.
The default is configure-cloud-routes=true, and it will only try to
configure routes if allocate-node-cidrs is also configured, so the
default behaviour is unchanged.
This is useful because on AWS the cloud provider configures routes by
setting up VPC routing table entries, but there is a limit of 50
entries. So setting configure-cloud-routes on AWS would allow us to
continue to allocate node CIDRs as today, but replace the VPC
route-table mechanism with something not limited to 50 nodes.
We can't just turn off the cloud-provider entirely because it also
controls other things - node discovery, load balancer creation etc.
Fix#25602
Automatic merge from submit-queue
Push public key to project if not already present
Fixes https://github.com/kubernetes/kubernetes/issues/26128
Fixes https://github.com/kubernetes/kubernetes/issues/26129
Whenever the ssh key changes (which happens right now whenever we add/change a new jenkins agent) this key will not get pushed to the project until either `gcloud compute ssh` or `gcloud compute config-ssh` runs. As a result instances on this project will reject ssh attempts with this key.
In many cases this will not happen until after a test on a project fails and we attempt to ssh to the nodes to grab logs.
This verifies the presence of the ssh key before starting tests, and attempts to add it if it is missing.
Jenkins relies on junit.xml to identify test failures
and non-0 exit codes to indentify infrastructure failures.
Test failures in kubemark tests should not cause the test
script to exit non-0. Infrastructure failures should.
- Add function to dump cluster logs without exiting (refactor)
- Change `test/kubemark/stop-kubemark.sh` to be run regardless of whether tests fail or not
- Exit code for failed tests overwritten to be the exit code of dumping the cluster logs
This code used to actually reach out to the internet to look for files. This
is flaky, slow, and semantically WRONG. The license that is upstream might
actually be different than what we have vendored. Only look at local files.
This now passes back-to-back updates and verifies.
This allows kube-controller-manager to allocate CIDRs to nodes (with
allocate-node-cidrs=true), but will not try to configure them on the
cloud provider, even if the cloud provider supports Routes.
The default is configure-cloud-routes=true, and it will only try to
configure routes if allocate-node-cidrs is also configured, so the
default behaviour is unchanged.
This is useful because on AWS the cloud provider configures routes by
setting up VPC routing table entries, but there is a limit of 50
entries. So setting configure-cloud-routes on AWS would allow us to
continue to allocate node CIDRs as today, but replace the VPC
route-table mechanism with something not limited to 50 nodes.
We can't just turn off the cloud-provider entirely because it also
controls other things - node discovery, load balancer creation etc.
Fix#25602
Automatic merge from submit-queue
Add release_1_3 clientset in update-codegen
Add release_1_3 clientset in update-codegen to keep it update-to-date; update the generated clientset.
Automatic merge from submit-queue
Attach Detach Controller Business Logic
This PR adds the meat of the attach/detach controller proposed in #20262.
The PR splits the in-memory cache into a desired and actual state of the world.
Automatic merge from submit-queue
use kubectl-with-retry on pause & resume
attempts to fix https://github.com/kubernetes/kubernetes/issues/25645 by using `kubectl-with-retry` on `rollout {pause,resume}` (`resume` is for safe measures) instead of `kubectl` directly, as is done with other `rollout {pause,resume}` tests in this same script.
Automatic merge from submit-queue
e2e-runner: Drop Trusty support in favor of GCI
Now that GCI (Google Container-VM Image) is out, we will start running e2e tests
with it instead of Ubuntu Trusty. This change updates `e2e-runner.sh` to replace
the Trusty related logic with GCI.
_Note that this change has to go in the same time as https://github.com/kubernetes/test-infra/pull/54_
@spxtr Can you review?
cc/ @andyzheng0831 @kubernetes/goog-image
Split controller cache into actual and desired state of world.
Controller will only operate on volumes scheduled to nodes that
have the "volumes.kubernetes.io/controller-managed-attach" annotation.
- Add junit test reported
- Write etcd.log, kubelet.log and kube-apiserver.log to files instead of stdout
- Scp artifacts to the jenkins WORKSPACE
Fixes#25966
Automatic merge from submit-queue
Renaming federated-apiserver to federation-apiserver
Its not a "federated" apiserver. Its an apiserver for federation.
The corresponding controller manager is also called federation-controller-manager.
If we are going to rename, it should be done in 1.3 before people write scripts referencing the binary name.
Automatic merge from submit-queue
Add a 'kubectl clusterinfo dump' option
Ref: #3500
@bgrant0607 @smarterclayton @jszczepkowski
Usage:
```
# Dump current cluster state to stdout
kubectl clusterinfo dump
# Dump current cluster state to /tmp
kubectl clusterinfo dump --output-directory=/tmp
# Dump all namespaces to stdout
kubectl clusterinfo dump --all-namespaces
# Dump a set of namespaces to /tmp
kubectl clusterinfo dump --namespaces default,kube-system --output-directory=/tmp
```
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/20672)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Improve the speed of do-nothing build.
As @thockin found out here https://github.com/kubernetes/kubernetes/issues/24518, vast majority of the do-nothing build time is spent in rebuilding the test binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them while building packages.
On my workstation, do-nothing hack/build-go.sh time goes from ~20 secs to ~4 secs, of which only ~1 sec is from doing test binary staleness check now (as opposed to ~17 secs it took to build the test binaries before). I did some experiments to bring this time down to <1 sec. I measured using go test -bench, but it was not very useful in this case. I believe, a vast majority of that ~1 second is being spent in fork/exec and piping the results back to the staleness check program along with the ser-deser involved, but it needs to be validated. Not a proof, but to provide some supporting evidence to this claim, running `go list -f format packages` in the shell takes about 600ms irrespective of what's in the format.
Tests are TBD. I am still trying to figure out how to test this, but I would like to get early feedback
cc @mikedanese @mml
Automatic merge from submit-queue
Cache Webhook Authentication responses
Add a simple LRU cache w/ 2 minute TTL to the webhook authenticator.
Kubectl is a little spammy, w/ >= 4 API requests per command. This also prevents a single unauthenticated user from being able to DOS the remote authenticator.
Automatic merge from submit-queue
Updaing QoS policy to be at the pod level
Quality of Service will be derived from an entire Pod Spec, instead of being derived from resource specifications of individual resources per-container.
A Pod is `Guaranteed` iff all its containers have limits == requests for all the first-class resources (cpu, memory as of now).
A Pod is `BestEffort` iff requests & limits are not specified for any resource across all containers.
A Pod is `Burstable` otherwise.
Note: Existing pods might be more susceptible to OOM Kills on the node due to this PR! To protect pods from being OOM killed on the node, set `limits` for all resources across all containers in a pod.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/14943)
<!-- Reviewable:end -->
Automatic merge from submit-queue
add CIDR allocator for NodeController
This PR:
* use pkg/controller/framework to watch nodes and reduce lists when allocate CIDR for node
* decouple the cidr allocation logic from monitoring status logic
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/19242)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Add 'kubectl set image'
```release-note
Add "kubectl set image" for easier updating container images (for pods or resources with pod templates).
```
**Usage:**
```
kubectl set image (-f FILENAME | TYPE NAME) CONTAINER_NAME_1=CONTAINER_IMAGE_1 ... CONTAINER_NAME_N=CONTAINER_IMAGE_N
```
**Example:**
```console
# Set a deployment's nginx container image to 'nginx:1.9.1', and its busybox container image to 'busybox'.
$ kubectl set image deployment/nginx busybox=busybox nginx=nginx:1.9.1
# Update all deployments' nginx container's image to 'nginx:1.9.1'
$ kubectl set image deployments nginx=nginx:1.9.1 --all
# Update image of all containers of daemonset abc to 'nginx:1.9.1'
$ kubectl set image daemonset abc *=nginx:1.9.1
# Print result (in yaml format) of updating nginx container image from local file, without hitting the server
$ kubectl set image -f path/to/file.yaml nginx=nginx:1.9.1 --local -o yaml
```
I abandoned the `--container=xxx --image=xxx` flags in the [deploy proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/deploy.md#kubectl-set) since it's much easier to use with just KEY=VALUE (CONTAINER_NAME=CONTAINER_IMAGE) pairs.
Ref #21648
@kubernetes/kubectl @bgrant0607 @kubernetes/sig-config
[]()
Automatic merge from submit-queue
Fix JENKINS_USE_SKEW_KUBECTL
I got this logic wrong; the first is a NOT comparison, so the second should only be available if that NOT comparison returns true.
Automatic merge from submit-queue
Add a timeout to the node e2e Ginkgo test runner
Also add a few debugging statements to indicate progress.
Should help prevent #25639, since we'll timeout tests before Jenkins times out the build.
Automatic merge from submit-queue
Cassandra examples updating images to v9
- this is a version bump for the C* image
- I also increased the cpu to .5 because .1 is slow like warm death
Who can actually run the build to get the container to the examples repo?
Now that GCI (Google Container-VM Image) is out, we will start running e2e tests
with it instead of Ubuntu Trusty. This change updates `e2e-runner.sh` to replace
the Trusty related logic with GCI.
This patch adds the --exit-on-lock-contention flag, which must be used
in conjunction with the --lock-file flag. When provided, it causes the
kubelet to wait for inotify events for that lock file. When an 'open'
event is received, the kubelet will exit.
Automatic merge from submit-queue
Implement the cluster-controller of kubernetes federation
This PR is the cluster-controller of kubernetes federation.
This controller of federation phase 1 just collect the status of federated clusters.
Automatic merge from submit-queue
Remove call to setup_env from hack/e2e-node-test.sh
[]()
This messes with the GOPATH in a way that confuses ginkgo,
and doesn't appear to be necessary to run the tests
Removes likely unnecessary use of sudo, though it should be noted that
e2e_node_suite_test.go uses sudo to do something with locksmithd, so you
currently end up using it anyway.
Updates the error for not having ginkgo built to recommend the new vendor/
path for ginkgo, and to recommend using make instead of hack/build-go.sh
Should resolve#25682
This messes with the GOPATH in a way that confuses ginkgo,
and doesn't appear to be necessary to run the tests
Removes likely unnecessary use of sudo, though it should be noted that
e2e_node_suite_test.go uses sudo to do something with locksmithd, so you
currently end up using it anyway.
Updates the error for not having ginkgo built to recommend the new vendor/
path for ginkgo, and to recommend using make instead of hack/build-go.sh
Should resolve#25682
Automatic merge from submit-queue
Build hyperkube in hack/local-up-cluster instead of separate binaries
This speeds up the build time a lot.
Also add arm64 as a valid arch
This is quite trivial. Let's get it in asap
@thockin @pmorie @liggitt @eparis @zmerlynn @smarterclayton
Automatic merge from submit-queue
Use --format='value(name)' with gcloud instead of grep/awk/cut
Fixing our fragile parsing of `gcloud` is getting old (#24746, #25159, maybe others?).
Instead, let's just get the proper output out of `gcloud` in the first place.
Automatic merge from submit-queue
Support struct,array,slice types when sorting kubectl output
Fixes https://github.com/kubernetes/kubernetes/issues/24328.
Briefly, `sorting_printer` only take cares of the following type kinds:
* `reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64`
* `reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64`
* `reflect.Float32, reflect.Float64`
* `reflect.String`
* `reflect.Ptr`
This commit aims to add `reflect.Struct, reflect.Slice, reflect.Array`.
/cc @bgrant0607
Automatic merge from submit-queue
Adding Services to federation clientset
Commits:
1. Regenerate the client without any changes to client-gen
2. Update clientgen to add a parameter to specify generating client only for Services v1 object.
3. Regenerate federation_internalclientset
4. Regenerate federation_release_1_3
Second commit is the most important one. Other 3 commits are auto generated by running client-gen.
I have added a command line argument to client-gen that takes in a list of group/version/resource. If a group version is part of this list, then only the resources in this list are included in the client. For other group versions, the existing check of genclient=true in types.go is used.
Other alternatives considered were:
* Update genclient in types.go to mention the clientset name in which it should be included instead of just saying genclient=true (so Services will say genclient=core,federation while all other v1 resources will say genclient=core). This requires a code change in types.go to change a client set.
* Create another types.go which will only include Services and use that to generate federation clientset. This will lead to duplicate Service definition.
cc @caesarxuchao @lavalamp @jianhuiz @mfanjie @kubernetes/sig-cluster-federation
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/25443)
<!-- Reviewable:end -->
As thockin found out here https://github.com/kubernetes/kubernetes/issues/24518,
vast majority of the do-nothing build time is spent in rebuilding the test
binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them
while building packages.
Tests are TBD. I am still trying to figure out how to test this.
Automatic merge from submit-queue
The remaining API changes for PodDisruptionBudget.
It's mostly the boilerplate required for the registry, some extra codegen, and a few tests.
Will squash once we're sure it's good.
