Automatic merge from submit-queue
Separate federation build.sh into development and deployment scripts.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.
Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.
This change also copies deploy/* files into the release distribution.
cc @kubernetes/sig-cluster-federation @colhom
```release-note
Federation can now be deployed using the `federation/deploy/deploy.sh` script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current `federation-up.sh`/`federation-down.sh` scripts. Both the scripts are going to co-exist in this release, but the `federation-up.sh`/`federation-down.sh` scripts might be removed in a future release in favor of `federation/deploy/deploy.sh` script.
```
Automatic merge from submit-queue
Node Conformance Test: Refactor node e2e framework
For #30122, #30174.
Based on #30348.
**Please only review the last 3 commits.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30348, it removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.
The PR refactored the node e2e framework. Moving different utilities into different packages under `pkg/`.
We need to do this because:
1) Files like e2e_remote.go and e2e_build.go should only be used by runner, but they were compiled into the test suite because they were placed in the same package. The worst thing is that it will introduce some never used flags in the test suite binary.
2) Make the directory structure more clear. Only test should be placed in `test/e2e_node`, other utilities should be placed in different packages in `pkg/`.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Add kubelet --network-plugin-mtu flag for MTU selection
* Add network-plugin-mtu option which lets us pass down a MTU to a network provider (currently processed by kubenet)
* Add a test, and thus make sysctl testable
MTU selection is difficult, and if there is a transport such as IPSEC in
use may be impossible. So we allow specification of the MTU with the
network-plugin-mtu flag, and we pass this down into the network
provider.
Currently implemented by kubenet.
Automatic merge from submit-queue
Kubelet: add --container-runtime-endpoint and --image-service-endpoint
Flag `--container-runtime-endpoint` (overrides `--container-runtime`) is introduced to identify the unix socket file of the remote runtime service. And flag `--image-service-endpoint` is introduced to identify the unix socket file of the image service.
This PR is part of #28789 Milestone 0.
CC @yujuhong @Random-Liu
Automatic merge from submit-queue
Implement TLS bootstrap for kubelet using `--experimental-bootstrap-kubeconfig` (2nd take)
Ref kubernetes/features#43 (comment)
cc @gtank @philips @mikedanese @aaronlevy @liggitt @deads2k @errordeveloper @justinsb
Continue on the older PR https://github.com/kubernetes/kubernetes/pull/30094 as there are too many comments on that one and it's not loadable now.
New flag --container-runtime-endpoint (overrides --container-runtime)
is introduced to kubelet which identifies the unix socket file of
the remote runtime service. And new flag --image-service-endpoint is
introduced to kubelet which identifies the unix socket file of the
image service.
Automatic merge from submit-queue
Expose flags for new NodeEviction logic in NodeController
Fix#28832
Last PR from the NodeController NodeEviction logic series.
cc @davidopp @lavalamp @mml
Automatic merge from submit-queue
Let load and density e2e tests use GC if it's on
I've run the 100 and 500 nodes tests and they both pass.
The test-infra half of the PR is https://github.com/kubernetes/test-infra/pull/369
cc @lavalamp
Automatic merge from submit-queue
A build and deploy script to manage cluster federation lifecycle and a Makefile wrapper to drive that process.
Also includes a sample config file to describe clusters.
The build script implements the following things:
1. Generates the required configs.
2. Builds the hyperkube binary and the corresponding docker image.
3. Pushes the image to a specified repository.
4. Pulls the federation installer docker images.
5. Builds the Kubernetes clusters described the config.json file.
6. Pushes the federation components to one of the Kubernetes clusters
built in the previous step.
7. Also turns down the federation components and the Kubernetes
clusters.
**NOTE**: Installer images are right now being pulled from my public repository of docker images. I am working on pushing them to our release repository.
```release-note
Cluster Federation components can now be built and deployed using the make command. Please see federation/README.md for details.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes: Issue #26655
Automatic merge from submit-queue
[kubelet] Introduce --protect-kernel-defaults flag to make the tunable behaviour configurable
Let's make the default behaviour of kernel tuning configurable. The default behaviour is kept modify as has been so far.
Automatic merge from submit-queue
[Kubelet] Rename `--config` to `--pod-manifest-path`. `--config` is deprecated.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose. I didn't change the flag name because that
API should remain stable.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29999)
<!-- Reviewable:end -->
Also provide a new --pod-manifest-path flag and deprecate the old
--config one.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose.
Automatic merge from submit-queue
kubectl config set-crentials: add arguments for auth providers
This PR adds `--auth-provider` and `--auth-provider-arg` flags to the
`kubectl config set-credentials` sub-command.
There's currently no way of interacting with the new auth provider framework added in #23066 through kubectl. You have to render a custom kubeconfig to use them. Additionally `kubectl config set` just sort of craps out when attempting to interact with authentication info objects (#29312).
This is a minimal implementation of allowing `kubect config set-credentials` to set fields for client auth providers.
cc @cjcullen @kubernetes/kubectl
Automatic merge from submit-queue
E2E & Node E2E: Move configmap, docker_containers, downward_api, expansion and secrets test into common directory.
This is the 3rd part of #29494.
For #29081.
Based on #29092, #29806.
The first commit is squash of all dependent commits. Please only review the second commit.
The second PR added 17 lines.
@vishh @timstclair
Automatic merge from submit-queue
Cleanup k8s script noise with a verbosity concept
Fixes https://github.com/kubernetes/kubernetes/issues/30109
The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.
V=2 kube::log::info foo bar
Would only print "foo bar" if $KUBE_VERBOSE >= 2.
Examples:
Default verbosity (1 for make commands):
```
$ make kubelet
+++ [0804 17:23:32] Generating bindata:
/usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0804 17:23:37] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0804 17:23:37] Building go targets for linux/amd64:
cmd/kubelet
# k8s.io/kubernetes/pkg/kubelet
pkg/kubelet/kubelet.go:247: undefined: a
make: *** [kubelet] Error 1
```
Extra verbose (5, comparable to previous levels):
<details>
```
$ make kubelet KUBE_VERBOSE=5
I0804 17:31:05.083395 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/trace.go:151:30: cannot use (traceBufHeader literal) (value of type traceBufHeader) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.083503 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/trace.go:151:7: array length 64 << 10 - unsafe.Sizeof((traceBufHeader literal)) (value of type uintptr) must be constant
I0804 17:31:05.083600 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mgcwork.go:269:37: cannot use (workbufhdr literal) (value of type workbufhdr) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.083654 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mgcwork.go:269:7: array length (_WorkbufSize - unsafe.Sizeof((workbufhdr literal))) / sys.PtrSize (value of type uintptr) must be constant
I0804 17:31:05.084006 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:28: cannot use (hchan literal) (value of type hchan) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.084040 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:66: cannot use (hchan literal) (value of type hchan) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.084076 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/chan.go:21:14: unsafe.Sizeof((hchan literal)) + uintptr(-int(unsafe.Sizeof((hchan literal))) & (maxAlign - 1)) (value of type uintptr) is not constant
I0804 17:31:05.085536 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:80:31: cannot use (struct{b bmap; v int64} literal).v (value of type int64) as unsafe.ArbitraryType value in argument to unsafe.Offsetof
I0804 17:31:05.085567 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:80:15: unsafe.Offsetof((struct{b bmap; v int64} literal).v) (value of type uintptr) is not constant
I0804 17:31:05.085788 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/hashmap.go:1053:45: cannot convert &zeroinitial (value of type *[1024]byte) to unsafe.Pointer
I0804 17:31:05.086995 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mfinal.go:20:65: cannot use (finalizer literal) (value of type finalizer) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.087031 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mfinal.go:20:11: array length (_FinBlockSize - 2 * sys.PtrSize - 2 * 4) / unsafe.Sizeof((finalizer literal)) (value of type uintptr) must be constant
I0804 17:31:05.087957 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mstats.go:170:39: cannot use memstats.by_size (variable of type [67]struct{size uint32; nmalloc uint64; nfree uint64}) as unsafe.ArbitraryType value in argument to unsafe.Offsetof
I0804 17:31:05.087999 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/mstats.go:170:76: cannot use memstats.by_size[0] (variable of type struct{size uint32; nmalloc uint64; nfree uint64}) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.088483 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/panic.go:118:34: cannot use (_defer literal) (value of type _defer) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.088510 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/panic.go:118:20: unsafe.Sizeof((_defer literal)) (value of type uintptr) is not constant
I0804 17:31:05.089812 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/sema.go:42:42: cannot use (semaRoot literal) (value of type semaRoot) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.089845 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/runtime/sema.go:42:8: array length sys.CacheLineSize - unsafe.Sizeof((semaRoot literal)) (value of type uintptr) must be constant
I0804 17:31:05.094634 2601 parse.go:307] type checking encountered some errors in "runtime", but ignoring.
I0804 17:31:05.875185 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/davecgh/go-spew/spew/bypass.go:33:26: cannot use (*byte)(nil) (value of type *byte) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.875234 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/davecgh/go-spew/spew/bypass.go:33:12: unsafe.Sizeof((*byte)(nil)) (value of type uintptr) is not constant
I0804 17:31:05.875838 2601 parse.go:307] type checking encountered some errors in "github.com/davecgh/go-spew/spew", but ignoring.
I0804 17:31:05.897216 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/crypto/cipher/xor.go:12:36: cannot use uintptr(0) (constant 0 of type uintptr) as unsafe.ArbitraryType value in argument to unsafe.Sizeof
I0804 17:31:05.897261 2601 parse.go:353] type checker error: /usr/local/google/home/stclair/.gvm/gos/go1.6.2/src/crypto/cipher/xor.go:12:18: int(unsafe.Sizeof(uintptr(0))) (value of type int) is not constant
I0804 17:31:05.897360 2601 parse.go:307] type checking encountered some errors in "crypto/cipher", but ignoring.
I0804 17:31:06.400904 2601 conversion.go:227] considering pkg "k8s.io/kubernetes/federation/apis/core/v1"
I0804 17:31:06.401138 2601 conversion.go:243] tags: ["k8s.io/kubernetes/federation/apis/core"]
I0804 17:31:06.427408 2601 conversion.go:283] no viable conversions, not generating for this package
I0804 17:31:06.427508 2601 main.go:73] Completed successfully.
Go version: go version go1.6.2 linux/amd64
+++ [0804 17:31:06] Generating bindata:
/usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/test/e2e/framework/gobindata_util.go
Generated bindata file : 11536 ../../..//test/e2e/generated/bindata.go lines of lovely automated artifacts
+++ [0804 17:31:12] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0804 17:31:12] Building go targets for linux/amd64:
cmd/kubelet
# k8s.io/kubernetes/pkg/kubelet
pkg/kubelet/kubelet.go:247: undefined: a
!!! Error in /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:506
'go install "${goflags[@]:+${goflags[@]}}" -gcflags "${gogcflags}" -ldflags "${goldflags}" "${nonstatics[@]:+${nonstatics[@]}}"' exited with status 2
Call stack:
1: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:506 kube::golang::build_binaries_for_platform(...)
2: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:692 kube::golang::build_binaries(...)
3: hack/make-rules/build.sh:27 main(...)
Exiting with status 1
!!! Error in /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:596
'( kube::golang::setup_env; echo "Go version: $(go version)"; local host_platform; host_platform=$(kube::golang::host_platform); local goflags goldflags gogcflags; eval "goflags=(${KUBE_GOFLAGS:-})"; goldflags="${KUBE_GOLDFLAGS:-} $(kube::version::ldflags)"; gogcflags="${KUBE_GOGCFLAGS:-}"; local use_go_build; local -a targets=(); local arg; readonly BINDATAS=("${KUBE_ROOT}/test/e2e/framework/gobindata_util.go"); kube::log::status "Generating bindata:" "${BINDATAS[@]}"; for bindata in ${BINDATAS[@]};
do
if [[ -f $bindata ]]; then
go generate "${bindata}";
fi;
done; for arg in "$@";
do
if [[ "${arg}" == "--use_go_build" ]]; then
use_go_build=true;
else
if [[ "${arg}" == -* ]]; then
goflags+=("${arg}");
else
targets+=("${arg}");
fi;
fi;
done; if [[ ${#targets[@]} -eq 0 ]]; then
targets=("${KUBE_ALL_TARGETS[@]}");
fi; local -a platforms=(${KUBE_BUILD_PLATFORMS:-}); if [[ ${#platforms[@]} -eq 0 ]]; then
platforms=("${host_platform}");
fi; local binaries; binaries=($(kube::golang::binaries_from_targets "${targets[@]}")); local parallel=false; if [[ ${#platforms[@]} -gt 1 ]]; then
local gigs; gigs=$(kube::golang::get_physmem); if [[ ${gigs} -ge ${KUBE_PARALLEL_BUILD_MEMORY} ]]; then
kube::log::status "Multiple platforms requested and available ${gigs}G >= threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in parallel"; parallel=true;
else
kube::log::status "Multiple platforms requested, but available ${gigs}G < threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in serial"; parallel=false;
fi;
fi; kube::golang::build_kube_toolchain; if [[ "${parallel}" == "true" ]]; then
kube::log::status "Building go targets for ${platforms[@]} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}";
do
( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: go build started"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-}; kube::log::status "${platform}: go build finished" ) &> "/tmp//${platform//\//_}.build" &
done; local fails=0; for job in $(jobs -p);
do
wait ${job} || let "fails+=1";
done; for platform in "${platforms[@]}";
do
cat "/tmp//${platform//\//_}.build";
done; exit ${fails};
else
for platform in "${platforms[@]}";
do
kube::log::status "Building go targets for ${platform}:" "${targets[@]}"; kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-};
done;
fi )' exited with status 1
Call stack:
1: /usr/local/google/home/stclair/go/k8s3/src/k8s.io/kubernetes/hack/lib/golang.sh:596 kube::golang::build_binaries(...)
2: hack/make-rules/build.sh:27 main(...)
Exiting with status 1
make: *** [kubelet] Error 1
```
</details>
Remaining work: Add a verbosity label to more log messages.
/cc @kubernetes/sig-api-machinery @kubernetes/contributor-experience
The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.
V=2 kube::log::info foo bar
Would only print "foo bar" if $KUBE_VERBOSE >= 2.
Automatic merge from submit-queue
Add a flag for `kubectl expose`to set ClusterIP and allow headless services
- Use `--cluster-ip=None` to create a headless service
- Fixes#10294
