Automatic merge from submit-queue
goflags must be after subcommand
**What this PR does / why we need it**:
if GOFLAGS is set when calling make, kubernetes will fail to build
as an example, I often have `GOFLAGS=-v` so I can have some idea of progress during compilation
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
no known issue
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
- Remaining spaghetti untangled
- Missed bazel update and a few hardcoded refs
- New instance of framework.ReadOrDie reference removed post rebase
- Resolve new clientset rebase
- Fixed e2e/generated BUILD dep
- A space
- Missed gobindata ref in golang.sh
Automatic merge from submit-queue
Build vendored copy of go-bindata and use that in go generate step
**What this PR does / why we need it**: as the title says, uses the vendored version of `go-bindata` rather than expecting developers to `go get` it (when building outside docker).
**Which issue this PR fixes**: fixes#34067, partially addresses #36655
**Special notes for your reviewer**: we still call `go generate` far too many times:
```console
~/.../src/k8s.io/kubernetes $ which go-bindata
~/.../src/k8s.io/kubernetes $ make
+++ [1116 17:35:28] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:29] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:30] Building go targets for linux/amd64:
cmd/libs/go2idl/deepcopy-gen
+++ [1116 17:35:35] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:35] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:36] Building go targets for linux/amd64:
cmd/libs/go2idl/defaulter-gen
+++ [1116 17:35:41] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:41] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:42] Building go targets for linux/amd64:
cmd/libs/go2idl/conversion-gen
+++ [1116 17:35:47] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:47] Generating bindata:
test/e2e/framework/gobindata_util.go
+++ [1116 17:35:48] Building go targets for linux/amd64:
cmd/libs/go2idl/openapi-gen
+++ [1116 17:35:56] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [1116 17:35:56] Generating bindata:
test/e2e/framework/gobindata_util.go
```
Fixing that is a separate effort, though.
cc @sebgoa @ZhangBanger
If dependencies are touched, the Makefile will detect that tools like
deepcopy-gen need to be rebuilt. It will rebuild them, but Go decides the
dependencies are not actually updated (maybe it was a test file - known
limitation of the current Makefile) and does not touch the resulting binary.
Then the Makefile touches the result explicitly to mark thatthe dep change has
been handled. Then rsync blows away that change with the older file, and
helpfully preserves the timestamp.
Now rsync ignores it if the checksum is the same. Result = faster build.
We weren't getting incremental builds because of new test only subpackages. Our
voodoo combo of 'go install' and 'go test -c' didn't cache things like
'test/e2e_node/services'. Add the '-i' flag to 'go test' to install test only
dependencies too.
Automatic merge from submit-queue
build kube-discovery and kubeadm with release
This will start publishing binaries from our ci builds.
@errordeveloper @dgoodwin
Automatic merge from submit-queue
Add separate build process for node test.
This PR is part of https://github.com/kubernetes/kubernetes/pull/31093. However, because currently node e2e is built on `KUBE_TEST_PLATFORMS`, which includes linux/amd64, darwin/amd64, windows/amd64 and linux/arm, it caused #32251 to fail.
In fact, node e2e is running on the same node with kubelet, and it also has built-in apiserver, etcd and namespace controller. All of them are only built on `KUBE_SERVER_PLATFORMS`, so node e2e should also only be built on those platforms.
```
KUBE_SERVER_PLATFORMS=(
linux/amd64
linux/arm
linux/arm64
)
```
This PR added a separate build process for node e2e to address this.
@vishh Do you need this for v1.4? because this blocks your #32251. /cc @dchen1107
Automatic merge from submit-queue
Don't error out if ${go_pkg_dir} already exists
I have 'make' aliased to 'make -j4' for various reasons, and that's always worked fine for Kubernetes in the past. But it recently broke because EEXIST errors from this symlink creation are fatal. So make them not fatal, because if the path already exists, why try to create it again? It exists because two invocations of setup_env() are run in parallel due to the -j4, and they race to create the symlink and one obviously fails.
ln: failed to create symbolic link ‘/home/dcbw/Development/containers/kubernetes/_output/local/go/src/k8s.io/kubernetes’: File exists
Makefile.generated_files:285: recipe for target '_output/bin/deepcopy-gen' failed
make[1]: *** [_output/bin/deepcopy-gen] Error 1
make[1]: *** Waiting for unfinished jobs....
+++ [0826 10:18:02] Generating bindata:
/home/dcbw/Development/containers/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0826 10:18:02] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0826 10:18:03] Building go targets for linux/amd64:
cmd/libs/go2idl/conversion-gen
Makefile:281: recipe for target 'generated_files' failed
make: *** [generated_files] Error 2
Also build the hyperkube docker image on-the-fly.
This is only a temporary fix until the proposal in issue
https://github.com/kubernetes/kubernetes/issues/28630 is implemented.
Also, the new build/deployment method completely obviates this step.
We use debian image instead of busybox and do not build hyperkube as a
static binary yet. Wait until PR
https://github.com/kubernetes/kubernetes/pull/26028 is merged to build
static hyperkube binaries.
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.
V=2 kube::log::info foo bar
Would only print "foo bar" if $KUBE_VERBOSE >= 2.
bindata and yaml, Gobindata automation
bindata utils for generating, go generate
match server version
gitignore for dirty, ca, rbase, KUBE_ROOT, buildfix
(rebased jul-25,29)
Automatic merge from submit-queue
Build Darwin test binaries on OS X when KUBE_FASTBUILD is set
Without this, on OS X, test binaries would not be built for the local ARCH, and test scripts fail with an obscure error due to not being able to find the correct binary to run.
This makes the behavior on OS X more consistent with Linux.
Tested on OS X and Linux.
Automatic merge from submit-queue
Improve the speed of do-nothing build.
As @thockin found out here https://github.com/kubernetes/kubernetes/issues/24518, vast majority of the do-nothing build time is spent in rebuilding the test binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them while building packages.
On my workstation, do-nothing hack/build-go.sh time goes from ~20 secs to ~4 secs, of which only ~1 sec is from doing test binary staleness check now (as opposed to ~17 secs it took to build the test binaries before). I did some experiments to bring this time down to <1 sec. I measured using go test -bench, but it was not very useful in this case. I believe, a vast majority of that ~1 second is being spent in fork/exec and piping the results back to the staleness check program along with the ser-deser involved, but it needs to be validated. Not a proof, but to provide some supporting evidence to this claim, running `go list -f format packages` in the shell takes about 600ms irrespective of what's in the format.
Tests are TBD. I am still trying to figure out how to test this, but I would like to get early feedback
cc @mikedanese @mml
As thockin found out here https://github.com/kubernetes/kubernetes/issues/24518,
vast majority of the do-nothing build time is spent in rebuilding the test
binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them
while building packages.
Tests are TBD. I am still trying to figure out how to test this.
Our `realpath` and `readlink -f` functions (required only because of MacOS,
thanks Steve) were poor substitutes at best. Mostly they were downright
broken. This thoroughly overhauls them and adds a test (in comments, since we
don't seem to have shell tests). For all the interesting cases I could think
of, the fakes act just like the real thing.
Then use those and canonicalize KUBE_ROOT. In order to make recursive calls of
our shell tool not additively grow `pwd` we have to essentially make the
sourcing of init.sh idempotent.
Automatic merge from submit-queue
Fix spacing in usage_from_stdin and info_from_stdin (issue #24186).
If "a" is a bash array, then the syntax to append the contents of $line as a
new element to the array is a+=("$line"), not messages+=$line
Using the former syntax just seems to append to the first element, creating a
long string and thus losing newline information.
Fixing this allows us to drop some empty lines from invocations of
usage_from_stdin.
If "a" is a bash array, then the syntax to append the contents of $line as a
new element to the array is a+=("$line"), not messages+=$line
Using the former syntax just seems to append to the first element, creating a
long string and thus losing newline information.
Fixing this allows us to drop some empty lines from invocations of
usage_from_stdin.
- Add Makefile targets
- Start services in the test harness and connect locally
- Build test into binary and copy to remote host to start services
- Use tar to copy binaries to remote hosts to simplify design
Adds cmd/mungedocs which is framework for processing
all files under docs/ and either verifying that no changes needed or
making in-place changes.
Did not reuse kube::util::gen-docs because that seemed to be
centered around handling added files, and this pass does not
add files.
Planned uses:
- table of contents automatic updating
- linkification
- internal link checker
- link-path-relativizer or absolutizer
- example file syncer
- header inserter.
Just table-of-contents updating in this PR.
Added Table of Contents to docs/networking.md.
Demonstrates use of new TOC generator presubmit.
Other docs will be added in future PRs.
Additional development will be needed to handle some
of the more complex cases.
hack/build-go.sh --use_go_build would fail because e2e.test was not being
excluded from the set of things that we were "go build"ing
Instead of walking all binaries, only walk static and nonstatic
```
$ hack/build-go.sh
+++ [0524 18:09:51] Building go targets for linux/amd64:
cmd/kube-proxy
cmd/kube-apiserver
cmd/kube-controller-manager
cmd/kubelet
cmd/hyperkube
cmd/kubernetes
plugin/cmd/kube-scheduler
cmd/kubectl
cmd/integration
cmd/gendocs
cmd/genman
cmd/genbashcomp
examples/k8petstore/web-server
github.com/onsi/ginkgo/ginkgo
test/e2e/e2e.test
+++ [0524 18:09:51] +++ Warning: stdlib pkg with cgo flag not found.
+++ [0524 18:09:51] +++ Warning: stdlib pkg cannot be rebuilt since /usr/lib/golang/pkg is not writable by eparis
+++ [0524 18:09:51] +++ Warning: Make /usr/lib/golang/pkg writable for eparis for a one-time stdlib install, Or
+++ [0524 18:09:51] +++ Warning: Rebuild stdlib using the command 'CGO_ENABLED=0 go install -a -installsuffix cgo std'
+++ [0524 18:09:51] +++ Falling back to go build, which is slower
**************can't load package: package github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test: cannot find package "github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test" in any of:
/usr/lib/golang/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test (from $GOROOT)
/storage/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test (from $GOPATH)
/storage/kubernetes/Godeps/_workspace/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test
!!! Error in /storage/kubernetes/hack/lib/golang.sh:339
'go build -o "${outfile}" "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${binary}"' exited with status 1
Call stack:
1: /storage/kubernetes/hack/lib/golang.sh:339 kube::golang::build_binaries_for_platform(...)
2: /storage/kubernetes/hack/lib/golang.sh:488 kube::golang::build_binaries(...)
3: hack/build-go.sh:26 main(...)
Exiting with status 1
!!! Error in /storage/kubernetes/hack/lib/golang.sh:406
'( kube::golang::setup_env; local version_ldflags; version_ldflags=$(kube::version::ldflags); local host_platform; host_platform=$(kube::golang::host_platform); local goflags; eval "goflags=(${KUBE_GOFLAGS:-})"; local use_go_build; local -a targets=(); local arg; for arg in "$@";
do
if [[ "${arg}" == "--use_go_build" ]]; then
use_go_build=true;
else
if [[ "${arg}" == -* ]]; then
goflags+=("${arg}");
else
targets+=("${arg}");
fi;
fi;
done; if [[ ${#targets[@]} -eq 0 ]]; then
targets=("${KUBE_ALL_TARGETS[@]}");
fi; local -a platforms=("${KUBE_BUILD_PLATFORMS[@]:+${KUBE_BUILD_PLATFORMS[@]}}"); if [[ ${#platforms[@]} -eq 0 ]]; then
platforms=("${host_platform}");
fi; local binaries; binaries=($(kube::golang::binaries_from_targets "${targets[@]}")); local parallel=false; if [[ ${#platforms[@]} -gt 1 ]]; then
local gigs; gigs=$(kube::golang::get_physmem); if [[ ${gigs} -gt ${KUBE_PARALLEL_BUILD_MEMORY} ]]; then
kube::log::status "Multiple platforms requested and available ${gigs}G > threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in parallel"; parallel=true;
else
kube::log::status "Multiple platforms requested, but available ${gigs}G < threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in serial"; parallel=false;
fi;
fi; if [[ "${parallel}" == "true" ]]; then
kube::log::status "Building go targets for ${platforms[@]} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}";
do
( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: go build started"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-}; kube::log::status "${platform}: go build finished" ) &> "/tmp//${platform//\//_}.build" &
done; local fails=0; for job in $(jobs -p);
do
wait ${job} || let "fails+=1";
done; for platform in "${platforms[@]}";
do
cat "/tmp//${platform//\//_}.build";
done; exit ${fails};
else
for platform in "${platforms[@]}";
do
kube::log::status "Building go targets for ${platform}:" "${targets[@]}"; kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-};
done;
fi )' exited with status 1
Call stack:
1: /storage/kubernetes/hack/lib/golang.sh:406 kube::golang::build_binaries(...)
2: hack/build-go.sh:26 main(...)
Exiting with status 1
```
Go 1.4 added the -o flag to the "go test" command as well as support for
the TestMain() function, so we must work around these not existing in
Go 1.3.
The version of readlink on OS X does not have the -f flag - so we'll
just skip canonicalizing the path.
This commit deletes cmd/e2e and updates hack/ginkgo-e2e.sh to use the
'ginkgo' command instead. All logic from cmd/e2e/e2e.go and
test/e2e/driver.go have been combined into the new file
test/e2e/e2e_test.go.
The test tarball now includes a built version of the test/e2e test
binary, which includes all tests under test/e2e. This was accomplished
by updating the build scripts to use 'go test -c' when a target name
ended with '.test', and adding a dependency on test/e2e/e2e.test.
This prebuilt test binary is passed to the Ginkgo runner in
hack/ginkgo-e2e.sh. In a future change, we can add support to run
Ginkgo against the source tree if it is available.
This change is generally intended to have no externally visible changes,
aside from the following caveats:
- The -t/--tests flag has been removed
- Calling cmd/e2e/e2e directly obviously won't work, but that was never
intended to be supported anyway
- If the GINKGO_PARALLEL environment variable is set to y, then ginkgo
will run test specs in parallel. (Currently defaults to n, since some
tests are broken in this mode.)
Additionally, several tests which made poor assumptions about cwd or
used testContext before it had been set have been fixed.
This commit deletes cmd/e2e and updates hack/ginkgo-e2e.sh to use the
'ginkgo' command instead. All logic from cmd/e2e/e2e.go and
test/e2e/driver.go have been combined into the new file
test/e2e/e2e_test.go.
Additionally, several tests which made poor assumptions about cwd or
used testContext before it was set have been fixed.
This change is generally intended to have no externally visible changes,
aside from the following caveats:
- The -t/--tests flag has been removed
- Calling cmd/e2e/e2e directly obviously won't work, but that was never
supported anyway
- If the GINKGO_PARALLEL environment variable is set to y, then ginkgo
will run test specs in parallel. (Currently defaults to n, since some
tests are broken in this mode.)
We allow cgo when available since some of our server components (mainly
Kubelet) rely on it. The current driving factor for the static binaries
was building Docker images from scratch. We will instead use images
built from small busybox bases.
Fixes#5517
It includes many of the server components which we only expect to be
able to build in the server target architectures, but it is included in
the client build targets.
