The alias for vendor/github.com/onsi/ginkgo/ginkgo ensures that code like
30e99cb2a9/experiment/kind-conformance-image-e2e.sh (L110)
continues to work. The one without "vendor/" is there just in case that it
was used because it also worked.
Long term, "ginkgo" is a nicer, version independent alias. It gets used
internally to avoid future churn and gets documented also publicly in the
Makefile help.
The caveat is that there's no guarantee that a future v3 CLI will be compatible
with current invocations. But the most common usage is through
hack/ginkgo-e2e.sh, which can deal with such differences.
This should fix the following error when running
./hack/update-generated-stable-metrics.sh:
'go get' is no longer supported outside a module.
To build and install a command, use 'go install' with a version,
like 'go install example.com/cmd@latest'
For more information, see https://golang.org/doc/go-get-install-deprecation
or run 'go help get' or 'go help install'.
Using `go get` to download gopkg.in/yaml.v2 package into
KUBE_EXTRA_GOPATH directory no longer works. Interestingly, main repo
already has gopkg.in/yaml.v2@v2.4.0, same version that was installed by
that go get.
I guess that GOPATH with multiple elements no longer works either,
and since this code was the only user of KUBE_EXTRA_GOPATH, let's remove
it as well.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This has somewhat subtle implications. For a concrete example, this
changes the `-trimpath` behavior from only affecting the named pkg to
affecting all pkgs, which broke ginkgo, which seems to try to strip its
own `pwd` prefix. But since that runs in run-in-gopath, and not in
KUBE_ROOT, it fails to strip anything.
e.g.
before this, strings in the binary would be like
/home/user/kube/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/...
Ginkgo would find its own root as
/home/user/kube/_output/local/go/src/k8s.io/kubernetes/
so it would produce
vendor/github.com/onsi/ginkgo/...
in logs.
after this, strings in the binary strip the KUBE_ROOT and be like:
_output/local/go/src/k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/...
Ginkgo would find its own root as
/home/user/kube/_output/local/go/src/k8s.io/kubernetes/
so it would not strip anything, and produce
_output/local/go/src/k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/...
in logs.
before:
```
$ make generated_files
+++ [0226 13:42:17] Building go targets for linux/amd64:
hack/make-rules/helpers/go2make
> non-static build: k8s.io/kubernetes/hack/make-rules/helpers/go2make
```
after:
```
$ make generated_files
+++ [0226 14:30:08] Building go targets for linux/amd64
k8s.io/kubernetes/hack/make-rules/helpers/go2make (non-static)
```
The --log-file parameter will be deprecated as of Kubernetes 1.23 and should be
avoided. The replacement for distroless images is the image with go-runner, a
tool that handles output redirection.
For kubemark to run in that image it must be built as static binary.
kube-log-runner was formerly known as go-runner when it was originally introduced
in 393e0952e9
It was moved to kubernetes/release/images/build/go-runner later but is now
needed again in Kubernetes itself as replacement for the deprecated --log-file
klog feature: when bringing up a Windows node, kube-proxy.exe and kubelet.exe must be wrapped
with the helper binary to redirect output.
It got renamed to avoid a naming conflict with
test/conformance/image/go-runner and because the name was too vague.
Other downstream Kubernetes users may have a similar need, therefore it makes
sense to provide a prebuilt binary also in the release archives.
On a ppc64le host, fast build was failing:
# make KUBE_FASTBUILD=true quick-release
+++ [0908 15:56:36] Verifying Prerequisites....
+++ [0908 15:56:36] Building Docker image kube-build:build-fd009aaa81-5-v1.23.0-go1.17-buster.0
+++ [0908 15:56:44] Syncing sources to container
+++ [0908 15:56:48] Running build command...
+++ [0908 15:56:56] Building go targets for linux/ppc64le:
./vendor/k8s.io/code-generator/cmd/prerelease-lifecycle-gen
> non-static build: k8s.io/kubernetes/./vendor/k8s.io/code-generator/cmd/prerelease-lifecycle-gen
touch: cannot touch '_output/bin/prerelease-lifecycle-gen': No such file or directory
make[2]: *** [Makefile.generated_files:209: _output/bin/prerelease-lifecycle-gen] Error 1
make[1]: *** [Makefile:552: generated_files] Error 2
make: *** [Makefile:512: cross] Error 1
!!! [0908 15:56:58] Call tree:
!!! [0908 15:56:58] 1: build/../build/common.sh:476 kube::build::run_build_command_ex(...)
!!! [0908 15:56:58] 2: build/release.sh:36 kube::build::run_build_command(...)
make: *** [Makefile:454: quick-release] Error 1
This error happened because the _output/bin/ directory didn't exist at
the moment the file _output/bin/prerelease-lifecycle-gen was touched, so
the path didn't exist.
The _output/bin symlink was not created by kube::golang::place_bins()
because kube::golang::setup_platforms() assumed
KUBE_CLIENT_PLATFORMS=linux/amd64 despite being on a ppc64le host.
Fix build error by not assuming host_arch=amd64 when on ppc64le in
kube::golang::setup_platforms().
Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
The tag extraction from GOFLAGS doesn't do anything if -tags is not
present in GOFLAGS. Not only does that mean non-sensical tags are
passed, but because GOFLAGS is a space-separated variable, the build
will fail because the build flags because -tags must be comma-separated.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
When building inside a docker container, we fail to pass along enough
information for KUBE_CGO_OVERRIDES. The original PR was geared towards
building the binary from the command line
- dca376a03e
- e4ded2b3ec
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
We now add the `-buildmode pie` flag when building non-static binaries,
which enables the ASLR security mechanism.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Golang fixed the issue with the non-reproducible buildid already in
v1.15.3:
c5f6920212a3e965ce8a
This means we can now use the internal buildid instead of the unset one.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
A benchmark with go1.16 on amd64 resulted in an overall maximum memory
usage of 15GiB. This means we now lower the `KUBE_PARALLEL_BUILD_MEMORY`
to `20` to still have some room left.
The benchmark has been done with the following `Dockerfile`:
```dockerfile
FROM k8s.gcr.io/build-image/kube-cross:v1.16.0-1
RUN apt-get update && apt-get install -y time
WORKDIR /go/src/k8s.io/kubernetes
RUN git clone https://github.com/kubernetes/kubernetes
ENV KUBE_PARALLEL_BUILD_MEMORY=0
RUN /usr/bin/time -v make -C kubernetes cross-in-a-container
```
The value of interest from the output:
```
Maximum resident set size (kbytes): 1847108
```
The RSS is not the only value we have to consider. During the runtime I
measured the usage of the cached memory, which gets peaks up to 15GiB.
Test machine specs:
- CPU: 2 x Intel Scalable Gold 5120 28-Core Processor @ 2.2GHz
- RAM: 384GB
- Disk: 1 x 3.8TB NVME
- OS: Debian 10 (buster)
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Currently target CC can only be set if the host platform
is linux/amd64 . If target is already set in the environment
it is always safe to use it and enable cgo.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The amount of memory required to build binaries in parallel is right now
set to 40GiB. We now make this variable to be able to build artifacts in
parallel even with a lower amount of memory.
This enables SIG Release to speed-up the build time drastically in
Google Cloud Build (GCB).
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
While the umask was previously set for builds (and in fact `docker run`
sets it to 0022 by default), the release pipeline `cp`'s files (with
`-a`) which respect the user's umask, and is outside of the build path.
This sets the umask in a common function called by all build tools,
which Stephen agreed was the simplest and most complete fix. The
`verify-prereqs` function is already poorly named for what it does, and
this only makes it epsilon worse.
Some binaries now run as non-root (kube-scheduler). When umask is 0027,
for example, the container image we build has the binary 0750, which is
not executable by the non-root UID.
We have been having issues with making builds reproducible, especially
with the `.note.go.buildid` ELF section. One tip from a golang issue was
to set `-ldflags=-buildid=` which seems to work well. You can confirm
that the buildid is set to empty by inspecting the binaries with the go
command example `go tool buildid _output/local/go/bin/kubectl`
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
In 24d105995d, a fix was made in bazel
based builds to ensure that we add `selinux` tag when we build all
binaries especially the `kubelet`. We need to do the same for in our
hack scripts so things like `make release` will work properly as well.
Some scripts use `GOFLAGS=-tags=providerless` for example, So we should
support the tags to be specified in GOFLAGS as well. We parse out the
tags from there and ensure selinux is added to the list of tags we used
for building the binaries. Note that we add our own `-tags` with the
full set of tags and since we specify our parameter at the end, ours
full list takes precendence
Adds a go app which runs the e2e tests with ginkgo.
- Supports all the existing env vars of the bash script
- Improved flow control to avoid and better report issues
regarding the process PID
- Adds flags for modifying where to find the test binary and
ginkgo binary so that you can run it locally
- Adds 3 flags for specifying extra args before the double-dash,
extra args after the double-dash, and the seperator to use between
values in those env vars. This allows setting arbitrary, complex
values for use on the command such as flags which include spaces
or other characters.
when compile kubectl on platform other than
linux/amd64, we need to check the KUBE_SERVER_PLATFORMS
array emptiness before assign it.
the example command is:
make WHAT=cmd/kubectl KUBE_BUILD_PLATFORMS="darwin/amd64 windows/amd64"
Prior to this change, including windows/amd64 in KUBE_BUILD_PLATFORMS
would, for example, attempt to build the server binaries/tars/images for
Windows, which is not supported. This can break downstream build steps.
Recent change to hack/lib/golang.sh broke the build on MacOS this way:
$ make clean && make generated_files
+++ [0325 13:38:22] Verifying Prerequisites....
+++ [0325 13:38:23] Removing _output directory
k8s.io/kubernetes/vendor/github.com/spf13/pflag
k8s.io/kubernetes/hack/make-rules/helpers/go2make
+++ [0325 13:38:40] Building go targets for darwin/amd64:
./vendor/k8s.io/code-generator/cmd/deepcopy-gen
can't load package: package k8s.io/kubernetes: no Go files in k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes
!!! [0325 13:38:40] Call tree:
!!! [0325 13:38:40] 1: k8s.io/kubernetes/hack/lib/golang.sh:629 kube::golang::build_some_binaries(...)
!!! [0325 13:38:40] 2: k8s.io/kubernetes/hack/lib/golang.sh:764 kube::golang::build_binaries_for_platform(...)
!!! [0325 13:38:40] 3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make[1]: *** [_output/bin/deepcopy-gen] Error 1
make: *** [generated_files] Error 2
It was caused by 'binaries' array not being declared with 'local -a'.
It looks like MacOS' old bash version makes an array to contain first
empty element if declared this way.
The fix has been tested on MacOS High Sierra and Linux openSUSE 42.3 (x86_64)
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Otherwise, calling make followed by bazel might fail, requiring one to
run make clean first.
Additionally, add comments explaining why we must do this.
In go 1.11, go commands will use `GOFLAGS` as default flags, see
https://golang.org/doc/go1.11#go_command.
There is no need to pass GOFLAGS to $goflags, and if we do, go commands
will fail with "duplicate flags" error, e.g.
```
$ make test-integration WHAT=./test/integration/scheduler GOFLAGS="-v"
...
go test: v flag may be set only once
run "go help test" or "go help testflag" for more information
...
```
Pick up some code from https://github.com/heptio/kube-conformance
Fix up build scripts for the new conformance image
Fix Header template and Copyright to make verify job go green
update README and add execute permissions for script
Change-Id: Ib6509acd816cc2fb3a516bfb8e0ff9e32bff8f79
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.
Add ability to build with runtime coverage instrumentation
**What this PR does / why we need it**:
This PR adds the ability to instrument a subset of kubernetes binaries to report code coverage information. The specific use-case is to help determine coverage of our end-to-end Conformance tests, as well as provide data that can be used to help determine where to focus. This PR focuses on making it possible to build with instrumentation; collecting and using the generated coverage data will be done in later PRs. For more details as to the intent, see the [design doc](https://docs.google.com/document/d/1FKMBFxz7vtA-6ZgUkA47F8m6yR00fwqLcXMVJqsHt0g/edit?usp=sharing) (google doc; requires kubernetes-dev membership).
Specifically, this PR adds a new `KUBE_BUILD_WITH_COVERAGE` make variable, which when set will cause `kube-apiserver`, `kube-controller-manager`, `kube-scheduler`, `kube-proxy` and `kubelet` to be built with coverage instrumentation. These coverage-instrumented binaries will flush coverage information to disk every five seconds, defaulting to a temporary directory unless the `KUBE_COVERAGE_FILE` environment variable is set at launch, in which case it will write to that file instead.
The mechanism used to achieve coverage instrumentation is to build the targeted binaries as "unit tests" with coverage enabled, and then rigging the unit tests to just execute the binary's usual entry point. This is implemented only for the bash build system.
/sig testing
```release-note
NONE
```
First version of this was only standalone, now the code is being
vendored by anyone who wants to use it. So the standalone binary and
container are no longer useful.
Change-Id: Ib9369de66b4ecb3451f73ba2a252526d6615b96f
