Before the workspace changes, it was possible to include proto files via the
vendor path and the symlinks there. Now those symlinks are gone, so the search
path for them must be given separately.
Prior to this it would download Go, setup the environment, then (maybe)
download Go again.
I renamed verify_go_version to be internal and fixed all callers.
This makes "new" and "old" setup_env functions. In subsequent commits,
all callers of the "old" form will be fixed, and the "new" will be
renamed back.
The old and new functions diff:
```diff
--- /tmp/a 2023-12-14 09:02:57.804092696 -0800
+++ /tmp/b 2023-12-14 09:03:09.679999585 -0800
@@ -1,4 +1,4 @@
-kube::golang::old::setup_env() {
+kube::golang:🆕:setup_env() {
kube::golang::verify_go_version
# Set up GOPATH. We have tools which depend on being in a GOPATH (see
@@ -7,9 +7,9 @@
# Even in module mode, we need to set GOPATH for `go build` and `go install`
# to work. We build various tools (usually via `go install`) from a lot of
# scripts.
- # * We can't set GOBIN because that does not work on cross-compiles.
- # * We could use `go build -o <something>`, but it's subtle when it comes
- # to cross-compiles and whether the <something> is a file or a directory,
+ # * We can't just set GOBIN because that does not work on cross-compiles.
+ # * We could always use `go build -o <something>`, but it's subtle wrt
+ # cross-compiles and whether the <something> is a file or a directory,
# and EVERY caller has to get it *just* right.
# * We could leave GOPATH alone and let `go install` write binaries
# wherever the user's GOPATH says (or doesn't say).
@@ -20,16 +20,6 @@
#
# Eventually, when we no longer rely on run-in-gopath.sh we may be able to
# simplify this some.
- local go_pkg_dir="${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}"
- local go_pkg_basedir
- go_pkg_basedir=$(dirname "${go_pkg_dir}")
-
- mkdir -p "${go_pkg_basedir}"
-
- # TODO: This symlink should be relative.
- if [[ ! -e "${go_pkg_dir}" || "$(readlink "${go_pkg_dir}")" != "${KUBE_ROOT}" ]]; then
- ln -snf "${KUBE_ROOT}" "${go_pkg_dir}"
- fi
export GOPATH="${KUBE_GOPATH}"
# If these are not set, set them now. This ensures that any subsequent
@@ -40,24 +30,10 @@
# Make sure our own Go binaries are in PATH.
export PATH="${KUBE_GOPATH}/bin:${PATH}"
- # Change directories so that we are within the GOPATH. Some tools get really
- # upset if this is not true. We use a whole fake GOPATH here to collect the
- # resultant binaries.
- local subdir
- subdir=$(kube::realpath . | sed "s|${KUBE_ROOT}||")
- cd "${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}/${subdir}" || return 1
-
- # Set GOROOT so binaries that parse code can work properly.
- GOROOT=$(go env GOROOT)
- export GOROOT
-
# Unset GOBIN in case it already exists in the current session.
# Cross-compiles will not work with it set.
unset GOBIN
- # This seems to matter to some tools
- export GO15VENDOREXPERIMENT=1
-
- # Disable workspaces
- export GOWORK=off
+ # Explicitly turn on modules.
+ export GO111MODULE=on
}
```
Result: `make` works for k/k:
```
$ make kubectl
+++ [1211 11:07:31] Building go targets for linux/amd64
k8s.io/kubernetes/cmd/kubectl (static)
$ make WHAT=./cmd/kubectl/
+++ [1211 11:08:19] Building go targets for linux/amd64
k8s.io/kubernetes/./cmd/kubectl/ (non-static)
$ make WHAT=k8s.io/kubernetes/cmd/kubectl
+++ [1211 11:08:52] Building go targets for linux/amd64
k8s.io/kubernetes/cmd/kubectl (static)
```
Result: `make` works for staging by package:
```
$ make WHAT=k8s.io/api
+++ [1211 11:11:37] Building go targets for linux/amd64
k8s.io/api (non-static)
```
Result: `make` fails for staging by path:
```
$ make WHAT=./staging/src/k8s.io/api
+++ [1211 11:12:44] Building go targets for linux/amd64
k8s.io/kubernetes/./staging/src/k8s.io/api (non-static)
cannot find module providing package k8s.io/kubernetes/staging/src/k8s.io/api: import lookup disabled by -mod=vendor
(Go version in go.work is at least 1.14 and vendor directory exists.)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44] 1: /home/thockin/src/kubernetes/hack/lib/golang.sh:850 kube::golang::build_some_binaries(...)
!!! [1211 11:12:44] 2: /home/thockin/src/kubernetes/hack/lib/golang.sh:1012 kube::golang::build_binaries_for_platform(...)
!!! [1211 11:12:44] 3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:96: all] Error 1
```
Result: `make test` fails:
```
$ make test WHAT=./cmd/kubectl
+++ [1211 11:13:38] Set GOMAXPROCS automatically to 6
+++ [1211 11:13:38] Running tests without code coverage and with -race
cmd/kubectl/kubectl.go:25:2: cannot find package "k8s.io/client-go/plugin/pkg/client/auth" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/plugin/pkg/client/auth (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOPATH)
cmd/kubectl/kubectl.go:20:2: cannot find package "k8s.io/component-base/cli" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/component-base/cli (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/component-base/cli (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/component-base/cli (from $GOPATH)
cmd/kubectl/kubectl.go:21:2: cannot find package "k8s.io/kubectl/pkg/cmd" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd (from $GOPATH)
cmd/kubectl/kubectl.go:22:2: cannot find package "k8s.io/kubectl/pkg/cmd/util" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd/util (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd/util (from $GOPATH)
make: *** [Makefile:191: test] Error 1
```
Now update-generated-proto-bindings rules all the api.pb.go generation.
Running this shows no delta on the runtime.pb.go
This exposes an issue in how protoc is called for protos that specify
`go_package` which is fixed here.
Not all of our protos specify that option (even though it is
recommended), which will be fixed subsequently.
Each of these scripts is basically identical, and all were too brittle.
Now they should be more resilient and easier to manage. The script
still needs to be updated if we add new ones, which I do not love.
More cleanup to follow.
