Automatic merge from submit-queue
Add --force to kubectl delete and explain force deletion
--force is required for --grace-period=0. --now is == --grace-period=1.
Improve command help to explain what graceful deletion is and warn about
force deletion.
Part of #34160 & #29033
```release-note
In order to bypass graceful deletion of pods (to immediately remove the pod from the API) the user must now provide the `--force` flag in addition to `--grace-period=0`. This prevents users from accidentally force deleting pods without being aware of the consequences of force deletion. Force deleting pods for resources like StatefulSets can result in multiple pods with the same name having running processes in the cluster, which may lead to data corruption or data inconsistency when using shared storage or common API endpoints.
```
Automatic merge from submit-queue
add script to check for updates to the files for generation
Adds a verify script to help people writing APIs remember to wire all the individual pieces up. Eventually it will be better to avoid this altogether, but this is a start.
@ncdc fyi
Automatic merge from submit-queue
Move Statefulset (previously PetSet) to v1beta1
**What this PR does / why we need it**: #28718
**Which issue this PR fixes** _(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)_: fixes #
**Special notes for your reviewer**: depends on #35663 (PetSet rename)
cc @erictune @foxish @kubernetes/sig-apps
**Release note**:
``` release-note
v1beta1/StatefulSet replaces v1alpha1/PetSet.
```
--force is required for --grace-period=0. --now is == --grace-period=1.
Improve command help to explain what graceful deletion is and warn about
force deletion.
Automatic merge from submit-queue
Build: Use checksum to rsync results
If dependencies are touched, the Makefile will detect that tools like
deepcopy-gen need to be rebuilt. It will rebuild them, but if Go decides the
dependencies are not ACTUALLY updated (maybe it was a test file - known
limitation of the current Makefile), it does not touch the resulting binary.
The Makefile touches the result explicitly to mark that the dep change has
been handled. But rsync blows away that change with the older file, and
helpfully preserves the timestamp. This repeats on every build.
Now rsync ignores it if the checksum is the same. Result = faster build.
@wojtek-t @gmarek
If dependencies are touched, the Makefile will detect that tools like
deepcopy-gen need to be rebuilt. It will rebuild them, but Go decides the
dependencies are not actually updated (maybe it was a test file - known
limitation of the current Makefile) and does not touch the resulting binary.
Then the Makefile touches the result explicitly to mark thatthe dep change has
been handled. Then rsync blows away that change with the older file, and
helpfully preserves the timestamp.
Now rsync ignores it if the checksum is the same. Result = faster build.
Automatic merge from submit-queue
Retry curl when it failed during download
Fixes#34769
For now I'm adding retry only to etcd installation as I saw it recently to fail in one of my PRs. We may change it also in other places if it also happens.
Generate a kubeconfig for use with controller-manager, kubelet,
scheduler etc. This kubeconfig should use the secure https
port of the api server with appropriate ca cert for the components
to talk to api server.
With this change, one can set API_PORT=0 to completely switch off
insecure access for testing admission controllers etc.
Fixes#33375
Automatic merge from submit-queue
build: Fix version generation.
This PR provides update to version string generation procedure to handle additional case - working on branch delivered from release tag, instead of working on `-(alpha|beta).no` suffixed tag.
Reason of it is that actually if someone would start working on branch forked from release tag (`v1.4.1` for example), will add some commits and then would try to build k8s/run conformance tests - will end up with broken version string in form similar to `v1.4.1.X+YYYY` where `X` is a dictance/number of commits from base tag and `YYYY` will be first 14 characters from commit hash.
Such version - containing four dotted parts is rejected during conformance tests with error similar to:
```
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubectl.go:843
Oct 11 12:38:33.726: Failed to get server version: Unable to parse server version "v1.4.1.5+somecommithash": Invalid character(s) found in patch number "1.5"
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:413
```
This PR provides a cure for this situation creating version string in form `v1.4.1-X+YYYY` which then should be readable for version validation checks.
Release note:
```release-note
Fix version string generation for local version different from release and not based on `-alpha.no` or `-beta.no` suffixed tag.
```
Automatic merge from submit-queue
Fix a typo in openapi spec generation
A typo in util.sh resulted in the wrong spec stored in source tree. The commit should be explanatory.
We weren't getting incremental builds because of new test only subpackages. Our
voodoo combo of 'go install' and 'go test -c' didn't cache things like
'test/e2e_node/services'. Add the '-i' flag to 'go test' to install test only
dependencies too.
Automatic merge from submit-queue
Use patched golang1.7.1 for cross-builds targeting darwin
This PR extends #32517 to use the patched go1.7.1 introduced by that PR to build all darwin targets (e.g. kubectl).
This is necessary because binaries built with earlier versions of Go regularly segfault on macOS Sierra (see #32999 and #33070).
This solution is somewhat hacky, but we intend to cherry-pick this to 1.4, and switching all of 1.4 to build with go1.7.1 is very high risk.
I haven't yet pushed the cross build image yet, so this will fail to build. Will test locally and update with results.
First step of fixing #33801.
cc @luxas @pwittrock @david-mcmahon @liggitt @smarterclayton @jfrazelle @Starefossen @gerred
Automatic merge from submit-queue
build kube-discovery and kubeadm with release
This will start publishing binaries from our ci builds.
@errordeveloper @dgoodwin
Automatic merge from submit-queue
Automatic generation of man pages
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Currently genman supports only kubectl command, so this PR will enable genman to generate man pages for rest of the command like - kube-apiserver, kube-controller-manager, kube-proxy, kube-scheduler, kubelet
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
N/A
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Add separate build process for node test.
This PR is part of https://github.com/kubernetes/kubernetes/pull/31093. However, because currently node e2e is built on `KUBE_TEST_PLATFORMS`, which includes linux/amd64, darwin/amd64, windows/amd64 and linux/arm, it caused #32251 to fail.
In fact, node e2e is running on the same node with kubelet, and it also has built-in apiserver, etcd and namespace controller. All of them are only built on `KUBE_SERVER_PLATFORMS`, so node e2e should also only be built on those platforms.
```
KUBE_SERVER_PLATFORMS=(
linux/amd64
linux/arm
linux/arm64
)
```
This PR added a separate build process for node e2e to address this.
@vishh Do you need this for v1.4? because this blocks your #32251. /cc @dchen1107
Automatic merge from submit-queue
Don't error out if ${go_pkg_dir} already exists
I have 'make' aliased to 'make -j4' for various reasons, and that's always worked fine for Kubernetes in the past. But it recently broke because EEXIST errors from this symlink creation are fatal. So make them not fatal, because if the path already exists, why try to create it again? It exists because two invocations of setup_env() are run in parallel due to the -j4, and they race to create the symlink and one obviously fails.
ln: failed to create symbolic link ‘/home/dcbw/Development/containers/kubernetes/_output/local/go/src/k8s.io/kubernetes’: File exists
Makefile.generated_files:285: recipe for target '_output/bin/deepcopy-gen' failed
make[1]: *** [_output/bin/deepcopy-gen] Error 1
make[1]: *** Waiting for unfinished jobs....
+++ [0826 10:18:02] Generating bindata:
/home/dcbw/Development/containers/kubernetes/test/e2e/framework/gobindata_util.go
+++ [0826 10:18:02] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
+++ [0826 10:18:03] Building go targets for linux/amd64:
cmd/libs/go2idl/conversion-gen
Makefile:281: recipe for target 'generated_files' failed
make: *** [generated_files] Error 2
Also build the hyperkube docker image on-the-fly.
This is only a temporary fix until the proposal in issue
https://github.com/kubernetes/kubernetes/issues/28630 is implemented.
Also, the new build/deployment method completely obviates this step.
We use debian image instead of busybox and do not build hyperkube as a
static binary yet. Wait until PR
https://github.com/kubernetes/kubernetes/pull/26028 is merged to build
static hyperkube binaries.
Automatic merge from submit-queue
Keep vendor/ and Godep/ when creating the staging client, add a readme
In copy.sh, instead of removing the vendor/, moving it to _vendor. vendor/ is needed when we publish the staging client to its own repository.
Automatic merge from submit-queue
fix path handling in hack/lib/init.sh
Jenkinsfile pipeline jobs get cloned into "\<project\> (\<branch\>)". As a result, I can't use certain things in `hack/lib/init.sh`.
This is a small fix for that problem.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Automatic merge from submit-queue
Disable linux/ppc64le compilation by default
Work-around for #30384.
I'm still testing this locally to see if it actually works. The build is slow. (PR Jenkins won't tell us whether this fixes ppc.)
cc @Random-Liu @spxtr @david-mcmahon @luxas
Automatic merge from submit-queue
Fix subtle build breakage
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
NB: this requires a `make clean` to activate.
@lavalamp @jbeda @quinton-hoole @david-mcmahon
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh
This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT. The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary. The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it. The in-container build tried to use the existing binaries,
but GOROOT is wrong.
This change whites-out the whole _output dir.
I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build. So I just put
it in the data container. This seems to work. The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
The KUBE_VERBOSE environment variable sets the verbosity level to
use. Log messages can specify a verbosity by setting the V
variable. e.g.
V=2 kube::log::info foo bar
Would only print "foo bar" if $KUBE_VERBOSE >= 2.
bindata and yaml, Gobindata automation
bindata utils for generating, go generate
match server version
gitignore for dirty, ca, rbase, KUBE_ROOT, buildfix
(rebased jul-25,29)
Automatic merge from submit-queue
adds source debug build options
See issue & discussion here: #28227
Enables source debugging the Kubernetes binaries with tools like delve by providing the user with the ability to provide debug build options to the glang compiler.
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
Automatic merge from submit-queue
Move KUBE_GIT_UPSTREAM out of init.sh and into *-munge-docs.sh.
It is only used in those 2 scripts and this way we can set the value dynamically.
Clean up a bit too (80col, formatting)
Automatic merge from submit-queue
Add -g curl option to hack/lib/util.sh wait_for_url
Add `-g` curl option
```
This option switches off the "URL globbing parser". When you set this option,
you can specify URLs that contain the letters {}[] without having them being
interpreted by curl itself. Note that these letters are not normal legal URL
contents but they should be encoded according to the URI standard.
```
> Why?
So that IPv6 addresses with `[` and `]`, like `[::1]:8080`, work with that script.
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
Build Darwin test binaries on OS X when KUBE_FASTBUILD is set
Without this, on OS X, test binaries would not be built for the local ARCH, and test scripts fail with an obscure error due to not being able to find the correct binary to run.
This makes the behavior on OS X more consistent with Linux.
Tested on OS X and Linux.
Automatic merge from submit-queue
Enable all ppc64le builds, except for hyperkube
Partially fixes: #25886
Talked to @Pensu, and all other binaries seem to work fine
@david-mcmahon @ixdy @Pensu @smarterclayton
Auto generated docs are **NO LONGER CHECKED IN**, only placeholders.
To generate them, e.g. before exporting docs, run hack/generate-docs.sh.
hack/verify-generated-docs.sh ensures that generated docs are merely the
placeholder text.
hack/update-generated-docs.sh puts the placeholder text in the proper
places.
The old munge behavior is moved into hack/{update|verify}-munge-docs.sh.
Automatic merge from submit-queue
Get install-etcd.sh to work on OSX
Check the platform before we download the etcd binary. On OSX, save
the zip file, unzip it and link the directory with the correct
name.
Fixes#26041
Automatic merge from submit-queue
Add more global options to local-up-cluster.sh
I added a couple more global variables to the `local-up-cluster.sh` script.
The `KUBELET_HOST` environment name make sense to me. If it isn't the best name I will change it.
The `ETCD` ones make sense since they are being used under `hack/lib/etcd.sh`
Signed-off-by: André Martins <aanm90@gmail.com>
Automatic merge from submit-queue
Improve the speed of do-nothing build.
As @thockin found out here https://github.com/kubernetes/kubernetes/issues/24518, vast majority of the do-nothing build time is spent in rebuilding the test binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them while building packages.
On my workstation, do-nothing hack/build-go.sh time goes from ~20 secs to ~4 secs, of which only ~1 sec is from doing test binary staleness check now (as opposed to ~17 secs it took to build the test binaries before). I did some experiments to bring this time down to <1 sec. I measured using go test -bench, but it was not very useful in this case. I believe, a vast majority of that ~1 second is being spent in fork/exec and piping the results back to the staleness check program along with the ser-deser involved, but it needs to be validated. Not a proof, but to provide some supporting evidence to this claim, running `go list -f format packages` in the shell takes about 600ms irrespective of what's in the format.
Tests are TBD. I am still trying to figure out how to test this, but I would like to get early feedback
cc @mikedanese @mml
Automatic merge from submit-queue
Implement the cluster-controller of kubernetes federation
This PR is the cluster-controller of kubernetes federation.
This controller of federation phase 1 just collect the status of federated clusters.
As thockin found out here https://github.com/kubernetes/kubernetes/issues/24518,
vast majority of the do-nothing build time is spent in rebuilding the test
binaries. There is no staleness check support for test binaries.
This commit implements the staleness checks for test binaries and uses them
while building packages.
Tests are TBD. I am still trying to figure out how to test this.
Our `realpath` and `readlink -f` functions (required only because of MacOS,
thanks Steve) were poor substitutes at best. Mostly they were downright
broken. This thoroughly overhauls them and adds a test (in comments, since we
don't seem to have shell tests). For all the interesting cases I could think
of, the fakes act just like the real thing.
Then use those and canonicalize KUBE_ROOT. In order to make recursive calls of
our shell tool not additively grow `pwd` we have to essentially make the
sourcing of init.sh idempotent.
Automatic merge from submit-queue
Introduce events flag for describers
Printing events for a given object is not always needed. Thus, introducing --show-events=false to ``kubectl describe`` to skip events printing.
Fixes: #24239
Introduce DescriberSettings for Describer display options
Introduce --show-events flag and DescriberSettings in Describer methods
Introduce unit-tests
Regenerated kubectl describe docs
Add events flag tests to test-cmd.sh
Signed-off-by: dhodovsk@redhat.com
Signed-off-by: jchaloup@redhat.com
Automatic merge from submit-queue
Fix spacing in usage_from_stdin and info_from_stdin (issue #24186).
If "a" is a bash array, then the syntax to append the contents of $line as a
new element to the array is a+=("$line"), not messages+=$line
Using the former syntax just seems to append to the first element, creating a
long string and thus losing newline information.
Fixing this allows us to drop some empty lines from invocations of
usage_from_stdin.
If "a" is a bash array, then the syntax to append the contents of $line as a
new element to the array is a+=("$line"), not messages+=$line
Using the former syntax just seems to append to the first element, creating a
long string and thus losing newline information.
Fixing this allows us to drop some empty lines from invocations of
usage_from_stdin.
Out tests for features that didn't exist in old golang versions (like
gofmt didn't exist in 1.3 or the linker didn't want an = in 1.4) checked
for the new version. But this means every time there is a new version we
have to update the test. This PR inverts the test. We do it the old way
for the old version and if a new version comes along we just assume it
is going to work the new way.
- Add Makefile targets
- Start services in the test harness and connect locally
- Build test into binary and copy to remote host to start services
- Use tar to copy binaries to remote hosts to simplify design
Some functionality in hack/lib is currently depended on by
cluster/common.sh so kube-up from the full release tar (which
does not include hack/) is currently broken. With this PR we
create cluster/lib/ and move the necessary bits from hack/
over to get kube-up working again.
Fixes: 96d1b8d1b2
Signed-off-by: Mike Danese <mikedanese@google.com>
Just do all doc generation in the hack::util::gen-docs instead of spread
around. We also only track the generated docs in a single file for the
whole tree.
kube::util:ensure-temp-dir claims that it will automatically clean it
up. But it obviously doesn't. Since we cannot add multiple trap in bash
add a function that lets us trap and clean up KUBE_TEMP even if someone
already set a trap.
Instead of calling rsync over and over and over and over and over and
over and over and over and over (and probably over) use one `cp`
Before:
real 0m5.247s
user 0m2.294s
sys 0m1.300s
After:
real 0m2.260s
user 0m2.230s
sys 0m0.936s
This PR changes how we version going forward in the following ways:
* mark-new-version.sh is changed to a new policy of just splitting
branches, rather than the old backmerge policy, as discussed in
vX.Y.0, and a tag for vX.(Y+1).0-alpha.0 back to master.
* I eliminated PRs back to master by making the version/base.go
gitVersion and gitCommit just be `export-subst`. I testing that this
works with GitHub's source export tarballs. There's no reason to
bother with forcing the version into `base.go` (especially twice). The
tarball versions outside a git tree aren't perfect (master looks like
"v0.0.0+hash", and the release branches look more accurate), but our
build contract has never allowed that version is perfect in this
situation, so I think we can relax this.
* That master tag gets picked up by "git describe" on master, so e.g.
master would have immediately become v1.1.0-alpha.0
* In order to be more semVer compatible, the gitVersion field for the
master branch now looks something like 1.1.0-alpha.0.6+84c76d1142ea4d.
This is a tiny translation of the "git describe". I did this because
there are a ton of consumers out there of the "gitVersion" field
expecting it to be the full version, but it would be nice if this
field were actually semver compliant. (1.1.0-alpha.0-6-84c76d1142ea4d
is, but it's not *usefully* so.)
Fixes#11495
* Add analytics munger w/ munge heading
* More link autofixes
* Allow running a subset of munges
* Fix repo root detection
* Only process non-preformatted blocks
* Gendocs no longer adds the analytics link; mungedocs does that in a
second pass.
Adds cmd/mungedocs which is framework for processing
all files under docs/ and either verifying that no changes needed or
making in-place changes.
Did not reuse kube::util::gen-docs because that seemed to be
centered around handling added files, and this pass does not
add files.
Planned uses:
- table of contents automatic updating
- linkification
- internal link checker
- link-path-relativizer or absolutizer
- example file syncer
- header inserter.
Just table-of-contents updating in this PR.
Added Table of Contents to docs/networking.md.
Demonstrates use of new TOC generator presubmit.
Other docs will be added in future PRs.
Additional development will be needed to handle some
of the more complex cases.
hack/build-go.sh --use_go_build would fail because e2e.test was not being
excluded from the set of things that we were "go build"ing
Instead of walking all binaries, only walk static and nonstatic
```
$ hack/build-go.sh
+++ [0524 18:09:51] Building go targets for linux/amd64:
cmd/kube-proxy
cmd/kube-apiserver
cmd/kube-controller-manager
cmd/kubelet
cmd/hyperkube
cmd/kubernetes
plugin/cmd/kube-scheduler
cmd/kubectl
cmd/integration
cmd/gendocs
cmd/genman
cmd/genbashcomp
examples/k8petstore/web-server
github.com/onsi/ginkgo/ginkgo
test/e2e/e2e.test
+++ [0524 18:09:51] +++ Warning: stdlib pkg with cgo flag not found.
+++ [0524 18:09:51] +++ Warning: stdlib pkg cannot be rebuilt since /usr/lib/golang/pkg is not writable by eparis
+++ [0524 18:09:51] +++ Warning: Make /usr/lib/golang/pkg writable for eparis for a one-time stdlib install, Or
+++ [0524 18:09:51] +++ Warning: Rebuild stdlib using the command 'CGO_ENABLED=0 go install -a -installsuffix cgo std'
+++ [0524 18:09:51] +++ Falling back to go build, which is slower
**************can't load package: package github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test: cannot find package "github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test" in any of:
/usr/lib/golang/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test (from $GOROOT)
/storage/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test (from $GOPATH)
/storage/kubernetes/Godeps/_workspace/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/e2e.test
!!! Error in /storage/kubernetes/hack/lib/golang.sh:339
'go build -o "${outfile}" "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${binary}"' exited with status 1
Call stack:
1: /storage/kubernetes/hack/lib/golang.sh:339 kube::golang::build_binaries_for_platform(...)
2: /storage/kubernetes/hack/lib/golang.sh:488 kube::golang::build_binaries(...)
3: hack/build-go.sh:26 main(...)
Exiting with status 1
!!! Error in /storage/kubernetes/hack/lib/golang.sh:406
'( kube::golang::setup_env; local version_ldflags; version_ldflags=$(kube::version::ldflags); local host_platform; host_platform=$(kube::golang::host_platform); local goflags; eval "goflags=(${KUBE_GOFLAGS:-})"; local use_go_build; local -a targets=(); local arg; for arg in "$@";
do
if [[ "${arg}" == "--use_go_build" ]]; then
use_go_build=true;
else
if [[ "${arg}" == -* ]]; then
goflags+=("${arg}");
else
targets+=("${arg}");
fi;
fi;
done; if [[ ${#targets[@]} -eq 0 ]]; then
targets=("${KUBE_ALL_TARGETS[@]}");
fi; local -a platforms=("${KUBE_BUILD_PLATFORMS[@]:+${KUBE_BUILD_PLATFORMS[@]}}"); if [[ ${#platforms[@]} -eq 0 ]]; then
platforms=("${host_platform}");
fi; local binaries; binaries=($(kube::golang::binaries_from_targets "${targets[@]}")); local parallel=false; if [[ ${#platforms[@]} -gt 1 ]]; then
local gigs; gigs=$(kube::golang::get_physmem); if [[ ${gigs} -gt ${KUBE_PARALLEL_BUILD_MEMORY} ]]; then
kube::log::status "Multiple platforms requested and available ${gigs}G > threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in parallel"; parallel=true;
else
kube::log::status "Multiple platforms requested, but available ${gigs}G < threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in serial"; parallel=false;
fi;
fi; if [[ "${parallel}" == "true" ]]; then
kube::log::status "Building go targets for ${platforms[@]} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}";
do
( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: go build started"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-}; kube::log::status "${platform}: go build finished" ) &> "/tmp//${platform//\//_}.build" &
done; local fails=0; for job in $(jobs -p);
do
wait ${job} || let "fails+=1";
done; for platform in "${platforms[@]}";
do
cat "/tmp//${platform//\//_}.build";
done; exit ${fails};
else
for platform in "${platforms[@]}";
do
kube::log::status "Building go targets for ${platform}:" "${targets[@]}"; kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform ${platform} ${use_go_build:-};
done;
fi )' exited with status 1
Call stack:
1: /storage/kubernetes/hack/lib/golang.sh:406 kube::golang::build_binaries(...)
2: hack/build-go.sh:26 main(...)
Exiting with status 1
```
Go 1.4 added the -o flag to the "go test" command as well as support for
the TestMain() function, so we must work around these not existing in
Go 1.3.
The version of readlink on OS X does not have the -f flag - so we'll
just skip canonicalizing the path.
This commit deletes cmd/e2e and updates hack/ginkgo-e2e.sh to use the
'ginkgo' command instead. All logic from cmd/e2e/e2e.go and
test/e2e/driver.go have been combined into the new file
test/e2e/e2e_test.go.
The test tarball now includes a built version of the test/e2e test
binary, which includes all tests under test/e2e. This was accomplished
by updating the build scripts to use 'go test -c' when a target name
ended with '.test', and adding a dependency on test/e2e/e2e.test.
This prebuilt test binary is passed to the Ginkgo runner in
hack/ginkgo-e2e.sh. In a future change, we can add support to run
Ginkgo against the source tree if it is available.
This change is generally intended to have no externally visible changes,
aside from the following caveats:
- The -t/--tests flag has been removed
- Calling cmd/e2e/e2e directly obviously won't work, but that was never
intended to be supported anyway
- If the GINKGO_PARALLEL environment variable is set to y, then ginkgo
will run test specs in parallel. (Currently defaults to n, since some
tests are broken in this mode.)
Additionally, several tests which made poor assumptions about cwd or
used testContext before it had been set have been fixed.
This commit deletes cmd/e2e and updates hack/ginkgo-e2e.sh to use the
'ginkgo' command instead. All logic from cmd/e2e/e2e.go and
test/e2e/driver.go have been combined into the new file
test/e2e/e2e_test.go.
Additionally, several tests which made poor assumptions about cwd or
used testContext before it was set have been fixed.
This change is generally intended to have no externally visible changes,
aside from the following caveats:
- The -t/--tests flag has been removed
- Calling cmd/e2e/e2e directly obviously won't work, but that was never
supported anyway
- If the GINKGO_PARALLEL environment variable is set to y, then ginkgo
will run test specs in parallel. (Currently defaults to n, since some
tests are broken in this mode.)
Right now, our doc generation scripts notice if you make changes and
don't regen docs, don't include new docs, etc. But they miss it if your
changes should have REMOVED a doc. Both kubectl-apiversion and
kubectl-clusterinfo should have been removed, but weren't.
This patch starts tracking all generated doc files and will cause
problems if files should be removed and aren't.
We allow cgo when available since some of our server components (mainly
Kubelet) rely on it. The current driving factor for the static binaries
was building Docker images from scratch. We will instead use images
built from small busybox bases.
Fixes#5517
It includes many of the server components which we only expect to be
able to build in the server target architectures, but it is included in
the client build targets.
`hack/test-cmd.sh` contains a list of `kubectl` commands to test.
However, the effect of the execution of those commands was not systematically
checked. As a consequence, it was possible to break a command without breaking
those tests.
This commit surrounds every single `kubectl` command with assertions on
pre and post condition.
This gets rid of bashWrap() and of running bash -s passing it a script
on stdin. Also get rid of most of the runBash*() helpers.
In particular, `go run hack/e2e.go -build` will now work when it needs
to ask whether it is OK to download a large docker image, it used to
fail since stdin was not available for the bash script using the `read`
command.
Tested by running a full `hack/e2e-test.sh` run including the build
stage without the docker image loaded, which used to fail before this
change.
This is another step in removing external dependencies of the Go e2e tests.
Remove references to this file on list of files required to run e2e tests.
Also use an unique name for the pod, so that failure in cleanup of a
previous run should not break a new run with a name conflict.
Tested by running cmd/e2e -t TestPodUpdate against an API server in GCE.
This is another step in removing external dependencies of the Go e2e tests.
Also remove other references to this file.
Tested by running cmd/e2e -t TestKubeletSendsEvent against an API server in GCE.
Add test artifacts to the build. This lets you do:
tar -xzf kubernetes.tar.gz
tar -xzf kubernetes-test.tar.gz
cd kubernetes
go run ./hack/e2e.go -up -test -down
without having a git checkout.
apiserver becomes kube-apiserver
controller-manager -> kube-controller-manager
scheduler and proxy similarly.
Only thing I promise is that right now hack/build-go.sh and
build/release.sh exit with 0. That's it. Who knows if any of this
actually works....
It's actually used unset a couple of lines after this patch, but this
just resolves it in one place instead of calling
$(kube::golang::host_platform) twice.
Because of Fedora's crazy packaging guidelines, go install basically
isn't allowed. So allow us to use go build instead of go install.
test-integration also needs to accept the flag to be able to pass it
along to the build function.
* Rewrite a bunch of the hack/ directory with modular reusable bash libraries.
* Have 'build/*' build on 'hack/*'. The stuff in build now just runs hack/* in a docker container.
* Use a docker data container to enable faster incremental builds.
* Standardize output to _output/{local,dockerized}/bin/OS/ARCH/*. This regularized placement makes cross compilation work.
* Move travis specific scripts under hack/travis
With new dockerized incremental builds, I can do a no-op `make quick-release` in ~30s. This is a significant improvement.
