Commit Graph

175 Commits

Author SHA1 Message Date
Claudiu Belu
6d3a18b0be test images: Image Promoter sed fix
The image used by the Image Promoter (gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4)
is based on busybox, and thus, the sed binary is actually busybox. image-util.sh calls
kube::util::ensure-gnu-sed several times, which ensures that a GNU sed binary exists
(it checks by greping GNU in its --help output). Obviously, it won't match the busybox sed
binary. But the sed usage in image-util.sh is fairly simple, and the busybox sed is sufficient.

This was previously fixed in: #87188, but it was reverted by #87653 as it was failing
on Mac (sed does not exist). This commit fixes that issue as well.
2020-02-18 19:58:35 -08:00
JieJhih Jhang
8bd9c994ca fix command variable exited with status 1 2020-01-30 12:52:09 +08:00
Claudiu Belu
4c51eb9063 test images: Image Promoter fixes
Prior to the Image Centralization part 4 (https://github.com/kubernetes/kubernetes/pull/81170),
a PR merged that enables the Image Promoter to run on the k/k test images.

The Image Promoter currently only builds the Conformance-related images, but the
Image Centralization part 4 centralized some of those images into agnhost, so they
need to be removed from the conformance_images list.

Additionally, https://github.com/kubernetes/kubernetes/pull/81226 proposes mounttest-user
image to be removed, and RunAsUser to be used in tests instead.

The image used by the Image Promoter (gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4)
is based on busybox, and thus, the sed binary is actually busybox. image-util.sh calls
kube::util::ensure-gnu-sed several times, which ensures that a GNU sed binary exists
(it checks by greping GNU in its --help output). Obviously, it won't match the busybox sed
binary. But the sed usage in image-util.sh is fairly simple, and the busybox sed is sufficient.

Bumps image versions for: jessie-dnsutils, nonewprivs, resource-consumer, sample-apiserver. These
images are included in the conformance_images that are being built by the Image Promoter, so
we're bumping them just to make sure we're not breaking anything and cause all the CIs to fall.
We're going to bump the image versions used in tests in a subsequent PR. The image version was not
bumped for: agnhost, kitten, nautilus, as they were already bumped by the Image Centralization part 4
PR.
2020-01-06 09:08:51 -08:00
howard
6ca5e0ed4d local-up-cluster kube-proxy terminated error
When using hack/local-up-cluster.sh deploy local cluster, it
failed with following message "kube-proxy terminated unexpectedly"
and "Failed to retrieve node info: nodes "127.0.0.1" not found" in
kube-proxy.log.

The root reason for this error is miss boot order of kubernetes
services in local-up-cluster.sh, kube-proxy and kubectl daemon.

When starting kube-proxy, it would check node information. And
these information are collected by kubelet daemon. However, in
the shell script, kube-proxy service start before kubelet daemon.

This patch changed the boot order of kubelet daemon and kube-proxy
and check if node stats ready for kube-proxy start.

Signed-off-by: Howard Zhang <howard.zhang@arm.com>
2019-09-16 10:36:42 +08:00
Davanum Srinivas
adbd1800c8 Drop cloud-controller-manager artifacts from k/k release 2019-08-06 13:51:27 -04:00
Benjamin Elder
4aaa7b4da2 document KUBE_ROOT in util.sh 2019-06-24 11:43:04 -07:00
Benjamin Elder
29b7b2c4f8 add kube::util::list_staging_repos 2019-06-24 11:43:04 -07:00
Benjamin Elder
7b0ea99a3e copy kube::release:md5 to kube::util::md5 2019-06-23 12:08:05 -07:00
Christoph Blecker
eaafa5952f
Don't use mapfile as it isn't bash 3 compatible 2019-05-06 15:55:12 -07:00
Jordan Liggitt
2ea3cbdcbc Update hack scripts to use go mod 2019-04-03 10:19:39 -04:00
Jordan Liggitt
7f246d461b Remove third_party/forked/godep 2019-04-03 10:19:39 -04:00
Ismo Puustinen
ecb6d13253 hack/lib/util.sh: fix empty array expansion with bash 3. 2019-03-05 21:34:08 +02:00
Ismo Puustinen
1e34d9df7d hack/lib/util.sh: replace sed with bash replace.
Test script:

  #!/bin/bash

  rev1="foo"
  rev2="\"bar\""
  rev3="'bar'"

  newrev1="${rev1//[\'\"]}"
  newrev2="${rev2//[\'\"]}"
  newrev3="${rev3//[\'\"]}"

  oldrev1=$(echo "${rev1}" | sed "s/['\"]//g")
  oldrev2=$(echo "${rev2}" | sed "s/['\"]//g")
  oldrev3=$(echo "${rev3}" | sed "s/['\"]//g")

  echo "$newrev1 vs. $oldrev1"
  echo "$newrev2 vs. $oldrev2"
  echo "$newrev3 vs. $oldrev3"

expected output:

  foo vs. foo
  bar vs. bar
  bar vs. bar
2019-03-04 23:23:20 +02:00
Ismo Puustinen
0078fce5bd hack/lib/util.sh: don't implicitly convert "find" results into array.
Also fix array item comparison. Test script for the comparison change:

  #!/bin/bash

  staging_apis=(extensions/v1beta1 extensions/v1 extensions/v1alpha)
  group_versions=(v1 extensions/v1beta1 extensions/v1 extensions.k8s.io/v1)

  for group_version in ${group_versions[@]}; do
    # original code
    if [[ " ${staging_apis[@]} " =~ " ${group_version/.*k8s.io/} " ]]; then
      echo "orig: vendor/k8s.io/api/${group_version/.*k8s.io/}"
    fi
    # new code
    for api in ${staging_apis[@]}; do
      if [[ "${api}" = "${group_version/.*k8s.io/}" ]]; then
        echo "new: vendor/k8s.io/api/${group_version/.*k8s.io/}"
      fi
    done
  done

Expected output:

  orig: vendor/k8s.io/api/extensions/v1beta1
  new: vendor/k8s.io/api/extensions/v1beta1
  orig: vendor/k8s.io/api/extensions/v1
  new: vendor/k8s.io/api/extensions/v1
  orig: vendor/k8s.io/api/extensions/v1
  new: vendor/k8s.io/api/extensions/v1
2019-03-04 23:23:20 +02:00
Ismo Puustinen
edd806330a hack/lib/util.sh: mark variables to be used in a sourcing context.
"Decorate" the variables with a no-op function to prevent shellcheck
from complaining that they are not being used. This method provides
visibility to which variables are supposed to be used in a sourcing
script compared to just disabling the warning.
2019-03-04 23:23:20 +02:00
Ismo Puustinen
24b5c67723 hack/lib/util.sh: various shellcheck-reported cleanups.
Use "command -v" instead of "which".  Also remove the redirections,
since "command -v" does not return an error message if the command isn't
found. Also use "read -r" instead of "read" and quote variables
properly. Do some error handling if "pushd" or "popd" fail. Read values
properly into arrays.

However, one shellcheck error is ignored in trap mechanism.  The logic
in trap_add function requires the trap command to be expanded when run.
Just storing the variable into trap doesn't work. Add a shellcheck
disable directive to ignore the error.

An alternative to ignoring could be tricking shellcheck with:

  trap ''"${new_cmd}" "${trap_add_name}"
2019-03-04 23:18:54 +02:00
Dr. Stefan Schimanski
2393799e2e hack/update-openapi-spec.sh: normalize indention of spec through jq 2019-02-11 13:16:36 +01:00
Kubernetes Prow Robot
92db54cc53
Merge pull request #73335 from oomichi/cleanup-hack
Add check-file-in-alphabetical-order for cleanup
2019-02-07 18:24:41 -08:00
Bin Lu
44a36ea36e Bug fix: ./hack/local-up-cluster.sh is not work on Arm64 platform
Signed-off-by: Bin Lu <bin.lu@arm.com>
2019-01-30 09:49:46 +08:00
Kenichi Omichi
c32d1acbb9 Add check-file-in-alphabetical-order for cleanup
Both verify-golint.sh and verify-shellcheck.sh have the same logic
which checks failure_file in alphabetical order.
In addition, we'd like to add another script which requires the
same logic. So this add a common function for cleanup.
2019-01-26 02:09:06 +00:00
Lubomir I. Ivanov
54e8d73920 docs: stop tracking placeholder documentation
The placeholder documentation introduces a couple of problems:
- it complicates the contributor-experience (forces the CI to run
N times before the contributor finds out that they need to call an .sh
script and include certain files from docs/)
- it forces CLI related pull requests for tools like kubeadm and kubectl
to require top level approval from docs/OWNERS as such PRs still need
to touch the .generated_docs file

Stop tracking the placeholder documentation by applying the
following actions:
- remove the utility set-placeholder-gen-docs()
- make verify-generated-docs.sh only generate in a temporary folder
and not match .generated_docs
- mark generate-docs.sh as an alias for update-generated-docs.sh
- remove all current placeholder files in docs folders admin, man,
user-guide, yaml
- ignore the above folders and .generated_docs in a .gitignore file
2019-01-26 01:07:10 +02:00
Roy Lenferink
b18bc2ea79 Improved some more bash script variable definitions 2019-01-21 23:11:58 +01:00
Kubernetes Prow Robot
4a6ac500fb
Merge pull request #72807 from dixudx/hack_host_arch
add method to get host os/arch in hack scripts
2019-01-17 13:51:24 -08:00
Di Xu
88740dfb9e add method to get host os/arch in hack scripts 2019-01-17 23:44:58 +08:00
Jordan Liggitt
9229399bd6 Remove build/verify scripts for swagger 1.2 API docs, API server swagger ui / swagger 1.2 config 2019-01-15 13:33:06 -05:00
Sen Lu
d6801b8211 Diff between PULL_BASE_SHA and HEAD when detecting file changes from
Prow
2019-01-08 11:17:00 -08:00
Christoph Blecker
b1aae87327
Don't use hermetic go environment for installing godep
This is unneeded, and causes this build environment to persist throughout the lifetime of any godep script.
2018-11-28 16:31:25 -08:00
Christoph Blecker
35cdbdf16c
Use ensured godep instead of relying on PATH 2018-11-07 11:02:01 -08:00
Christoph Blecker
c0c62821d7
Install ensured godep inside hermetic GOPATH 2018-11-06 12:23:27 -08:00
Tim Allclair
275212bbc9 Fix find-binary to locate bazel e2e tests 2018-10-08 17:20:02 -07:00
Katharine Berry
8fe6467013 Improve bash formatting. 2018-08-30 16:18:15 -07:00
Katharine Berry
aa2a7d001a Build relevant binaries with coverage. 2018-08-29 14:48:24 -07:00
Benjamin Elder
dc9def6f93 Revert "Update kube::util::ensure-cfssl"
This reverts commit 7a10073e4a.
2018-08-10 19:01:17 -07:00
Christoph Blecker
7a10073e4a
Update kube::util::ensure-cfssl 2018-08-09 14:42:57 -07:00
Jordan Liggitt
cc1a7735e8
move easyrsa check to gce prereq check 2018-08-06 19:56:10 -04:00
Aaron Crickenberger
a9aa892177 Exit gce kube-up.sh early if openssl is LibreSSL
macOS has an openssl binary, but it's actually LibreSSL, which
doesn't play well with the easyrsa script that cluster/gce/util.sh
uses to generate certs

Instead of waiting until we generate certs to discover easyrsa doesn't
work, consider openssl a prereq for gce, and include a check for the
version string starting with OpenSSL

Also, mirror kube-up.sh's "... calling" output in kube-down.sh
2018-07-26 14:48:51 -07:00
Brandon Dalla Rosa
510f6b8916 Adding IKS functionality to kubemark 2018-07-23 11:42:22 -04:00
Christoph Blecker
50fd906f74
Update vendored tool go install location to use GOPATH 2018-06-25 15:45:14 -07:00
ruicao
95c232ee07 Typo fix: toto -> to 2018-06-12 23:12:39 +08:00
Matthias Bertschy
9b15af19b2 Update all script to use /usr/bin/env bash in shebang 2018-04-19 13:20:13 +02:00
Kubernetes Submit Queue
22440e1576
Merge pull request #62037 from cblecker/godep-bump
Automatic merge from submit-queue (batch tested with PRs 61959, 62037). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump godep version to v80

**What this PR does / why we need it**:
Update the minimum godep, to v80 (supposed to be the final version).

**Release note**:
```release-note
NONE
```
2018-04-04 02:45:08 -07:00
Kubernetes Submit Queue
3ea4e449c3
Merge pull request #57600 from thockin/vendor-gazelle
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Vendor gazelle and kazel

Rather that relying on upstream git repos that can break, vendor it all.  These are NOT head of tree, respectively - they are some backrev forms that were previously being used.

```release-note
NONE
```
2018-04-02 12:13:06 -07:00
Christoph Blecker
ba71e6307a
Bump godep version to v80 2018-03-30 15:54:52 -07:00
Tim Hockin
4a5e554704 Don't change GOPATH or PATH in a script lib 2018-03-29 17:01:32 -07:00
Tim Hockin
f72484df88 Remove kube::util::go_install_from_commit 2018-03-29 17:01:32 -07:00
Davanum Srinivas
1f4246957f Allow curl --max-time to be configurable
In nested virutalized environments, the 1 second max-time is too
low. Just bumping up WAIT_FOR_URL_API_SERVER does not work unless
we bump up the max-time too. Let us just make it configurable like
so folks can customize to their environment.
2018-03-28 10:21:15 -04:00
hzxuzhonghu
f12647e16d pkg/api/unversioned related cleanup 2018-03-13 17:20:16 +08:00
Ismo Puustinen
afc459758e hack/lib/util.sh: remove shadowed case statements.
The topmost case statements shadow those coming after them. Since they
are never run, remove them.
2018-02-26 16:46:19 +02:00
Ismo Puustinen
f03730682c hack/lib/util.sh: do not iterate over ls output.
Use globs to make the code more resilient for directory names with
special characters or whitespace.
2018-02-26 16:46:19 +02:00
Ismo Puustinen
4a82a011a0 hack/lib/util.sh: improve staging api finding.
Do not bother with pushd/popd construct -- there is no need to go back
to original directory in a subshell. Use "find --exec" instead of "find
| xargs" to handle special file names better.
2018-02-26 16:46:19 +02:00