we should only use this env var for `arm`, since `arm64` is fully
supported by etcd folks, let us drop this!
(ex - https://github.com/etcd-io/etcd/releases/tag/v3.5.6)
ppc64le comment should be dropped as well
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Several files that I maintain (primarily logcheck.conf right now, but
I'm also working on golangci-lint) are in this directory. It would be
useful to have approval rights to avoid delays.
The previous approach was based on the observation that some Prow jobs use the
--report-dir parameter instead of the E2E_REPORT_DIR env variable. Parsing the
command line was necessary to use the --json-report and --junit-report
parameters.
But that is complex and can be avoided by triggering the creation of complete
reports in the E2E test suite. The paths are hard-coded and relative to the
report directory to keep the code simple.
There was a report that k8s-triage started processing more data after
6db4b741dd was merged. It's unclear whether
that was because of the new <report-dir>/ginkgo_report.xml file. To avoid
this potential problem, the reports are now in a "ginkgo" sub-directory.
While at it, error checking gets enhanced:
- Create directories at the start of
the suite and bail out early if that fails.
- *All* e2e suites using the framework do this, not just test/e2e.
- Added missing error checking of truncated JUnit report writing.
Bump golangci-lint version. This version
adds support for go1.20.
This consequently also bumps the version of
staticcheck. Note that this was changed to
use master to be compatible with 1.20. This
commit reverts back to using a tagged release.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Swagger "docs" are actually Go code, which is used by other codegen
tools, so if you really want to regen EVERYTHING, this is part of it and
sequence matters.
Calling update-codegen.sh with no arguments runs all the functions in
definition order. Client-generation depends on protobuf, so protobuf
codegen needs to be near the beginning.
Also add some debug output for protobuf generation.
Also hide some old, verbose debug output.
Now update-generated-proto-bindings rules all the api.pb.go generation.
Running this shows no delta on the runtime.pb.go
This exposes an issue in how protoc is called for protos that specify
`go_package` which is fixed here.
Not all of our protos specify that option (even though it is
recommended), which will be fixed subsequently.
Each of these scripts is basically identical, and all were too brittle.
Now they should be more resilient and easier to manage. The script
still needs to be updated if we add new ones, which I do not love.
More cleanup to follow.
The `find` tool has hard to comprehend syntax and does not consider
things excluded by .gitignore. I keep tripping over this in my own
repos, where I have __stuff which gets found.
This converts update-codegen to use `git ls-files` in a seemingly
equivalent way (`-cmo --exclude-standard`). I verified it finds the
same set of files as before.
This also drops some obsolete filtering.
Also hide grep errors for not-found files, which can happen if a file is
removed but git ls-files still knows it.
Re-running update-codegen shows no diffs.
This will make subsequent changes easier.
Don't just grep for DO NOT EDIT - anchor it in something that looks like
a comment and alone on a line.
Also ignore __* dirs
Prevent it from triggering on update-generated-swagger-docs (hack, but
better than before)
The env vars are needed until go workspaces lands, then it can get
simpler.
Downsides to this:
1) If you don't call kube::golang::setup_env, it might work but will
just splat results somewhere
2) The resultant binaries are not in _output/bin but instead in the
phony GOPATH/bin (which setup_env puts in PATH)
hack/pin-dependency.sh github.com/moby/ipvs v1.1.0
- go to a fixed tag for `vishvananda/netns`
- no more references to `pkg/errors`
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
This came up when updating go-oidc. After updating go-oidc (with its
dependency tree), cloud.google.com/go was no longer used as a package
import, but still listed in the module dependency graph; as a result,
"go mod vendor" no longer pulled in cloud.google.com/go itself, but
update-vendor-licenses.sh still wanted a license file for it since it
appeared in the list of modules.
This scenario is already supposed to be handled: when a module doesn't
contain any *files* as first-level content, if the number of
subdirectories it contains *equals* the number of submodules it
contains (excluding itself), the module is skipped. This fails for
cloud.google.com/go because several submodules are included in the
module dependency graph but aren't actually used, and therefore not
vendored.
Updating the test to check that the number of subdirectories is less
than or equal to the number of expected submodules fixes this.
The correct fix would be to process the submodules first, keeping a
note of which ones really have content, then check that the top-level
module only contains subdirectories corresponding to those modules;
but it's not clear to me that this is worth the effort (especially in
a shell script).
Signed-off-by: Stephen Kitt <skitt@redhat.com>
From the warning message that ginkgo now emits:
--slow-spec-threshold is deprecated --slow-spec-threshold has been deprecated
and will be removed in a future version of Ginkgo. This feature has proved
to be more noisy than useful. You can use --poll-progress-after, instead, to
get more actionable feedback about potentially slow specs and understand
where they might be getting stuck.
We already use --poll-progress-after.
This is a fix for 104aab81a4: because
the default was not set for E2E_TEST_DEBUG_TOOL, all parameters were always
also passed to the E2E suite.
That wasn't wrong for the parameters so far, but breaks when using something
like --output-dir which is only understood by the CLI.
If the script was called with no arguments, it passed "${@:-}" to the suite,
which expands to one empty argument. That's not right, "${@}" should be used
instead because it expands to nothing when empty.
Every ginkgo callback should return immediately when a timeout occurs or the
test run manually gets aborted with CTRL-C. To do that, they must take a ctx
parameter and pass it through to all code which might block.
This is a first automated step towards that: the additional parameter got added
with
sed -i 's/\(framework.ConformanceIt\|ginkgo.It\)\(.*\)func() {$/\1\2func(ctx context.Context) {/' \
$(git grep -l -e framework.ConformanceIt -e ginkgo.It )
$GOPATH/bin/goimports -w $(git status | grep modified: | sed -e 's/.* //')
log_test.go was left unchanged.
The kubectl-convert binary itself only depends on the following runtime
libraries when linking dynamically:
```
> ldd _output/bin/kubectl-convert
linux-vdso.so.1 (0x00007ffef0786000)
libpthread.so.0 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libpthread.so.0 (0x00007f5f4ac25000)
libdl.so.2 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libdl.so.2 (0x00007f5f4ac20000)
libc.so.6 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libc.so.6 (0x00007f5f4aa00000)
/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/ld-linux-x86-64.so.2 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib64/ld-linux-x86-64.so.2 (0x00007f5f4ac2c000)
```
We now move kubectl-convert to become a static binary as well to achieve
maximum portability.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
The kube-aggregator itself only depends on the following runtime
libraries when linking dynamically:
```
> ldd _output/bin/kube-aggregator
linux-vdso.so.1 (0x00007fff1616f000)
libpthread.so.0 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libpthread.so.0 (0x00007fad9339a000)
libc.so.6 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libc.so.6 (0x00007fad93000000)
/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/ld-linux-x86-64.so.2 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib64/ld-linux-x86-64.so.2 (0x00007fad933a1000)
```
We now move the kube-aggregator to become a static binary as well to
achieve maximum portability.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
We need to better catch problems like the one we fixed in:
7b6a3d6a33
Essentially a copy-paste error where had a branch pointing to staging/src/k8s.io/code-generator instead of staging/src/k8s.io/apimachinery
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
The call to `i18n.LoadTranslations` needs to occur on init so
that package-level variables that call `i18n.T()` to initialize
their values will be able to receive the translated string.
Added new integration tests to test help output translation.
github.com/container-orchestrated-devices/container-device-interface and
github.com/opencontainers/runtime-tools depend on some problematic packages in
code that isn't used by Kubernetes. Explicitly listing those dependencies
ensures that they do not get pulled in accidentally.
Dependencies need to be updated to use
github.com/container-orchestrated-devices/container-device-interface.
It's not decided yet whether we will implement Topology support
for DRA or not. Not having any toppology-related code
will help to avoid wrong impression that DRA is used as a hint
provider for the Topology Manager.
This adds a new resource.k8s.io API group with v1alpha1 as version. It contains
four new types: resource.ResourceClaim, resource.ResourceClass, resource.ResourceClaimTemplate, and
resource.PodScheduling.
This was originally added in the early days of go.mod in order to ensure that the root k8s.io/kubernetes
go.mod file reflected all the versions of dependencies we were using, and to ensure that go commands didn't spuriously
increment these versions.
With module pruning in go1.17+ and go defaulting to mod=readonly mode in go1.16+, this is no longer necessary.
Most parameters can be passed to both the CLI and the suite, but some
(for example, --ginkgo.slow-spec-threshold) had no effect when only
passed to the suite.
DOCKER is otherwise used to be the command name (perhaps podman), but we were conflating DOCKER_OPTS in kube::util::ensure_docker_daemon_connectivity.
Split out docker opts.
This fixes shellcheck warning that docker is assigned an array and then a string in some scripts.
Adding the ability to ignore no schedule flags in testing.
Specifically node.cloudprovider.kubernetes.io/uninitialized:NoSchedule
Fix shellcheck complaint.
The `make` rules which auto-generate some of our API stuff are
incredibly baroque, and hard to maintain. They were originally added on
the assumption that we would stop checking generated files into git.
Since then we have moved away from that goal, and the worst problems
with generated files have been resolved.
Reasons to kill this:
* It is slow on every build, as opposed to just being slow when running
the generators. It is even slow to calculate that there's nothing to
update.
* Most development work doesn't involve changing APIs.
* It only covers about half (or less) of the generated code, and making
it cover more would be even slower.
* Approximately 1 person knows how this all works.
* We have CI to make sure changes do not get merged without updating
this code.
* We have corner cases where this does the WRONG thing and tracking
those down is ugly and hard in perpetuity.
So this commit puts all the same logic that WAS in the
Makefile.generated_files into update-codegen.sh.
I do not love this script, especially WRT sub-packages, but I am trying
not to boil the ocean. I hope to follow up with some more cleanups over
time.
I have tested this manually and with the scripts and it still seems to
catch errors properly.
This includes a change to kube::util::read-array to make it not unset
variables and not over-write non-array variables.
A staging repo which just got created with only the doc.go file in it won't
have any dependencies yet, which caused the script to fail because the
dependency files didn't get created:
+++ [0926 14:33:22] go.mod: tidying
cat: /tmp/update-vendor.1VTv/group_replace.ZbIT/go.mod.require_direct.tmp: No such file or directory
!!! [0926 14:33:23] Call tree:
!!! [0926 14:33:23] 1: hack/update-vendor.sh:354 group_directives(...)
- Moves kms proto apis to the staging repo
- Updates generate and verify kms proto scripts to check staging repo
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
The new support in ginkgo for progress reports while a test runs dumps
information about where a test is stuck when it runs too long. This can provide
additional insights into what the test is waiting for.
For the Kubernetes jobs using ginkgo-e2e.sh, such dumps are now enabled after
300 seconds and then get repeated every 20 seconds. The initial delay is
intentionally the same as for warning about a slow test. The rationale is that
such test runtimes are unexpected and may need further information to diagnose
why they are slow.
With -ginkgo.source-root, Ginkgo is able to locate the Kubernetes source code
and display small source code snippets for functions that are related to the
test, determined through a heuristic that assumes that all files under the test
suite are for the tests in it.
Set intercept mode to none will help to reveal more information when the
test hangs,
- https://github.com/onsi/ginkgo/issues/970
or circumvent cases where the code grabbing the stdout/stderr pipe
is not under the framework control and may cause hangs,
- https://github.com/onsi/ginkgo/issues/851
The flag `output-interceptor-mode` is set to `none` as we were trying to
figure out of the rootcase of the test flaky, it's only intended for debugging.
- https://github.com/kubernetes/kubernetes/issues/111086
But this set also has some side effect, since it will turn off stdout/stderr
capture completely, any output to stdout/stderr will be lost.
Now that the root cause is not caused by Ginkgo bump nor how the intercept
mode was set, we'd better to follow the default value.
Signed-off-by: Dave Chen <dave.chen@arm.com>
