In the KEP 1710 we promised to have all SELinux metrics with access mode
label, so cluster admin is able to distinguish when RWOP volumes are
failing to mount (-> SELinuxMountReadWriteOncePod feature gate must be
disabled) or volumes with any other access modes are failing (->
SELinuxMount feature gate must be disabled).
Adding the label to kubelet is quite straightforward, there were some
changes needed in the e2e test. Now grabMetrics() collects values of all
SELinux related metrics with all labels. It only skips unrelated volume
plugins. And waitForMetricIncrease gets metric with all labels on input, so
it can check that say RWOP metric increased and RWX one did not.
This is not relevant for namespaced objects, but matters for the cluster-scoped
ResourceClass during unit testing. This works right now because there is only
one such unit test, but will fail when adding a second one.
Instead of passing a boolean flag down into all functions where it might be
needed, it's now a context value.
This change removes the environment variable set during
the Alpha phase and prepares the feature for beta graduation.
xref: https://github.com/kubernetes/enhancements/pull/4419
Signed-off-by: Sohan Kunkerkar <sohank2602@gmail.com>
The return type of ktesting.NewTestContext is now a TContext. Code
which combined it WithCancel often didn't compile anymore (cannot overwrite
ktesting.TContext with context.Context). This is a good thing because all of
that code can be simplified to let ktesting handle the cancelation.
1) Fail if we can't read critical env vars
2) Don't rely on KUBE_ROOT env var when `go list` works
3) Don't rely on GOOS env var when `go env` works
4) Don't quietly ignore "can't read" errors
Once we stop ignoring errors, some tests fail for real (and should
always have failed).
The "Imported k8s.io/staging constant" test seems to not be allowed at
all anymore. Han said to nix it and he'd look async.
Oversall this test is dodgy. You REALLY can't glue strings together and
expect valid Go module paths. We should consider a deeper rework.
Instead of walking paths ourselves, just let Go's packages library do
it. This is a slight CLI change - it wants "./foo" rather than "foo".
This also flagged a few things which seem to be legit failures.
Prior to this it would download Go, setup the environment, then (maybe)
download Go again.
I renamed verify_go_version to be internal and fixed all callers.
The YAML files get decoded into an unstructured object, without validation, and
then sent to the apiserver with a generic client. The default behavior is to
issue a warning to the client, which gets logged by client-go. What we want
instead is an error that causes the test to fail in a clean way right at the
beginning.
This makes "new" and "old" setup_env functions. In subsequent commits,
all callers of the "old" form will be fixed, and the "new" will be
renamed back.
The old and new functions diff:
```diff
--- /tmp/a 2023-12-14 09:02:57.804092696 -0800
+++ /tmp/b 2023-12-14 09:03:09.679999585 -0800
@@ -1,4 +1,4 @@
-kube::golang::old::setup_env() {
+kube::golang:🆕:setup_env() {
kube::golang::verify_go_version
# Set up GOPATH. We have tools which depend on being in a GOPATH (see
@@ -7,9 +7,9 @@
# Even in module mode, we need to set GOPATH for `go build` and `go install`
# to work. We build various tools (usually via `go install`) from a lot of
# scripts.
- # * We can't set GOBIN because that does not work on cross-compiles.
- # * We could use `go build -o <something>`, but it's subtle when it comes
- # to cross-compiles and whether the <something> is a file or a directory,
+ # * We can't just set GOBIN because that does not work on cross-compiles.
+ # * We could always use `go build -o <something>`, but it's subtle wrt
+ # cross-compiles and whether the <something> is a file or a directory,
# and EVERY caller has to get it *just* right.
# * We could leave GOPATH alone and let `go install` write binaries
# wherever the user's GOPATH says (or doesn't say).
@@ -20,16 +20,6 @@
#
# Eventually, when we no longer rely on run-in-gopath.sh we may be able to
# simplify this some.
- local go_pkg_dir="${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}"
- local go_pkg_basedir
- go_pkg_basedir=$(dirname "${go_pkg_dir}")
-
- mkdir -p "${go_pkg_basedir}"
-
- # TODO: This symlink should be relative.
- if [[ ! -e "${go_pkg_dir}" || "$(readlink "${go_pkg_dir}")" != "${KUBE_ROOT}" ]]; then
- ln -snf "${KUBE_ROOT}" "${go_pkg_dir}"
- fi
export GOPATH="${KUBE_GOPATH}"
# If these are not set, set them now. This ensures that any subsequent
@@ -40,24 +30,10 @@
# Make sure our own Go binaries are in PATH.
export PATH="${KUBE_GOPATH}/bin:${PATH}"
- # Change directories so that we are within the GOPATH. Some tools get really
- # upset if this is not true. We use a whole fake GOPATH here to collect the
- # resultant binaries.
- local subdir
- subdir=$(kube::realpath . | sed "s|${KUBE_ROOT}||")
- cd "${KUBE_GOPATH}/src/${KUBE_GO_PACKAGE}/${subdir}" || return 1
-
- # Set GOROOT so binaries that parse code can work properly.
- GOROOT=$(go env GOROOT)
- export GOROOT
-
# Unset GOBIN in case it already exists in the current session.
# Cross-compiles will not work with it set.
unset GOBIN
- # This seems to matter to some tools
- export GO15VENDOREXPERIMENT=1
-
- # Disable workspaces
- export GOWORK=off
+ # Explicitly turn on modules.
+ export GO111MODULE=on
}
```
Result: `make` works for k/k:
```
$ make kubectl
+++ [1211 11:07:31] Building go targets for linux/amd64
k8s.io/kubernetes/cmd/kubectl (static)
$ make WHAT=./cmd/kubectl/
+++ [1211 11:08:19] Building go targets for linux/amd64
k8s.io/kubernetes/./cmd/kubectl/ (non-static)
$ make WHAT=k8s.io/kubernetes/cmd/kubectl
+++ [1211 11:08:52] Building go targets for linux/amd64
k8s.io/kubernetes/cmd/kubectl (static)
```
Result: `make` works for staging by package:
```
$ make WHAT=k8s.io/api
+++ [1211 11:11:37] Building go targets for linux/amd64
k8s.io/api (non-static)
```
Result: `make` fails for staging by path:
```
$ make WHAT=./staging/src/k8s.io/api
+++ [1211 11:12:44] Building go targets for linux/amd64
k8s.io/kubernetes/./staging/src/k8s.io/api (non-static)
cannot find module providing package k8s.io/kubernetes/staging/src/k8s.io/api: import lookup disabled by -mod=vendor
(Go version in go.work is at least 1.14 and vendor directory exists.)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44] 1: /home/thockin/src/kubernetes/hack/lib/golang.sh:850 kube::golang::build_some_binaries(...)
!!! [1211 11:12:44] 2: /home/thockin/src/kubernetes/hack/lib/golang.sh:1012 kube::golang::build_binaries_for_platform(...)
!!! [1211 11:12:44] 3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [1211 11:12:44] Call tree:
!!! [1211 11:12:44] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:96: all] Error 1
```
Result: `make test` fails:
```
$ make test WHAT=./cmd/kubectl
+++ [1211 11:13:38] Set GOMAXPROCS automatically to 6
+++ [1211 11:13:38] Running tests without code coverage and with -race
cmd/kubectl/kubectl.go:25:2: cannot find package "k8s.io/client-go/plugin/pkg/client/auth" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/plugin/pkg/client/auth (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/client-go/plugin/pkg/client/auth (from $GOPATH)
cmd/kubectl/kubectl.go:20:2: cannot find package "k8s.io/component-base/cli" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/component-base/cli (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/component-base/cli (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/component-base/cli (from $GOPATH)
cmd/kubectl/kubectl.go:21:2: cannot find package "k8s.io/kubectl/pkg/cmd" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd (from $GOPATH)
cmd/kubectl/kubectl.go:22:2: cannot find package "k8s.io/kubectl/pkg/cmd/util" in any of:
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util (vendor tree)
/home/thockin/sdk/gotip/src/k8s.io/kubectl/pkg/cmd/util (from $GOROOT)
/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubectl/pkg/cmd/util (from $GOPATH)
make: *** [Makefile:191: test] Error 1
```
Previously, SELinuxMount started two pods and in laboratory conditions
waited for the second Pod to get stuck (because of RWOP) and observed
kubelet behavor after the test unstuck them (i.e. deleted the first Pod).
When testing RWO volumes, the second Pod may not get stuck, it may actually
run. So update the tests to allow the second Pod to run and start counting
CSI calls for it earlier.
very simple tests to check that the node controllers removes the taint.
Test will be used later for add more coverage without having to depend
on e2e tests for validating complex scenarios and combinations of flags
and controllers.
Change-Id: I55ce01488352cce9d673585e7959678c2bfb5924
- Increase the global level for broadcaster's logging to 3 so that users can ignore event messages by lowering the logging level. It reduces information noise.
- Making sure the context is properly injected into the broadcaster, this will allow the -v flag value to be used also in that broadcaster, rather than the above global value.
- test: use cancellation from ktesting
- golangci-hints: checked error return value
Extending the duration and the allowed delta in f6682370b1 was still not enough
to make the unit test run reliably in pull-kubernetes-unit.
Now it uses the original, stricter timing again, but only when run locally. In
Prow (detected by checking the "CI" env variable), the duration check is
skipped.
The dead code was found with:
deadcode -test -filter=k8s.io/kubernetes/test/e2e/framework/... ./test/e2e ./test/e2e_node ./test/e2e_node ./test/e2e_kubeadm
See https://go.dev/blog/deadcode for an introduction.
This removes everything that was reported as dead code for
test/e2e/framework/providers.
* Forensic Container Checkpointing as described in KEP 2008 moves from
Alpha to Beta. This is corresponding code change.
* Adapt e2e test to handle
'(rpc error: code = Unimplemented desc = unknown method CheckpointContainer'
and
'(rpc error: code = Unimplemented desc = method CheckpointContainer not implemented)'
and
'(rpc error: code = Unknown desc = checkpoint/restore support not available)'
One error message is if the CRI implementation does
not implement the CRI RPC (too old) and the second is
if the CRI implementation does explicitly not support the feature.
The third error message can be seen if the container engine
explicitly disabled the checkpoint/restore support,
* As described in the corresponding KEP 2008 explicitly test for
disabled functionality.
* Extended test to look for the checkpoint kubelet metric.
* Extended test to look for the CRI error metric.
* Add separate sub-resource permission to control permissions on
the checkpoint kubelet API endpoint
Signed-off-by: Adrian Reber <areber@redhat.com>
When failing inside the `ginkgo.By` callback function, skipping intermediate
stack frames didn't work properly because `ginkgo.By` itself and other internal
code is also on the stack.
To fix this, the code which can fail now runs outside of such a
callback. That's not a big loss, the only advantage of the callback was getting
timing statistics from Ginkgo which weren't used in practice.
