Commit Graph

20744 Commits

Author SHA1 Message Date
Tim Hockin
f772410082 Make code-gen subprojects work on gengo/v2 2024-02-29 22:06:38 -08:00
Tim Hockin
b0a70dec4a Fix go-to-protobuf wrt gengo/v2
There's some very fishy-smelling logic in here, but this commit is
trying to be as focused as possible.

The *.pb.go diffs are the "name" encoded in the descriptor.  The
descriptor blobs can be decoded by this program (thanks StackOverflow!):

```
package main

import (
	"bytes"
	"compress/gzip"
	"encoding/json"
	"fmt"
	"os"

	"io/ioutil"

	proto "github.com/golang/protobuf/proto"
	dpb "github.com/golang/protobuf/protoc-gen-go/descriptor"
)

func main() {
	m := map[string][]byte{
		"before": blobv1,
		"after":  blobv2,
	}
	arg := os.Args[1]
	dump(m[arg])
}

func dump(bytes []byte) {
	fd, err := decodeFileDesc(bytes)
	if err != nil {
		panic(err)
	}
	b, err := json.MarshalIndent(fd, "", "  ")
	if err != nil {
		panic(err)
	}
	fmt.Println(string(b))
}

// decompress does gzip decompression.
func decompress(b []byte) ([]byte, error) {
	r, err := gzip.NewReader(bytes.NewReader(b))
	if err != nil {
		return nil, fmt.Errorf("bad gzipped descriptor: %v", err)
	}
	out, err := ioutil.ReadAll(r)
	if err != nil {
		return nil, fmt.Errorf("bad gzipped descriptor: %v", err)
	}
	return out, nil
}

func decodeFileDesc(enc []byte) (*dpb.FileDescriptorProto, error) {
	raw, err := decompress(enc)
	if err != nil {
		return nil, fmt.Errorf("failed to decompress enc: %v", err)
	}

	fd := new(dpb.FileDescriptorProto)
	if err := proto.Unmarshal(raw, fd); err != nil {
		return nil, fmt.Errorf("bad descriptor: %v", err)
	}
	return fd, nil
}

var blobv1 = []byte{
	// insert proto "before" blob here
}

var blobv2 = []byte{
	// insert proto "after" blob here
}
```

Running this with "before" and "after" args, and diffing the output
yields something like:

```diff
--- /tmp/a	2023-12-23 23:57:04.748090836 -0800
+++ /tmp/b	2023-12-23 23:57:11.000040973 -0800
@@ -1,5 +1,5 @@
 {
-  "name": "k8s.io/kubernetes/vendor/k8s.io/api/admission/v1/generated.proto",
+  "name": "k8s.io/api/admission/v1/generated.proto",
   "package": "k8s.io.api.admission.v1",
   "dependency": [
     "github.com/gogo/protobuf/gogoproto/gogo.proto",
```
2024-02-29 22:06:37 -08:00
Tim Hockin
b961b39efa Fix client-gen wrt gengo/v2 2024-02-29 22:06:35 -08:00
Tim Hockin
5ddd6772a4 Fix applyconfiguration-gen wrt gengo/v2 2024-02-29 22:06:33 -08:00
Tim Hockin
0aa98ee8e3 Fix informer-gen wrt gengo/v2 2024-02-29 22:06:31 -08:00
Tim Hockin
b82527b6fb Fix lister-gen wrt gengo/v2 2024-02-29 22:06:30 -08:00
Tim Hockin
5475797f43 Make update-codegen conversion work on gengo/v2 2024-02-29 22:06:28 -08:00
Tim Hockin
ed35118664 Fix conversion-gen wrt gengo/v2 2024-02-29 22:06:27 -08:00
Tim Hockin
fd2ca0a344 Fix prerelease-lifecycle-gen wrt gengo/v2 2024-02-29 22:06:25 -08:00
Tim Hockin
514dd0e1f6 Convert all gengo references to gengo/v2
TO REPEAT:

    git ls-files ':!:vendor/k8s.io/gengo' ':!:gengo-v2' ':(glob)**/*.go' \
        | xargs grep -l 'k8s.io/gengo/' \
        | while read F; do \
            sed -i 's|k8s.io/gengo/\([^v]\)|k8s.io/gengo/v2/\1|' $F
        done
2024-02-29 22:06:20 -08:00
Tim Hockin
6f2f3735e0 Re-vendor latest kube-openapi and gengo/v2
./hack/pin-dependency.sh k8s.io/kube-openapi latest
./hack/pin-dependency.sh k8s.io/gengo/v2 latest
./hack/update-vendor.sh
2024-02-29 22:06:19 -08:00
Tim Hockin
b3dd724174 Remove dep on set-gen: gone in gengo/v2 2024-02-29 22:06:18 -08:00
Tim Hockin
7a7dff1eeb Change verify-imports to use new setup_env
Also clean up the tool a bit and drop vendor as an explicit thing to
be handled.
2024-02-29 22:06:16 -08:00
Kubernetes Prow Robot
6f42da171b Merge pull request #123565 from ah8ad3/update-help-text-kubectl-create-token
Implement to accept zero as a default value for kubectl create token duration
2024-02-29 21:51:41 -08:00
Kubernetes Prow Robot
5cf4fbe524 Merge pull request #123458 from aramase/aramase/i/min_jwt_payload
add min valid jwt payload to API docs for structured authn config
2024-02-29 15:11:40 -08:00
Kubernetes Prow Robot
0d50a398df Merge pull request #123598 from liggitt/remotecommand-cleanup
Remotecommand test flake cleanup
2024-02-29 13:40:48 -08:00
Jordan Liggitt
6c1a935da2 Keep streams from being set up after closeAllStreamReaders is called 2024-02-29 15:31:55 -05:00
Igor Velichkovich
a51a5b4622 kep-3716 GA, remove feature gate 2024-02-29 14:14:19 -06:00
Jordan Liggitt
26484df210 Make websocket heartbeat test timing less flaky 2024-02-29 15:10:28 -05:00
Jordan Liggitt
fc86811cbe Avoid logging binary junk for frame write failure 2024-02-29 15:09:34 -05:00
Anish Ramasekar
b57d7d6ad7 add min valid jwt payload to API docs for structured authn config
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
2024-02-29 11:08:39 -08:00
Siddhant Tibrewal
cb560a72e0 Improving Legibility of Kubectl describe configmap by adding new line to output 2024-02-29 18:46:27 +00:00
Sascha Grunert
e38531e9a2 Add image_id to CRI ContainerStatus message
There is a conversion function `ConvertPodStatusToRunningPod`, which
can override the `Container.ImageID` into a digested reference from the
`ContainerStatus` CRI RPC, which gets mapped from the `image_ref`:

411c29c39f/pkg/kubelet/container/helpers.go (L259-L292)

To avoid that failure case, we now introduce the same `image_id` into
the container status and let runtimes separate the fields.

We also add a note that the mapping from the digested reference of the
CRI to the Kubernetes Pod API `ImageID` field is intentional and should
not change.

Follow-up on: https://github.com/kubernetes/kubernetes/pull/123508

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
2024-02-29 12:41:55 +01:00
Sunil Shivanand
e6ed0f37c6 Add apiserver_watch_cache_read_wait metric to cache refresh time
Signed-off-by: Sunil Shivanand <padlar@live.com>
2024-02-29 11:07:13 +01:00
Kubernetes Prow Robot
411c29c39f Merge pull request #123331 from aojea/ccm_update
CCM wait for providerID to initialize the Node object
2024-02-29 01:50:02 -08:00
Kubernetes Prow Robot
234f0fcfc3 Merge pull request #122676 from p0lyn0mial/upstream-watch-cache-init-events-ordering
apiserver/storage: improve RunWatchSemanticInitialEventsExtended test
2024-02-29 01:49:48 -08:00
Patrick Ohly
65b841c077 Generate go.work files
This creates go.work and enables Go Workspaces.  This is a file that
includes info on k/k and all the staging modules.

This depends on go 1.22 and setting FORCE_HOST_GO=true (for kube
scripts, which try to be hermetic).

Make this part of the normal update/verify sequence.

The top-level go.work file contains no replace statements. Instead, the
replace statements in the individual go.mod files are used. For this to
work, replace statements in the individual go.mod files have to be
consistent.

hack/tools has different dependencies and can't be in the main
workspace, so this adds a go.work just for that.  Without this, go tries
to consider all deps in all modules and pick one that works for all.
This is problematic because there are so many of them that it is
difficult to manage.

Likewise for k8s.io/code-generator/examples and
k8s.io/kms/internal/plugins/_mock - add trivial go.work files.

For example k/k depends on an older version of a lib that gloangci-lint
needs (transitively) and it breaks.

This also updates vendor (needed to make go happy), and removes
vendor'ed symlinks.  This breaks a LOT of our build tools, which will be
fixed subsequently.

Result: `go` commands work across modules:

Before:
```
$ go list ./pkg/proxy/iptables/ ./staging/src/k8s.io/api/core/v1/
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api/core/v1

$ go build ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api

$ go test ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api
```

After:
```
$ go list ./pkg/proxy/iptables/ ./staging/src/k8s.io/api/core/v1/
k8s.io/kubernetes/pkg/proxy/iptables
k8s.io/api/core/v1

$ go build ./pkg/proxy/iptables/ ./staging/src/k8s.io/api

$ go test ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
ok  	k8s.io/kubernetes/pkg/proxy/iptables	0.360s
ok  	k8s.io/api	2.302s
```

Result: `make` fails:

```
$ make
go version go1.22rc1 linux/amd64
+++ [0106 12:11:03] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kube-proxy (static)
    k8s.io/kubernetes/cmd/kube-apiserver (static)
    k8s.io/kubernetes/cmd/kube-controller-manager (static)
    k8s.io/kubernetes/cmd/kubelet (non-static)
    k8s.io/kubernetes/cmd/kubeadm (static)
    k8s.io/kubernetes/cmd/kube-scheduler (static)
    k8s.io/component-base/logs/kube-log-runner (static)
    k8s.io/kube-aggregator (static)
    k8s.io/apiextensions-apiserver (static)
    k8s.io/kubernetes/cluster/gce/gci/mounter (static)
    k8s.io/kubernetes/cmd/kubectl (static)
    k8s.io/kubernetes/cmd/kubectl-convert (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    k8s.io/kubernetes/test/e2e/e2e.test (test)
    k8s.io/kubernetes/test/conformance/image/go-runner (non-static)
    k8s.io/kubernetes/cmd/kubemark (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    k8s.io/kubernetes/test/e2e_node/e2e_node.test (test)
test/e2e/e2e.go:35:2: cannot find package "k8s.io/api/apps/v1" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/api/apps/v1 (vendor tree)
	/home/thockin/src/kubernetes/_output/local/.gimme/versions/go1.22rc1.linux.amd64/src/k8s.io/api/apps/v1 (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/api/apps/v1 (from $GOPATH)
	... more ...
	... more ...
	... more ...
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: /home/thockin/src/kubernetes/hack/lib/golang.sh:948 kube::golang::build_binaries_for_platform(...)
!!! [0106 12:13:41]  2: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:96: all] Error 1
```

Again, this requires go 1.22 (e.g. gotip), as go 1.21.x does not have
`go work vendor` support.

TO REPEAT:
    ( \
      ./hack/update-go-workspace.sh; \
      ./hack/update-vendor.sh; \
      ./hack/update-go-workspace.sh; \
    )
2024-02-29 00:22:06 -08:00
Wei Huang
ffa183a7a4 generated files 2024-02-28 23:18:44 -08:00
Wei Huang
01db4ae9e7 Graduate PodSchedulingReadiness to stable 2024-02-28 23:18:44 -08:00
ah8ad3
3b5a79f647 cleanup: add zero as a value for token create duration and act as default, added test, updated the help text
change duration validator text message in create token

unit bug fixed
2024-02-29 08:28:05 +03:30
Kubernetes Prow Robot
a67973a45c Merge pull request #122717 from jpbetz/crd-object-filters
KEP-4358: Custom Resource Field Selectors
2024-02-28 20:07:12 -08:00
Jiahui Feng
64ee859aa8 make ValidatingAdmissionPolicy ignore excluded resources. 2024-02-28 15:31:44 -08:00
Jiahui Feng
5b1fffa3e4 add resource filter to admission initializer. 2024-02-28 15:31:18 -08:00
Joe Betz
a0dc949ce3 Improve error handling for json path results 2024-02-28 16:22:34 -05:00
Joe Betz
61d6026d23 Fix field paths 2024-02-28 15:49:44 -05:00
Kubernetes Prow Robot
fe8a12d264 Merge pull request #123562 from jpbetz/bump-cel-go-0_17_8
Bump cel-go to v0.17.8 to pick up CEL estimated cost fix
2024-02-28 12:45:41 -08:00
Joe Betz
45742d1d9a Add integration tests 2024-02-28 14:46:47 -05:00
Joe Betz
31f7efab20 Add cost stability tests for chained and nested CEL comprehensions 2024-02-28 14:20:16 -05:00
Joe Betz
7ef3eea8a7 Generate code 2024-02-28 14:06:46 -05:00
Joe Betz
291703482d Add selectableFields to CRDs 2024-02-28 14:06:46 -05:00
Kubernetes Prow Robot
e21a2f5d4f Merge pull request #123542 from liggitt/websocket-round-tripper-protocol
Use the websocket protocol header, verify selected protocol
2024-02-28 11:01:44 -08:00
Kubernetes Prow Robot
68a47053d1 Merge pull request #123508 from saschagrunert/image-id-container
Add `image_id` to CRI `Container` message
2024-02-28 11:01:35 -08:00
Joe Betz
d49949b642 Bump cel-go to v0.17.8 to pick up CEL estimated cost fix 2024-02-28 10:52:36 -05:00
Kensei Nakada
f46df21cad update comments on API 2024-02-28 12:32:32 +00:00
Wojciech Tyczyński
92bdc7b387 Ensure that initial events are sorted for WatchList 2024-02-28 12:50:53 +01:00
Lukasz Szaszkiewicz
20ded27570 apiserver/storage: improve RunWatchSemanticInitialEventsExtended test
changes the test to populate the underlying data store with
more data to trigger potential ordering issues.
2024-02-28 10:57:11 +01:00
Kubernetes Prow Robot
d2b4928669 Merge pull request #122830 from p0lyn0mial/upstream-watch-cache-wati-for-bk-after-rv
storage/cacher: ensure the cache is at the Most Recent ResourceVersion when streaming was requested
2024-02-28 01:43:37 -08:00
Sascha Grunert
e663285ccf Add image_id to CRI Container message
This new field allows fixing the kubelet image garbage collection in
container runtimes. The `image_ref` has been historically used by
container runtimes to reference images by digest.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
2024-02-28 10:05:07 +01:00
Kubernetes Prow Robot
f7ca532472 Merge pull request #123281 from seans3/remote-command-websocket-beta
RemoteCommand over WebSockets to Beta
2024-02-27 21:01:55 -08:00
Jordan Liggitt
b394aac4ce Use the websocket protocol header, verify selected protocol 2024-02-27 22:47:19 -05:00