Commit Graph

20320 Commits

Author SHA1 Message Date
Tim Hockin
fcd6c19c24
Rename gengo generator.Package -> Target 2024-02-29 22:07:10 -08:00
Tim Hockin
d2c7401910
Drop generator.Packages type 2024-02-29 22:07:10 -08:00
Tim Hockin
b2832697d5
Rename DefaultPackage to SimplePackage 2024-02-29 22:07:09 -08:00
Tim Hockin
4627a15ac0
Get rid of vendor baggage in go-to-proto 2024-02-29 22:07:07 -08:00
Tim Hockin
dd4e456875
codegen:proto: Use revised API to gengo parser 2024-02-29 22:07:03 -08:00
Tim Hockin
4b55936ab0
Remove defunct trim-path-prefix 2024-02-29 22:07:02 -08:00
Tim Hockin
2348e94908
Conversion: Better logging 2024-02-29 22:07:01 -08:00
Tim Hockin
e84adf611f
Remove references to "GO111MODULE" 2024-02-29 22:06:55 -08:00
Tim Hockin
d772f7719d
Remove defunct references to "vendor" 2024-02-29 22:06:52 -08:00
Tim Hockin
7f5982e708
code-generator: Remove now unused helper function 2024-02-29 22:06:51 -08:00
Tim Hockin
10c32b3e2f
Get rid of most references to GOPATH 2024-02-29 22:06:51 -08:00
Tim Hockin
91efe7e571
Fix import-boss wrt gengo/v2 2024-02-29 22:06:40 -08:00
Tim Hockin
c8b61bb98b
Fix register-gen wrt gengo/v2 2024-02-29 22:06:39 -08:00
Tim Hockin
f772410082
Make code-gen subprojects work on gengo/v2 2024-02-29 22:06:38 -08:00
Tim Hockin
b0a70dec4a
Fix go-to-protobuf wrt gengo/v2
There's some very fishy-smelling logic in here, but this commit is
trying to be as focused as possible.

The *.pb.go diffs are the "name" encoded in the descriptor.  The
descriptor blobs can be decoded by this program (thanks StackOverflow!):

```
package main

import (
	"bytes"
	"compress/gzip"
	"encoding/json"
	"fmt"
	"os"

	"io/ioutil"

	proto "github.com/golang/protobuf/proto"
	dpb "github.com/golang/protobuf/protoc-gen-go/descriptor"
)

func main() {
	m := map[string][]byte{
		"before": blobv1,
		"after":  blobv2,
	}
	arg := os.Args[1]
	dump(m[arg])
}

func dump(bytes []byte) {
	fd, err := decodeFileDesc(bytes)
	if err != nil {
		panic(err)
	}
	b, err := json.MarshalIndent(fd, "", "  ")
	if err != nil {
		panic(err)
	}
	fmt.Println(string(b))
}

// decompress does gzip decompression.
func decompress(b []byte) ([]byte, error) {
	r, err := gzip.NewReader(bytes.NewReader(b))
	if err != nil {
		return nil, fmt.Errorf("bad gzipped descriptor: %v", err)
	}
	out, err := ioutil.ReadAll(r)
	if err != nil {
		return nil, fmt.Errorf("bad gzipped descriptor: %v", err)
	}
	return out, nil
}

func decodeFileDesc(enc []byte) (*dpb.FileDescriptorProto, error) {
	raw, err := decompress(enc)
	if err != nil {
		return nil, fmt.Errorf("failed to decompress enc: %v", err)
	}

	fd := new(dpb.FileDescriptorProto)
	if err := proto.Unmarshal(raw, fd); err != nil {
		return nil, fmt.Errorf("bad descriptor: %v", err)
	}
	return fd, nil
}

var blobv1 = []byte{
	// insert proto "before" blob here
}

var blobv2 = []byte{
	// insert proto "after" blob here
}
```

Running this with "before" and "after" args, and diffing the output
yields something like:

```diff
--- /tmp/a	2023-12-23 23:57:04.748090836 -0800
+++ /tmp/b	2023-12-23 23:57:11.000040973 -0800
@@ -1,5 +1,5 @@
 {
-  "name": "k8s.io/kubernetes/vendor/k8s.io/api/admission/v1/generated.proto",
+  "name": "k8s.io/api/admission/v1/generated.proto",
   "package": "k8s.io.api.admission.v1",
   "dependency": [
     "github.com/gogo/protobuf/gogoproto/gogo.proto",
```
2024-02-29 22:06:37 -08:00
Tim Hockin
b961b39efa
Fix client-gen wrt gengo/v2 2024-02-29 22:06:35 -08:00
Tim Hockin
5ddd6772a4
Fix applyconfiguration-gen wrt gengo/v2 2024-02-29 22:06:33 -08:00
Tim Hockin
0aa98ee8e3
Fix informer-gen wrt gengo/v2 2024-02-29 22:06:31 -08:00
Tim Hockin
b82527b6fb
Fix lister-gen wrt gengo/v2 2024-02-29 22:06:30 -08:00
Tim Hockin
5475797f43
Make update-codegen conversion work on gengo/v2 2024-02-29 22:06:28 -08:00
Tim Hockin
ed35118664
Fix conversion-gen wrt gengo/v2 2024-02-29 22:06:27 -08:00
Tim Hockin
fd2ca0a344
Fix prerelease-lifecycle-gen wrt gengo/v2 2024-02-29 22:06:25 -08:00
Tim Hockin
514dd0e1f6
Convert all gengo references to gengo/v2
TO REPEAT:

    git ls-files ':!:vendor/k8s.io/gengo' ':!:gengo-v2' ':(glob)**/*.go' \
        | xargs grep -l 'k8s.io/gengo/' \
        | while read F; do \
            sed -i 's|k8s.io/gengo/\([^v]\)|k8s.io/gengo/v2/\1|' $F
        done
2024-02-29 22:06:20 -08:00
Tim Hockin
6f2f3735e0
Re-vendor latest kube-openapi and gengo/v2
./hack/pin-dependency.sh k8s.io/kube-openapi latest
./hack/pin-dependency.sh k8s.io/gengo/v2 latest
./hack/update-vendor.sh
2024-02-29 22:06:19 -08:00
Tim Hockin
b3dd724174
Remove dep on set-gen: gone in gengo/v2 2024-02-29 22:06:18 -08:00
Tim Hockin
7a7dff1eeb
Change verify-imports to use new setup_env
Also clean up the tool a bit and drop vendor as an explicit thing to
be handled.
2024-02-29 22:06:16 -08:00
Patrick Ohly
65b841c077
Generate go.work files
This creates go.work and enables Go Workspaces.  This is a file that
includes info on k/k and all the staging modules.

This depends on go 1.22 and setting FORCE_HOST_GO=true (for kube
scripts, which try to be hermetic).

Make this part of the normal update/verify sequence.

The top-level go.work file contains no replace statements. Instead, the
replace statements in the individual go.mod files are used. For this to
work, replace statements in the individual go.mod files have to be
consistent.

hack/tools has different dependencies and can't be in the main
workspace, so this adds a go.work just for that.  Without this, go tries
to consider all deps in all modules and pick one that works for all.
This is problematic because there are so many of them that it is
difficult to manage.

Likewise for k8s.io/code-generator/examples and
k8s.io/kms/internal/plugins/_mock - add trivial go.work files.

For example k/k depends on an older version of a lib that gloangci-lint
needs (transitively) and it breaks.

This also updates vendor (needed to make go happy), and removes
vendor'ed symlinks.  This breaks a LOT of our build tools, which will be
fixed subsequently.

Result: `go` commands work across modules:

Before:
```
$ go list ./pkg/proxy/iptables/ ./staging/src/k8s.io/api/core/v1/
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api/core/v1

$ go build ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api

$ go test ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
main module (k8s.io/kubernetes) does not contain package k8s.io/kubernetes/staging/src/k8s.io/api
```

After:
```
$ go list ./pkg/proxy/iptables/ ./staging/src/k8s.io/api/core/v1/
k8s.io/kubernetes/pkg/proxy/iptables
k8s.io/api/core/v1

$ go build ./pkg/proxy/iptables/ ./staging/src/k8s.io/api

$ go test ./pkg/proxy/iptables/ ./staging/src/k8s.io/api
ok  	k8s.io/kubernetes/pkg/proxy/iptables	0.360s
ok  	k8s.io/api	2.302s
```

Result: `make` fails:

```
$ make
go version go1.22rc1 linux/amd64
+++ [0106 12:11:03] Building go targets for linux/amd64
    k8s.io/kubernetes/cmd/kube-proxy (static)
    k8s.io/kubernetes/cmd/kube-apiserver (static)
    k8s.io/kubernetes/cmd/kube-controller-manager (static)
    k8s.io/kubernetes/cmd/kubelet (non-static)
    k8s.io/kubernetes/cmd/kubeadm (static)
    k8s.io/kubernetes/cmd/kube-scheduler (static)
    k8s.io/component-base/logs/kube-log-runner (static)
    k8s.io/kube-aggregator (static)
    k8s.io/apiextensions-apiserver (static)
    k8s.io/kubernetes/cluster/gce/gci/mounter (static)
    k8s.io/kubernetes/cmd/kubectl (static)
    k8s.io/kubernetes/cmd/kubectl-convert (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    k8s.io/kubernetes/test/e2e/e2e.test (test)
    k8s.io/kubernetes/test/conformance/image/go-runner (non-static)
    k8s.io/kubernetes/cmd/kubemark (static)
    github.com/onsi/ginkgo/v2/ginkgo (non-static)
    k8s.io/kubernetes/test/e2e_node/e2e_node.test (test)
test/e2e/e2e.go:35:2: cannot find package "k8s.io/api/apps/v1" in any of:
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/api/apps/v1 (vendor tree)
	/home/thockin/src/kubernetes/_output/local/.gimme/versions/go1.22rc1.linux.amd64/src/k8s.io/api/apps/v1 (from $GOROOT)
	/home/thockin/src/kubernetes/_output/local/go/src/k8s.io/api/apps/v1 (from $GOPATH)
	... more ...
	... more ...
	... more ...
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: /home/thockin/src/kubernetes/hack/lib/golang.sh:948 kube::golang::build_binaries_for_platform(...)
!!! [0106 12:13:41]  2: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
!!! [0106 12:13:41] Call tree:
!!! [0106 12:13:41]  1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...)
make: *** [Makefile:96: all] Error 1
```

Again, this requires go 1.22 (e.g. gotip), as go 1.21.x does not have
`go work vendor` support.

TO REPEAT:
    ( \
      ./hack/update-go-workspace.sh; \
      ./hack/update-vendor.sh; \
      ./hack/update-go-workspace.sh; \
    )
2024-02-29 00:22:06 -08:00
Kubernetes Prow Robot
a67973a45c
Merge pull request #122717 from jpbetz/crd-object-filters
KEP-4358: Custom Resource Field Selectors
2024-02-28 20:07:12 -08:00
Joe Betz
a0dc949ce3 Improve error handling for json path results 2024-02-28 16:22:34 -05:00
Joe Betz
61d6026d23 Fix field paths 2024-02-28 15:49:44 -05:00
Kubernetes Prow Robot
fe8a12d264
Merge pull request #123562 from jpbetz/bump-cel-go-0_17_8
Bump cel-go to v0.17.8 to pick up CEL estimated cost fix
2024-02-28 12:45:41 -08:00
Joe Betz
45742d1d9a Add integration tests 2024-02-28 14:46:47 -05:00
Joe Betz
31f7efab20 Add cost stability tests for chained and nested CEL comprehensions 2024-02-28 14:20:16 -05:00
Joe Betz
7ef3eea8a7 Generate code 2024-02-28 14:06:46 -05:00
Joe Betz
291703482d Add selectableFields to CRDs 2024-02-28 14:06:46 -05:00
Kubernetes Prow Robot
e21a2f5d4f
Merge pull request #123542 from liggitt/websocket-round-tripper-protocol
Use the websocket protocol header, verify selected protocol
2024-02-28 11:01:44 -08:00
Kubernetes Prow Robot
68a47053d1
Merge pull request #123508 from saschagrunert/image-id-container
Add `image_id` to CRI `Container` message
2024-02-28 11:01:35 -08:00
Joe Betz
d49949b642 Bump cel-go to v0.17.8 to pick up CEL estimated cost fix 2024-02-28 10:52:36 -05:00
Wojciech Tyczyński
92bdc7b387 Ensure that initial events are sorted for WatchList 2024-02-28 12:50:53 +01:00
Kubernetes Prow Robot
d2b4928669
Merge pull request #122830 from p0lyn0mial/upstream-watch-cache-wati-for-bk-after-rv
storage/cacher: ensure the cache is at the Most Recent ResourceVersion when streaming was requested
2024-02-28 01:43:37 -08:00
Sascha Grunert
e663285ccf
Add image_id to CRI Container message
This new field allows fixing the kubelet image garbage collection in
container runtimes. The `image_ref` has been historically used by
container runtimes to reference images by digest.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
2024-02-28 10:05:07 +01:00
Kubernetes Prow Robot
f7ca532472
Merge pull request #123281 from seans3/remote-command-websocket-beta
RemoteCommand over WebSockets to Beta
2024-02-27 21:01:55 -08:00
Jordan Liggitt
b394aac4ce
Use the websocket protocol header, verify selected protocol 2024-02-27 22:47:19 -05:00
Kubernetes Prow Robot
286cdad32d
Merge pull request #123538 from jiahuif-forks/fix/cel/mutation-library-map-support
CEL mutation library: add support for map
2024-02-27 18:46:10 -08:00
Kubernetes Prow Robot
872c621bb7
Merge pull request #123475 from alexzielenski/apiserver/apiextensions/cel-error-fieldpath
bugfix: incorrect fieldpath when using multiple crd validation rules
2024-02-27 18:45:57 -08:00
Kubernetes Prow Robot
72c92e975d
Merge pull request #123411 from hunshcn/fix/volume-link
fix persistent volume docs link
2024-02-27 16:47:27 -08:00
Sean Sullivan
03812ddb16 streamtranslator counter metric by status code 2024-02-27 23:21:55 +00:00
Monis Khan
e89dddd4af
jwt: strictly support compact serialization only
Signed-off-by: Monis Khan <mok@microsoft.com>
2024-02-27 17:16:08 -05:00
Jiahui Feng
dc4c92f5a5 add support for map
to CEL mutation library.
2024-02-27 13:55:08 -08:00
Lukasz Szaszkiewicz
ecaf2093f5 storage/watch_cache: rework getAllEventsSinceLocked 2024-02-27 14:26:46 +01:00