Commit Graph

17795 Commits

Author SHA1 Message Date
Patrick Ohly
f131cabfa0 test: use go-uber/goleak for strict leak checking
It provides more readable output and has additional APIs for using it inside a
unit test. goleak.IgnoreCurrent is needed to filter out the goroutine that gets
started when importing go.opencensus.io/stats/view.

In order to handle background goroutines that get created on demand and cannot
be stopped (like the one for LogzHealth), a helper function ensures that those
are running before calling goleak.IgnoreCurrent. Keeping those goroutines
running is not a problem and thus not worth the effort of adding new APIs to
stop them.

Other goroutines are genuine leaks for which no fix is available. Those get
suppressed via IgnoreTopFunction, which works as long as that function
is unique enough.

Example output for the leak fixed in https://github.com/kubernetes/kubernetes/pull/115423:

    E0202 09:30:51.641841   74789 etcd.go:205] "EtcdMain goroutine check" err=<
        found unexpected goroutines:
        [Goroutine 4889 in state chan receive, with k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop on top of the stack:
        goroutine 4889 [chan receive]:
        k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc0076183c0)
        	/nvme/gopath/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/watch/mux.go:268 +0x65
        created by k8s.io/apimachinery/pkg/watch.NewBroadcaster
        	/nvme/gopath/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/watch/mux.go:77 +0x116
    >
2023-02-14 12:11:37 +01:00
Kubernetes Prow Robot
d0db9a959b Merge pull request #115677 from aramase/kmsv2-middleware
[KMSv2] implement local KEK service
2023-02-10 16:21:29 -08:00
Monis Khan
ee2e1ff99a implement service.Service interface and update localKEK generation
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
Co-authored-by: Monis Khan <mok@microsoft.com>
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
2023-02-10 22:12:34 +00:00
Kubernetes Prow Robot
16bd7e6a79 Merge pull request #115674 from mengjiao-liu/update-podsecurity-registry
PodSecurity test: update registry from k8s.gcr.io to registry.k8s.io
2023-02-10 10:45:30 -08:00
Arda Güçlü
c40f9f8bbb Add explicitNamespace into ignore list in debug unit tests
Since, `explicitNamespace` is set by kubeconfig automatically, we can
safely add this field into ignore list.
2023-02-10 11:31:19 +03:00
Arda Güçlü
e0fedec69d (kubectl debug): Support debugging via files
Currently `kubectl debug` only supports passing names in command line.
However, users might want to pass resources in files by passing `-f` flag like
in all other kubectl commands.

This PR adds this ability.
2023-02-10 10:21:30 +03:00
Krzysztof Ostrowski
711a985217 kmsv2: add LocalKEKService
Signed-off-by: Krzysztof Ostrowski <kostrows@redhat.com>
2023-02-10 07:18:57 +00:00
Mengjiao Liu
51089767a6 PodSecurity test: update registry from k8s.gcr.io to registry.k8s.io 2023-02-10 11:02:42 +08:00
Anish Ramasekar
b6b00e65bc logcheck.conf: ensure that kms and value/encrypt uses structured logging
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
2023-02-10 02:28:39 +00:00
Anish Ramasekar
6db7496db6 [KMS] Use structured logging in value/encrypt
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
2023-02-09 23:57:07 +00:00
Kubernetes Prow Robot
9a51625ebe Merge pull request #115665 from aramase/aramase/f/update_vendor_k8s_utils
Update k8s.io/utils to `a36077c30491`
2023-02-09 15:50:53 -08:00
Kubernetes Prow Robot
0698d9eb82 Merge pull request #115649 from aramase/grpc-metrics
[KMSv2] Add metrics for grpc service
2023-02-09 15:50:45 -08:00
Anish Ramasekar
09e02052fd Update k8s.io/utils to a36077c30491
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
2023-02-09 20:09:41 +00:00
Anish Ramasekar
de3b2d525b [KMSv2] Add metrics for grpc service
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
2023-02-09 18:51:37 +00:00
Anish Ramasekar
f72cf5c510 [KMSv2] store hash of encrypted DEK as key in cache
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
2023-02-09 17:26:27 +00:00
Shang Jian Ding
d35da348c6 Implement kubectl debug profiles: general, baseline, and restricted (#114280)
* feat(debug): add more profiles

Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>

* feat(debug): implment serveral debugging profiles

Including `general`, `baseline` and `restricted`.

I plan to add more profiles afterwards, but I'd like to get early
reviews.

Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>

* test: add some basic tests

Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>

* chore: add some helper functions

Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>

* ensure pod copies always get their probes cleared

not wanting probes to be present is something we want
for all the debug profiles; so an easy place to implement
this is at the time of pod copy generation.

* ensure debug container in pod copy is added before the profile application

The way that the container list modification was defered causes the
debug container to be added after the profile applier runs. We now
make sure to have the container list modification happen before
the profile applier runs.

* make switch over pod copy, ephemeral, or node more clear

* use helper functions

added a helper function to modify a container out of a list that
matches the provided container name.

also added a helper function that adds capabilities to container
security.

* add tests for the debug profiles

* document new debugging profiles in command line help text

* add file header to profiles_test.go

* remove URL to KEP from help text

* move probe removal to the profiles

* remove mustNewProfileApplier in tests

* remove extra whiteline from import block

* remove isPodCopy helper func

* switch baselineProfile to using the modifyEphemeralContainer helper

* rename addCap to addCapability, and don't do deep copy

* fix godoc on modifyEphemeralContainer

* export DebugOptions.Applier for extensibility

* fix unit test

* fix spelling on overriden

* remove debugStyle facilities

* inline setHostNamespace helper func

* remove modifyContainer, modifyEphemeralContainer, and remove probes

their logic have been in-lined at call sites

* remove DebugApplierFunc convenience facility

* fix baseline profile implementation

it shouldn't have SYS_PTRACE base on
https://github.com/kubernetes/enhancements/tree/master/keps/sig-cli/1441-kubectl-debug#profile-baseline

* remove addCapability helper, in-lining at call sites

* address Arda's code review comments

1 use Bool instead of BoolPtr (now deprecated)
2 tweak for loop to continue when container name is not what we expect
3 use our knowledge on how the debug container is generated to simplify
  our modification to the security context
4 use our knowledge on how the pod for node debugging is generated to no
  longer explicit set pod's HostNework, HostPID and HostIPC fields to
  false

* remove tricky defer in generatePodCopyWithDebugContainer

* provide helper functions to make debug profiles more readable

* add note to remind people about updating --profile's help text when adding new profiles

* Implement helper functions with names that improve readability

* add styleUnsupported to replace debugStyle(-1)

* fix godoc on modifyContainer

* drop style prefix from debugStyle values

* put VisitContainers in podutils & use that from debug

* cite source for ContainerType and VisitContainers

* pull in AllContainers ContainerType value

* have VisitContainer take pod spec rather than pod

* in-line modifyContainer

* unexport helper funcs

* put debugStyle at top of file

* merge profile_applier.go into profile.go

* tweak dropCapabilities

* fix allowProcessTracing & add a test for it

* drop mask param from help funcs, since we can already unambiguous identify the container by name

* fix grammar in code comment

---------

Signed-off-by: Jian Zeng <anonymousknight96@gmail.com>
Co-authored-by: Jian Zeng <anonymousknight96@gmail.com>
2023-02-09 09:18:22 -08:00
Kubernetes Prow Robot
19eddfc567 Merge pull request #110953 from Abirdcfly/220705
cleanup kubectl describe cronjob v1beta1 in 1.26
2023-02-09 02:40:24 -08:00
Kubernetes Prow Robot
b0171f77f6 Merge pull request #115514 from pohly/event-broadcaster-shutdown
client-go: don't wait too long after EventBroadcaster.Shutdown
2023-02-08 04:38:58 -08:00
Kubernetes Prow Robot
dfb976e25a Merge pull request #114376 from baomingwang/serialization-error-metric
Added serialization from etcd error metric
2023-02-07 13:46:59 -08:00
Kubernetes Prow Robot
22b88dea36 Merge pull request #115315 from enj/enj/i/kas_kubelet_conn_close
kubelet/client: collapse transport wiring onto standard approach
2023-02-07 07:01:14 -08:00
Kubernetes Prow Robot
b30ff25fdb Merge pull request #112662 from Iceber/fix_sample_apiserver_readme
sample-apiserver: create the client cert with SHA256
2023-02-07 07:01:01 -08:00
Kubernetes Prow Robot
4f321041bd Merge pull request #115537 from MadhavJivrajani/bump-tools-deps-go120
*: Bump golangci-lint version and adapt to new linters
2023-02-07 05:53:12 -08:00
Madhav Jivrajani
5e1f440d0a *: Fix linter warnings
Adapt to newly improved linters in golangci-lint v1.51.1

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2023-02-07 13:01:41 +05:30
Kubernetes Prow Robot
6eb008620c Merge pull request #115379 from artemvmin/serial-mkfs
Add an option to limit the number of concurrent mkfs calls
2023-02-06 18:26:58 -08:00
Monis Khan
754cb3d601 kubelet/client: collapse transport wiring onto standard approach
Signed-off-by: Monis Khan <mok@microsoft.com>
2023-02-06 20:34:49 -05:00
Kubernetes Prow Robot
2ab3151936 Merge pull request #108396 from tkashem/retry-metric
client-go: add a metric to count request retries
2023-02-06 15:30:58 -08:00
Artem Minyaylov
d2dd415f00 Add option to limit the number of concurrent mkfs calls 2023-02-06 14:27:57 -08:00
baomingwang
92c490f0bc Added serialization from etcd error metric 2023-02-06 12:27:31 -08:00
Kubernetes Prow Robot
daf0d8d14f Merge pull request #115542 from SataQiu/clean-apiserver-20230206
Using BuildOpenAPISpecFromRoutes instead of deprecated BuildOpenAPISpec
2023-02-06 11:30:58 -08:00
Kubernetes Prow Robot
31d6d9a275 Merge pull request #108984 from LittleFox94/cloud-provider/logging-format
cloud-provider: add log options, allow setting logging-format via CLI option
2023-02-06 10:15:01 -08:00
Kubernetes Prow Robot
06914bdaf5 Merge pull request #115521 from aojea/cloudprovidergcp
Improve performance on the cloud provider node-controller
2023-02-06 03:53:00 -08:00
SataQiu
5fbd49c102 using BuildOpenAPISpecFromRoutes instead of deprecated BuildOpenAPISpec 2023-02-06 17:47:44 +08:00
Antonio Ojea
80d21e5929 parallelize node-controller
The node-controllers has 2 reconcilation methods:
- workqueue with workers, is using during bootstrap and process
nodes until the cloud provider taint is removed
- periodic loop, that runs every certain period polling the cloud
provider to get the instances metadata to update the node addresses,
since nodes can Update its addresses anytime during its lifecycle.

These follows up on the parallelization of the node-controller, that
previously increased the number of workers that handle the bootstrap.

This parallelize the periodic loop based on the input value of the
number of workers, and also uses the informer lister instead of doing
a new List to the apiserver.

Added an unit test that can used to evaluate the performance improvement
with different workers values:

=== RUN   TestUpdateNodeStatus/single_thread
    node_controller_test.go:2537: 1 workers: processed 100 nodes int 1.055595262s
=== RUN   TestUpdateNodeStatus/5_workers
    node_controller_test.go:2537: 5 workers: processed 100 nodes int 216.990972ms
=== RUN   TestUpdateNodeStatus/10_workers
    node_controller_test.go:2537: 10 workers: processed 100 nodes int 112.422435ms
=== RUN   TestUpdateNodeStatus/30_workers
    node_controller_test.go:2537: 30 workers: processed 100 nodes int 46.243204ms

Change-Id: I38870993431d38fc81a2dc6a713321cfa2e40d85
2023-02-06 09:06:29 +00:00
Kubernetes Prow Robot
f3562d9dea Merge pull request #115403 from apelisse/hasschema-pool
apiextensions: Benchmark escaping in SchemaHas and pool Schemas
2023-02-05 22:35:02 -08:00
Artem Minyaylov
f573e14942 Update k8s.io/utils to latest version
Update all usages of FakeExec to pointer to avoid copying the mutex
2023-02-04 11:05:22 -08:00
Antonio Ojea
bdd3e1d8c8 fake cloud provider don't lock emulating delay
Change-Id: Icf0cf5d67a4c1d53556f93bbda5f286faaa456b2
2023-02-04 16:15:44 +00:00
cpanato
b9ddf07a75 [go] Bump images, dependencies and versions to go 1.20
Signed-off-by: cpanato <ctadeu@gmail.com>
2023-02-03 22:55:24 +01:00
Kubernetes Prow Robot
0e3818e027 Merge pull request #115515 from dgrisonnet/scrape-and-compare
Add wrapper around ScrapeAndCompare
2023-02-03 13:46:29 -08:00
Kubernetes Prow Robot
1ded677b2a Merge pull request #114894 from dipankardas011/validate-admission-policy-error-handling
Added error handling in ValidateAdmissionPolicy
2023-02-03 12:14:30 -08:00
Damien Grisonnet
c5774a703d component-base: add ScrapeAndCompare wrapper
Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2023-02-03 20:35:52 +01:00
Patrick Ohly
15b01af9c1 client-go: don't wait too long after EventBroadcaster.Shutdown
When Shutdown was called, delivery of each pending event would still be retried
12 times with a delay of ~10s between each retry. In apiserver integration
tests that caused the goroutine to linger long after the corresponding
apiserver of the test was shut down.
2023-02-03 20:02:48 +01:00
Kubernetes Prow Robot
c34ddb5054 Merge pull request #115485 from SergeyKanzhelev/epmeneralContainersQoS
small comment the ephemeral containers are not accounted for the QoS …
2023-02-03 08:46:29 -08:00
Abu Kashem
b6c369f5c9 client-go: add metric to count retries 2023-02-03 09:04:10 -05:00
Kubernetes Prow Robot
0ebf9a3a1b Merge pull request #115434 from ehashman/requests-limits-ratio-docs
Document relationship between requests/limits
2023-02-02 11:02:53 -08:00
Sergey Kanzhelev
b517d640a4 small comment the ephemeral containers are not accounted for the QoS calculation 2023-02-02 18:21:39 +00:00
Kubernetes Prow Robot
05121454ba Merge pull request #115455 from dgrisonnet/verb
Rename "method" label to "verb" in rest_client_requests_total
2023-02-02 06:35:31 -08:00
Damien Grisonnet
c9944709bc restclient: rename "method" label to "verb"
Verb is the commonly used label when referring to HTTP verbs.
rest_client_requests_total is the only metric in the rest package using
`method` instead of `verb` which makes it inconsistent and confusing.

Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2023-02-02 14:09:21 +01:00
Kubernetes Prow Robot
fc7b25cf72 Merge pull request #113104 from pawbana/add-workers-to-cloud-controller-manager
Added workerCount flag to node controller in cloud controller manager
2023-02-01 19:21:28 -08:00
Max Smythe
0ed74145fb make CEL admission controller code consumable (#115412)
* Make policy decision object public

Signed-off-by: Max Smythe <smythe@google.com>

* Separate version conversion from validation

Signed-off-by: Max Smythe <smythe@google.com>

* Address review comments

Signed-off-by: Max Smythe <smythe@google.com>

* Fix variable name

Signed-off-by: Max Smythe <smythe@google.com>

---------

Signed-off-by: Max Smythe <smythe@google.com>
2023-02-01 17:29:30 -08:00
Kubernetes Prow Robot
d475085776 Merge pull request #114439 from alexzielenski/apiserver/smd/conversion-smaller
add direct construction of TypeConverter from OpenAPI
2023-02-01 14:51:35 -08:00