Commit Graph

15 Commits

Author SHA1 Message Date
Mike Spreitzer
008576da07 Improve and simplify maintenance of APF bootstrap objects
Prepare to make deletion of unwanted object conditional on ResourceVersion.

Remove unnecessary split between finding unwanted objects and removing
them.

Remove unnecessary layers of indirection to reach constant logic.

Use interfaces to remove need for type assertions.

Threaded context into APF object maintenance

Note and respect immutability of desired bootstrap objects
2023-05-05 09:36:48 -04:00
Patrick Ohly
5e1c6cd0d4 pkg/registry/flowcontrol: avoid race condition during Create
k8s.io/kubernetes/test/integration/controlplane.TestReconcilerAPIServerLeaseMultiCombined
suffered from race conditions. The underlying reason is that
330b5a2b8d/staging/src/k8s.io/apimachinery/pkg/runtime/helper.go (L221-L243)
temporarily modifies the object that it is meant to encode. Callers of
client-go Create calls must be aware of that and pass in unique object if they
might get called concurrently.

It's not clear where these goroutines came from, but the data race seems genuine:

WARNING: DATA RACE
Read at 0x00c0001d66f0 by goroutine 70907:
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/apis/meta/v1.(*TypeMeta).GroupVersionKind()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/meta.go:126 +0x64
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime.WithVersionEncoder.Encode()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime/helper.go:231 +0x176
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime.(*WithVersionEncoder).Encode()
      <autogenerated>:1 +0xfb
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime.Encode()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime/codec.go:50 +0xb3
  k8s.io/kubernetes/vendor/k8s.io/client-go/rest.(*Request).Body()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/rest/request.go:469 +0x884
  k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes/typed/flowcontrol/v1beta3.(*flowSchemas).Create()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes/typed/flowcontrol/v1beta3/flowschema.go:118 +0x23c
  k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer.(*flowSchemaWrapper).Create()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer/flowschema.go:156 +0x12b
  k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer.ensureConfiguration()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer/strategy.go:235 +0x147
  k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer.(*fsEnsurer).Ensure()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer/flowschema.go:121 +0xd2
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.ensureSuggestedConfiguration()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:211 +0x417
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.ensure()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:186 +0x99
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.(*bootstrapConfigurationEnsurer).ensureAPFBootstrapConfiguration.func1()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:157 +0xb4
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:154 +0x7b
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:245 +0x57
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x59
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.(*bootstrapConfigurationEnsurer).ensureAPFBootstrapConfiguration()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:153 +0x237
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.(*bootstrapConfigurationEnsurer).ensureAPFBootstrapConfiguration-fm()
      <autogenerated>:1 +0x58
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.runPostStartHook.func1()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:199 +0xa1
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.runPostStartHook()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:200 +0xda
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.(*GenericAPIServer).RunPostStartHooks.func2()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:166 +0xb4

Previous write at 0x00c0001d66f0 by goroutine 69811:
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/apis/meta/v1.(*TypeMeta).SetGroupVersionKind()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/meta.go:121 +0x193
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime.WithVersionEncoder.Encode()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime/helper.go:241 +0x3d9
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime.(*WithVersionEncoder).Encode()
      <autogenerated>:1 +0xfb
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime.Encode()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime/codec.go:50 +0xb3
  k8s.io/kubernetes/vendor/k8s.io/client-go/rest.(*Request).Body()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/rest/request.go:469 +0x884
  k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes/typed/flowcontrol/v1beta3.(*flowSchemas).Create()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes/typed/flowcontrol/v1beta3/flowschema.go:118 +0x23c
  k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer.(*flowSchemaWrapper).Create()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer/flowschema.go:156 +0x12b
  k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer.ensureConfiguration()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer/strategy.go:235 +0x147
  k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer.(*fsEnsurer).Ensure()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/ensurer/flowschema.go:121 +0xd2
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.ensureSuggestedConfiguration()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:211 +0x417
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.ensure()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:186 +0x99
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.(*bootstrapConfigurationEnsurer).ensureAPFBootstrapConfiguration.func1()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:157 +0xb4
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:154 +0x7b
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.poll()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:245 +0x57
  k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/poll.go:200 +0x59
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.(*bootstrapConfigurationEnsurer).ensureAPFBootstrapConfiguration()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/registry/flowcontrol/rest/storage_flowcontrol.go:153 +0x237
  k8s.io/kubernetes/pkg/registry/flowcontrol/rest.(*bootstrapConfigurationEnsurer).ensureAPFBootstrapConfiguration-fm()
      <autogenerated>:1 +0x58
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.runPostStartHook.func1()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:199 +0xa1
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.runPostStartHook()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:200 +0xda
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.(*GenericAPIServer).RunPostStartHooks.func2()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:166 +0xb4

Goroutine 70907 (running) created at:
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.(*GenericAPIServer).RunPostStartHooks()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:166 +0x167
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.preparedGenericAPIServer.NonBlockingRun()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/genericapiserver.go:729 +0x21a
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.preparedGenericAPIServer.Run()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/genericapiserver.go:578 +0x907
  k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/apiserver.preparedAPIAggregator.Run()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/apiserver/apiserver.go:447 +0xf8
  k8s.io/kubernetes/cmd/kube-apiserver/app/testing.StartTestServer.func3()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/testing/testserver.go:260 +0x109
  k8s.io/kubernetes/cmd/kube-apiserver/app/testing.StartTestServer.func9()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/testing/testserver.go:263 +0x47

Goroutine 69811 (running) created at:
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.(*GenericAPIServer).RunPostStartHooks()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/hooks.go:166 +0x167
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.preparedGenericAPIServer.NonBlockingRun()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/genericapiserver.go:729 +0x21a
  k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.preparedGenericAPIServer.Run()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/genericapiserver.go:578 +0x907
  k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/apiserver.preparedAPIAggregator.Run()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/apiserver/apiserver.go:447 +0xf8
  k8s.io/kubernetes/cmd/kube-apiserver/app/testing.StartTestServer.func3()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/testing/testserver.go:260 +0x109
  k8s.io/kubernetes/cmd/kube-apiserver/app/testing.StartTestServer.func9()
      /home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/testing/testserver.go:263 +0x47
2023-04-05 15:59:22 +02:00
Abu Kashem
424b23bb15 apiserver: fix defaulting for apf bootstrap configuration 2022-11-08 13:23:09 -08:00
Abu Kashem
66fc0d7037 rename assuredConcurrencyShares for flowcontrol v1beta3 2022-09-26 15:34:10 -04:00
Abu Kashem
0a99e6ebb1 apiserver: update apf logic to use v1beta3 2022-09-21 18:54:20 -04:00
Davanum Srinivas
a9593d634c Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-07-26 13:14:05 -04:00
Abu Kashem
df41fe5d84 apf: clarify with comment 2022-01-19 17:38:31 -05:00
Patrick Ohly
9eaa2dc554 avoid klog Info calls without verbosity
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:

   if klog.V(5).Enabled() {
       klog.Info("hello world")
   }

Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.

Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.
2022-01-12 07:48:36 +01:00
Mike Spreitzer
72403c8814 Concurrentize pkg/registry/flowcontrol/ensurer/strategy.go
Stop logging error messages when surprised by concurrent activity.
Concurrent activity is not an error, it is normal.
2022-01-04 22:55:21 -05:00
Kubernetes Prow Robot
9e209efd65 Merge pull request #107104 from tkashem/apf-use-v1beta2
apf: use v1beta2 in registry package
2022-01-04 13:06:42 -08:00
Abu Kashem
f30a34bc01 apf: use v1beta2 for registry 2021-12-17 10:35:37 -05:00
Ben Luddy
3f929b1634 Use a lister for bootstrap flowcontrol config objects.
Instead of listing all flowschemas and prioritylevelconfigurations on
each tick, read from local informer caches.
2021-10-20 11:40:00 -04:00
Abu Kashem
28f2b42a41 apf: update apf logic to use v1beta2 2021-09-09 08:28:58 -04:00
Abu Kashem
f9ee64007e apf: always create missing bootstrap configuration object(s) 2021-05-17 12:08:39 -04:00
Abu Kashem
759a64136b add auto update for apf bootstrap configuration
Take the following approach:
On a fresh install, all bootstrap configuration objects will
have auto update enabled via the following annotation :
`apf.kubernetes.io/autoupdate: 'true'`

The kube-apiserver periodically checks the bootstrap configuration
objects on the cluster and applies update if necessary.

We enforce an 'always auto-update' policy for the mandatory
configuration object(s).

We update the suggested configuration objects when:
- auto update is enabled (`apf.kubernetes.io/autoupdate: 'true'`) or
- auto update annotation key is missing but `generation` is `1`

If the configuration object is missing the annotation key, we add
it appropriately:
it is set to `true` if `generation` is `1`, `false` otherwise.

The above approach ensures that we don't squash changes made by an
operator. Please note, we can't protect the changes made by the
operator in the following scenario:
- the user changes the spec and then deletes and recreates
  the same object. (generation resets to 1)

remove using a marker
2021-05-07 14:23:17 -04:00