While currently those objects only get published by the kubelet for node-local
resources, this could change once we also support network-attached
resources. Dropping the "Node" prefix enables such a future extension.
The NodeName in ResourceSlice and StructuredResourceHandle then becomes
optional. The kubelet still needs to provide one and it must match its own node
name, otherwise it doesn't have permission to access ResourceSlice objects.
This adds support for semantic version comparison to the CEL support in the
"named resources" structured parameter model. For example, it can be used to
check that an instance supports a certain API level.
To minimize the risk, the new "semver" type is only defined in the CEL
environment for DRA expressions, not in the base library. See
https://github.com/kubernetes/kubernetes/pull/123664 for a PR which
adds it to the base library.
Validation of semver strings is done with the regular expression from
semver.org. The actual evaluation at runtime then uses semver/v4.
Like the current device plugin interface, a DRA driver using this model
announces a list of resource instances. In contrast to device plugins, this
list is made available to the scheduler together with attributes that can be
used to select suitable instances when they are not all alike.
Because this is the first structured parameter model, some checks that
previously were not possible, in particular "is one structured parameter field
set", now gets enabled. Adding another structured parameter model will be
similar.
The applyconfigs code generator assumes that all types in an API are defined in
a single package. If it wasn't for that, it would be possible to place the
"named resources" types in separate packages, which makes their names in the Go
code more natural and provides an indication of their stability level because
the package name could include a version.
NodeResourceSlice will be used by kubelet to publish resource information on
behalf of DRA drivers on the node. NodeName and DriverName in
NodeResourceSlice must be immutable. This simplifies tracking the different
objects because what they are for cannot change after creation.
The new field in ResourceClass tells scheduler and autoscaler that they are
expected to handle allocation.
ResourceClaimParameters and ResourceClassParameters are new types for telling
in-tree components how to handle claims.
When moving the reservation of a claim for a pod into the PreBind phase in a
future commit, multiple different update attempts will be executed
concurrently. We want an attempt to succeed if and only if adding the entry
passes validation. Without patch strategy and key, strategic-merge-patch
replaces the entire ReservedFor instead of adding new entries.
Server-side-apply cannot be used because each attempt may start with a stale
ResourceClaim (thus cannot send the entire ReservedFor) and SSA doesn't support
merging when using the same manager string. Using different managers (one for
each entry) would work, but sounds like a bad hack.
This assumes that any such field is atomic, except:
* OwnerReferences: because it has a `+patchStrategy=merge`, but it
probably needs a `+listMapKey=...` ?
* Finalizers: because it hs a `+patchStrategy=merge`, but is a
primitive type (string).
* []byte fields, which should not be failing this anyway (fixed
subsequently).
An alternative approach could be just to turn off the API warnings for
these fields, but it felt more correct to declare the semantics.
The "set" list type was chosen because it seemed appropriate (no duplicates!)
but that made tracking of managed fields more expensive (each entry in the list
is tracked, not the entire field) and for no good reason (one client is
responsible for the entire list).
Therefore the type gets changed to "atomic". Server-side-apply has not been
used in the past and PodSchedulingContext objects are short-lived and still in
alpha, so the any potential compatibility issues should be minor.
The scheduling throughput in scheduler_perf increases:
name old SchedulingThroughput/Average new SchedulingThroughput/Average
PerfScheduling/SchedulingWithResourceClaimTemplate/2000pods_100nodes-36 18.8 ± 8% 24.0 ±37%
PerfScheduling/SchedulingWithMultipleResourceClaims/2000pods_100nodes-36 13.7 ±81% 18.5 ±40%
