6e856480c0
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. apiextensions: add subresources for custom resources Fixes #38113 Fixes #58778 **Related**: - Proposal: https://github.com/kubernetes/community/pull/913 - For custom resources to work with `kubectl scale`: https://github.com/kubernetes/kubernetes/pull/58283 **Add types**: - Add `CustomResourceSubResources` type to CRD. - Fix proto generation for `CustomResourceSubResourceStatus`: https://github.com/kubernetes/kubernetes/pull/55970. - Add feature gate for `CustomResourceSubResources`. - Update CRD strategy: if feature gate is disabled, this feature is dropped (i.e. set to `nil`). - Add validation for `CustomResourceSubResources`: - `SpecReplicasPath` should not be empty and should be a valid json path under `.spec`. If there is no value under the given path in the CustomResource, the `/scale` subresource will return an error on GET. - `StatusReplicasPath` should not be empty and should be a valid json path under `.status`. If there is no value under the given path in the CustomResource, the status replica value in the /scale subresource will default to 0. - If present, `LabelSelectorPath` should be a valid json path. If there is no value under `LabelSelectorPath` in the CustomResource, the status label selector value in the `/scale` subresource will default to the empty string. - `ScaleGroupVersion` should be `autoscaling/v1`. - If `CustomResourceSubResources` is enabled, only `properties` is allowed under the root schema for CRD validation. **Add status and scale subresources**: - Use helper functions from `apimachinery/pkg/apis/meta/v1/unstructured/helpers.go`. - Improve error handling: https://github.com/kubernetes/kubernetes/pull/56563, https://github.com/kubernetes/kubernetes/pull/58215. - Introduce Registry interface for storage. - Update storage: - Introduce `CustomResourceStorage` which acts as storage for the custom resource and its status and scale subresources. Note: storage for status and scale is only enabled when the feature gate is enabled _and_ the respective fields are enabled in the CRD. - Introduce `StatusREST` and its `New()`, `Get()` and `Update()` methods. - Introduce `ScaleREST` and its `New()`, `Get()` and `Update()` methods. - Get and Update use the json paths from the CRD and use it to return an `autoscaling/v1.Scale` object. - Update strategy: - In `PrepareForCreate`, - Clear `.status`. - Set `.metadata.generation` = 1 - In `PrepareForUpdate`, - Do not update `.status`. - If both the old and new objects have `.status` and it is changed, set it back to its old value. - If the old object has a `.status` but the new object doesn't, set it to the old value. - If old object did not have a `.status` but the new object does, delete it. - Increment generation if spec changes i.e. in the following cases: - If both the old and new objects had `.spec` and it changed. - If the old object did not have `.spec` but the new object does. - If the old object had a `.spec` but the new object doesn't. - In `Validate` and `ValidateUpdate`, - ensure that values at `specReplicasPath` and `statusReplicasPath` are >=0 and < maxInt32. - make sure there are no errors in getting the value at all the paths. - Introduce `statusStrategy` with its methods. - In `PrepareForUpdate`: - Do not update `.spec`. - If both the old and new objects have `.spec` and it is changed, set it back to its old value. - If the old object has a `.spec` but the new object doesn't, set it to the old value. - If old object did not have a `.spec` but the new object does, delete it. - Do not update `.metadata`. - In `ValidateStatusUpdate`: - For CRD validation, validate only under `.status`. - Validate value at `statusReplicasPath` as above. If `labelSelectorPath` is a path under `.status`, then validate it as well. - Plug into the custom resource handler: - Store all three storage - customResource, status and scale in `crdInfo`. - Use the storage as per the subresource in the request. - Use the validator as per the subresource (for status, only use the schema for `status`, if present). - Serve the endpoint as per the subresource - see `serveResource`, `serveStatus` and `serveScale`. - Update discovery by adding the `/status` and `/scale` resources, if enabled. **Add tests**: - Add unit tests in `etcd_test.go`. - Add integration tests. - In `subresources_test.go`, use the [polymporphic scale client](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/client-go/scale) to get and update `Scale`. - Add a test to check everything works fine with yaml in `yaml_test.go`. **Release note**: ```release-note `/status` and `/scale` subresources are added for custom resources. ```
314 lines
12 KiB
Go
314 lines
12 KiB
Go
/*
|
|
Copyright 2017 The Kubernetes Authors.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
*/
|
|
|
|
package features
|
|
|
|
import (
|
|
apiextensionsfeatures "k8s.io/apiextensions-apiserver/pkg/features"
|
|
genericfeatures "k8s.io/apiserver/pkg/features"
|
|
utilfeature "k8s.io/apiserver/pkg/util/feature"
|
|
)
|
|
|
|
const (
|
|
// Every feature gate should add method here following this template:
|
|
//
|
|
// // owner: @username
|
|
// // alpha: v1.X
|
|
// MyFeature utilfeature.Feature = "MyFeature"
|
|
|
|
// owner: @tallclair
|
|
// beta: v1.4
|
|
AppArmor utilfeature.Feature = "AppArmor"
|
|
|
|
// owner: @mtaufen
|
|
// alpha: v1.4
|
|
DynamicKubeletConfig utilfeature.Feature = "DynamicKubeletConfig"
|
|
|
|
// owner: @pweil-
|
|
// alpha: v1.5
|
|
//
|
|
// Default userns=host for containers that are using other host namespaces, host mounts, the pod
|
|
// contains a privileged container, or specific non-namespaced capabilities (MKNOD, SYS_MODULE,
|
|
// SYS_TIME). This should only be enabled if user namespace remapping is enabled in the docker daemon.
|
|
ExperimentalHostUserNamespaceDefaultingGate utilfeature.Feature = "ExperimentalHostUserNamespaceDefaulting"
|
|
|
|
// owner: @vishh
|
|
// alpha: v1.5
|
|
//
|
|
// Ensures guaranteed scheduling of pods marked with a special pod annotation `scheduler.alpha.kubernetes.io/critical-pod`
|
|
// and also prevents them from being evicted from a node.
|
|
// Note: This feature is not supported for `BestEffort` pods.
|
|
ExperimentalCriticalPodAnnotation utilfeature.Feature = "ExperimentalCriticalPodAnnotation"
|
|
|
|
// owner: @vishh
|
|
// alpha: v1.6
|
|
//
|
|
// This is deprecated and will be removed in v1.11. Use DevicePlugins instead.
|
|
//
|
|
// Enables support for GPUs as a schedulable resource.
|
|
// Only Nvidia GPUs are supported as of v1.6.
|
|
// Works only with Docker Container Runtime.
|
|
Accelerators utilfeature.Feature = "Accelerators"
|
|
|
|
// owner: @jiayingz
|
|
// alpha: v1.8
|
|
//
|
|
// Enables support for Device Plugins
|
|
// Only Nvidia GPUs are tested as of v1.8.
|
|
DevicePlugins utilfeature.Feature = "DevicePlugins"
|
|
|
|
// owner: @gmarek
|
|
// alpha: v1.6
|
|
//
|
|
// Changes the logic behind evicting Pods from not ready Nodes
|
|
// to take advantage of NoExecute Taints and Tolerations.
|
|
TaintBasedEvictions utilfeature.Feature = "TaintBasedEvictions"
|
|
|
|
// owner: @jcbsmpsn
|
|
// alpha: v1.7
|
|
//
|
|
// Gets a server certificate for the kubelet from the Certificate Signing
|
|
// Request API instead of generating one self signed and auto rotates the
|
|
// certificate as expiration approaches.
|
|
RotateKubeletServerCertificate utilfeature.Feature = "RotateKubeletServerCertificate"
|
|
|
|
// owner: @jcbsmpsn
|
|
// alpha: v1.7
|
|
//
|
|
// Automatically renews the client certificate used for communicating with
|
|
// the API server as the certificate approaches expiration.
|
|
RotateKubeletClientCertificate utilfeature.Feature = "RotateKubeletClientCertificate"
|
|
|
|
// owner: @msau42
|
|
// alpha: v1.7
|
|
//
|
|
// A new volume type that supports local disks on a node.
|
|
PersistentLocalVolumes utilfeature.Feature = "PersistentLocalVolumes"
|
|
|
|
// owner: @jinxu
|
|
// alpha: v1.7
|
|
//
|
|
// New local storage types to support local storage capacity isolation
|
|
LocalStorageCapacityIsolation utilfeature.Feature = "LocalStorageCapacityIsolation"
|
|
|
|
// owner: @gnufied
|
|
// alpha: v1.8
|
|
// Ability to Expand persistent volumes
|
|
ExpandPersistentVolumes utilfeature.Feature = "ExpandPersistentVolumes"
|
|
|
|
// owner: @verb
|
|
// alpha: v1.10
|
|
//
|
|
// Allows running a "debug container" in a pod namespaces to troubleshoot a running pod.
|
|
DebugContainers utilfeature.Feature = "DebugContainers"
|
|
|
|
// owner: @verb
|
|
// alpha: v1.10
|
|
//
|
|
// Allows all containers in a pod to share a process namespace.
|
|
PodShareProcessNamespace utilfeature.Feature = "PodShareProcessNamespace"
|
|
|
|
// owner: @bsalamat
|
|
// alpha: v1.8
|
|
//
|
|
// Add priority to pods. Priority affects scheduling and preemption of pods.
|
|
PodPriority utilfeature.Feature = "PodPriority"
|
|
|
|
// owner: @resouer
|
|
// alpha: v1.8
|
|
//
|
|
// Enable equivalence class cache for scheduler.
|
|
EnableEquivalenceClassCache utilfeature.Feature = "EnableEquivalenceClassCache"
|
|
|
|
// owner: @k82cn
|
|
// alpha: v1.8
|
|
//
|
|
// Taint nodes based on their condition status for 'NetworkUnavailable',
|
|
// 'MemoryPressure', 'OutOfDisk' and 'DiskPressure'.
|
|
TaintNodesByCondition utilfeature.Feature = "TaintNodesByCondition"
|
|
|
|
// owner: @jsafrane
|
|
// beta: v1.10
|
|
//
|
|
// Enable mount propagation of volumes.
|
|
MountPropagation utilfeature.Feature = "MountPropagation"
|
|
|
|
// owner: @ConnorDoyle
|
|
// alpha: v1.8
|
|
//
|
|
// Alternative container-level CPU affinity policies.
|
|
CPUManager utilfeature.Feature = "CPUManager"
|
|
|
|
// owner: @derekwaynecarr
|
|
// beta: v1.10
|
|
//
|
|
// Enable pods to consume pre-allocated huge pages of varying page sizes
|
|
HugePages utilfeature.Feature = "HugePages"
|
|
|
|
// owner @brendandburns
|
|
// alpha: v1.9
|
|
//
|
|
// Enable nodes to exclude themselves from service load balancers
|
|
ServiceNodeExclusion utilfeature.Feature = "ServiceNodeExclusion"
|
|
|
|
// owner @brendandburns
|
|
// deprecated: v1.10
|
|
//
|
|
// Enable the service proxy to contact external IP addresses. Note this feature is present
|
|
// only for backward compatibility, it will be removed in the 1.11 release.
|
|
ServiceProxyAllowExternalIPs utilfeature.Feature = "ServiceProxyAllowExternalIPs"
|
|
|
|
// owner: @jsafrane
|
|
// alpha: v1.9
|
|
//
|
|
// Enable running mount utilities in containers.
|
|
MountContainers utilfeature.Feature = "MountContainers"
|
|
|
|
// owner: @msau42
|
|
// alpha: v1.9
|
|
//
|
|
// Extend the default scheduler to be aware of PV topology and handle PV binding
|
|
// Before moving to beta, resolve Kubernetes issue #56180
|
|
VolumeScheduling utilfeature.Feature = "VolumeScheduling"
|
|
|
|
// owner: @vladimirvivien
|
|
// alpha: v1.9
|
|
//
|
|
// Enable mount/attachment of Container Storage Interface (CSI) backed PVs
|
|
CSIPersistentVolume utilfeature.Feature = "CSIPersistentVolume"
|
|
|
|
// owner @MrHohn
|
|
// alpha: v1.9
|
|
//
|
|
// Support configurable pod DNS parameters.
|
|
CustomPodDNS utilfeature.Feature = "CustomPodDNS"
|
|
|
|
// owner: @screeley44
|
|
// alpha: v1.9
|
|
//
|
|
// Enable Block volume support in containers.
|
|
BlockVolume utilfeature.Feature = "BlockVolume"
|
|
|
|
// owner: @pospispa
|
|
// beta: v1.10
|
|
//
|
|
// Postpone deletion of a PV or a PVC when they are being used
|
|
StorageObjectInUseProtection utilfeature.Feature = "StorageObjectInUseProtection"
|
|
|
|
// owner: @aveshagarwal
|
|
// alpha: v1.9
|
|
//
|
|
// Enable resource limits priority function
|
|
ResourceLimitsPriorityFunction utilfeature.Feature = "ResourceLimitsPriorityFunction"
|
|
|
|
// owner: @m1093782566
|
|
// beta: v1.9
|
|
//
|
|
// Implement IPVS-based in-cluster service load balancing
|
|
SupportIPVSProxyMode utilfeature.Feature = "SupportIPVSProxyMode"
|
|
|
|
// owner: @dims
|
|
// alpha: v1.10
|
|
//
|
|
// Implement support for limiting pids in pods
|
|
SupportPodPidsLimit utilfeature.Feature = "SupportPodPidsLimit"
|
|
|
|
// owner: @feiskyer
|
|
// alpha: v1.10
|
|
//
|
|
// Enable Hyper-V containers on Windows
|
|
HyperVContainer utilfeature.Feature = "HyperVContainer"
|
|
|
|
// owner: @joelsmith
|
|
// deprecated: v1.10
|
|
//
|
|
// Mount secret, configMap, downwardAPI and projected volumes ReadOnly. Note: this feature
|
|
// gate is present only for backward compatibility, it will be removed in the 1.11 release.
|
|
ReadOnlyAPIDataVolumes utilfeature.Feature = "ReadOnlyAPIDataVolumes"
|
|
|
|
// owner: @k82cn
|
|
// alpha: v1.10
|
|
//
|
|
// Schedule DaemonSet Pods by default scheduler instead of DaemonSet controller
|
|
NoDaemonSetScheduler utilfeature.Feature = "NoDaemonSetScheduler"
|
|
|
|
// owner: @mikedanese
|
|
// alpha: v1.10
|
|
//
|
|
// Implement TokenRequest endpoint on service account resources.
|
|
TokenRequest utilfeature.Feature = "TokenRequest"
|
|
)
|
|
|
|
func init() {
|
|
utilfeature.DefaultFeatureGate.Add(defaultKubernetesFeatureGates)
|
|
}
|
|
|
|
// defaultKubernetesFeatureGates consists of all known Kubernetes-specific feature keys.
|
|
// To add a new feature, define a key for it above and add it here. The features will be
|
|
// available throughout Kubernetes binaries.
|
|
var defaultKubernetesFeatureGates = map[utilfeature.Feature]utilfeature.FeatureSpec{
|
|
AppArmor: {Default: true, PreRelease: utilfeature.Beta},
|
|
DynamicKubeletConfig: {Default: false, PreRelease: utilfeature.Alpha},
|
|
ExperimentalHostUserNamespaceDefaultingGate: {Default: false, PreRelease: utilfeature.Beta},
|
|
ExperimentalCriticalPodAnnotation: {Default: false, PreRelease: utilfeature.Alpha},
|
|
Accelerators: {Default: false, PreRelease: utilfeature.Alpha},
|
|
DevicePlugins: {Default: false, PreRelease: utilfeature.Alpha},
|
|
TaintBasedEvictions: {Default: false, PreRelease: utilfeature.Alpha},
|
|
RotateKubeletServerCertificate: {Default: false, PreRelease: utilfeature.Alpha},
|
|
RotateKubeletClientCertificate: {Default: true, PreRelease: utilfeature.Beta},
|
|
PersistentLocalVolumes: {Default: true, PreRelease: utilfeature.Beta},
|
|
LocalStorageCapacityIsolation: {Default: false, PreRelease: utilfeature.Alpha},
|
|
HugePages: {Default: true, PreRelease: utilfeature.Beta},
|
|
DebugContainers: {Default: false, PreRelease: utilfeature.Alpha},
|
|
PodShareProcessNamespace: {Default: false, PreRelease: utilfeature.Alpha},
|
|
PodPriority: {Default: false, PreRelease: utilfeature.Alpha},
|
|
EnableEquivalenceClassCache: {Default: false, PreRelease: utilfeature.Alpha},
|
|
TaintNodesByCondition: {Default: false, PreRelease: utilfeature.Alpha},
|
|
MountPropagation: {Default: true, PreRelease: utilfeature.Beta},
|
|
ExpandPersistentVolumes: {Default: false, PreRelease: utilfeature.Alpha},
|
|
CPUManager: {Default: true, PreRelease: utilfeature.Beta},
|
|
ServiceNodeExclusion: {Default: false, PreRelease: utilfeature.Alpha},
|
|
MountContainers: {Default: false, PreRelease: utilfeature.Alpha},
|
|
VolumeScheduling: {Default: true, PreRelease: utilfeature.Beta},
|
|
CSIPersistentVolume: {Default: true, PreRelease: utilfeature.Beta},
|
|
CustomPodDNS: {Default: false, PreRelease: utilfeature.Alpha},
|
|
BlockVolume: {Default: false, PreRelease: utilfeature.Alpha},
|
|
StorageObjectInUseProtection: {Default: true, PreRelease: utilfeature.Beta},
|
|
ResourceLimitsPriorityFunction: {Default: false, PreRelease: utilfeature.Alpha},
|
|
SupportIPVSProxyMode: {Default: false, PreRelease: utilfeature.Beta},
|
|
SupportPodPidsLimit: {Default: false, PreRelease: utilfeature.Alpha},
|
|
HyperVContainer: {Default: false, PreRelease: utilfeature.Alpha},
|
|
NoDaemonSetScheduler: {Default: false, PreRelease: utilfeature.Alpha},
|
|
TokenRequest: {Default: false, PreRelease: utilfeature.Alpha},
|
|
|
|
// inherited features from generic apiserver, relisted here to get a conflict if it is changed
|
|
// unintentionally on either side:
|
|
genericfeatures.StreamingProxyRedirects: {Default: true, PreRelease: utilfeature.Beta},
|
|
genericfeatures.AdvancedAuditing: {Default: true, PreRelease: utilfeature.Beta},
|
|
genericfeatures.APIResponseCompression: {Default: false, PreRelease: utilfeature.Alpha},
|
|
genericfeatures.Initializers: {Default: false, PreRelease: utilfeature.Alpha},
|
|
genericfeatures.APIListChunking: {Default: true, PreRelease: utilfeature.Beta},
|
|
|
|
// inherited features from apiextensions-apiserver, relisted here to get a conflict if it is changed
|
|
// unintentionally on either side:
|
|
apiextensionsfeatures.CustomResourceValidation: {Default: true, PreRelease: utilfeature.Beta},
|
|
apiextensionsfeatures.CustomResourceSubresources: {Default: false, PreRelease: utilfeature.Alpha},
|
|
|
|
// features that enable backwards compatibility but are scheduled to be removed
|
|
ServiceProxyAllowExternalIPs: {Default: false, PreRelease: utilfeature.Deprecated},
|
|
ReadOnlyAPIDataVolumes: {Default: true, PreRelease: utilfeature.Deprecated},
|
|
}
|