Automatic merge from submit-queue
Add validation that detects repeated keys in the labels and annotations maps
Fixes#2965 (a nearly 2 year old feature request!)
@kubernetes/kubectl
@eparis
Automatic merge from submit-queue
Adding default StorageClass annotation printout for resource_printer and describer and some refactoring
adding ISDEFAULT for _kubectl get storageclass_ output
```
[root@screeley-sc1 gce]# kubectl get storageclass
NAME TYPE ISDEFAULT
another-class kubernetes.io/gce-pd NO
generic1-slow kubernetes.io/gce-pd YES
generic2-fast kubernetes.io/gce-pd YES
```
```release-note
Add ISDEFAULT to kubectl get storageClass output
```
@kubernetes/sig-storage
Automatic merge from submit-queue
Split conversion and defaulting
Separate conversion and defaulting. Defaulting occurs mixed with conversion today - change the server so that the `VersioningCodec` performs defaulting on the external type during decoding.
* Add a new method to `Scheme` - `func (*runtime.Scheme) Default(runtime.Object)` - that takes an object and performs defaulting.
* Call `Default` during decoding and at static initialization time
* Use the new `defaulter-gen` to generate top level object defaulters (`v1.Pod`) at build time for any type that needs to perform defaulting.
* Add tests and alter the existing code to adapt as necessary
* Fix a few bugs in conversions that depended on defaulting behavior
---
Step 1 of decoupling conversion and defaulting. The generator will assist in creating top level defaulters that in a single method invoke all nested defaulters, preventing the need to recurse via reflection or conversion. These top level defaulters will be registered in the scheme and invoked instead of the nested recursion path. This will set the stage for a future generator, capable of creating defaulters from embedded struct tags on external types. However, we must gradually switch these over.
The immediate goal here is to split defaulting and conversion so that the unsafe convertor can be used to maximum potential (we would be able to use direct memory conversion for any identical nested struct, even those that must be defaulted).
The generator uses `k8s:defaulter-gen=TypeMeta` on most public packages to flag any top level type that has defaulters to get a `SetObjectDefaults_NAME` function created (types that don't have defaulters won't have functions). This also creates a `RegisterDefaults` method that applies a default to an interface{} and returns true if the object was handled. Existing defaults are left as is.
Add a test to verify old and new path generate the same outcomes. Defaulter will move to gengo before this is merged, and subsequent PRs will remove defaulting during conversion and have the VersioningCodec apply defaults.
Most normal codec use should perform defaulting. DirectCodecs should not
perform defaulting. Update the defaulting_test to fuzz the list of known
defaulters. Use the new versioning.NewDefaultingCodec() method.
Automatic merge from submit-queue
Add PSP support for seccomp profiles
Seccomp support for PSP. There are still a couple of TODOs that need to be fixed but this is passing tests.
One thing of note, since seccomp is all being stored in annotations right now it breaks some of the assumptions we've stated for the provider in terms of mutating the passed in pod. I've put big warning comments around the pieces that do that to make sure it's clear and covered the rollback in admission if the policy fails to validate.
@sttts @pmorie @erictune @smarterclayton @liggitt
Automatic merge from submit-queue
Add unit test for bad ReclaimPolicy and valid ReclaimPolicy in /pkg/api/validation
unit tests for validation.go regarding PersistentVolumeReclaimPolicy (bad value and good value)
see PR: #30304
Automatic merge from submit-queue
Add PVC storage to LimitRange
This PR adds the ability to add a LimitRange to a namespace that enforces min/max on `pvc.Spec.Resources.Requests["storage"]`.
@derekwaynecarr @abhgupta @kubernetes/sig-storage
Examples forthcoming.
```release-note
pvc.Spec.Resources.Requests min and max can be enforced with a LimitRange of type "PersistentVolumeClaim" in the namespace
```
Automatic merge from submit-queue
Match GroupVersionKind against specific version
Currently when multiple GVK match a specific kind in `KindForGroupVersionKinds` only the first will be matched, which not necessarily will be the correct one. I'm proposing to extend this to pick the best match, instead.
Here's my problematic use-case, of course it involves ScheduledJobs 😉:
I have a `GroupVersions` with `batch/v1` and `batch/v2alpha1` in that order. I'm calling `KindForGroupVersionKinds` with kind `batch/v2alpha1 ScheduledJob` and that currently results this matching first `GroupVersion`, instead of picking more concrete one. There's a [clear description](ee77d4e6ca/pkg/api/unversioned/group_version.go (L183)) why it is on single `GroupVersion`, but `GroupVersions` should pick this more carefully.
@deads2k this is your baby, wdyt?
Contination of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
I rolled back some of this from a previous commit because it just got
to big/messy. Will follow up with additional PRs
Signed-off-by: Doug Davis <dug@us.ibm.com>
Automatic merge from submit-queue
Dynamic provisioning for flocker volume plugin
Refactor flocker volume plugin
* [x] Support provisioning beta (#29006)
* [x] Support deletion
* [x] Use bind mounts instead of /flocker in containers
* [x] support ownership management or SELinux relabeling.
* [x] adds volume specification via datasetUUID (this is guranted to be unique)
I based my refactor work to replicate pretty much GCE-PD behaviour
**Related issues**: #29006#26908
@jsafrane @mattbates @wallrj @wallnerryan
Automatic merge from submit-queue
Revert "Work around the etcd watch issue"
Reverts kubernetes/kubernetes#33101
Since #33393 is merged, the bug should have been fixed.
We had another bug where we confused the hostname with the NodeName.
To avoid this happening again, and to make the code more
self-documenting, we use types.NodeName (a typedef alias for string)
whenever we are referring to the Node.Name.
A tedious but mechanical commit therefore, to change all uses of the
node name to use types.NodeName
Also clean up some of the (many) places where the NodeName is referred
to as a hostname (not true on AWS), or an instanceID (not true on GCE),
etc.
* flocker datasets should be attached using an unique identifier. This
is not the case for the name metadata used by datasetName
* allow only one of datasetUUID / datasetName specified
Automatic merge from submit-queue
Allow garbage collection to work against different API prefixes
The GC needs to build clients based only on Resource or Kind. Hoist the
restmapper out of the controller and the clientpool, support a new
ClientForGroupVersionKind and ClientForGroupVersionResource, and use the
appropriate one in both places.
Allows OpenShift to use the GC
Automatic merge from submit-queue
unify available api group versions in our scripts
There are currently many parallel lists of available group versions with slightly different syntaxes in each one. This collapses them into a single list for us to maintain.
Also caught spots where the lists didn't match before.
@sttts @ncdc
The GC needs to build clients based only on Resource or Kind. Hoist the
restmapper out of the controller and the clientpool, support a new
ClientForGroupVersionKind and ClientForGroupVersionResource, and use the
appropriate one in both places.
Automatic merge from submit-queue
Fix backward compatibility issue caused by promoting initcontainers f…
#31026 moves init-container feature from alpha to beta, but only took care the backward compatibility for pod specification, not deal with status. For status, it simply moved from `pods.beta.kubernetes.io/init-container-statuses` to
`pods.beta.kubernetes.io/init-container-statuses` instead of introducing one more pods.beta.kubernetes.io/init-container-statuses. This breaks when the cluster is running with 1.4, but the user is still running with kubectl 1.3.x.
Fixed#32711
Automatic merge from submit-queue
Specific error message on failed rolling update issued by older kubectl against 1.4 master
Fix#32706
`kubernetes-e2e-gke-1.4-1.3-kubectl-skew` (1.3 kubectl and 1.4 master) test suite failed with:
```
k8s.io] Kubectl client [k8s.io] Kubectl rolling-update should support rolling-update to same image [Conformance]
...
Error from server: object is being deleted: replicationcontrollers "e2e-test-nginx-rc" already exists error: exit status 1 not to have occurred
```
It's because the old RC had an orphanFinalizer, so it is not deleted from the key-value store immediately. In turn, the creation of the new RC of the same name failed.
In this failure, the RC and pods are updated, it's just that the RC is of different name, i.e., original name + a hash generated based on podTemplate. The error is confusing to user, but not that bad. So this PR just prints a warning message to instruct users how to work around.
1.4 kubectl rolling-update uses different logic so it's working.
@lavalamp @gmarek @janetkuo @pwittrock
cc @liggitt for the ctx changes.
Automatic merge from submit-queue
Centralize install code
Trying to figure out a way to do this that makes the changes as painless to roll out as possible. This is going to be a multi-step process...
Automatic merge from submit-queue
Allow webhook authenticator to use TokenReviewsInterface
Refactors the authentication webhook to be able to be fed a kubeconfig file or a TokenReviewsInterface
Fixes a bug with webhooks not retrying on 429/500 errors
Related to https://github.com/kubernetes/kubernetes/pull/32518#discussion_r78450170
Automatic merge from submit-queue
Change rbac roleref to reflect the information we want
@liggitt @ericchiang This is a version of https://github.com/kubernetes/kubernetes/pull/31359 which updates the `RoleRef` to be (I think) the type that we want, with a group, resource, and name.
This is **not** backwards compatible with any existing data. I'm ok with doing this since rbac was considered alpha, but its something to consider.
If we want this instead, I'll close the previous pull (or update it with this content).
Automatic merge from submit-queue
update taints e2e, restrict taints operation with key, effect
Since taints are now unique by key, effect on a node, this PR is to restrict existing taints adding/removing/updating operations in taints e2e.
Also fixes https://github.com/kubernetes/kubernetes/issues/31066#issuecomment-242870101
Related prior Issue/PR #29362 and #30590
Automatic merge from submit-queue
add group prefix matching for kubectl usage
Adds prefix matching for groups so that `kubectl get storageclass.storage` matches `storageclass.storage.k8s.io`.
@kubernetes/kubectl
Automatic merge from submit-queue
Split path validation into a separate library
This PR splits path segment validation into it's own package. This cuts off one of the restclient's dependency paths to some docker packages, and completely eliminates its dependency on go-restful swagger validation.
cc @kubernetes/sig-api-machinery
Automatic merge from submit-queue
Generate versioned client for pvc
I noticed while trying to use client-go that there is no way to get PVC with versioned clientset. I don't know why it is excluded, I cannot find any discussion about this, so I am creating this assuming it is just an oversight...
Automatic merge from submit-queue
Add AppArmor feature gate
Add option to disable AppArmor via a feature gate. This PR treats AppArmor as Beta, and thus depends on https://github.com/kubernetes/kubernetes/pull/31471 (I will remove `do-not-merge` once that merges).
Note that disabling AppArmor means that pods with AppArmor annotations will be rejected in validation. It does not mean that the components act as though AppArmor was never implemented. This is by design, because we want to make it difficult to accidentally run a Pod with an AppArmor annotation without AppArmor protection.
/cc @dchen1107
Automatic merge from submit-queue
Add validation preventing recycle of / in a hostPath PV
Adds a validation that prevents a user from recycling `/` when it is used in a hostPath PV
cc @kubernetes/sig-storage
Automatic merge from submit-queue
add validation for PV spec to ensure correct values are used for ReclaimPolicy on initial create
k8 currently allows invalid values for ReclaimPolicy (i.e. "scotto") - this allows the PV to be created and even bound, however, when the pvc or pod is deleted and the recycler is triggered, an error is thrown
```
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
36s 36s 1 {persistentvolume-controller } Warning VolumeUnknownReclaimPolicy Volume has unrecognized PersistentVolumeReclaimPolicy
```
New behavior will not allow the user to create the PV:
```
[root@k8dev nfs]# kubectl create -f nfs-pv-bad.yaml
The PersistentVolume "pv-gce" is invalid: spec.persistentVolumeReclaimPolicy: Unsupported value: "scotto": supported values: Delete, Recycle, Retain
```
Automatic merge from submit-queue
support Azure data disk volume
This is a WIP of supporting azure data disk volume. Will add test and dynamic provisioning support once #29006 is merged
replace #25915fix#23259
@kubernetes/sig-storage
@colemickens @brendandburns
Automatic merge from submit-queue
[GarbageCollector] Allow per-resource default garbage collection behavior
What's the bug:
When deleting an RC with `deleteOptions.OrphanDependents==nil`, garbage collector is supposed to treat it as `deleteOptions.OrphanDependents==true", and orphan the pods created by it. But the apiserver is not doing that.
What's in the pr:
Allow each resource to specify the default garbage collection behavior in the registry. For example, RC registry's default GC behavior is Orphan, and Pod registry's default GC behavior is CascadingDeletion.
Automatic merge from submit-queue
Add readyReplicas to replica sets
@bgrant0607 for the api changes
@bprashanth for the controllers changes
@deads2k fyi
Automatic merge from submit-queue
Move init-container feature from alpha to beta.
```release-note
Moved init-container feature from alpha to beta.
In 1.3, an init container is specified with this annotation key
on the pod or pod template: `pods.alpha.kubernetes.io/init-containers`.
In 1.4, either that key or this key: pods.beta.kubernetes.io/init-containers`,
can be used.
When you GET an object, you will see both annotation keys with the same values.
You can safely roll back from 1.4 to 1.3, and things with init-containers
will still work (pods, deployments, etc).
If you are running 1.3, only use the alpha annotation, or it may be lost when
rolling forward.
The status has moved from annotation key
`pods.beta.kubernetes.io/init-container-statuses` to
`pods.beta.kubernetes.io/init-container-statuses`.
Any code that inspects this annotation should be changed to use the new key.
State of Initialization will continue to be reported in both pods.alpha.kubernetes.io/initialized
and in `podStatus.conditions.{status: "True", type: Initialized}`
```
Mini-design for this change:
Goals:
1. A user can create an object with the beta annotation
on 1.4, and it works. The fact that the annotation has beta
in it communicates to the user that the feature is beta,
and so the user should have confidence in using it. Preferably,
when the user gets the annotation back, he see the beta
annotation.
1) If someone had an existing alpha object in their apiserver,
such as a RS with a pod template with an init-containers
annotation on it, it should continue to work (init containers
run) when stack upgraded to 1.4.
2) If someone is using a chart or blog post that has alpha
annotation on it and they create it on a 1.4 cluster, it should
work.
3) If someone had something with an init container in 1.4
and they roll back stack to 1.3, it should not silently stop
working (init containers don't run anymore).
To meet all these, we mirror an absent beta label from the alpha
key and vice versa. If they are out of sync, we use the alpha
one. We do this in conversion since there was already logic there.
In 1.3 code, all annotations are preserved across a round trip
(v1 -> api -> v1), and the alpha annotation turns into the internal
field that kubelet uses.
In 1.4 code, the alpha annotation is always preserved across
a round trip, and a beta annotation is always set equal to
the alpha one, after a round trip.
Currently, the kubelet always sees the object after a round trip
when it GETs it. But, we don't want to rely on that behavior,
since it will break when fastpath is implemented.
So, we rely on this:
all objects either are created with an alpha annotation (1.3 or 1.4
code) or are created with a beta annotation under 1.4. In the later
case, they are round tripped at creation time, and so get both
annotations. So all subsequent GETs see both labels.
The serviceAccountName is occasionally useful for clients running on
Kube that need to know who they are when talking to other components.
The nodeName is useful for PetSet or DaemonSet pods that need to make
calls back to the API to fetch info about their node.
Both fields are immutable, and cannot easily be retrieved in another
way.
Automatic merge from submit-queue
Make labels, fields expose selectable requirements
What?
This is to change the labels/fields Selector interface and make them expose selectable requirements. We reuse labels.Requirement struct for label selector and add fields.Requirement for field selector.
Why?
In order to index labels/fields, we need them to tell us three things: index key (a field or a label), operator (greater, less, or equal), and value (string, int, etc.). By getting selectable requirements, we are able to pass them down and use them for indexing in storage layer.
Automatic merge from submit-queue
Basic scaler/reaper for petset
Currently scaling or upgrading a petset is more complicated than it should be. Would be nice if this made code freeze on friday. I'm planning on a follow up change with generation number and e2es post freeze.
Automatic merge from submit-queue
change all PredicateFunc to use SelectionPredicate
What?
- This PR changes all PredicateFunc in registry to return SelectionPredicate instead of Matcher interface.
Why?
- We want to pass SelectionPredicate to storage layer. Matcher interface did not expose enough information for indexing.
```relnote
Moved init-container feature from alpha to beta.
In 1.3, an init container is specified with this annotation key
on the pod or pod template: `pods.alpha.kubernetes.io/init-containers`.
In 1.4, either that key or this key: pods.beta.kubernetes.io/init-containers`,
can be used.
When you GET an object, you will see both annotation keys with the same values.
You can safely roll back from 1.4 to 1.3, and things with init-containers
will still work (pods, deployments, etc).
If you are running 1.3, only use the alpha annotation, or it may be lost when
rolling forward.
The status has moved from annotation key
`pods.beta.kubernetes.io/init-container-statuses` to
`pods.beta.kubernetes.io/init-container-statuses`.
Any code that inspects this annotation should be changed to use the new key.
State of Initialization will continue to be reported in both pods.alpha.kubernetes.io/initialized
and in `podStatus.conditions.{status: "True", type: Initialized}`
```
Mini-design for this change:
Goals:
1. A user can create an object with the beta annotation
on 1.4, and it works. The fact that the annotation has beta
in it communicates to the user that the feature is beta,
and so the user should have confidence in using it. Preferably,
when the user gets the annotation back, he see the beta
annotation.
1) If someone had an existing alpha object in their apiserver,
such as a RS with a pod template with an init-containers
annotation on it, it should continue to work (init containers
run) when stack upgraded to 1.4.
2) If someone is using a chart or blog post that has alpha
annotation on it and they create it on a 1.4 cluster, it should
work.
3) If someone had something with an init container in 1.4
and they roll back stack to 1.3, it should not silently stop
working (init containers don't run anymore).
To meet all these, we mirror an absent beta label from the alpha
key and vice versa. If they are out of sync, we use the alpha
one. We do this in conversion since there was already logic there.
In 1.3 code, all annotations are preserved across a round trip
(v1 -> api -> v1), and the alpha annotation turns into the internal
field that kubelet uses.
In 1.4 code, the alpha annotation is always preserved across
a round trip, and a beta annotation is always set equal to
the alpha one, after a round trip.
Currently, the kubelet always sees the object after a round trip
when it GETs it. But, we don't want to rely on that behavior,
since it will break when fastpath is implemented.
So, we rely on this:
all objects either are created with an alpha annotation (1.3 or 1.4
code) or are created with a beta annotation under 1.4. In the later
case, they are round tripped at creation time, and so get both
annotations. So all subsequent GETs see both labels.
Automatic merge from submit-queue
Fix pvc requests.storage validation
A `PersistentVolumeClaim` should not be able to request a negative amount of storage.
/cc @kubernetes/sig-storage @kubernetes/rh-cluster-infra @deads2k
Convert single GV and lists of GVs into an interface that can handle
more complex scenarios (everything internal, nothing supported). Pass
the interface down into conversion.
Automatic merge from submit-queue
Allow setting permission mode bits on secrets, configmaps and downwardAPI files
cc @thockin @pmorie
Here is the first round to implement: https://github.com/kubernetes/kubernetes/pull/28733.
I made two commits: one with the actual change and the other with the auto-generated code. I think it's easier to review this way, but let me know if you prefer in some other way.
I haven't written any tests yet, I wanted to have a first glance and not write them till this (and the API) are more close to the "LGTM" :)
There are some things:
* I'm not sure where to do the "AND 0777". I'll try to look better in the code base, but suggestions are always welcome :)
* The write permission on group and others is not set when you do an `ls -l` on the running container. It does work with write permissions to the owner. Debugging seems to show that is something happening after this is correctly set on creation. Will look closer.
* The default permission (when the new fields are not specified) are the same that on kubernetes v1.3
* I do realize there are conflicts with master, but I think this is good enough to have a look. The conflicts is with the autog-enerated code, so the actual code is actually the same (and it takes like ~30 minutes to generate it here)
* I didn't generate the docs (`generated-docs` and `generated-swagger-docs` from `hack/update-all.sh`) because my machine runs out of mem. So that's why it isn't in this first PR, will try to investigate and see why it happens.
Other than that, this works fine here with some silly scripts I did to create a secret&configmap&downwardAPI, a pod and check the file permissions. Tested the "defaultMode" and "mode" for all. But of course, will write tests once this is looking fine :)
Thanks a lot again!
Rodrigo
