Automatic merge from submit-queue
Fix GPU resource validation
This fixes scheduling of pods with GPU resources. The change was never upstreamed during the 1.3 beta period, as it got lost in the noise of other changes in our fork. Ooops. I'll submit a cherry-pick request for 1.3.1 as soon as this lands in master.
Because of defaulting, requests are always set if limits are. Thus, the check can never succeed. Instead, make sure that the two values are equal.
Also, remove a few other error messages and remove unnecessary Sprintf calls.
Automatic merge from submit-queue
Undelete generated files
There's been enough people broken by not committing generated code, that we
should undo that until we have a proper client that is `go get` compatible.
This is temporary.
Fixes#28920
Automatic merge from submit-queue
move api.ObjectReference.ObjectKind to pkg/api/ref.go
This is needed to convert k8s components to use versioned clientset.
To let components use versioned client, we need to convert `pkg/client/record` to v1, which depends on `pkg/api/ref.go`, so we need to make a `pkg/api/v1/ref.go`. And this [line](https://github.com/kubernetes/kubernetes/blob/master/pkg/api/ref.go#L44) requires v1.ObjectReference to be a runtime.Object. Moving `api.ObjectReference.ObjectKind` to `pkg/api/ref.go` will make the writing a conversion script easier because all the necessary changes will be restricted in `ref.go`.
There's been enough people broken by not committing generated code, that we
should undo that until we have a proper client that is `go get` compatible.
This is temporary.
Automatic merge from submit-queue
Generate a better Stringer method for proto types
This replaces the bad string output generated by golang/proto with gogo/protobuf stringer generation. Makes the output similar to %#v and more debuggable. We have to have a String() method to implement proto.Message, so this is strictly better.
@wojtek-t, @thockin for after your PR merges
Fixes#28756
Automatic merge from submit-queue
Implement a RESTMappings method
With the introduction of batch/v1 and batch/v2alpha1, any
MultiRESTMapper that has per groupversion mappers (as a naive discovery
client would create) would end up being unable to call RESTMapping() on
batch.Jobs. As we finish up discovery we will need to be able to choose
prioritized RESTMappings based on the service discovery doc.
This change implements RESTMappings(groupversion) which returns all
possible RESTMappings for that kind. That allows a higher level call to
prioritize the returned mappings by server or client preferred version.
@deads2k
Update external KubeletConfiguration type
Add defaults for new KubeletConfiguration fields
Modify some defaults to match upstream settings
Add/rename some conversion functions
Updated codegen
Fixed typos
Mike Danese caught that s.NodeLabels wasn't allocated, fix on line 118
of cmd/kubelet/app/options/options.go.
Provide list of valid sources in comment for HostNetworkSources field
Automatic merge from submit-queue
Support terminal resizing for exec/attach/run
```release-note
Add support for terminal resizing for exec, attach, and run. Note that for Docker, exec sessions
inherit the environment from the primary process, so if the container was created with tty=false,
that means the exec session's TERM variable will default to "dumb". Users can override this by
setting TERM=xterm (or whatever is appropriate) to get the correct "smart" terminal behavior.
```
Fixes#13585
With the introduction of batch/v1 and batch/v2alpha1, any
MultiRESTMapper that has per groupversion mappers (as a naive discovery
client would create) would end up being unable to call RESTMapping() on
batch.Jobs. As we finish up discovery we will need to be able to choose
prioritized RESTMappings based on the service discovery doc.
This change implements RESTMappings(groupversion) which returns all
possible RESTMappings for that kind. That allows a higher level call to
prioritize the returned mappings by server or client preferred version.
Add support for terminal resizing for exec, attach, and run. Note that for Docker, exec sessions
inherit the environment from the primary process, so if the container was created with tty=false,
that means the exec session's TERM variable will default to "dumb". Users can override this by
setting TERM=xterm (or whatever is appropriate) to get the correct "smart" terminal behavior.
Because of defaulting, requests are always set if limits are. Thus, the check can never succeed. Instead, make sure that the two values are equal.
Also, remove a few other error messages and remove unnecessary Sprintf calls.
This mostly takes the previously checked in files and removes them, and moves
the generation to be on-demand instead of manual. Manually verified no change
in generated output.
This allows us to start building real dependencies into Makefile.
Leave old hack/* scripts in place but advise to use 'make'. There are a few
rules that call things like 'go run' or 'build/*' that I left as-is for now.
Automatic merge from submit-queue
Deepcopy: avoid struct copies and reflection Call
- make signature of generated deepcopy methods symmetric with `in *type, out *type`, avoiding copies of big structs on the stack
- switch to `in interface{}, out interface{}` which allows us to call them with without `reflect.Call`
The first change reduces runtime of BenchmarkPodCopy-4 from `> 3500ns` to around `2300ns`.
The second change reduces runtime to around `1900ns`.
Automatic merge from submit-queue
ListOptions: add test for ResourceVersion > 0 in List
ref: #28472
Done:
- Add a test for ResourceVersion > 0 in registry (cache) store List()
- Fix the docs.
Search and replace for references to moved examples
Reverted find and replace paths on auto gen docs
Reverting changes to changelog
Fix bugs in test-cmd.sh
Fixed path in examples README
ran update-all successfully
Updated verify-flags exceptions to include renamed files
This fixes PodSpec to generate cleanly. No other types only half-generate (so
now we Fatalf), though several fail to generate at all (only Errorf for now).
There are ample opportunities to optimize and streamline here. For example,
there's no reason to have a function to convert IntStr to IntStr. Removing the
function does generate the right assignment, but it is unclear whether the
registered function is needed or not. I opted to leave it alone for now.
Another example is Convert_Slice_byte_To_Slice_byte, which just seems silly.
This drives conversion generation from file tags like:
// +conversion-gen=k8s.io/my/internal/version
.. rather than hardcoded lists of packages.
The only net change in generated code can be explained as correct. Previously
it didn't know that conversion was available.
Automatic merge from submit-queue
Prep for not checking in generated, part 1/2
This PR is extracted from #25978 - it is just the deep-copy related parts. All the Makefile and conversion stuff is excluded.
@wojtek-t this is literally branched, a bunch of commits deleted, and a very small number of manual fixups applied. If you think this is easier to review (and if it passes CI) you can feel free to go over it again. I will follow this with a conversion-related PR to build on this.
Or if you prefer, just close this and let the mega-PR ride.
@lavalamp
Automatic merge from submit-queue
Remove reflection path in meta.Accessor
Callers are required to implement their interfaces, removes the
potential for mistakes. We have a reflective test
pkg/api/meta_test.go#TestAccessorImplementations that verifies that all
objects registered to the scheme properly implement their interfaces.
@wojtek-t this may need an additional test or two, but I think this is a
net win over the current state (where people fallback to reflection without
being aware of it).
This is the last piece of Clayton's #26179 to be implemented with file tags.
All diffs are accounted for. Followup will use this to streamline some
packages.
Also add some V(5) debugging - it was helpful in diagnosing various issues, it
may be helpful again.
This drives most of the logic of deep-copy generation from tags like:
// +deepcopy-gen=package
..rather than hardcoded lists of packages. This will make it possible to
subsequently generate code ONLY for packages that need it *right now*, rather
than all of them always.
Also remove pkgs that really do not need deep-copies (no symbols used
anywhere).
This is in prep to simplify tag logic. Don't rely on processing commas as new
tag delimiters. Put new tags on new lines. This had zero effect on generated
code (as intended).
In bringing back Clayton's PR piece-by-piece this was almost as easy to
implement as his version, and is much more like what I think we should be
doing.
Specifically, any time which defines a .DeepCopy() method will have that method
called preferentially. Otherwise we generate our own functions for
deep-copying. This affected exactly one type - resource.Quantity. In applying
this heuristic, several places in the generated code were simplified.
To achieve this I had to convert types.Type.Methods from a slice to a map,
which seems correct anyway (to do by-name lookups).
His PR cam during the middle of this development cycle, and it was easier to
burn it down and recreate it than try to patch it into an existing series and
re-test every assumption. This behavior will be re-introduced in subsequent
commits.
Automatic merge from submit-queue
Validation: Make validation func return error strings
Part of an ongoing series of validation cleanups.
This centralizes the error strings next to the code that checks the error conditions. Future commits will refine the messages further and provide more utility validators.
I'm OK if this doesn't go into 1.2, but I am tired of rebasing :) I suggest commit-by-commit review, which should go pretty quickly. This was largely mechanical.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/21240)
<!-- Reviewable:end -->
Callers are required to implement their interfaces, removes the
potential for mistakes. We have a reflective test
pkg/api/meta_test.go#TestAccessorImplementations that verifies that all
objects registered to the scheme properly implement their interfaces.
Automatic merge from submit-queue
selector: make sure value of GT and LT is integer
GT and LT in selector has been introduced in Node Affinity feature: https://github.com/kubernetes/kubernetes/pull/19758, https://github.com/kubernetes/kubernetes/pull/18261
According to the API:
> If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer.
But the implementation has parsed it as float64:
ef0c9f0c5b/pkg/labels/selector.go (L183)
Modeling integer as float is dangerous. We don't even have fixed precision guarantee when doing comparison.
This PR is to get rid of this pre-optimization and convert **integer** to int64.
Automatic merge from submit-queue
Use `CreatedByAnnotation` constant
A nit but didn't want the strings to get out of sync.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Automatic merge from submit-queue
Add additional testing scenarios for compute resource requests=0
I was asked about the qos tier of a pod that specified
`--requests=cpu=0,memory=0 --limits=cpu=100m,memory=1Gi`
and in just investigating current behavior, realized we should have an explicit test case to ensure that 0 values are preserved in defaulting passes, and that this is still a burstable pod (but the lowest for that tier as it related to eviction)
/cc @vishh
Automatic merge from submit-queue
TLS bootstrap API group (alpha)
This PR only covers the new types and related client/storage code- the vast majority of the line count is codegen. The implementation differs slightly from the current proposal document based on discussions in design thread (#20439). The controller logic and kubelet support mentioned in the proposal are forthcoming in separate requests.
I submit that #18762 ("Creating a new API group is really hard") is, if anything, understating it. I've tried to structure the commits to illustrate the process.
@mikedanese @erictune @smarterclayton @deads2k
```release-note-experimental
An alpha implementation of the the TLS bootstrap API described in docs/proposals/kubelet-tls-bootstrap.md.
```
[]()
Automatic merge from submit-queue
make quota validation re-useable
Break quota validation into smaller functions to allow for object re-use.
@derekwaynecarr
Automatic merge from submit-queue
Add WrapUpdatedObjectInfo helper
This makes it easier to attach checks/transformations to the updated object in storage Update functions, while still keeping the data flow intact (so admission, patch, and other injected checks continue to work as intended), without needing to do anything tricky to get the updated object out of the UpdatedObjectInfo introduced in https://github.com/kubernetes/kubernetes/pull/25787
This is especially useful when one storage is delegating to another, but wants its checks to be run in the heart of the eventual GuaranteedUpdate call.
Automatic merge from submit-queue
Check for an empty value in validateField
```release-note
* Fix a panic when args was not supplied with any values.
```
reflect.TypeOf() can take a nil (it then returns a nil), but
Kind() panics on a nil.
Now the user gets the following output:
./kubectl.sh --server=http://localhost:8080 create -f ../../test-files/test-rc.yaml
error validating "../../test-files/test-rc.yaml": error validating data: unexpected nil value for field spec.template.spec.containers[0].args[0]; if you choose to ignore these errors, turn validation off with --validate=false
fixes#20627 and fixes#26927
Automatic merge from submit-queue
let dynamic client handle non-registered ListOptions
And register v1.ListOptions in the policy group.
Fix#27622
@lavalamp @smarterclayton @krousey
Automatic merge from submit-queue
Kubelet Volume Manager Wait For Attach Detach Controller and Backoff on Error
* Closes https://github.com/kubernetes/kubernetes/issues/27483
* Modified Attach/Detach controller to report `Node.Status.AttachedVolumes` on successful attach (unique volume name along with device path).
* Modified Kubelet Volume Manager wait for Attach/Detach controller to report success before proceeding with attach.
* Closes https://github.com/kubernetes/kubernetes/issues/27492
* Implemented an exponential backoff mechanism for for volume manager and attach/detach controller to prevent operations (attach/detach/mount/unmount/wait for controller attach/etc) from executing back to back unchecked.
* Closes https://github.com/kubernetes/kubernetes/issues/26679
* Modified volume `Attacher.WaitForAttach()` methods to uses the device path reported by the Attach/Detach controller in `Node.Status.AttachedVolumes` instead of calling out to cloud providers.
Automatic merge from submit-queue
Allow emitting PersistentVolume events.
Similarly to Nodes, PersistentVolumes are not in any namespace and we should
not block events on them. Currently, these events are rejected with
`Event "nfs.145841cf9c8cfaf0" is invalid: involvedObject.namespace: Invalid value: "": does not match involvedObject`
Automatic merge from submit-queue
Remove EncodeToStream(..., []unversioned.GroupVersion)
Was not being used. Is a signature change and is necessary for post 1.3 work on Templates and other objects that nest objects.
Extracted from #26044
Automatic merge from submit-queue
ObjectMeta, ListMeta, and TypeMeta should implement their interfaces
Make unversioned.ListMeta implement List. Update all the *List types so they implement GetListMeta.
This helps avoid using reflection to get list information.
Remove all unnecessary boilerplate, move the interfaces to the right
places, and add a test that verifies that objects implement one, the
other, but never both.
@ncdc @lavalamp this supercedes #26964 with the boilerplate removed. Added tests
Automatic merge from submit-queue
Filter seccomp profile path from malicious .. and /
Without this patch with `localhost/<some-releative-path>` as seccomp profile one can load any file on the host, e.g. `localhost/../../../../dev/mem` which is not healthy for the kubelet.
/cc @jfrazelle
Unit tests depend on https://github.com/kubernetes/kubernetes/pull/26710.
This commit adds a new volume manager in kubelet that synchronizes
volume mount/unmount (and attach/detach, if attach/detach controller
is not enabled).
This eliminates the race conditions between the pod creation loop
and the orphaned volumes loops. It also removes the unmount/detach
from the `syncPod()` path so volume clean up never blocks the
`syncPod` loop.
Similarly to Nodes, PersistentVolumes are not in any namespace and we should
not block events on them. Currently, these events are rejected with
'Event "nfs.145841cf9c8cfaf0" is invalid: involvedObject.namespace: Invalid value: "": does not match involvedObject'
Make unversioned.ListMeta implement List. Update all the *List types so they implement GetListMeta.
This helps avoid using reflection to get list information.
Remove all unnecessary boilerplate, move the interfaces to the right
places, and add a test that verifies that objects implement one, the
other, but never both.
