Automatic merge from submit-queue
Unit test quota for nodeport associated with loadbalancer
**What this PR does / why we need it**:
This PR adds unit tests to ensure node ports associated with loadbalancers are charged to quota appropriately. The original PR that added that feature to quota lacked a unit test (https://github.com/kubernetes/kubernetes/pull/39364)
Automatic merge from submit-queue
allow fallthrough handling from go-restful routes
This sets up the gorestful routes to fall through to a default handler and reorders the API to be ahead of the other endpoints. This makes it possible to cleanly support cases of "match, fail, try this other handler" which we'll need for API server composition.
@kubernetes/sig-api-machinery-pr-reviews @ncdc
Automatic merge from submit-queue (batch tested with PRs 42672, 42770, 42818, 42820, 40849)
Return early from eviction debug helpers if !glog.V(3)
Should keep us from running a bunch of loops needlessly.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43653, 43654, 43652)
CRI: Check nil pointer to avoid kubelet panic.
When working on the containerd kubernetes integration, I casually returns an empty `sandboxStatus.Linux{}`, but it cause kubelet to panic.
This won't happen when runtime returns valid data, but we should not make the assumption here.
/cc @yujuhong @feiskyer
When ctrl.kubeClient.Core().PersistentVolumes().Create(volume) returns no err, but storeVolumeUpdate() failed, we save PV sucessfully ,but here err is not nil,
we should not run the codes next in block if err != nil {}
same in the deletion retries below
Automatic merge from submit-queue (batch tested with PRs 43642, 43170, 41813, 42170, 41581)
Enable storage class support in Azure File volume
**What this PR does / why we need it**:
Support StorageClass in Azure file volume
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Support StorageClass in Azure file volume
```
Automatic merge from submit-queue (batch tested with PRs 43642, 43170, 41813, 42170, 41581)
Be able to specify the timeout to wait for pod for kubectl logs/attach
Fixes https://github.com/kubernetes/kubernetes/issues/41786
current flag is `get-pod-timeout`, we can have a discussion if you have better one, default unit is seconds, above 0
@soltysh @kargakis ptal, thanks
@kubernetes/sig-cli-feature-requests
Automatic merge from submit-queue (batch tested with PRs 42522, 42545, 42556, 42006, 42631)
optimize the binding logic of bindClaimToVolume
extract var shouldSetBoundByController and do not need to judge volumename twice
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42522, 42545, 42556, 42006, 42631)
Fixes MountVolume.NewMounter errors not displayed to users via describe events
Fixes#42004
This fixes the problem of mount errors being eaten and not displayed to users again. Specifically erros caught in MountVolume.NewMounter (like missing endpoints, etc...)
Current behavior for any mount failure:
```
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
12m 12m 1 default-scheduler Normal Scheduled Successfully assigned glusterfs-bb-pod1 to 127.0.0.1
10m 1m 5 kubelet, 127.0.0.1 Warning FailedMount Unable to mount volumes for pod "glusterfs-bb-pod1_default(67c9dfa7-f9f5-11e6-aee2-5254003a59cf)": timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". list of unattached/unmounted volumes=[glusterfsvol]
10m 1m 5 kubelet, 127.0.0.1 Warning FailedSync Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". list of unattached/unmounted volumes=[glusterfsvol]
```
New Behavior:
For example on glusterfs - deliberately didn't create endpoints, now correct message is displayed:
```
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
2m 2m 1 default-scheduler Normal Scheduled Successfully assigned glusterfs-bb-pod1 to 127.0.0.1
54s 54s 1 kubelet, 127.0.0.1 Warning FailedMount Unable to mount volumes for pod "glusterfs-bb-pod1_default(8edd2c25-fa09-11e6-92ae-5254003a59cf)": timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". With error timed out waiting for the condition. list of unattached/unmounted volumes=[glusterfsvol]
54s 54s 1 kubelet, 127.0.0.1 Warning FailedSync Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"glusterfs-bb-pod1". With error timed out waiting for the condition. list of unattached/unmounted volumes=[glusterfsvol]
2m 6s 814 kubelet, 127.0.0.1 Warning FailedMount MountVolume.NewMounter failed for volume "kubernetes.io/glusterfs/8edd2c25-fa09-11e6-92ae-5254003a59cf-glusterfsvol" (spec.Name: "glusterfsvol") pod "8edd2c25-fa09-11e6-92ae-5254003a59cf" (UID: "8edd2c25-fa09-11e6-92ae-5254003a59cf") with: endpoints "glusterfs-cluster" not found
```
Automatic merge from submit-queue (batch tested with PRs 42522, 42545, 42556, 42006, 42631)
Use pod sandbox id in checkpoint
**What this PR does / why we need it**: we should log out sandbox id when checkpoint error
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 42237, 42297, 42279, 42436, 42551)
should replace errors.New(fmt.Sprintf(...)) with fmt.Errorf(...)
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)
Add support for bring-your-own ip address for Services on Azure
@colemickens @codablock
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)
Make kubelet never delete files on mounted filesystems
With bug #27653, kubelet could remove mounted volumes and delete user data.
The bug itself is fixed, however our trust in kubelet is significantly lower.
Let's add an extra version of RemoveAll that does not cross mount boundary
(rm -rf --one-file-system).
It calls lstat(path) three times for each removed directory - once in
RemoveAllOneFilesystem and twice in IsLikelyNotMountPoint, however this way
it's platform independent and the directory that is being removed by kubelet
should be almost empty.
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)
create configmap from-env-file
Allow ConfigMaps to be created from Docker based env files.
See proposal https://github.com/kubernetes/community/issues/165
**Release-note:**
```release-note
1. create configmap has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line.
2. create secret has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line.
```
Automatic merge from submit-queue
config set cluster api value
Related bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1420280
**Release note**:
```release-note
release-note-none
```
This patch sets a specified api version if one is given while setting cluster values in kubeconfig
cc @AdoHe @pweil-
Automatic merge from submit-queue
Add approvers to the aws OWNERS file
Without this it was picking up reviewers from a much higher directory.
```release-note
NONE
```
Automatic merge from submit-queue
fix typos
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
small change to ControlleeExpectations result judgement
GetByKey() will never return err, so err != nil {} is redundant,remove it and remove the err return too
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Add gnufied as reviewer for pkg/volume
I have helped review and contributed code to this
area already.
cc @saad-ali @jsafrane @childsb
Automatic merge from submit-queue
fc: Drop multipath.conf snippet
**What this PR does / why we need it**:
Removes multipath.conf - The code does not make use of it - or ensure s that it's getting used - and it should in addition be handled elsewehre.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
A minimalistic multipath.conf got written, but it was useless, as
it is unclear if multipathd is running and there was also no
config reload triggered.
This patch drops this snippet. In general it's probably a better idea
to leave the multipath.conf to the component managing the host.
Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>
Out-of-tree external provisioners have the same purpose as in-tree provisioners. As external provisioners work with PV and PVC datastructures it's an advantage to import certain Kubernetes packages instead of copy-pasting the Kubernetes code.
That's why the constants are made public so that they can be used in an external provisioner.
Each major interface is now in its own file. Any package private
functions that are only referenced by a particular module was also moved
to the corresponding file. All common helper functions were moved to
gce_util.go.
This change is a pure movement of code; no semantic changes were made.
Automatic merge from submit-queue
Fix adding disks to more than one scsi adapter. Fixes#42399
**What this PR does / why we need it**: Allows a single node to use more than 16 disks.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#42399
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix adding disks to more than one scsi adapter.
```
Auto-generated via:
git grep -l [Ss]uccesfully | xargs sed -ri 's/([sS])uccesfully/\1uccessfully/g'
I noticed this when running kube-scheduler with --v4 and it is annoying.
Then manually reverted changed to the vendored bits.
Automatic merge from submit-queue (batch tested with PRs 43533, 43539)
kuberuntime: don't override the pod IP for pods using host network
This fixes the issue of not passing pod IP via downward API for host network pods.
Automatic merge from submit-queue (batch tested with PRs 43465, 43529, 43474, 43521)
kubelet/cni: hook network plugin Status() up to CNI network discovery
Ensure that the plugin returns NotReady status until there is a
CNI network available which can be used to set up pods.
Fixes: https://github.com/kubernetes/kubernetes/issues/43014
I think the only reason it wasn't done like this in the first place was that the dynamic "reread /etc/cni/net.d every 10s forever" was added long after the Status() hook was. What do you think?
@freehan @caseydavenport @luxas @jbeda
Automatic merge from submit-queue
Disable readyReplicas validation for Deployments
Because there is no field in 1.5, when we update to 1.6 and the
controller tries to update the Deployment, it will be denied by
validation because the pre-existing availableReplicas field is greater
than readyReplicas (normally readyReplicas should always be greater or
equal).
Fixes https://github.com/kubernetes/kubernetes/issues/43392
@kubernetes/sig-apps-bugs
Because there is no field in 1.5, when we update to 1.6 and the
controller tries to update the Deployment, it will be denied by
validation because the pre-existing availableReplicas field is greater
than readyReplicas (normally readyReplicas should always be greater or
equal).
Automatic merge from submit-queue
Add validation for affinities and taints/tolerations annotations.
It fixes annotations validation issues for pod/node affinities and taints/tolerations annotations for 1.5 to 1.6 upgrade tests as discussed in the issue https://github.com/kubernetes/kubernetes/issues/43228 .
@davidopp @derekwaynecarr @kubernetes/sig-scheduling-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43481, 43419, 42741, 43480)
controller: work around milliseconds skew in AddAfter
AddAfter is not requeueing precisely after the provided time and may
skew for some millieseconds. This is really important because controllers
don't relist often so a missed check because of ms difference is
essentially dropping the key. For example, in [1] the test requeues a
Deployment for a progress check after 10s[2] but the Deployment is synced
9ms earlier ending up in the controller not recognizing the Deployment as
failed thus dropping it from the queue w/o any error. The drop is fixed by
forcing the controller to resync the Deployment but we are going to resync
after the full duration.
@deads2k if you don't like this I am going to handle this on a case by case basis
[1] https://github.com/kubernetes/kubernetes/issues/39785#issuecomment-279959133
[2] c48b2cab0f/test/e2e/deployment.go (L1122)
Automatic merge from submit-queue
Rate limit HPA controller to sync period
Since the HPA controller pulls information from an external source that
makes no guarantees about consistency, it's possible for the HPA
to get into an infinite update loop -- if the metrics change with
every query, the HPA controller will run it's normal reconcilation,
post a status update, see that status update itself, fetch new metrics,
and if those metrics are different, post another status update, and
repeat. This can lead to continuously updating a single HPA.
By rate-limiting each HPA to once per sync interval, we prevent this
from happening.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Ensure slices are serialized as zero-length, not null
Fixes https://github.com/kubernetes/kubernetes/issues/43203 null serialization of slices to prevent NPE errors in clients that store and expect to receive non-null JSON values in these fields.
Ensures when we are converting to an external slice field that will be serialized even if empty (has `json` tag that does not include `omitempty`), we populate it with `[]`, not `nil`
Other places I considered putting this logic instead:
* When unmarshaling
* Would have to be done for both protobuf and ugorji
* Would still have to be done here (or on marshal) to handle cases where we construct objects to return
* When marshaling
* Would have to switch to use custom json marshaler (currently we use stdlib)
* When defaulting
* Defaulting isn't run on some fields, notably, pod template in rc/deployment spec
* Would still have to be done here (or on marshal) to handle cases where we construct objects to return
```release-note
API fields that previously serialized null arrays as `null` and empty arrays as `[]` no longer distinguish between those values and always output `[]` when serializing to JSON.
```
Automatic merge from submit-queue
Install a REJECT rule for nodeport with no backend
Rather than actually accepting the connection, REJECT. This will avoid
CLOSE_WAIT.
Fixes#43212
@justinsb @felipejfc @spiddy
Automatic merge from submit-queue
Use uid in config.go instead of pod full name.
For https://github.com/kubernetes/kubernetes/issues/43397.
In config.go, use pod uid in pod cache.
Previously, if we update the static pod, even though a new UID is generated in file.go, config.go will only reference the pod with pod full name, and never update the pod UID in the internal cache. This causes:
1) If we change container spec, kubelet will restart the corresponding container because the container hash is changed.
2) If we change pod spec, kubelet will do nothing.
With this fix, kubelet will always restart pod whenever pod spec (including container spec) is changed.
@yujuhong @bowei @dchen1107
/cc @kubernetes/sig-node-bugs
Automatic merge from submit-queue
Use Semantic.DeepEqual to compare DaemonSet template on updates
Switch to `Semantic.DeepEqual` when comparing templates on DaemonSet updates, since we can't distinguish between `null` and `[]` in protobuf. This avoids unnecessary DaemonSet pods restarts.
I didn't touch `reflect.DeepEqual` used in controller because it's close to release date, and the DeepEqual in the controller doesn't cause serious issues (except for maybe causing more enqueues than needed).
Fixes#43218
@liggitt @kargakis @lukaszo @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42452, 43399)
Modify getInstanceByName to avoid calling getInstancesByNames
This PR modify getInstanceByname to loop through all management zones
directly instead of calling getInstancesByNames. Currently
getInstancesByNames use a node name prefix as a filter to list the
instances. If the prefix does not match, it will return all instances
which is very wasteful since getInstanceByName only query one instance
with a specific name.
Partially fix issue #42445
Automatic merge from submit-queue (batch tested with PRs 43398, 43368)
CRI: add support for dns cluster first policy
**What this PR does / why we need it**:
PR #29378 introduces ClusterFirstWithHostNet policy but only dockertools was updated to support the feature.
This PR updates kuberuntime to support it for all runtimes.
**Which issue this PR fixes**
fixes#43352
**Special notes for your reviewer**:
Candidate for v1.6.
**Release note**:
```release-note
NONE
```
cc @thockin @luxas @vefimova @Random-Liu
Automatic merge from submit-queue
Deflake TestSyncDeploymentDeletionRace
**What this PR does / why we need it**:
The cache was sometimes catching up while we were testing the case
where the cache is not yet caught up.
Before this fix, I could reproduce the failure with the following
command. After the fix, it passes.
```
go test -count 100000 -run TestSyncDeploymentDeletionRace
```
I checked the other controllers, and they all were already not starting informers for the deletion race test. I also checked that the deletion race tests for other controllers all pass with `-count 100000`.
**Which issue this PR fixes**:
Fixes#43390
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42659, 43370)
dockershim: process protocol correctly for port mapping
**What this PR does / why we need it**:
dockershim: process protocol correctly for port mapping.
**Which issue this PR fixes**
Fixes#43365.
**Special notes for your reviewer**:
Should be included in v1.6.
**Release note**:
```release-note
NONE
```
cc/ @Random-Liu @justinsb @kubernetes/sig-node-pr-reviews
The cache was sometimes catching up while we were testing the case
where the cache is not yet caught up.
Before this fix, I could reproduce the failure with the following
command. After the fix, it passes.
```
go test -count 100000 -run TestSyncDeploymentDeletionRace
```
lsblk reports FSTYPE of devices with partition tables as empty string "",
which is indistinguishable from empty devices. We must look for dependent
devices (i.e. partitions) to see that the device is really empty and report
error otherwise.
I checked that LVM, LUKS and MD RAID have their own FSTYPE in lsblk output,
so it should be only a partition table that has empty FSTYPE.
The main point of this patch is to run lsblk without "-n", i.e. print all
dependent devices and check if they're there.
PR #29378 introduces ClusterFirstWithHostNet, but docker doesn't support
setting dns options togather with hostnetwork. This commit rewrites
resolv.conf same as dockertools.
PR #29378 introduces ClusterFirstWithHostNet policy but only dockertools
was updated to support the feature. This PR updates kuberuntime to
support it for all runtimes.
Also fixes#43352.
Automatic merge from submit-queue
GC: Fix re-adoption race when orphaning dependents.
**What this PR does / why we need it**:
The GC expects that once it sees a controller with a non-nil
DeletionTimestamp, that controller will not attempt any adoption.
There was a known race condition that could cause a controller to
re-adopt something orphaned by the GC, because the controller is using a
cached value of its own spec from before DeletionTimestamp was set.
This fixes that race by doing an uncached quorum read of the controller
spec just before the first adoption attempt. It's important that this
read occurs after listing potential orphans. Note that this uncached
read is skipped if no adoptions are attempted (i.e. at steady state).
**Which issue this PR fixes**:
Fixes#42639
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42828, 43116)
Apply taint tolerations for NoExecute for all static pods.
Fixed https://github.com/kubernetes/kubernetes/issues/42753
**Release note**:
```
Apply taint tolerations for NoExecute for all static pods.
```
cc/ @davidopp
The GC expects that once it sees a controller with a non-nil
DeletionTimestamp, that controller will not attempt any adoption.
There was a known race condition that could cause a controller to
re-adopt something orphaned by the GC, because the controller is using a
cached value of its own spec from before DeletionTimestamp was set.
This fixes that race by doing an uncached quorum read of the controller
spec just before the first adoption attempt. It's important that this
read occurs after listing potential orphans. Note that this uncached
read is skipped if no adoptions are attempted (i.e. at steady state).
Automatic merge from submit-queue (batch tested with PRs 43313, 43257, 43271, 43307)
Fix AWS untagged instances
To revert to 1.5 behaviour we need to consider untagged
instances if no clusterID has been specified or found.
Fixes https://github.com/kubernetes/kubernetes/issues/43063
cc @justinsb
Automatic merge from submit-queue (batch tested with PRs 43313, 43257, 43271, 43307)
Remove 'all namespaces' meaning of empty list in PodAffinityTerm
Removes the distinction between `null` and `[]` for the PodAffinityTerm#namespaces field (option 4 discussed in https://github.com/kubernetes/kubernetes/issues/43203#issuecomment-287237992), since we can't distinguish between them in protobuf (and it's a less than ideal API)
Leaves the door open to reintroducing "all namespaces" function via a dedicated field or a dedicated token in the list of namespaces
Wanted to get a PR open and tests green in case we went with this option.
Not sure what doc/release-note is needed if the "all namespaces" function is not present in 1.6
Automatic merge from submit-queue
kubectl: Use v1.5-compatible ownership logic when listing dependents.
**What this PR does / why we need it**:
This restores compatibility between kubectl 1.6 and clusters running Kubernetes 1.5.x. It introduces transitional ownership logic in which the client considers ControllerRef when it exists, but does not require it to exist.
If we were to ignore ControllerRef altogether (pre-1.6 client behavior), we would introduce a new failure mode in v1.6 because controllers that used to get stuck due to selector overlap will now make progress. For example, that means when reaping ReplicaSets of an overlapping Deployment, we would risk deleting ReplicaSets belonging to a different Deployment that we aren't about to delete.
This transitional logic avoids such surprises in 1.6 clusters, and does no worse than kubectl 1.5 did in 1.5 clusters. To prevent this when kubectl 1.5 is used against 1.6 clusters, we can cherrypick this change.
**Which issue this PR fixes**:
Fixes#43159
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Fix revision when SetDeploymentRevision
When some oldRSs be deleted or cleared(eg. revisionHistoryLimit set 0), the revision for SetDeploymentRevision is incorrect
Automatic merge from submit-queue
protobuf generation modifies types.go, which needs to be copied out
This was broken when we moved to the build container, but no one
noticed. Made it so that we get a test error if a field in a registered type has a json tag with no protobuf tag.
Fixes#35486
Automatic merge from submit-queue
construction of GC should not fail for restmapper error caused by tpr
Fix https://github.com/kubernetes/kubernetes/issues/43147.
The issue is that GC will fail its initialization due to an RESTMapper error cause by tpr. This PR lets GC log the error instead of failing.
Automatic merge from submit-queue
Fix polarity of a test in NodePort allocation
The result of this was that an update to a Service would release the
NodePort temporarily (the repair loop would fix it in a minute). During
that window, another Service could get allocated that Port.
Fixes#43233
Automatic merge from submit-queue (batch tested with PRs 43254, 43255, 43184, 42509)
Update comment on the default DeletionPropagationPolicy
This is fixing the API doc, so apply the 1.6 milestone.
I'll run update-all.sh before merge.
The result of this was that an update to a Service would release the
NodePort temporarily (the repair loop would fix it in a minute). During
that window, another Service could get allocated that Port.
In particular, we should not assume ControllerRefs are necessarily set.
However, we can still use ControllerRefs that do exist to avoid
interfering with controllers that do use it.
This effectively reverts the client-side changes in
cec3899b96.
We have to maintain the old behavior on the client side to support
version skew when talking to old servers that set the annotation.
However, the new server-side behavior is still to NOT set the
annotation.
Automatic merge from submit-queue
recycle pod can't get the event since channel closed
What this PR does / why we need it:
We create a hostPath type PV with "Recycle" persistentVolumeReclaimPolicy, and bind a PVC to it, but after deleted the PVC, the PV cannot become to available status. This is happened after we upgrade etcd to 3.0. The reason is:
If the channel used to get the pod message and events been abnormal closed(for example, the event channel maybe closed because of "required revision has been compacted" error), the function internalRecycleVolumeByWatchingPodUntilCompletion will stuck in a loop, and the recycle pod will not been deleted, the PV can not become into available status
Special notes for your reviewer:
None
Release note:
Automatic merge from submit-queue
Prevent protobuf storage with etcd2
Prevents accidentally storing protobuf content in etcd2 when upgrading to 1.6
c.f. https://github.com/kubernetes/kubernetes/issues/42976#issuecomment-286537139
```release-note
if kube-apiserver is started with `--storage-backend=etcd2`, the media type `application/json` is used.
```
Automatic merge from submit-queue (batch tested with PRs 40964, 42967, 43091, 43115)
Improve code coverage for pkg/kubelet/status/generate.go
**What this PR does / why we need it**:
Improve code coverage for pkg/kubelet/status/generate.go from #39559
Thanks.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Fix Deployment upgrade test.
**What this PR does / why we need it**:
When the upgrade test operates on Deployments in a pre-1.6 cluster (i.e. during the Setup phase), it needs to use the v1.5 deployment/util logic. In particular, the v1.5 logic does not filter children to only those with a matching ControllerRef.
**Which issue this PR fixes**:
Fixes#42738
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 43106, 43110)
Wait for garbagecollector to be synced in test
Fix#42952
Without the `cache.WaitForCacheSync` in the test, it's possible for the GC to get a merged event of RC's creation and its update (update to deletionTimestamp != 0), before GC gets the creation event of the pod, so it's possible the GC will handle the foreground deletion of the RC before it adds the Pod to the dependency graph, thus the race.
With the `cache.WaitForCacheSync` in the test, because GC runs a single thread to process graph changes, it's guaranteed the Pod will be added to the dependency graph before GC handles the foreground deletion of the RC.
Note that this pull fixes the race in the test. The race described in the first point of #26120 still exists.
Automatic merge from submit-queue (batch tested with PRs 42747, 43030)
dockershim: remove corrupted sandbox checkpoints
This is a workaround to ensure that kubelet doesn't block forever when
the checkpoint is corrupted.
This is a workaround for #43021
Automatic merge from submit-queue (batch tested with PRs 42747, 43030)
kube-proxy/iptables: optimize endpoint map creation by excluding invalid endpoints earlier
We don't need to do as much work as we were doing, if we exclude invalid endpoints earlier in the endpoints processing.
Fixes: https://github.com/kubernetes/kubernetes/issues/42210
@freehan @liggitt @thockin if you could review this with a fine-toothed comb... I can't immediately think of why invalid endpoints would be useful for the HealthChecker, and this PR prevents the HC from seeing these endpoints.
Automatic merge from submit-queue
dockershim: call sync() after writing the checkpoint
This ensures the checkpoint files are persisted.
This fixes#43021
Automatic merge from submit-queue (batch tested with PRs 42854, 43105, 43090)
Update ScaleIO volume plugin default readOnly value
This commit updates the code to set readOnly attribute to be set to false.
**What this PR does / why we need it**:
This PR is a minor fix that updates the default value of `readOnly` attribute to `false`.
**Release note**:
```release-note
NONE
```
When the upgrade test operates on Deployments in a pre-1.6 cluster
(i.e. during the Setup phase), it needs to use the v1.5 deployment/util
logic. In particular, the v1.5 logic does not filter children to only
those with a matching ControllerRef.
Automatic merge from submit-queue (batch tested with PRs 42775, 42991, 42968, 43029)
Remove 'beta' from default storage class annotation
I forgot to update default storage class annotation in my storage.k8s.io/v1beta1 -> v1 PRs. Let's fix it before 1.6 is released.
I consider it as a bugfix, in #40088 I already updated the release notes to include non-beta annotation `storageclass.kubernetes.io/is-default-class`
```release-note
NONE
```
@kubernetes/sig-storage-pr-reviews
@msau42, please help with merging.
Automatic merge from submit-queue (batch tested with PRs 42775, 42991, 42968, 43029)
`kubectl taint` update for `NoExecute`.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Fixes#42774
**Release note**:
```release-note
```
Automatic merge from submit-queue
Add DaemonSet templateGeneration validation and tests, and fix a bunch of validation test errors
For DaemonSet update:
1. Validate that templateGeneration is increased when and only when template is changed
1. Validate that templateGeneration is never decreased
1. Added validation tests for templateGeneration
1. Fix a bunch of errors in validate tests
- fake tests: almost all validation test error cases failed on "missing resource version", "name changes", "missing update strategy", "selector/template labels mismatch", not on the real validation we wanted to test
- some error cases should be success cases
@kargakis @lukaszo @kubernetes/sig-apps-bugs
*I've verified locally that all DaemonSet e2e tests pass with this change.*
This commit updates the code to set the default value of the readOnly attribute to false.
It also updates the example docs to add full list of supported plugin attributes and doc.
Automatic merge from submit-queue (batch tested with PRs 42942, 42935)
pkg/util/flock: Fix the flock so it actually locks.
With this PR, the second call to `Acquire()` will block unless the lock is released (process exits).
Also removed the memory mutex in the previous code since we don't need `Release()` here so no need to save and protect the local fd.
Fix#42929.
Automatic merge from submit-queue (batch tested with PRs 42942, 42935)
[Bug] Handle container restarts and avoid using runtime pod cache while allocating GPUs
Fixes#42412
**Background**
Support for multiple GPUs is an experimental feature in v1.6.
Container restarts were handled incorrectly which resulted in stranding of GPUs
Kubelet is incorrectly using runtime cache to track running pods which can result in race conditions (as it did in other parts of kubelet). This can result in same GPU being assigned to multiple pods.
**What does this PR do**
This PR tracks assignment of GPUs to containers and returns pre-allocated GPUs instead of (incorrectly) allocating new GPUs.
GPU manager is updated to consume a list of active pods derived from apiserver cache instead of runtime cache.
Node e2e has been extended to validate this failure scenario.
**Risk**
Minimal/None since support for GPUs is an experimental feature that is turned off by default. The code is also isolated to GPU manager in kubelet.
**Workarounds**
In the absence of this PR, users can mitigate the original issue by setting `RestartPolicyNever` in their pods.
There is no workaround for the race condition caused by using the runtime cache though.
Hence it is worth including this fix in v1.6.0.
cc @jianzhangbjz @seelam @kubernetes/sig-node-pr-reviews
Replaces #42560
Currently, TCPSocketAction always uses Pod's IP in connection. But when a
pod uses the host network, sometimes firewall rules may prevent kubelet
from connecting through the Pod's IP. This PR introduces the 'Host' field
for TCPSocketAction, and if it is set to non-empty string, the probe will
be performed on the configured host rather than the Pod's IP. This gives
users an opportunity to explicitly specify 'localhost' as the target for
the above situations.
With this PR, the second call to `Acquire()` will block unless the lock is released (process exits).
Also removed the memory mutex in the previous code since we don't need `Release()` here so no need to save and protect the local fd.
Fix#42929.
Automatic merge from submit-queue
Fixed incorrect result of getMinTolerationTime.
For the following case, `getMinTolerationTime` should return one; but it returned -1 ：
1. for tolerations[0], TolerationSeconds is nil, minTolerationTime is not set
2. for tolerations[1], it's TolerationSeconds (1) is bigger than `minTolerationTime`, so minTolerationTime is still -1 which means infinite.
```
+ {
+ tolerations: []v1.Toleration{
+ {
+ TolerationSeconds: nil,
+ },
+ {
+ TolerationSeconds: &one,
+ },
+ },
+ },
```
Automatic merge from submit-queue
Fix taint based pod eviction for clusters where controller manager is not running with allocate-node-cidrs set
Fixes https://github.com/kubernetes/kubernetes/issues/42733
In my cluster, I have not set allocate-node-cidr, and It is causing taint based pod eviction to fail.
@gmarek @kubernetes/sig-scheduling-bugs @davidopp @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 41794, 42349, 42755, 42901, 42933)
Fix DefaultTolerationSeconds admission plugin
DefaultTolerationSeconds is not working as expected. It is supposed to add default tolerations (for unreachable and notready conditions). but no pod was getting these toleration. And api server was throwing this error:
```
Mar 08 13:43:57 fedora25 hyperkube[32070]: E0308 13:43:57.769212 32070 admission.go:71] expected pod but got Pod
Mar 08 13:43:57 fedora25 hyperkube[32070]: E0308 13:43:57.789055 32070 admission.go:71] expected pod but got Pod
Mar 08 13:44:02 fedora25 hyperkube[32070]: E0308 13:44:02.006784 32070 admission.go:71] expected pod but got Pod
Mar 08 13:45:39 fedora25 hyperkube[32070]: E0308 13:45:39.754669 32070 admission.go:71] expected pod but got Pod
Mar 08 14:48:16 fedora25 hyperkube[32070]: E0308 14:48:16.673181 32070 admission.go:71] expected pod but got Pod
```
The reason for this error is that the input to admission plugins is internal api objects not versioned objects so expecting versioned object is incorrect. Due to this, no pod got desired tolerations and it always showed:
```
Tolerations: <none>
```
After this fix, the correct tolerations are being assigned to pods as follows:
```
Tolerations: node.alpha.kubernetes.io/notReady=:Exists:NoExecute for 300s
node.alpha.kubernetes.io/unreachable=:Exists:NoExecute for 300s
```
@davidopp @kevin-wangzefeng @kubernetes/sig-scheduling-pr-reviews @kubernetes/sig-scheduling-bugs @derekwaynecarr
Fixes https://github.com/kubernetes/kubernetes/issues/42716
Automatic merge from submit-queue
Invalid environment var names are reported and pod starts
When processing EnvFrom items, all invalid keys are collected and
reported as a single event.
The Pod is allowed to start.
fixes#42583
Automatic merge from submit-queue
[Federation] Kubefed Init should use the right RBAC API version clientset
**What this PR does / why we need it**:
Implements the need as described in https://github.com/kubernetes/kubernetes/issues/41263
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/41263
**Special notes for your reviewer**:
@madhusudancs @shashidharatd @marun
cc @kubernetes/sig-federation-bugs
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 38805, 42362, 42862)
Let GC print specific message for RESTMapping failure
Make the error messages reported in https://github.com/kubernetes/kubernetes/issues/39816 to be more specific, also only print the message once.
I'll also update the garbage collector's doc to clearly state we don't support tpr yet.
We'll wait for the watchable discovery feature (@sttts are you going to work on that?) to land in 1.7, and then enable the garbage collector to handle TPR.
cc @hongchaodeng @MikaelCluseau @djMax
Automatic merge from submit-queue (batch tested with PRs 38805, 42362, 42862)
Fix deployment generator after introducing deployments in apps/v1beta1
This PR does two things:
1. Switches all generator to produce versioned objects, to bypass the problem of having an object in multiple versions, which then results in not having stable generator (iow. producing exactly the same object).
2. Introduces new generator for `apps/v1beta1` deployments.
@kargakis @janetkuo ptal
@kubernetes/sig-apps-pr-reviews @kubernetes/sig-cli-pr-reviews ptal
This is a followup to https://github.com/kubernetes/kubernetes/pull/39683, so I'm adding 1.6 milestone.
```release-note
Introduce new generator for apps/v1beta1 deployments
```
Automatic merge from submit-queue (batch tested with PRs 42608, 42444)
Return nil when deleting non-exist GCE PD
When gce cloud tries to delete a disk, if the disk could not be found
from the zones, the function should return nil error. This modified behavior is also consistent with AWS
Automatic merge from submit-queue (batch tested with PRs 42877, 42853)
Remove unused functions and make logs slightly better
Zero risk cleanup, removing function that are not used anymore, and adding few more logs to help debugging problems.
cc @aveshagarwal
Automatic merge from submit-queue
Use Prometheus instrumentation conventions
The `System` and `Subsystem` parameters are subject to removal.
(x-ref: https://github.com/prometheus/client_golang/issues/240)
All metrics should use base units, which is seconds in the duration
case.
Counters should always end in `_total` and metrics should avoid
referring to potential label dimensions. Those should rather be
mentioned in the documentation string.
@kubernetes/sig-instrumentation
Reference docs:
https://prometheus.io/docs/practices/instrumentation/https://prometheus.io/docs/practices/naming/
**Release note**:
```
Breaking change: Renamed REST client Prometheus metrics to follow the instrumentation conventions ("request_latency_microseconds" -> "rest_client_request_latency_seconds", "request_status_codes" -> "rest_client_requests_total"). Please update your alerting pipeline if you rely on them.
```
Automatic merge from submit-queue
Add new DaemonSetStatus to kubectl printer and describer
@kargakis @lukaszo @kubernetes/sig-apps-pr-reviews @kubernetes/sig-cli-pr-reviews
```release-note
Add new DaemonSet status fields to kubectl printer and describer.
```
The `System` and `Subsystem` parameters are subject to removal.
(x-ref: https://github.com/prometheus/client_golang/issues/240)
All metrics should use base units, which is seconds in the duration
case.
Counters should always end in `_total` and metrics should avoid
referring to potential label dimensions. Those should rather be
mentioned in the documentation string.
Automatic merge from submit-queue (batch tested with PRs 42811, 42859)
Validation PVs for mount options
We are going to move the validation in its own package and we will be calling validation for individual volume types as needed.
Fixes https://github.com/kubernetes/kubernetes/issues/42573
Automatic merge from submit-queue (batch tested with PRs 42024, 42780, 42808, 42640)
kubectl: respect DaemonSet strategy parameters for rollout status
It handles "after-merge" comments from #41116
cc @kargakis @janetkuo
I will add one more e2e test later. I need to handle some in company stuff.
Automatic merge from submit-queue (batch tested with PRs 42024, 42780, 42808, 42640)
Node controller test flake 39975 with delay for try function
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39975
/cc @ncdc @gmarek @liggitt
Automatic merge from submit-queue
Remove VCenterPort from vsphere cloud provider.
**What this PR does / why we need it**:
Address a bug inside vsphere cloud provider when a port number other than 443 is specified inside the config file.
The url which is used for communicating with govmomi should not include port number.
A port number other than 443 will result in 404 error.
VCenterPort stays in VSphereConfig structure for backward compatibility.
**Which issue this PR fixes** : fixes https://github.com/kubernetes/kubernetes-anywhere/issues/338
Automatic merge from submit-queue (batch tested with PRs 42734, 42745, 42758, 42814, 42694)
Dropped docker 1.9.x support. Changed the minimumDockerAPIVersion to
1.22
cc/ @Random-Liu @yujuhong
We talked about dropping docker 1.9.x support for a while. I just realized that we haven't really done it yet.
```release-note
Dropped the support for docker 1.9.x and the belows.
```
Since the HPA controller pulls information from an external source that
makes no guarantees about consistency, it's possible for the HPA
to get into an infinite update loop -- if the metrics change with
every query, the HPA controller will run it's normal reconcilation,
post a status update, see that status update itself, fetch new metrics,
and if those metrics are different, post another status update, and
repeat. This can lead to continuously updating a single HPA.
By rate-limiting each HPA to once per sync interval, we prevent this
from happening.
Automatic merge from submit-queue (batch tested with PRs 42786, 42553)
Updated auto generated protobuf codes.
Generated by `./hack/update-generated-protobuf-dockerized.sh` in Mac.
Automatic merge from submit-queue
dockershim: Fix the race condition in ListPodSandbox
In ListPodSandbox(), we
1. List all sandbox docker containers
2. List all sandbox checkpoints. If the checkpoint does not have a
corresponding container in (1), we return partial result based on
the checkpoint.
The problem is that new PodSandboxes can be created between step (1) and
(2). In those cases, we will see the checkpoints, but not the sandbox
containers. This leads to strange behavior because the partial result
from the checkpoint does not include some critical information. For
example, the creation timestamp'd be zero, and that would cause kubelet's
garbage collector to immediately remove the sandbox.
This change fixes that by getting the list of checkpoints before listing
all the containers (since in RunPodSandbox we create them in the reverse
order).
Automatic merge from submit-queue (batch tested with PRs 42211, 38691, 42737, 42757, 42754)
Only create the symlink when container log path exists
When using `syslog` logging driver instead of `json-file`, there will not be container log files such as `<containerID-json.log>`. We should not create symlink in this case.
In ListPodSandbox(), we
1. List all sandbox docker containers
2. List all sandbox checkpoints. If the checkpoint does not have a
corresponding container in (1), we return partial result based on
the checkpoint.
The problem is that new PodSandboxes can be created between step (1) and
(2). In those cases, we will see the checkpoints, but not the sandbox
containers. This leads to strange behavior because the partial result
from the checkpoint does not include some critical information. For
example, the creation timestamp'd be zero, and that would cause kubelet's
garbage collector to immediately remove the sandbox.
This change fixes that by getting the list of checkpoints before listing
all the containers (since in RunPodSandbox we create them in the reverse
order).
1. Validate that templateGeneration is increased when and only when template is changed
2. Validate that templateGeneration is never decreased
3. Added validation tests for templateGeneration
4. Fix a bunch of errors in validate tests, for example, all validation test error cases failed
on lack of resource version, or on name changes, not on the real validation we wanted to test
Automatic merge from submit-queue
Use namespace from context
Fixes#42653
Updates rbac_test.go to submit objects without namespaces set, which matches how actual objects are submitted to the API.
Automatic merge from submit-queue (batch tested with PRs 42692, 42169, 42173)
DaemonSet: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings DaemonSet into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
This ensures that DaemonSet does not fight with other controllers over control of Pods.
**Special notes for your reviewer**:
**Release note**:
```release-note
DaemonSet now respects ControllerRef to avoid fighting over Pods.
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42692, 42169, 42173)
Add pprof trace support
Add support for `/debug/pprof/trace`
Can wait for master to reopen for 1.7.
cc @smarterclayton @wojtek-t @gmarek @timothysc @jeremyeder @kubernetes/sig-scalability-pr-reviews
The design of DaemonSet requires a relist before each phase (manage,
update, status) because it does not short-circuit and requeue for each
action triggered.
The DaemonSet Listers still use selectors, because this is the
behavior expected by callers. This clarifies the meaning of the
returned list. Some callers may need to switch to using
GetControllerOf() instead, but that is a separate, case-by-case issue.
We don't need the svcPortToInfoMap. Its only purpose was to
send "valid" local endpoints (those with valid IP and >0 port) to the
health checker. But we shouldn't be sending invalid endpoints to
the health checker anyway, because it can't do anything with them.
If we exclude invalid endpoints earlier, then we don't need
flattenValidEndpoints().
And if we don't need flattenValidEndpoints() it makes no sense to have
svcPortToInfoMap store hostPortInfo, since endpointsInfo is the same
thing as hostPortInfo except with a combined host:port.
And if svcPortToInfoMap now only stores valid endpointsInfos, it is
exactly the same thing as newEndpoints.
* Properly return ImageNotFoundError
* Support inject "Images" or "ImageInspects" and keep both in sync.
* Remove the FakeDockerPuller and let FakeDockerClient subsumes its
functinality. This reduces the overhead to maintain both objects.
* Various small fixes and refactoring of the testing utils.
Automatic merge from submit-queue (batch tested with PRs 41890, 42593, 42633, 42626, 42609)
Pods pending due to insufficient OIR should get scheduled once sufficient OIR becomes available (e2e disabled).
#41870 was reverted because it introduced an e2e test flake. This is the same code with the e2e for OIR disabled again.
We can attempt to enable the e2e test cases one-by-one in follow-up PRs, but it would be preferable to get the main fix merged in time for 1.6 since OIR is broken on master (see #41861).
cc @timothysc
Automatic merge from submit-queue (batch tested with PRs 41890, 42593, 42633, 42626, 42609)
Remove everything that is not new from batch/v2alpha1
Fixes#37166.
@lavalamp you've asked for it
@erictune this is a prereq for moving CronJobs to beta. I initially planned to put all in one PR, but after I did that I figured out it'll be easier to review separately. ptal
@kubernetes/api-approvers @kubernetes/sig-api-machinery-pr-reviews ptal
Automatic merge from submit-queue (batch tested with PRs 42506, 42585, 42596, 42584)
RC/RS: Fix ignoring inactive Pods.
**What this PR does / why we need it**:
Fix typo that broke ignoring of inactive Pods in RC, and add unit test for that case.
**Which issue this PR fixes**:
Fixes#37479
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 42506, 42585, 42596, 42584)
provide active pods to cgroup cleanup
**What this PR does / why we need it**:
This PR provides more information for when a pod cgroup is considered orphaned. The running pods cache is based on the runtime's view of the world. we create pod cgroups before containers so we should just be looking at activePods.
**Which issue this PR fixes**
Fixes https://github.com/kubernetes/kubernetes/issues/42431
Automatic merge from submit-queue
cgroup names created by kubelet should be lowercased
**What this PR does / why we need it**:
This PR modifies the kubelet to create cgroupfs names that are lowercased. This better aligns us with the naming convention for cgroups v2 and other cgroup managers in ecosystem (docker, systemd, etc.)
See: https://www.kernel.org/doc/Documentation/cgroup-v2.txt
"2-6-2. Avoid Name Collisions"
**Special notes for your reviewer**:
none
**Release note**:
```release-note
kubelet created cgroups follow lowercase naming conventions
```
Automatic merge from submit-queue (batch tested with PRs 42080, 41653, 42598, 42555)
Revert "Pods pending due to insufficient OIR should get scheduled once sufficient OIR becomes available."
Reverts kubernetes/kubernetes#41870 for stopping bleeding edge: #42597
cc/ @ConnorDoyle @kubernetes/release-team
Connor if there is a pending pr to fix the issue, please point it out to me. We can close this one, otherwise, I would like to revert the pr first. You can resubmit the fix. Thanks!
Automatic merge from submit-queue (batch tested with PRs 42080, 41653, 42598, 42555)
StatefulSet: Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings StatefulSet into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
Fixes#36859
**Special notes for your reviewer**:
**Release note**:
```release-note
StatefulSet now respects ControllerRef to avoid fighting over Pods. At the time of upgrade, **you must not have StatefulSets with selectors that overlap** with any other controllers (such as ReplicaSets), or else [ownership of Pods may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading).
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Deployment should ignore Pods that don't match the selector, even if
they have a ControllerRef pointing to one of the ReplicaSets it owns.
The ReplicaSet itself will orphan the Pod as soon as it syncs.
The Deployment Listers still use selectors, because this is the
behavior expected by callers. This clarifies the meaning of the
returned list. Some callers may need to switch to using
GetControllerOf() instead, but that is a separate, case-by-case issue.
The list functions in deployment/util are used outside the Deployment
controller itself. Therefore, they don't do actual adoption/orphaning.
However, they still need to avoid listing things that don't belong.
Although Deployment already applied its ControllerRef to adopt matching
ReplicaSets, it actually still used label selectors to list objects that
it controls. That meant it didn't actually follow the rules of
ControllerRef, so it could still fight with other controller types.
This should mean that the special handling for overlapping Deployments
is no longer necessary, since each Deployment will only see objects that
it owns (via ControllerRef).
Automatic merge from submit-queue
add "external" cloud provider
@thockin @justinsb @luxas @joonas
This is the discussion we had in our last meeting about moving the cloud provider dependent parts out of kubelet
1. Flex Volume v2 will not be done anytime soon, until then it is okay to leave the volume code as it is. i.e. Volume plugins with cloudprovider dependencies will continue to reside in the core kubernetes repository, and new volume plugins will be allowed.
2. We are moving forward with the previous decision to move the cloudprovider dependencies from kubelet into a new controller loop in the cloud-controller-manager.
3. The kubelet will set a taint using the flag (--register-with-taints, https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/options/options.go#L238) when it starts. @luxas, we decided to go with a different approach last week, but it turns out that the other approach will not work. There is a hold on PRs that set taints in the admission controller (kubernetes/kubernetes: Pull Request 40288). Due to the uncertain nature of this hold, the kubelet flag is our only alternative. Also, since this is an alpha feature that is not fully baked, kubeadm has a lot of time to make the changes to support this, i.e. it need not support it right away. It can support it in the future releases of k8s.
4. A new flag will be added in kubelet to explicitly denote that no cloudprovider initialization should be done (--no-cloud-init). This is required to clearly indicate to the user that kubelet will not perform any cloudprovider specific calls to initialize the node object.
5. A new flag will be added in kubelet to set uniquely identifying information about the host (--external-id?). This will be required for clouds like openstack, where uniquely identifying information is only available from the host itself.
6. The upgrade scenario with these changes is straightforward, as we all are working under the assumption that the user upgrades the master before upgrading the kubelet.
In this PR, I have added the `--do-cloud-init` flag. (Item 4)
Automatic merge from submit-queue (batch tested with PRs 31783, 41988, 42535, 42572, 41870)
Pods pending due to insufficient OIR should get scheduled once sufficient OIR becomes available.
This appears to be a regression since v1.5.0 in scheduler behavior for opaque integer resources, reported in https://github.com/kubernetes/kubernetes/issues/41861.
- [X] Add failing e2e test to trigger the regression
- [x] Restore previous behavior (pods pending due to insufficient OIR get scheduled once sufficient OIR becomes available.)
Automatic merge from submit-queue (batch tested with PRs 31783, 41988, 42535, 42572, 41870)
update names for kube plugin initializer to avoid conflicts
Fixes#42581
Other API servers are likely to create admission plugin initializers and so the names we choose for our interfaces matter (they may want to run multiple initializers in the chain). This updates the names for the plugin initializers to be more specific. No other changes.
@ncdc
Automatic merge from submit-queue (batch tested with PRs 31783, 41988, 42535, 42572, 41870)
controller: ensure deployment rollback is re-entrant
Make rollbacks re-entrant in the Deployment controller, otherwise
fast enqueues of a Deployment may end up in undesired behavior
- redundant rollbacks.
Fixes https://github.com/kubernetes/kubernetes/issues/36703
@kubernetes/sig-apps-bugs
The StatefulSet Listers still use selectors, because this is the
behavior expected by callers. This clarifies the meaning of the
returned list. Some callers may need to switch to using
GetControllerOf() instead, but that is a separate, case-by-case issue.
Automatic merge from submit-queue
make the system:authenticated group adder smarter
Fixes#42437
This prevents the group adder from adding the system:authenticated group when:
1. it's already in the list
2. the user is system:anonymous
3. system:unauthenticated is in the list
Smaller alternative to https://github.com/kubernetes/kubernetes/pull/42421 for 1.6.
@kubernetes/sig-auth-pr-reviews @enj @liggitt
Make rollbacks re-entrant in the Deployment controller, otherwise
fast enqueues of a Deployment may end up in undesired behavior
- redundant rollbacks.
Automatic merge from submit-queue
Fix Multizone pv creation on GCE
When Multizone is enabled static PV creation on GCE
fails because Cloud provider configuration is not
available in admission plugins.
cc @derekwaynecarr @childsb
Automatic merge from submit-queue
[Bug] Fix gpu initialization in Kubelet
Kubelet incorrectly fails if `AllAlpha=true` feature gate is enabled with container runtimes that are not `docker`.
Replaces #42407
Automatic merge from submit-queue
kubectl: respect deployment strategy parameters for rollout status
Fixes https://github.com/kubernetes/kubernetes/issues/40496
`rollout status` now respects the strategy parameters for a RollingUpdate Deployment. This means that it will exit as soon as minimum availability is reached for a rollout (note that if you allow maximum availability, `rollout status` will succeed as soon as the new pods are created)
@janetkuo @AdoHe ptal
Automatic merge from submit-queue
Small fix to the bootstrap TokenCleaner
Accidentally missed setting options and so the TokenCleaner was in a retry loop. Also moved from using an explicit timer over cached values vs. relying on a short resync timeout.
```release-note
```
Putting this in the 1.6 milestone as this is clearly a bug fix in a new feature.
- Added schedulercache.Resource.SetOpaque helper.
- Amend kubelet allocatable sync so that when OIRs are removed from capacity
they are also removed from allocatable.
- Fixes#41861.
Automatic merge from submit-queue (batch tested with PRs 42456, 42457, 42414, 42480, 42370)
In DaemonSet e2e test, don't check nodes with NoSchedule taints
Fixes#42345
For example, master node has a ismaster:NoSchedule taint. We don't expect pods to be created there without toleration.
cc @marun @lukaszo @kargakis @yujuhong @Random-Liu @davidopp @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42456, 42457, 42414, 42480, 42370)
controller: reduce log verbosity for deployments
Fixes https://github.com/kubernetes/kubernetes/issues/41187
Labeling as a bug fix since I think excessive logging should be considered as a bug.
@kubernetes/sig-apps-bugs
Automatic merge from submit-queue (batch tested with PRs 42369, 42375, 42397, 42435, 42455)
[Bug Fix]: Avoid evicting more pods than necessary by adding Timestamps for fsstats and ignoring stale stats
Continuation of #33121. Credit for most of this goes to @sjenning. I added volume fs timestamps.
**why is this a bug**
This PR attempts to fix part of https://github.com/kubernetes/kubernetes/issues/31362 which results in multiple pods getting evicted unnecessarily whenever the node runs into resource pressure. This PR reduces the chances of such disruptions by avoiding reacting to old/stale metrics.
Without this PR, kubernetes nodes under resource pressure will cause unnecessary disruptions to user workloads.
This PR will also help deflake a node e2e test suite.
The eviction manager currently avoids evicting pods if metrics are old. However, timestamp data is not available for filesystem data, and this causes lots of extra evictions.
See the [inode eviction test flakes](https://k8s-testgrid.appspot.com/google-node#kubelet-flaky-gce-e2e) for examples.
This should probably be treated as a bugfix, as it should help mitigate extra evictions.
cc: @kubernetes/sig-storage-pr-reviews @kubernetes/sig-node-pr-reviews @vishh @derekwaynecarr @sjenning
Automatic merge from submit-queue (batch tested with PRs 42369, 42375, 42397, 42435, 42455)
Kubelet: return container runtime's version instead of CRI's one
**What this PR does / why we need it**:
With CRI enabled by default, kubelet reports the version of CRI instead of container runtime version. This PR fixes this problem.
**Which issue this PR fixes**
Fixes#42396.
**Special notes for your reviewer**:
Should also cherry-pick to 1.6 branch.
**Release note**:
```release-note
NONE
```
cc @yujuhong @kubernetes/sig-node-bugs
Automatic merge from submit-queue
Output of `kubectl get` is inconsistent for pods
Builds on top of fixes from #42283, only the last two commits are new. Reverts behavior of #39042 which was inconsistent and confusing.
Fixes#15853
Accidentally missed setting options and so the TokenCleaner was in a retry loop. Also moved from using an explicit timer over cached values vs. relying on a short resync timeout.
Signed-off-by: Joe Beda <joe.github@bedafamily.com>
Automatic merge from submit-queue
Eviction Manager Enforces Allocatable Thresholds
This PR modifies the eviction manager to enforce node allocatable thresholds for memory as described in kubernetes/community#348.
This PR should be merged after #41234.
cc @kubernetes/sig-node-pr-reviews @kubernetes/sig-node-feature-requests @vishh
** Why is this a bug/regression**
Kubelet uses `oom_score_adj` to enforce QoS policies. But the `oom_score_adj` is based on overall memory requested, which means that a Burstable pod that requested a lot of memory can lead to OOM kills for Guaranteed pods, which violates QoS. Even worse, we have observed system daemons like kubelet or kube-proxy being killed by the OOM killer.
Without this PR, v1.6 will have node stability issues and regressions in an existing GA feature `out of Resource` handling.
Automatic merge from submit-queue (batch tested with PRs 42443, 38924, 42367, 42391, 42310)
Apply custom defaults to init containers
Adds overridden defaults to init containers. They were not being defaulted the same way normal containers were.
Automatic merge from submit-queue (batch tested with PRs 42443, 38924, 42367, 42391, 42310)
Dell EMC ScaleIO Volume Plugin
**What this PR does / why we need it**
This PR implements the Kubernetes volume plugin to allow pods to seamlessly access and use data stored on ScaleIO volumes. [ScaleIO](https://www.emc.com/storage/scaleio/index.htm) is a software-based storage platform that creates a pool of distributed block storage using locally attached disks on every server. The code for this PR supports persistent volumes using PVs, PVCs, and dynamic provisioning.
You can find examples of how to use and configure the ScaleIO Kubernetes volume plugin in [examples/volumes/scaleio/README.md](examples/volumes/scaleio/README.md).
**Special notes for your reviewer**:
To facilitate code review, commits for source code implementation are separated from other artifacts such as generated, docs, and vendored sources.
```release-note
ScaleIO Kubernetes Volume Plugin added enabling pods to seamlessly access and use data stored on ScaleIO volumes.
```
Automatic merge from submit-queue (batch tested with PRs 41919, 41149, 42350, 42351, 42285)
enable cgroups tiers and node allocatable enforcement on pods by default.
```release-note
Pods are launched in a separate cgroup hierarchy than system services.
```
Depends on #41753
cc @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 41919, 41149, 42350, 42351, 42285)
kubelet: enable qos-level memory limits
```release-note
Experimental support to reserve a pod's memory request from being utilized by pods in lower QoS tiers.
```
Enables the QoS-level memory cgroup limits described in https://github.com/kubernetes/community/pull/314
**Note: QoS level cgroups have to be enabled for any of this to take effect.**
Adds a new `--experimental-qos-reserved` flag that can be used to set the percentage of a resource to be reserved at the QoS level for pod resource requests.
For example, `--experimental-qos-reserved="memory=50%`, means that if a Guaranteed pod sets a memory request of 2Gi, the Burstable and BestEffort QoS memory cgroups will have their `memory.limit_in_bytes` set to `NodeAllocatable - (2Gi*50%)` to reserve 50% of the guaranteed pod's request from being used by the lower QoS tiers.
If a Burstable pod sets a request, its reserve will be deducted from the BestEffort memory limit.
The result is that:
- Guaranteed limit matches root cgroup at is not set by this code
- Burstable limit is `NodeAllocatable - Guaranteed reserve`
- BestEffort limit is `NodeAllocatable - Guaranteed reserve - Burstable reserve`
The only resource currently supported is `memory`; however, the code is generic enough that other resources can be added in the future.
@derekwaynecarr @vishh
When gce cloud tries to delete a disk, if the disk could not be found
from the zones, the function should return nil error. This modified behavior is also consistent with AWS
This PR modify getInstanceByname to loop through all management zones
directly instead of calling getInstancesByNames. Currently
getInstancesByNames use a node name prefix as a filter to list the
instances. If the prefix does not match, it will return all instances
which is very wasteful since getInstanceByName only query one instance
with a specific name.
Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)
Implement bulk polling of volumes
This implements Bulk volume polling using ideas presented by
justin in https://github.com/kubernetes/kubernetes/pull/39564
But it changes the implementation to use an interface
and doesn't affect other implementations.
cc @justinsb
Automatic merge from submit-queue (batch tested with PRs 42365, 42429, 41770, 42018, 35055)
controller: statefulsets respect observed generation
StatefulSets do not update ObservedGeneration even though the API field is in place. This means that clients can never be sure whether the StatefulSet controller has observed the latest spec of a StatefulSet.
@kubernetes/sig-apps-bugs
This implements Bulk volume polling using ideas presented by
justin in https://github.com/kubernetes/kubernetes/pull/39564
But it changes the implementation to use an interface
and doesn't affect other implementations.
Automatic merge from submit-queue (batch tested with PRs 41984, 41682, 41924, 41928)
make edit respect --save-config
New behavior:
edit:
1) the flag is set to true: it will create or update the last-applied-config annotation
2) the flag is set to false or is unspecified: NOP
replace:
1) If saveConfig is true, create or update the annotation.
2) If saveConfig is false and the local config file doesn't have the annotation, we save the annotation from the live object if there is one (Try to not surprise the users).
3) If saveConfig is false and the local config file has the annotation, we use the annotation in the config file.
4) Same behavior for force replacing
fixes#40626
```release-note
stop kubectl edit from updating the last-applied-configuration annotation when --save-config is unspecified or false.
```
@pwittrock @liggitt
This is a bug fix that prevented `edit` from being respected by `apply`
Automatic merge from submit-queue (batch tested with PRs 41984, 41682, 41924, 41928)
RC/RS: Fully Respect ControllerRef
**What this PR does / why we need it**:
This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings ReplicaSet and ReplicationController into full compliance with ControllerRef. See the individual commit messages for details.
**Which issue this PR fixes**:
Although RC/RS had partially implemented ControllerRef, they didn't use it to determine which controller to sync, or to update expectations. This could lead to instability or controllers getting stuck.
Ref: https://github.com/kubernetes/kubernetes/issues/24433
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @erictune @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42128, 42064, 42253, 42309, 42322)
Add storage.k8s.io/v1 API
This is combined version of reverted #40088 (first 4 commits) and #41646. The difference is that all controllers and tests use old `storage.k8s.io/v1beta1` API so in theory all tests can pass on GKE.
Release note:
```release-note
StorageClassName attribute has been added to PersistentVolume and PersistentVolumeClaim objects and should be used instead of annotation `volume.beta.kubernetes.io/storage-class`. The beta annotation is still working in this release, however it will be removed in a future release.
```
Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
ExecProbes should be able to do simple env var substitution
For containers that don't have bash, we should support env substitution
like we do on command and args. However, without major refactoring
valueFrom is not supportable from inside the prober. For now, implement
substitution based on hardcoded env and leave TODOs for future work.
Improves the state of #40846, will spawn a follow up issue for future refactoring after CRI settles down
Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
Don't sync IPtables before underlying store/reflector is fully synced
Ref #42108
Build on top of #42108 - only the second commit is unique.
Automatic merge from submit-queue (batch tested with PRs 42126, 42130, 42232, 42245, 41932)
fix LeaderTransitions always zero
on leader transition, LeaderTransitions is increased to 1, but
then cleared to zero by next renew.
External monitoring system may watch LeaderTransitions and rely
on it's facticity.
Automatic merge from submit-queue (batch tested with PRs 41980, 42192, 42223, 41822, 42048)
CRI: Make dockershim better implements CRI.
When thinking about CRI Validation test, I found that `PodSandboxStatus.Linux.Namespaces.Options.HostPid` and `PodSandboxStatus.Linux.Namespaces.Options.HostIpc` are not populated. Although they are not used by kuberuntime now, we should populate them to conform to CRI.
/cc @yujuhong @feiskyer
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Use `docker logs` directly if the docker logging driver is not `json-file`
Fixes https://github.com/kubernetes/kubernetes/issues/41996.
Post the PR first, I still need to manually test this, because we don't have test coverage for journald logging pluggin.
@yujuhong @dchen1107
/cc @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Admission Controller: Add Pod Preset
Based off the proposal in https://github.com/kubernetes/community/pull/254
cc @pmorie @pwittrock
TODO:
- [ ] tests
**What this PR does / why we need it**: Implements the Pod Injection Policy admission controller
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Added new Api `PodPreset` to enable defining cross-cutting injection of Volumes and Environment into Pods.
```
Automatic merge from submit-queue
Add RBAC roles for bootstrap controllers
Supercedes https://github.com/kubernetes/kubernetes/pull/42221
When locking down controllers to individual RBAC roles we need to make sure that the bootstrap controllers have the right permissions.
This adds the roles and bindings at the correct namespace scopes for the bootstrap-signer and token-cleaner controllers.
@liggitt ptal
@jbeda @luxas you got a good way to test this? It must not be covered in normal e2e or we'd've seen the issue before.
Automatic merge from submit-queue
AWS: run k8s master in different account or different provider
Currently the master and the nodes must run in the same account. With this change the master can run in a different AWS account, on a different cloud provider or on premise.
## Release Notes
```release-note
AWS cloud provider: allow to run the master with a different AWS account or even on a different cloud provider than the nodes.
```
The RC/RS Listers still use selectors, because this is the behavior
expected by callers. This clarifies the meaning of the returned list.
Some callers may need to switch to using GetControllerOf() instead,
but that is a separate, case-by-case issue.
Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)
Burstable QoS cgroup has cpu shares assigned
**What this PR does / why we need it**:
This PR sets the Burstable QoS cgroup cpu shares value to the sum of the pods cpu requests in that tier. We need it for proper evaluation of CPU shares in the new QoS hierarchy.
**Special notes for your reviewer**:
It builds against the framework proposed for https://github.com/kubernetes/kubernetes/pull/41833
Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)
Ensure pod cgroup is deleted prior to deletion of pod
**What this PR does / why we need it**:
This PR ensures that the kubelet removes the pod cgroup sandbox prior to deletion of a pod from the apiserver. We need this to ensure that the default behavior in the kubelet is to not leak resources.
Automatic merge from submit-queue
Re-writing of the resolv.conf file generated by docker
Fixes#17406
Docker 1.12 will contain feature "The option --dns and --net=host should not be mutually exclusive" (docker/docker#22408)
This patch adds optional support for this ability in kubelet (for now in case of "hostNetwork: true" set all dns settings are ignored if any).
To enable feature use newly added kubelet flag: --allow-dns-for-hostnet=true
export functions from pkg/api/validation
add settings API
add settings to pkg/registry
add settings api to pkg/master/master.go
add admission control plugin for pod preset
add new admission control plugin to kube-apiserver
add settings to import_known_versions.go
add settings to codegen
add validation tests
add settings to client generation
add protobufs generation for settings api
update linted packages
add settings to testapi
add settings install to clientset
add start of e2e
add pod preset plugin to config-test.sh
Signed-off-by: Jess Frazelle <acidburn@google.com>
Automatic merge from submit-queue
HPA Controller: Use Custom Metrics API
This commit switches over the HPA controller to use the custom metrics
API. It also converts the HPA controller to use the generated client
in k8s.io/metrics for the resource metrics API.
In order to enable support, you must enable
`--horizontal-pod-autoscaler-use-rest-clients` on the
controller-manager, which will switch the HPA controller's MetricsClient
implementation over to use the standard rest clients for both custom
metrics and resource metrics. This requires that at the least resource
metrics API is registered with kube-aggregator, and that the controller
manager is pointed at kube-aggregator. For this to work, Heapster
must be serving the new-style API server (`--api-server=true`).
Before this merges, this will need kubernetes/metrics#2 to merge, and a godeps update to pull that in.
It's also semi-dependent on kubernetes/heapster#1537, but that is not required in order for this to merge.
**Release note**:
```release-note
Allow the Horizontal Pod Autoscaler controller to talk to the metrics API and custom metrics API as standard APIs.
```
Automatic merge from submit-queue
statefulset: wait for pvc cache sync
#42056 switched the statefulset controller to use the pvc shared informer/lister, but accidentally left out waiting for its cache to sync.
cc @kubernetes/sig-apps-pr-reviews @kargakis @foxish @kow3ns @smarterclayton @deads2k
Automatic merge from submit-queue
fix rsListerSynced and podListerSynced for DeploymentController
**What this PR does / why we need it**:
There is a mistake when initializing `DeploymentController`'s `rsListerSynced` and `podListerSynced` in `NewDeploymentController`, they are all initialized to `Deployment`'s `Informer`, so the `DeploymentController` maybe running before the `ReplicaSet` cache and `Pod` cache has been synced.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Indeed according unit test is neccessary, but this bug fix is simple, and if the tests is neccessary I will submit another PR later.
**Release note**:
```release-note
```
Automatic merge from submit-queue
Implement support for mount options in PVs
**What this PR does / why we need it**:
This PR implements support for mount options in PersistentVolume via `volume.beta.kubernetes.io/mount-options` annotation.
**Which issue this PR fixes**
Fixes https://github.com/kubernetes/features/issues/168
**Release note**:
```
Enable additional, custom mount options to be passed to PersistentVolume objects via volume.beta.kubernetes.io/mount-options annotation.
```
Automatic merge from submit-queue
Extensible Userspace Proxy
This PR refactors the userspace proxy to allow for custom proxy socket implementations.
It changes the the ProxySocket interface to ensure that other packages can properly implement it (making sure all arguments are publicly exposed types, etc), and adds in a mechanism for an implementation to create an instance of the userspace proxy with a non-standard ProxySocket.
Custom ProxySockets are useful to inject additional logic into the actual proxying. For example, our idling proxier uses a custom proxy socket to hold connections and notify the cluster that idled scalable resources need to be woken up.
Also-Authored-By: Ben Bennett bbennett@redhat.com
Introduced chages:
1. Re-writing of the resolv.conf file generated by docker.
Cluster dns settings aren't passed anymore to docker api in all cases, not only for pods with host network:
the resolver conf will be overwritten after infra-container creation to override docker's behaviour.
2. Added new one dnsPolicy - 'ClusterFirstWithHostNet', so now there are:
- ClusterFirstWithHostNet - use dns settings in all cases, i.e. with hostNet=true as well
- ClusterFirst - use dns settings unless hostNetwork is true
- Default
Fixes#17406
Set the vpcID when dummy is created (+1 squashed commit)
Squashed commits:
[0b1ac6e83e] Use the VPC flag and KubernetesClusterTag as identifier (+1 squashed commit)
Squashed commits:
[962bc56e38] Remove again availabilityZone and fix naming (+1 squashed commit)
Squashed commits:
[e3d1b41807] Use the VCID flag as identifier (+1 squashed commit)
Squashed commits:
[5b99fe6243] Add flag for external master
Automatic merge from submit-queue
Add apps/v1beta1 deployments with new defaults
This pull introduces deployments under `apps/v1beta1` and fixes#23597 and #23304.
TODO:
* [x] - create new type `apps/v1beta1.Deployment`
* [x] - update kubectl (stop, scale)
* [ ] - ~~new `kubectl run` generator~~ - this will only duplicate half of generator code, I suggest replacing current to use new endpoint
* [ ] - ~~create extended tests~~ - I've added integration and cmd tests verifying new endpoints
* [ ] - ~~create `hack/test-update-storage-objects.sh`~~ - see above
This is currently blocked by https://github.com/kubernetes/kubernetes/pull/38071, due to conflicting name `v1beta1.Deployment`.
```release-note
Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments.
```
@kargakis @mfojtik @kubernetes/sig-apps-misc
Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)
Make 'docker-email' optional on dockercfg secrets
It is not required for most username/password registries.
Fixes#41727
Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)
Support unqualified and partially qualified domain name in DNS query in Windows kube-proxy
**What this PR does / why we need it**:
In Windows container networking, --dns-search is not currently supported on Windows Docker. Besides, even with --dns-suffix, inside Windows container DNS suffix is not appended to DNS query names. That makes unqualified domain name or partially qualified domain name in DNS query not able to resolve.
This PR provides a solution to resolve unqualified domain name or partially qualified domain name in DNS query for Windows container in Windows kube-proxy. It uses well-known Kubernetes DNS suffix as well host DNS suffix search list to append to the name in DNS query. DNS packet in kube-proxy UDP stream is modified as appropriate.
This PR affects the Windows kube-proxy only.
**Special notes for your reviewer**:
This PR is based on top of Anthony Howe's commit 48647fb, 0e37f0a and 7e2c71f which is already included in the PR 41487. Please only review commit b9dfb69.
**Release note**:
```release-note
Add DNS suffix search list support in Windows kube-proxy.
```
This commit switches over the HPA controller to use the custom metrics
API. It also converts the HPA controller to use the generated client
in k8s.io/metrics for the resource metrics API.
In order to enable support, you must enable
`--horizontal-pod-autoscaler-use-rest-clients` on the
controller-manager, which will switch the HPA controller's MetricsClient
implementation over to use the standard rest clients for both custom
metrics and resource metrics. This requires that at the least resource
metrics API is registered with kube-aggregator, and that the controller
manager is pointed at kube-aggregator. For this to work, Heapster
must be serving the new-style API server (`--api-server=true`).
Automatic merge from submit-queue
Prevent caching empty discovery docs
If kubectl makes requests before a server fully initialized, it could cache bad discovery responses (results of a partially forbidden discovery call, etc)
This changes to only cache responses that included at least one group or resource so that total failure doesn't get cached for 5 minutes
Automatic merge from submit-queue
Extend experimental support to multiple Nvidia GPUs
Extended from #28216
```release-note
`--experimental-nvidia-gpus` flag is **replaced** by `Accelerators` alpha feature gate along with support for multiple Nvidia GPUs.
To use GPUs, pass `Accelerators=true` as part of `--feature-gates` flag.
Works only with Docker runtime.
```
1. Automated testing for this PR is not possible since creation of clusters with GPUs isn't supported yet in GCP.
1. To test this PR locally, use the node e2e.
```shell
TEST_ARGS='--feature-gates=DynamicKubeletConfig=true' FOCUS=GPU SKIP="" make test-e2e-node
```
TODO:
- [x] Run manual tests
- [x] Add node e2e
- [x] Add unit tests for GPU manager (< 100% coverage)
- [ ] Add unit tests in kubelet package
Automatic merge from submit-queue (batch tested with PRs 41921, 41695, 42139, 42090, 41949)
Unify fake runtime helper in kuberuntime, rkt and dockertools.
Addresses https://github.com/kubernetes/kubernetes/pull/42081#issuecomment-282429775.
Add `pkg/kubelet/container/testing/fake_runtime_helper.go`, and change `kuberuntime`, `rkt` and `dockertools` to use it.
@yujuhong This is a small unit test refactoring PR. Could you help me review it?
Automatic merge from submit-queue (batch tested with PRs 41921, 41695, 42139, 42090, 41949)
AWS: Support shared tag `kubernetes.io/cluster/<clusterid>`
We recognize an additional cluster tag:
kubernetes.io/cluster/<clusterid>
This now allows us to share resources, in particular subnets.
In addition, the value is used to track ownership/lifecycle. When we
create objects, we record the value as "owned".
We also refactor out tags into its own file & class, as we are touching
most of these functions anyway.
```release-note
AWS: Support shared tag `kubernetes.io/cluster/<clusterid>`
```
Automatic merge from submit-queue (batch tested with PRs 38676, 41765, 42103, 41833, 41702)
AWS: Skip instances that are taggged as a master
We recognize a few AWS tags, and skip over masters when finding zones
for dynamic volumes. This will fix#34583.
This is not perfect, in that really the scheduler is the only component
that can correctly choose the zone, but should address the common
problem.
```release-note
AWS: Do not consider master instance zones for dynamic volume creation
```
Automatic merge from submit-queue (batch tested with PRs 38676, 41765, 42103, 41833, 41702)
kubelet: cm: refactor QoS logic into seperate interface
This commit has no functional change. It refactors the QoS cgroup logic into a new `QOSContainerManager` interface to allow for better isolation for QoS cgroup features coming down the pike.
This is a breakout of the refactoring component of my QoS memory limits PR https://github.com/kubernetes/kubernetes/pull/41149 which will need to be rebased on top of this.
@vishh @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 38676, 41765, 42103, 41833, 41702)
Support --context flag completion for kubectl
**What this PR does / why we need it**:
With this PR, `--context` flag completion is supported for kubectl.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 41597, 42185, 42075, 42178, 41705)
Honor output formats in kubectl patch
Currently, output formats other than `-o name` are only honored when in `--local` mode.
This PR also prints the result from the server when in regular mode
Automatic merge from submit-queue (batch tested with PRs 41597, 42185, 42075, 42178, 41705)
force rbd image unlock if the image is not used
**What this PR does / why we need it**:
Ceph RBD image could be locked if the host that holds the lock is down. In such case, the image cannot be used by other Pods.
The fix is to detect the orphaned locks and force unlock.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#31790
**Special notes for your reviewer**:
Note, previously, RBD volume plugin maps the image, mount it, and create a lock on the image. Since the proposed fix uses `rbd status` output to determine if the image is being used, the sequence has to change to: rbd lock checking (through `rbd lock list`), mapping check (through `rbd status`), forced unlock if necessary (through `rbd lock rm`), image lock, image mapping, and mount.
**Release note**:
```release-note
force unlock rbd image if the image is not used
```
Automatic merge from submit-queue (batch tested with PRs 42162, 41973, 42015, 42115, 41923)
kubectl drain: make code reusable
DrainOptions requires a few fields to be set, and the expectation is
that these are set as part of construction of the object. If they are
set, then the drain code can be reused in other kubernetes projects.
This does not create a contract that DrainOptions should fulfill going
forwards, any more than any of the other types that happen to be exposed
are part of the contract. Instead, this merely makes use outside the
package possible.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 42162, 41973, 42015, 42115, 41923)
Increase Min Timeout for kill pod
Should mitigate #41347, which describes flakes in the inode eviction test due to "GracePeriodExceeded" errors.
When we use gracePeriod == 0, as we do in eviction, the pod worker currently sets a timeout of 2 seconds to kill a pod.
We are hitting this timeout fairly often during eviction tests, causing extra pods to be evicted (since the eviction manager "fails" to evict that pod, and kills the next one).
This PR increases the timeout from 2 seconds to 4, although we could increase it even more if we think that would be appropriate.
cc @yujuhong @vishh @derekwaynecarr
Automatic merge from submit-queue
fix kubectl describe pod, show tolerationSeconds
**What this PR does / why we need it**:
tolerationSeconds is now not shown in kubectl describe resutl, this PR is to fix it.
With this fix, pod toleration with tolerationSeconds would like below:
```yaml
Name: bar
Namespace: foo
Node: /
Labels: <none>
Status:
IP:
Controllers: <none>
Containers: <none>
No volumes.
QoS Class:
Node-Selectors: <none>
Tolerations: key1=value1
key2=value2:NoSchedule
key3=value3:NoExecute for 300s
```
**Which issue this PR fixes** :
Related issue: #1574
Related PR: #39469
**Special notes for your reviewer**:
**Release note**:
```release-note
make kubectl describe pod show tolerationSeconds
```
Automatic merge from submit-queue
Don't filter items when resources requested by name
Add tracking on resource.Builder if a "named" item is requested (from
file, stream, url, or resource args) and use that in `get` to accurately
determine whether to filter resources. Add tests.
Fixes#41150, #40492
```release-note
Completed pods should not be hidden when requested by name via `kubectl get`.
```
Automatic merge from submit-queue (batch tested with PRs 42200, 39535, 41708, 41487, 41335)
Update kube-proxy support for Windows
**What this PR does / why we need it**:
The kube-proxy is built upon the sophisticated iptables NAT rules. Windows does not have an equivalent capability. This introduces a change to the architecture of the user space mode of the Windows version of kube-proxy to match the capabilities of Windows.
The proxy is organized around service ports and portals. For each service a service port is created and then a portal, or iptables NAT rule, is opened for each service ip, external ip, node port, and ingress ip. This PR merges the service port and portal into a single concept of a "ServicePortPortal" where there is one connection opened for each of service IP, external ip, node port, and ingress IP.
This PR only affects the Windows kube-proxy. It is important for the Windows kube-proxy because it removes the limited portproxy rule and RRAS service and enables full tcp/udp capability to services.
**Special notes for your reviewer**:
**Release note**:
```
Add tcp/udp userspace proxy support for Windows.
```
For containers that don't have bash, we should support env substitution
like we do on command and args. However, without major refactoring
valueFrom is not supportable from inside the prober. For now, implement
substitution based on hardcoded env and leave TODOs for future work.
Add tracking on resource.Builder if a "named" item is requested (from
file, stream, url, or resource args) and use that in `get` to accurately
determine whether to filter resources. Add tests.
- Add a new type PortworxVolumeSource
- Implement the kubernetes volume plugin for Portworx Volumes under pkg/volume/portworx
- The Portworx Volume Driver uses the libopenstorage/openstorage specifications and apis for volume operations.
Changes for k8s configuration and examples for portworx volumes.
- Add PortworxVolume hooks in kubectl, kube-controller-manager and validation.
- Add a README for PortworxVolume usage as PVs, PVCs and StorageClass.
- Add example spec files
Handle code review comments.
- Modified READMEs to incorporate to suggestions.
- Add a test for ReadWriteMany access mode.
- Use util.UnmountPath in TearDown.
- Add ReadOnly flag to PortworxVolumeSource
- Use hostname:port instead of unix sockets
- Delete the mount dir in TearDown.
- Fix link issue in persistentvolumes README
- In unit test check for mountpath after Setup is done.
- Add PVC Claim Name as a Portworx Volume Label
Generated code and documentation.
- Updated swagger spec
- Updated api-reference docs
- Updated generated code under pkg/api/v1
Godeps update for Portworx Volume Driver
- Adds github.com/libopenstorage/openstorage
- Adds go.pedge.io/pb/go/google/protobuf
- Updates Godep Licenses
Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)
Use chroot for containerized mounts
This PR is to modify the containerized mounter script to use chroot
instead of rkt fly. This will avoid the problem of possible large number
of mounts caused by rkt containers if they are not cleaned up.
Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)
Fix azure file secret reference
Follow up to https://github.com/kubernetes/kubernetes/pull/41957
Fixes nil dereference getting secret name from AzureFile volume source.
Adds unit tests to make sure all secret references are extracted correctly, and adds reflective tests to help catch drift if new secret references are added to the pod spec
Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)
client-gen: create private registry in fake clientset
This cuts of the laster `k8s.io/kubernetes/pkg/api.{Registry+Scheme+Codecs}` dependency from the clientsets. This enables clientset generation for packages that must not have a dependency onto kubernetes itself.
@deads2k there is more than the namespace checking we discussed: the RESTMapper built from the registry. This introduces a private registry. I try get that out from the normal versioned client as much as possible. I would even like to remove this private registry some day, at best remove all registry code from the client. But that's for another day...
Automatic merge from submit-queue (batch tested with PRs 40746, 41699, 42108, 42174, 42093)
Switch kube-proxy to informers & save 2/3 of cpu & memory of non-iptables related code.
Fix#42000
This PR should be no-op from the behavior perspective.
It is changing KubeProxy to use standard "informer" framework instead of combination of reflector + undelta store.
This is significantly reducing CPU usage of kube-proxy and number of memory allocations.
Previously, on every endpoints/service update, we were copying __all__ endpoints/services at least 3 times, now it is once (which should also be removed in the future).
In Kubemark-500, hollow-proxies were processing backlog from load test for an hour after the test was finishing. With this change, it is keeping up with the load.
@thockin @ncdc @derekwaynecarr
Automatic merge from submit-queue (batch tested with PRs 40746, 41699, 42108, 42174, 42093)
Output result of apply operation
Fixes#41690
Plumbs the resulting object from patch operations back to the top level so it can be output when printing
Automatic merge from submit-queue
numeric ordering of kubectl outputs
**What this PR does / why we need it**:
Instead of having kubectl listing the pods in a alphabetical way:
foobar-1-build
foobar-10-build
foobar-2-build
foobar-3-build
With the parameter --sort-by '{.metadata.name}' it now gives:
foobar-1-build
foobar-2-build
foobar-3-build
foobar-10-build
**Which issue this PR fixes**
https://github.com/openshift/origin/issues/7229
**Special notes for your reviewer**:
I have followed the dependencies requirements from https://github.com/kubernetes/community/blob/master/contributors/devel/godep.md
**Release note**:
```release-note
Import a natural sorting library and use it in the sorting printer.
```
With bug #27653, kubelet could remove mounted volumes and delete user data.
The bug itself is fixed, however our trust in kubelet is significantly lower.
Let's add an extra version of RemoveAll that does not cross mount boundary
(rm -rf --one-file-system).
It calls lstat(path) three times for each removed directory - once in
RemoveAllOneFilesystem and twice in IsLikelyNotMountPoint, however this way
it's platform independent and the directory that is being removed by kubelet
should be almost empty.
Automatic merge from submit-queue
clean up generic apiserver options
Clean up generic apiserver options before we tag any levels. This makes them more in-line with "normal" api servers running on the platform.
Also remove dead example code.
@sttts
Automatic merge from submit-queue
Reserve kubernetes.io and k8s.io namespace for flex volume options
Split from https://github.com/kubernetes/kubernetes/pull/39488.
Flex volume already stuffs system information into the options map, and assumes it is free to do so:
```
optionFSType = "kubernetes.io/fsType"
optionReadWrite = "kubernetes.io/readwrite"
optionKeySecret = "kubernetes.io/secret"
```
this formalizes that by reserving the `kubernetes.io` and `k8s.io` namespaces so that user-specified options are never stomped by the system, and flex plugins can know that options with those namespaces came from the system, not user-options.
```release-note
Parameter keys in a StorageClass `parameters` map may not use the `kubernetes.io` or `k8s.io` namespaces.
```
Automatic merge from submit-queue (batch tested with PRs 41937, 41151, 42092, 40269, 42135)
Add a unit test for idempotent applys to the TPR entries.
The test in apply_test follows the general pattern of other tests.
We load from a file in test/fixtures and mock the API server in the
function closure in the HttpClient call.
The apply operation expects a last-modified-configuration annotation.
That is written verbatim in the test/fixture file.
References #40841
**What this PR does / why we need it**:
Adds one unit test for TPR's using applies.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
References:
https://github.com/kubernetes/features/issues/95https://github.com/kubernetes/kubernetes/issues/40841#issue-204769102
**Special notes for your reviewer**:
I am not super proud of the tpr-entry name.
But I feel like we need to call the two objects differently.
The one which has Kind:ThirdPartyResource
and the one has Kind:Foo.
Is the name "ThirdPartyResource" used interchangeably for both ? I used tpr-entry for the Kind:Foo object.
Also I !assume! this is testing an idempotent apply because the last-applied-configuration annotation is the same as the object itself.
This is the state I see in the logs of kubectl if I do a proper idempotent apply of a third party resource entry.
I guess I will know more once I start playing around with apply command that change TPR objects.
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 41234, 42186, 41615, 42028, 41788)
apimachinery: handle duplicated and conflicting type registration
Double registrations were leading to duplications in `KnownKinds()`. Conflicting registrations with same gvk, but different types were not detected.
Automatic merge from submit-queue (batch tested with PRs 41234, 42186, 41615, 42028, 41788)
Make DaemonSet respect critical pods annotation when scheduling
**What this PR does / why we need it**: #41612
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#41612
**Special notes for your reviewer**:
**Release note**:
```release-note
Make DaemonSet respect critical pods annotation when scheduling.
```
cc @kubernetes/sig-apps-feature-requests @erictune @vishh @liggitt @kargakis @lukaszo @piosz @davidopp
on leader transition, LeaderTransitions is increased to 1, but
then cleared to zero by next renew.
External monitoring system may watch LeaderTransitions and rely
on it's facticity.
Automatic merge from submit-queue
Enforce Node Allocatable via cgroups
This PR enforces node allocatable across all pods using a top level cgroup as described in https://github.com/kubernetes/community/pull/348
This PR also provides an option to enforce `kubeReserved` and `systemReserved` on user specified cgroups.
This PR will by default make kubelet create top level cgroups even if `kubeReserved` and `systemReserved` is not specified and hence `Allocatable = Capacity`.
```release-note
New Kubelet flag `--enforce-node-allocatable` with a default value of `pods` is added which will make kubelet create a top level cgroup for all pods to enforce Node Allocatable. Optionally, `system-reserved` & `kube-reserved` values can also be specified separated by comma to enforce node allocatable on cgroups specified via `--system-reserved-cgroup` & `--kube-reserved-cgroup` respectively. Note the default value of the latter flags are "".
This feature requires a **Node Drain** prior to upgrade failing which pods will be restarted if possible or terminated if they have a `RestartNever` policy.
```
cc @kubernetes/sig-node-pr-reviews @kubernetes/sig-node-feature-requests
TODO:
- [x] Adjust effective Node Allocatable to subtract hard eviction thresholds
- [x] Add unit tests
- [x] Complete pending e2e tests
- [x] Manual testing
- [x] Get the proposal merged
@dashpole is working on adding support for evictions for enforcing Node allocatable more gracefully. That work will show up in a subsequent PR for v1.6
Automatic merge from submit-queue (batch tested with PRs 41205, 42196, 42068, 41588, 41271)
[CRI] enable kubenet traffic shaping
ref: https://github.com/kubernetes/kubernetes/issues/37316
Another way to do this is to expose another interface in network host to allow network plugins to retrieve annotation. But that seems unnecessary and more complicated.
This changes the userspace proxy so that it cleans up its conntrack
settings when a service is removed (as the iptables proxy already
does). This could theoretically cause problems when a UDP service
as deleted and recreated quickly (with the same IP address). As
long as packets from the same UDP source IP and port were going to
the same destination IP and port, the the conntrack would apply and
the packets would be sent to the old destination.
This is astronomically unlikely if you did not specify the IP address
to use in the service, and even then, only happens with an "established"
UDP connection. However, in cases where a service could be "switched"
between using the iptables proxy and the userspace proxy, this case
becomes much more frequent.
This commit makes the userspace proxy keep an ObjectReference to the
service being proxied. This allows the consumers of the `ServiceInfo`
struct, like `ProxySockets` to emit events about or otherwise refer to
the service.
This commit adds a new method for constructing userspace proxiers,
`NewCustomProxier`. `NewCustomProxier` functions identically to
`NewProxier`, except that it allows a custom constructor method to
be passed in to construct instances of ProxySocket.
This commit makes it possible for the `ProxySocket` interface to be
implemented by types outside of the `userspace` package. It mainly just
exposes relevant types and fields as public.
We recognize an additional cluster tag:
kubernetes.io/cluster/<clusterid>
This now allows us to share resources, in particular subnets.
In addition, the value is used to track ownership/lifecycle. When we
create objects, we record the value as "owned".
We also refactor out tags into its own file & class, as we are touching
most of these functions anyway.
Automatic merge from submit-queue (batch tested with PRs 42053, 41282, 42056, 41663, 40927)
Allow getting logs directly from deployment, job and statefulset
**Special notes for your reviewer**:
@smarterclayton you asked for it in OpenShift
```release-note
kubectl logs allows getting logs directly from deployment, job and statefulset
```
Automatic merge from submit-queue (batch tested with PRs 42053, 41282, 42056, 41663, 40927)
Update kubeadm token to work as expected
**What this PR does / why we need it**:
Follows up: https://github.com/kubernetes/kubernetes/pull/41509
Updates `kubeadm token` to work as discussed in https://docs.google.com/document/d/1deJYPIF4LmhGjDVaqrswErIrV7mtwJgovtLnPCDxP7U/edit#
Promotes the command from the `ex` subcommand which now is named `alpha` for clarity. (This will later become `kubeadm alpha phase`)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Example UX:
```console
sudo ./kubeadm token --help
This command will manage Bootstrap Token for you.
Please note this usage of this command is optional, and mostly for advanced users.
In short, Bootstrap Tokens are used for establishing bidirectional trust between a client and a server.
A Bootstrap Token can be used when a client (for example a node that's about to join the cluster) needs
to trust the server it is talking to. Then a Bootstrap Token with the "signing" usage can be used.
Bootstrap Tokens can also function as a way to allow short-lived authentication to the API Server
(the token serves as a way for the API Server to trust the client), for example for doing the TLS Bootstrap.
What is a Bootstrap Token more exactly?
- It is a Secret in the kube-system namespace of type "bootstrap.kubernetes.io/token".
- A Bootstrap Token must be of the form "[a-z0-9]{6}.[a-z0-9]{16}"; the former part is the public Token ID,
and the latter is the Token Secret, which must be kept private at all circumstances.
- The name of the Secret must be named "bootstrap-token-(token-id)".
You can read more about Bootstrap Tokens in this proposal:
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/bootstrap-discovery.md
Usage:
kubeadm token [flags]
kubeadm token [command]
Available Commands:
create Create bootstrap tokens on the server.
delete Delete bootstrap tokens on the server.
generate Generate and print a bootstrap token, but do not create it on the server.
list List bootstrap tokens on the server.
Flags:
--kubeconfig string The KubeConfig file to use for talking to the cluster (default "/etc/kubernetes/admin.conf")
Use "kubeadm token [command] --help" for more information about a command.
lucas@THENINJA:~/luxas/kubernetes$ sudo ./kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION
70c388.41a07b703aa4bedf <forever> <never> authentication,signing The default bootstrap token generated by 'kubeadm init'.
lucas@THENINJA:~/luxas/kubernetes$ sudo ./kubeadm token create
c57e6a.abb75fa1debe555f
lucas@THENINJA:~/luxas/kubernetes$ sudo ./kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION
70c388.41a07b703aa4bedf <forever> <never> authentication,signing The default bootstrap token generated by 'kubeadm init'.
c57e6a.abb75fa1debe555f <forever> <never> authentication,signing <none>
lucas@THENINJA:~/luxas/kubernetes$ sudo ./kubeadm token create s
token ["s"] was not of form ["^([a-z0-9]{6})\\.([a-z0-9]{16})$"]
lucas@THENINJA:~/luxas/kubernetes$ sudo ./kubeadm token create c57e6a.abb75fa1debe555f
a token with id "c57e6a" already exists
lucas@THENINJA:~/luxas/kubernetes$ sudo ./kubeadm token delete c57e6a.abb75fa1debe555f
bootstrap token with id "c57e6a" deleted
```
**Release note**:
```release-note
NONE
```
@dmmcquay @jbeda @mikedanese @errordeveloper @pires
Automatic merge from submit-queue (batch tested with PRs 42053, 41282, 42056, 41663, 40927)
Fully remove hand-written listers and informers
Note: the first commit is from #41927. Adding do-not-merge for now as we'll want that to go in first, and then I'll rebase this on top.
Update statefulset controller to use a lister for PVCs instead of a client request. Also replace a unit test's dependency on legacylisters with the generated ones. cc @kargakis @kow3ns @foxish @kubernetes/sig-apps-pr-reviews
Remove all references to pkg/controller/informers and pkg/client/legacylisters, and remove those packages.
@smarterclayton @deads2k this should be it!
cc @gmarek @wojtek-t @derekwaynecarr @kubernetes/sig-scalability-pr-reviews
The tests in apply_test follows the general pattern of other tests.
We load from a file in test/fixtures and mock the API server in the
function closure in the HttpClient call.
In PATCH request rount-tripper we check that the kubectl apply
implementation worked as expected.
References #40841
Automatic merge from submit-queue (batch tested with PRs 41116, 41804, 42104, 42111, 42120)
make kubectl taint command respect effect NoExecute
**What this PR does / why we need it**:
Part of feature forgiveness implementation, make kubectl taint command respect effect NoExecute.
**Which issue this PR fixes**:
Related Issue: #1574
Related PR: #39469
**Special notes for your reviewer**:
**Release note**:
```release-note
make kubectl taint command respect effect NoExecute
```
Automatic merge from submit-queue (batch tested with PRs 41116, 41804, 42104, 42111, 42120)
Remove SandboxReceived event
This PR removes SandboxReceived event in sync pod.
> This event seems somewhat meaningless, and clouds the event records for a pod. Do we actually need it? Pulling and pod received on the node are very relevant, this seems much less so. Would suggest we either remove it, or turn it into a message that clearly indicates why it has value.
Refer d65309399a (commitcomment-21052453).
cc @smarterclayton @yujuhong
Automatic merge from submit-queue (batch tested with PRs 41116, 41804, 42104, 42111, 42120)
Add support for attacher/detacher interface in Flex volume
Add support for attacher/detacher interface in Flex volume
This change breaks backward compatibility and requires to be release noted.
```release-note
Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts.
```
Automatic merge from submit-queue (batch tested with PRs 41962, 42055, 42062, 42019, 42054)
dockershim puts pause container in pod cgroup
**What this PR does / why we need it**:
The CRI was not launching the pause container in the pod level cgroup. The non-CRI code path was.
Automatic merge from submit-queue (batch tested with PRs 42044, 41694, 41927, 42050, 41987)
Add apply set-last-applied subcommand
implement part of https://github.com/kubernetes/community/pull/287, will rebase after https://github.com/kubernetes/kubernetes/pull/41699 got merged, EDIT: since bug output format has been confirmed, will update the behavior of output format soon
cc @kubernetes/sig-cli-pr-reviews @AdoHe @pwittrock
```release-note
Support kubectl apply set-last-applied command to update the applied-applied-configuration annotation
```
Automatic merge from submit-queue (batch tested with PRs 35408, 41915, 41992, 41964, 41925)
azure: document config file (+ remove unused field)
**What this PR does / why we need it**:
* documents the config file used by the Azure cloudprovider
* removes an unused field that shouldn't have been added
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 35408, 41915, 41992, 41964, 41925)
add secret option to flag
To resolve the issue of security(pr #35030 ),
> @smarterclayton commented 5 days ago
> This is unfortunately not all flags that could be secrets. The best option would be to add support in spf13/pflag to tag a flag as a secret, and then use that bit to determine the list.
>
> Also, Command() could be used in contexts that need exact parameters (for subshell execution), so we would need to add a new method or extend the signature here to allow exact flags to be retrieved.
we could add a secret option to the flags.
Automatic merge from submit-queue
make iscsi portals optional
**What this PR does / why we need it**: Make iSCSI portals optional
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41954, 40528, 41875, 41165, 41877)
Updating apiserver to return 202 when resource is being deleted asynchronously via cascading deletion
As per https://github.com/kubernetes/kubernetes/issues/33196#issuecomment-278440622.
cc @kubernetes/sig-api-machinery-pr-reviews @smarterclayton @caesarxuchao @bgrant0607 @kubernetes/api-reviewers
```release-note
Updating apiserver to return http status code 202 for a delete request when the resource is not immediately deleted because of user requesting cascading deletion using DeleteOptions.OrphanDependents=false.
```
Automatic merge from submit-queue (batch tested with PRs 41701, 41818, 41897, 41119, 41562)
Allow updates to pod tolerations.
Opening this PR to continue discussion for pod spec tolerations updates when a pod has been scheduled already. This PR is built on top of https://github.com/kubernetes/kubernetes/pull/38957.
@kubernetes/sig-scheduling-pr-reviews @liggitt @davidopp @derekwaynecarr @kubernetes/rh-cluster-infra
Automatic merge from submit-queue
Admit critical pods under resource pressure
And evict critical pods that are not static.
Depends on #40952.
For #40573
Automatic merge from submit-queue (batch tested with PRs 41994, 41969, 41997, 40952, 40576)
Updating kubectl to send delete requests with orphanDependents=false if --cascade is true
Ref https://github.com/kubernetes/kubernetes/issues/40568#38897
Updating kubectl to always set `DeleteOptions.orphanDependents=false` when deleting a resource with `--cascade=true`.
This is primarily for federation where we want to use server side cascading deletion.
Impact on kubernetes: kubectl will do another GET after sending a DELETE and wait till the resource is actually deleted. This can have an impact if the resource has a finalizer. kubectl will wait till the finalizer is removed and then the resource is deleted, which is the right thing to do but a notable change in behavior.
cc @caesarxuchao @lavalamp @smarterclayton @kubernetes/sig-federation-pr-reviews @kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41994, 41969, 41997, 40952, 40576)
Guaranteed admission for Critical Pods
This is the first step in implementing node-level preemption for critical pods.
It defines the AdmissionFailureHandler interface, which allows callers, like the kubelet, to define how failed predicates are handled, and take steps to correct failures if necessary.
In the kubelet's implementation, it triggers preemption if the pod being admitted is critical, and if the only failed predicates are InsufficientResourceErrors, then it prempts (not yet implemented) other other pods to allow admission of the critical pod.
cc: @vishh
Automatic merge from submit-queue
Add namespaced role to inspect particular configmap for delegated authentication
Builds on https://github.com/kubernetes/kubernetes/pull/41814 and https://github.com/kubernetes/kubernetes/pull/41922 (those are already lgtm'ed) with the ultimate goal of making an extension API server zero-config for "normal" authentication cases.
This part creates a namespace role in `kube-system` that can *only* look the configmap which gives the delegated authentication check. When a cluster-admin grants the SA running the extension API server the power to run delegated authentication checks, he should also bind this role in this namespace.
@sttts Should we add a flag to aggregated API servers to indicate they want to look this up so they can crashloop on startup? The alternative is sometimes having it and sometimes not. I guess we could try to key on explicit "disable front-proxy" which may make more sense.
@kubernetes/sig-api-machinery-misc
@ncdc I spoke to @liggitt about this before he left and he was ok in concept. Can you take a look at the details?
Automatic merge from submit-queue (batch tested with PRs 41857, 41864, 40522, 41835, 41991)
kubectl: Allow 'drain --force' to remove orphaned pods
If the managing resource of a given pod (e.g. DaemonSet/ReplicaSet/etc) is deleted (effectively orphaning the pod), and ``kubectl drain --force`` is invoked on the node hosting the pod, the command would fail with an error indicating that the managing resource was not found. This PR reduces the error to a warning if ``--force`` is specified, allowing nodes with orphaned pods to be drained.
Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1424678
cc: @derekwaynecarr
```release-note
Allow drain --force to remove pods whose managing resource is deleted.
```
Automatic merge from submit-queue (batch tested with PRs 41814, 41922, 41957, 41406, 41077)
add kubectl can-i to see if you can perform an action
Adds `kubectl auth can-i <verb> <resource> [<name>]` so that a user can see if they are allowed to perform an action.
@kubernetes/sig-cli-pr-reviews @fabianofranz
This particular command satisfies the immediate need of knowing if you can perform an action without trying that action. When using RBAC in a script that is adding permissions, there is a lag between adding the permission and the permission being realized in the RBAC cache. As a user on the CLI, you almost never see it, but as a script adding a binding and then using that new power, you hit it quite often.
There are natural follow-ons to the same area (hence the `auth` subcommand) to figure out if someone else can perform an action, what actions you can perform in total, and who can perform a given action. Someone else is an API we have already, what-can-i-do was a proposed API a while back and a very useful one for interfaces, and who-can is common question if someone is administering a namespace.
Automatic merge from submit-queue (batch tested with PRs 41814, 41922, 41957, 41406, 41077)
pv_controller: Do not report exponential backoff as error.
It's not an error when recycle/delete/provision operation cannot be started
because it has failed recently. It will be restarted automatically when
backoff expires.
This just pollutes logs without any useful information:
```
E0214 08:00:30.428073 77288 pv_controller.go:1410] error scheduling operaion "delete-pvc-1fa0e8b4-f2b5-11e6-a8bb-fa163ecb84eb[1fbd52ee-f2b5-11e6-a8bb-fa163ecb84eb]": Failed to create operation with name "delete-pvc-1fa0e8b4-f2b5-11e6-a8bb-fa163ecb84eb[1fbd52ee-f2b5-11e6-a8bb-fa163ecb84eb]". An operation with that name failed at 2017-02-14 08:00:15.631133152 -0500 EST. No retries permitted until 2017-02-14 08:00:31.631133152 -0500 EST (16s). Last error: "Cannot delete the volume \"11a4faea-bfc7-4713-88b3-dec492480dba\", it's still attached to a node".
```
```release-note
NONE
```
@kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41814, 41922, 41957, 41406, 41077)
Use consistent helper for getting secret names from pod
Kubelet secret-manager and mirror-pod admission both need to know what secrets a pod spec references. Eventually, a node authorizer will also need to know the list of secrets.
This creates a single (well, double, because api versions) helper that can be used to traverse the secret names referenced from a pod, optionally short-circuiting (for places that are just looking to see if any secrets are referenced, like admission, or are looking for a particular secret ref, like authorization)
Fixes:
* secret manager not handling secrets used by env/envFrom in initcontainers
* admission allowing mirror pods with secret references
@smarterclayton @wojtek-t
Automatic merge from submit-queue
make reconcilation generic to handle roles and clusterroles
We have a need to reconcile regular roles, so this pull moves the reconciliation code to use interfaces (still tightly coupled) rather than structs.
@liggitt @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue
add client-ca to configmap in kube-public
Client CA information is not secret and it's required for any API server trying to terminate a TLS connection. This pull adds the information to configmaps in `kube-public` that look like this:
```yaml
apiVersion: v1
data:
client-ca.crt: |
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
requestheader-allowed-names: '["system:auth-proxy"]'
requestheader-client-ca-file: |
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
requestheader-extra-headers-prefix: '["X-Remote-Extra-"]'
requestheader-group-headers: '["X-Remote-Group"]'
requestheader-username-headers: '["X-Remote-User"]'
kind: ConfigMap
metadata:
creationTimestamp: 2017-02-22T17:54:37Z
name: extension-apiserver-authentication
namespace: kube-system
resourceVersion: "6"
selfLink: /api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication
uid: fa1dd328-f927-11e6-8b0e-28d2447dc82b
```
@kubernetes/sig-auth-api-reviews @liggitt @kubernetes/sig-api-machinery-pr-reviews @lavalamp @sttts
There will need to be a corresponding pull for permissions
Automatic merge from submit-queue (batch tested with PRs 40932, 41896, 41815, 41309, 41628)
Add custom CA file to openstack cloud provider config
**What this PR does / why we need it**: Adds ability to specify custom CA bundle file to verify OpenStack endpoint against. Useful in tests and PoC deployments. Similar to what https://github.com/kubernetes/kubernetes/pull/35488 did for authentication.
**Which issue this PR fixes**: None
**Special notes for your reviewer**: Based on https://github.com/kubernetes/kubernetes/pull/35488 which added support for custom CA file for authentication.
**Release note**:
Automatic merge from submit-queue (batch tested with PRs 40932, 41896, 41815, 41309, 41628)
Make DaemonSets survive taint-based evictions when nodes turn unreachable/notReady
**What this PR does / why we need it**:
DaemonPods shouldn't be deleted by NodeController in case of Node problems.
This PR is to add infinite tolerations for Unreachable/NotReady NoExecute Taints, so that they won't be deleted by NodeController when a node goes unreachable/notReady.
**Which issue this PR fixes** :
fixes#41738
Related PR: #41133
**Special notes for your reviewer**:
**Release note**:
```release-note
Make DaemonSets survive taint-based evictions when nodes turn unreachable/notReady.
```
Automatic merge from submit-queue (batch tested with PRs 40932, 41896, 41815, 41309, 41628)
Modify CronJob API to add job history limits, cleanup jobs in controller
**What this PR does / why we need it**:
As discussed in #34710: this adds two limits to `CronJobSpec`, to limit the number of finished jobs created by a CronJob to keep.
**Which issue this PR fixes**: fixes#34710
**Special notes for your reviewer**:
cc @soltysh, please have a look and let me know what you think -- I'll then add end to end testing and update the doc in a separate commit. What is the timeline to get this into 1.6?
The plan:
- [x] API changes
- [x] Changing versioned APIs
- [x] `types.go`
- [x] `defaults.go` (nothing to do)
- [x] `conversion.go` (nothing to do?)
- [x] `conversion_test.go` (nothing to do?)
- [x] Changing the internal structure
- [x] `types.go`
- [x] `validation.go`
- [x] `validation_test.go`
- [x] Edit version conversions
- [x] Edit (nothing to do?)
- [x] Run `hack/update-codegen.sh`
- [x] Generate protobuf objects
- [x] Run `hack/update-generated-protobuf.sh`
- [x] Generate json (un)marshaling code
- [x] Run `hack/update-codecgen.sh`
- [x] Update fuzzer
- [x] Actual logic
- [x] Unit tests
- [x] End to end tests
- [x] Documentation changes and API specs update in separate commit
**Release note**:
```release-note
Add configurable limits to CronJob resource to specify how many successful and failed jobs are preserved.
```
Automatic merge from submit-queue (batch tested with PRs 41621, 41946, 41941, 41250, 41729)
Refactor printers and describers into their own package.
This sets the stage for using printer code from the server side (decoupled from kubectl) and loosens the coupling between kubectl and the printers. `pkg/printers` contains interfaces and has an import restriction against pulling in API specific code, while `pkg/printers/internalversion` can be used for internal types.
Add a method on `Factory` for retrieving PrinterForCommand which uses the Scheme and RESTMapper from the Factory, not the hardcoded ones. This further separates kubectl from the core API scheme and allows better composition.
Change NamePrinter to use RESTMapper (previously it was hardcoding those conversions). This means that we now return plural resource names (`pods/foo`) but is correct once aliases and shortnames start being returned by the mapper.
This is a prerequisite for server side get, but is pure refactor (contains no new features).
@deads2k @liggitt
Automatic merge from submit-queue (batch tested with PRs 41621, 41946, 41941, 41250, 41729)
bug fix for hostport-syncer
fix a bug introduced by the previous refactoring of hostport-syncer. https://github.com/kubernetes/kubernetes/pull/39443
and fix some nits
Automatic merge from submit-queue
BestEffort QoS class has min cpu shares
**What this PR does / why we need it**:
BestEffort QoS class is given the minimum amount of CPU shares per the QoS design.
Automatic merge from submit-queue (batch tested with PRs 42106, 42094, 42069, 42098, 41852)
Fix availableReplicas validation
An available replica is a ready replica, not the other way around
@kubernetes/sig-apps-bugs caught while testing https://github.com/kubernetes/kubernetes/pull/42097
DrainOptions requires a few fields to be set, and the expectation is
that these are set as part of construction of the object. If they are
set, then the drain code can be reused in other kubernetes projects.
This does not create a contract that DrainOptions should fulfill going
forwards, any more than any of the other types that happen to be exposed
are part of the contract. Instead, this merely makes use outside the
package possible.
Automatic merge from submit-queue (batch tested with PRs 41854, 41801, 40088, 41590, 41911)
Add storage.k8s.io/v1 API
v1 API is direct copy of v1beta1 API. This v1 API gets installed and exposed in this PR, I tested that kubectl can create both v1beta1 and v1 StorageClass.
~~Rest of Kubernetes (controllers, examples,. tests, ...) still use v1beta1 API, I will update it when this PR gets merged as these changes would get lost among generated code.~~ Most parts use v1 API now, it would not compile / run tests without it.
**Release note**:
```
Kubernetes API storage.k8s.io for storage objects is now fully supported and is available as storage.k8s.io/v1. Beta version of the API storage.k8s.io/v1beta1 is still available in this release, however it will be removed in a future Kubernetes release.
Together with the API endpoint, StorageClass annotation "storageclass.beta.kubernetes.io/is-default-class" is deprecated and "storageclass.kubernetes.io/is-default-class" should be used instead to mark a default storage class. The beta annotation is still working in this release, however it won't be supported in the next one.
```
@kubernetes/sig-storage-misc
Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)
Update i18n tools and process.
@fabianofranz @zen @kubernetes/sig-cli-pr-reviews
This is an update to the translation process based on feedback from folks.
The main changes are:
* `msgctx` is being removed from the files.
* String wrapping and string extraction have been separated.
* More tools from the `gettext` family of tools are being used
* Extracted strings are being sorted for canonical ordering
* A `.pot` template has been added.
Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)
Show specific error when a volume is formatted by unexpected filesystem.
kubelet now detects that e.g. xfs volume is being mounted as ext3 because of
wrong volume.Spec.
Mount error is left in the error message to diagnose issues with mounting e.g.
'ext3' volume as 'ext4' - they are different filesystems, however kernel should
mount ext3 as ext4 without errors.
Example kubectl describe pod output:
```
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
41s 3s 7 {kubelet ip-172-18-3-82.ec2.internal} Warning FailedMount MountVolume.MountDevice failed for volume "kubernetes.io/aws-ebs/aws://us-east-1d/vol-ba79c81d" (spec.Name: "pvc-ce175cbb-6b82-11e6-9fe4-0e885cca73d3") pod "3d19cb64-6b83-11e6-9fe4-0e885cca73d3" (UID: "3d19cb64-6b83-11e6-9fe4-0e885cca73d3") with: failed to mount the volume as "ext4", it's already formatted with "xfs". Mount error: mount failed: exit status 32
Mounting arguments: /dev/xvdba /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1d/vol-ba79c81d ext4 [defaults]
Output: mount: wrong fs type, bad option, bad superblock on /dev/xvdba,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so.
```
Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)
Switch scheduler to use generated listers/informers
Where possible, switch the scheduler to use generated listers and
informers. There are still some places where it probably makes more
sense to use one-off reflectors/informers (listing/watching just a
single node, listing/watching scheduled & unscheduled pods using a field
selector).
I think this can wait until master is open for 1.7 pulls, given that we're close to the 1.6 freeze.
After this and #41482 go in, the only code left that references legacylisters will be federation, and 1 bit in a stateful set unit test (which I'll clean up in a follow-up).
@resouer I imagine this will conflict with your equivalence class work, so one of us will be doing some rebasing 😄
cc @wojtek-t @gmarek @timothysc @jayunit100 @smarterclayton @deads2k @liggitt @sttts @derekwaynecarr @kubernetes/sig-scheduling-pr-reviews @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)
controller: fix requeueing progressing deployments
Drop the secondary queue and add either ratelimited or after the
required amount of time that we need to wait directly in the main
queue. In this way we can always be sure that we will sync back
the Deployment if its progress has yet to resolve into a complete
(NewReplicaSetAvailable) or TimedOut condition.
This should also simplify the deployment controller a bit.
Fixes https://github.com/kubernetes/kubernetes/issues/39785. Once this change soaks, I will move the test out of the flaky suite.
@kubernetes/sig-apps-misc
This PR is to modify the containerized mounter script to use chroot
instead of rkt fly. This will avoid the problem of possible large number
of mounts caused by rkt containers if they are not cleaned up.
Add some lines about how to enable multipath for block storage.
A new README was added, because multipath is relevant for at least
FC and iSCSI.
Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>
A minimalistic multipath.conf got written, but it was useless, as
it is unclear if multipathd is running and there was also no
config reload triggered.
This patch drops this snippet. In general it's probably a better idea
to leave the multipath.conf to the component managing the host.
Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>
is contacted if there is a failure in the connected server.
Mount option becomes:
mount -t glusterfs -o log-level=ERROR,log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/glustermount/glusterpod-glusterfs.log,backup-volfile-servers=192.168.100.0:192.168.200.0:192.168.43.149 ..
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Automatic merge from submit-queue
Fix for Support selection of datastore for dynamic provisioning in vS…
Fixes#40558
Current vSphere Cloud provider doesn't allow a user to select a datastore for dynamic provisioning. All the volumes are created in default datastore provided by the user in the global vsphere configuration file.
With this fix, the user will be able to provide the datastore in the storage class definition. This will allow the volumes to be created in the datastore specified by the user in the storage class definition. This field is optional. If no datastore is specified, the volume will be created in the default datastore specified in the global config file.
For example:
User creates a storage class with the datastore
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
name: slow
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: thin
datastore: VMFSDatastore
Now the volume will be created in the datastore - "VMFSDatastore" specified by the user.
If the user creates a storage class without any datastore
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
name: slow
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: thin
Now the volume will be created in the datastore which in the global configuration file (vsphere.conf)
@pdhamdhere @kerneltime
Automatic merge from submit-queue (batch tested with PRs 41667, 41820, 40910, 41645, 41361)
Refactor ControllerRefManager
**What this PR does / why we need it**:
To prepare for implementing ControllerRef across all controllers (https://github.com/kubernetes/community/pull/298), this pushes the common adopt/orphan logic into ControllerRefManager so each controller doesn't have to duplicate it.
This also shares the adopt/orphan logic between Pods and ReplicaSets, so it lives in only one place.
**Which issue this PR fixes**:
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @kubernetes/sig-apps-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41667, 41820, 40910, 41645, 41361)
Switch admission to use shared informers
Originally part of #40097
cc @smarterclayton @derekwaynecarr @deads2k @liggitt @sttts @gmarek @wojtek-t @timothysc @lavalamp @kubernetes/sig-scalability-pr-reviews @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41667, 41820, 40910, 41645, 41361)
Allow multiple mounts in StatefulSet volume zone placement
We have some heuristics that ensure that volumes (and hence stateful set
pods) are spread out across zones. Sadly they forgot to account for
multiple mounts. This PR updates the heuristic to ignore the mount name
when we see something that looks like a statefulset volume, thus
ensuring that multiple mounts end up in the same AZ.
Fix#35695
```release-note
Fix zone placement heuristics so that multiple mounts in a StatefulSet pod are created in the same zone
```
The url which is used for communicating with govmomi should not include
port number. A port number other than 443 will result in 404 error.
VCenterPort stays in VSphereConfig structure for backward compatibility.
This commit adds a method to the `LoadBalancer` interface in the
userspace proxy which allows consumers of the `LoadBalancer` to check if
it thinks a given service has endpoints available.
Automatic merge from submit-queue
add deads2k and sttts to kubeapiserver owners
Adds @deads2k and @sttts to packages we authored or significantly modified.
@lavalamp @smarterclayton
Automatic merge from submit-queue (batch tested with PRs 38702, 41810, 41778, 41858, 41872)
Always enable RBAC in kubeadm and make a pkg with authorization constants
**What this PR does / why we need it**:
This PR:
- Splits the authz constants out into a dedicated package, so consumers don't have to import lots of other things (informers, etc...)
- Makes a `IsValidAuthorizationMode` function for easy checking
- Hooks up kubeadm against the new constant package, for example using the validation method when validating the kubeadm API obj
- Always enables RBAC in kubeadm as discussed with @liggitt and @jbeda
- This because we have to grant some rules in all cases for kubeadm (for instance, making the cluster-info configmap public)
- Adds more unit tests
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@liggitt @jbeda @errordeveloper @dmmcquay @pires @deads2k
Automatic merge from submit-queue (batch tested with PRs 38702, 41810, 41778, 41858, 41872)
gce: Reuse unsuccessfully provisioned volumes.
GCE PD names generated by Kubernetes are guaranteed to be unique - they
contain name of the cluster and UID of the PVC that is behind it.
Presence of a GCE PD that has the same name as we want to provision
indicates that previous provisioning did not go well and most probably
the controller manager process was restarted in the meantime.
Kubernetes should reuse this volume and not provision a new one.
Fixes#38681
Where possible, switch the scheduler to use generated listers and
informers. There are still some places where it probably makes more
sense to use one-off reflectors/informers (listing/watching just a
single node, listing/watching scheduled & unscheduled pods using a field
selector).
Automatic merge from submit-queue
Add ClassName attributes to PV and PVC
This just adds new attributes to PV/PVC. Real code that uses the attributes instead of beta annotations will follow when we agree on the attribute names / style.
Automatic merge from submit-queue
Include all user.Info data in CSR object
In order to use authorization checks to auto-approve CSRs in the future, we need all the info from the user.Info interface.
This mirrors the API fields in the TokenReview API used to return user info, and in the SubjectAccessReview API we use to check authorization.
```release-note
The CertificateSigningRequest API added the `extra` field to persist all information about the requesting user. This mirrors the fields in the SubjectAccessReview API used to check authorization.
```
Automatic merge from submit-queue (batch tested with PRs 41812, 41665, 40007, 41281, 41771)
kube-apiserver: add a bootstrap token authenticator for TLS bootstrapping
Follows up on https://github.com/kubernetes/kubernetes/pull/36101
Still needs:
* More tests.
* To be hooked up to the API server.
- Do I have to do that in a separate PR after k8s.io/apiserver is synced?
* Docs (kubernetes.io PR).
* Figure out caching strategy.
* Release notes.
cc @kubernetes/sig-auth-api-reviews @liggitt @luxas @jbeda
```release-notes
Added a new secret type "bootstrap.kubernetes.io/token" for dynamically creating TLS bootstrapping bearer tokens.
```
Automatic merge from submit-queue (batch tested with PRs 41812, 41665, 40007, 41281, 41771)
Kubelet-rkt: Add useful informations for Ops on the Kubelet Host
Create a Systemd SyslogIdentifier inside the [Service]
Create a Systemd Description inside the [Unit]
**What this PR does / why we need it**:
#### Overview
Logged against the host, it's difficult to identify who's who.
This PR add useful information to quickly get straight to the point with the **DESCRIPTION** field:
```
systemctl list-units "k8s*"
UNIT LOAD ACTIVE SUB DESCRIPTION
k8s_b5a9bdf7-e396-4989-8df0-30a5fda7f94c.service loaded active running kube-controller-manager-172.20.0.206
k8s_bec0d8a1-dc15-4b47-a850-e09cf098646a.service loaded active running nginx-daemonset-gxm4s
k8s_d2981e9c-2845-4aa2-a0de-46e828f0c91b.service loaded active running kube-apiserver-172.20.0.206
k8s_fde4b0ab-87f8-4fd1-b5d2-3154918f6c89.service loaded active running kube-scheduler-172.20.0.206
```
#### Overview and Journal
Always on the host, to easily retrieve the pods logs, this PR add a SyslogIdentifier named as the PodBaseName.
```
# A DaemonSet prometheus-node-exporter is running on the Kubernetes Cluster
systemctl list-units "k8s*" | grep prometheus-node-exporter
k8s_c60a4b1a-387d-4fce-afa1-642d6f5716c1.service loaded active running prometheus-node-exporter-85cpp
# Get the logs from the prometheus-node-exporter DaemonSet
journalctl -t prometheus-node-exporter | wc -l
278
```
Sadly the `journalctl` flag `-t` / `--identifier` doesn't allow a pattern to catch the logs.
Also this field improve any queries made by any tools who exports the Journal (E.g: ES, Kibana):
```
{
"__CURSOR" : "s=86fd390d123b47af89bb15f41feb9863;i=164b2c27;b=7709deb3400841009e0acc2fec1ebe0e;m=1fe822ca4;t=54635e6a62285;x=b2d321019d70f36f",
"__REALTIME_TIMESTAMP" : "1484572200411781",
"__MONOTONIC_TIMESTAMP" : "8564911268",
"_BOOT_ID" : "7709deb3400841009e0acc2fec1ebe0e",
"PRIORITY" : "6",
"_UID" : "0",
"_GID" : "0",
"_SYSTEMD_SLICE" : "system.slice",
"_SELINUX_CONTEXT" : "system_u:system_r:kernel_t:s0",
"_MACHINE_ID" : "7bbb4401667243da81671e23fd8a2246",
"_HOSTNAME" : "Kubelet-Host",
"_TRANSPORT" : "stdout",
"SYSLOG_FACILITY" : "3",
"_COMM" : "ld-linux-x86-64",
"_CAP_EFFECTIVE" : "3fffffffff",
"SYSLOG_IDENTIFIER" : "prometheus-node-exporter",
"_PID" : "88827",
"_EXE" : "/var/lib/rkt/pods/run/c60a4b1a-387d-4fce-afa1-642d6f5716c1/stage1/rootfs/usr/lib64/ld-2.21.so",
"_CMDLINE" : "stage1/rootfs/usr/lib/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/systemd-nspawn [....]",
"_SYSTEMD_CGROUP" : "/system.slice/k8s_c60a4b1a-387d-4fce-afa1-642d6f5716c1.service",
"_SYSTEMD_UNIT" : "k8s_c60a4b1a-387d-4fce-afa1-642d6f5716c1.service",
"MESSAGE" : "[ 8564.909237] prometheus-node-exporter[115]: time=\"2017-01-16T13:10:00Z\" level=info msg=\" - time\" source=\"node_exporter.go:157\""
}
```
The factory knows all possible types, and should abstract the process of
creating all printers. A future refactor can further reduce the
dependencies between printer code and internal types.
Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)
fix issue #41746
**What this PR does / why we need it**:
**Which issue this PR fixes** : fixes#41746
**Special notes for your reviewer**:
cc @feiskyer
Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)
Added `kubectl create clusterrole` command.
Added `kubectl create clusterrole` command.
Fixed part of #39596
**Special notes for your reviewer**:
@deads2k, please help to review this patch, thanks
**Release note**:
```
Added one new command `kubectl create clusterrole` to help user create a single ClusterRole from command line.
```
Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)
Switch statefulset controller to shared informers
Originally part of #40097
I *think* the controller currently makes a deep copy of a StatefulSet before it mutates it, but I'm not 100% sure. For those who are most familiar with this code, could you please confirm?
@beeps @smarterclayton @ingvagabund @sttts @liggitt @deads2k @kubernetes/sig-apps-pr-reviews @kubernetes/sig-scalability-pr-reviews @timothysc @gmarek @wojtek-t
Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)
client-gen: independent scheme for clientsets
This PR adds a clientset internal scheme instead of using `pkg/api.Scheme`. **The clientset API stays the same.**
In detail:
- introduce a scheme for each clientset, i.e. do not use `pkg/api.Scheme+Registry+Codec+ParameterCodecs`.
This makes it easier to compose client-go's clientset (which is rewritten in `staging/copy.sh` and therefore hardcoded to use `k8s.io/client-go/pkg/api.Scheme+Registry+Codecs+ParameterCodecs`) with third-party clientsets (kube-aggregator, openshift, federation) which are not rewritten using `copy.sh` as all of them are self-contained and therefore relocatable.
This fixes https://github.com/kubernetes/kubernetes/pull/41403/files#diff-76edfb07dee54ff7ddeda25c33c10d29R81 and prepares client-gen for use in OpenShift.
- register types into the clientset scheme via `AddToScheme` for versioned clientsets. This decouples the client-go clients from announce+registration (internal clients continue using announce+registry and apigroup installers).
This reduces complexity for client-go, possibly remove the necessity for the announce+register machinery for many use-cases, maybe even to delete it mid-term.
- port federation and testgroup `install/install.go` to `announced.GroupMetaFactory` in order to have a proper `Install.Install(...)` func for registration.
With the first change it's easy to add the types of one clientset to the scheme of the other using the `clientset/scheme.AddToScheme` method. This allows to use cross-clientset `runtime.RawExtensions`:
```golang
import (
"k8s.io/client-go/kubernetes"
clientsetscheme "k8s.io/client-go/kuberentes/scheme"
aggregatorclientsetscheme "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/scheme"
)
kclientset, _ := kubernetes.NewForConfig(c)
aggregatorclientsetscheme.AddToScheme(clientsetscheme.Scheme)
```
Kubernetes types with a `RawExtension` can en/decode aggregator types after this.
TODO:
- [x] fix fake clientsets
- [x] get the `*Options` types registered correctly for core, compare DO-NOT-MERGE commit.
- [x] get prefered version right in internal client. Do we need all versions registered in the internal client to support negotiation?
- [x] run `staging/copy.sh` and run tests: https://github.com/kubernetes/kubernetes/pull/41744
- ~~[ ] fixup usage through-out the code-base~~
- **Follow-up**: move `import_known_versions.go` files somewhere such that import of the `api.Scheme` package automatically installs the apigroups. It looks like we depended on the import fo the clientset for this purpose.
Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)
Add apply view-last-applied subcommand
reopen pr https://github.com/kubernetes/kubernetes/pull/40984, implement part of https://github.com/kubernetes/community/pull/287
for now unit test all pass, the output looks like:
```console
shiywang@dhcp-140-33 template $ ./kubectl apply view last-applied deployment nginx-deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
creationTimestamp: null
name: nginx-deployment
spec:
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx:1.12.10
name: nginx
ports:
- containerPort: 80
resources: {}
status: {}
```
```release-note
Support new kubectl apply view-last-applied command for viewing the last configuration file applied
```
not sure if there is any flag I should updated or the some error handling I should changed.
will generate docs when you guys think is ok.
cc @pwittrock @jessfraz @AdoHe @ymqytw
Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)
Wrap and extract some flag description strings.
Moving on from help strings to flag descriptions.
@zen @fabianofranz @kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)
Fix deployment helper - no assumptions on only one new ReplicaSet
#40415
**Release note**:
```release-note
NONE
```
@kubernetes/sig-apps-bugs
Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)
Change taints/tolerations to api fields
This PR changes current implementation of taints and tolerations from annotations to API fields. Taint and toleration are now part of `NodeSpec` and `PodSpec`, respectively. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints` have been removed.
**Release note**:
Pod tolerations and node taints have moved from annotations to API fields in the PodSpec and NodeSpec, respectively. Pod tolerations and node taints that are defined in the annotations will be ignored. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints` have been removed.
Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)
client-go: don't import client auth provider packages
Both of these auth providers are useful for kubectl but not so much for everyone importing client-go. Let users optionally import them (example [0]) and reduce the overall number of imports that client-go requires.
Quick grep seems to imply it wont import it after.
```
$ grep -r 'client-go/plugin/pkg/client/auth' staging/
staging/src/k8s.io/client-go/plugin/pkg/client/auth/plugins.go: _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
staging/src/k8s.io/client-go/plugin/pkg/client/auth/plugins.go: _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
staging/src/k8s.io/client-go/examples/third-party-resources/main.go: _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
staging/src/k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/clientset.go: _ "k8s.io/client-go/plugin/pkg/client/auth"
staging/src/k8s.io/kube-aggregator/pkg/client/clientset_generated/internalclientset/clientset.go: _ "k8s.io/client-go/plugin/pkg/client/auth"
```
closes https://github.com/kubernetes/client-go/issues/49
updates https://github.com/kubernetes/client-go/issues/79 (removes cloud.google.com/go import)
cc @kubernetes/sig-api-machinery-pr-reviews @kubernetes/sig-auth-pr-reviews
```release-notes
client-go no longer imports GCP OAuth2 and OpenID Connect packages by default.
```
[0] 8b466d64c5/examples/third-party-resources/main.go (L34-L35)
Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)
Enable pod level cgroups by default
**What this PR does / why we need it**:
It enables pod level cgroups by default.
**Special notes for your reviewer**:
This is intended to be enabled by default on 2/14/2017 per the plan outlined here:
https://github.com/kubernetes/community/pull/314
**Release note**:
```release-note
Each pod has its own associated cgroup by default.
```
If a user attempts to use basic auth, and the username/password combination
is rejected, the authenticator should return an error. This distinguishes
requests that did not provide username/passwrod (and are unauthenticated
without error) from ones that attempted to, and failed.
Automatic merge from submit-queue
Log that debug handlers have been turned on.
**What this PR does / why we need it**: PR allows user to have a message in logs that debug handlers are on. It should allow the operator to know and automate a check for the case where debug has been left on.
**Release note**:
```
NONE
```
Conversions can mutate the underlying object (and ours were).
Make a deepcopy before our first conversion at the very start
of the reconciler method in order to avoid mutating the shared
informer cache during conversion.
Fixes#41768
Automatic merge from submit-queue (batch tested with PRs 41364, 40317, 41326, 41783, 41782)
changes to cleanup the volume plugin for recycle
**What this PR does / why we need it**:
Code cleanup. Changing from creating a new interface from the plugin, that then calls a function to recycle a volume, to adding the function to the plugin itself.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#26230
**Special notes for your reviewer**:
Took same approach from closed PR #28432.
Do you want the approach to be the same for NewDeleter(), NewMounter(), NewUnMounter() and should they be in this same PR or submit different PR's for those?
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 41709, 41685, 41754, 41759, 37237)
Projected volume plugin
This is a WIP volume driver implementation as noted in the commit for https://github.com/kubernetes/kubernetes/pull/35313.
Automatic merge from submit-queue (batch tested with PRs 41709, 41685, 41754, 41759, 37237)
Tolerate unknown fields in strategic merge patch
When using `apply` or `edit` with an object that has a compiled-in struct, if an unknown server-side field is sent, or is present in a provided file, the strategic merge patch computation fails looking up type info from the go struct
If the field only exists in one side of the patch (is being added or removed), or is identical in both sides of the patch, we should tolerate missing type info, since it doesn't affect the patch.
Automatic merge from submit-queue
kubeadm: Hide the unnecessary --fuzz-iters flag
super straightforward. We don't want this flag to leak into our UX.
cc @jbeda @dmmcquay @deads2k
change to GetOriginalConfiguration
add bazel
refactor apply view-last-applied command
update some changes
minor change
add unit tests, update
update some codes and genreate docs
update LongDesc
Automatic merge from submit-queue (batch tested with PRs 41706, 39063, 41330, 41739, 41576)
deprecate and disconnect --output-version
For at least two releases, there haven't been multiple versions of API groups and we don't plan to support conversions in commands other than `kubectl convert`. This disconnects the `--output-version` option to be consistent with conversion agnostic command before it becomes an issue.
@kubernetes/sig-cli-pr-reviews @fabianofranz @smarterclayton
```release-note
`--output-version` is ignored for all commands except `kubectl convert`. This is consistent with the generic nature of `kubectl` CRUD commands and the previous removal of `--api-version`. Specific versions can be specified in the resource field: `resource.version.group`, `jobs.v1.batch`.
```
Automatic merge from submit-queue (batch tested with PRs 41706, 39063, 41330, 41739, 41576)
Fix regex match doc of procfs.PidOf
Fixes#41247.
cc @bboreham
Automatic merge from submit-queue (batch tested with PRs 41756, 36344, 34259, 40843, 41526)
Update Photon Controller cloud provider for authentication support
Resolve Issue: [#40755](https://github.com/kubernetes/kubernetes/issues/40755)
1. Update the configuration file for Photon Controller cloud provider
2. Only master nodes can communicate with Photon Controller endpoint
3. Enable support for authentication-enabled Photon Controller endpoint
4. Update NodeAddresses function for query from local node
New format of photon controller config file:
```
[Global]
target = https://[LOAD_BALANCER_IP]:443
project = [PROJECT ID]
overrideIP = true
vmID = [LOCAL VM ID]
authentication = true
```
This config file will be automatically created by Photon Controller cluster management.
If authentication file is set to true, then a pc_login_info file should be placed under /etc/kubernetes with username and password.
This file can be created by user directly.
Or the user can choose to use kubernetes secret and a handling pod to avoid directly login to master nodes. This usage will be available with Photon Controller 1.2.
This is a temporary solution before metadata service becomes available in Photon Controller.
Automatic merge from submit-queue (batch tested with PRs 41756, 36344, 34259, 40843, 41526)
add InternalDNS/ExternalDNS node address types
This PR adds internal/external DNS names to the types of NodeAddresses that can be reported by the kubelet.
will spawn follow up issues for cloud provider owners to include these when possible
```release-note
Nodes can now report two additional address types in their status: InternalDNS and ExternalDNS. The apiserver can use `--kubelet-preferred-address-types` to give priority to the type of address it uses to reach nodes.
```
Automatic merge from submit-queue (batch tested with PRs 41756, 36344, 34259, 40843, 41526)
Migrate rackspace/gophercloud -> gophercloud/gophercloud
`gophercloud` is the go library we use to interact with openstack. It has recently been renamed and incorporated a number of breaking code cleanups. This change migrates the Kubernetes openstack code (openstack provider, keystone auth, cinder volumes) to the new library.
Fixes#30404
```release-note
The openstack cloud provider config no longer supports the deprecated `api-key` option.
```
Note this PR doesn't change the rackspace provider. It uses some rackspace-specific APIs and continues to use the older rackspace/gophercloud library.
Automatic merge from submit-queue (batch tested with PRs 41421, 41440, 36765, 41722)
Use watch param instead of deprecated /watch/ prefix
Switches clients to use watch param instead of /watch/ prefix
```release-note
Clients now use the `?watch=true` parameter to make watch API calls, instead of the `/watch/` path prefix
```
Drop the secondary queue and add either ratelimited or after the
required amount of time that we need to wait directly in the main
queue. In this way we can always be sure that we will sync back
the Deployment if its progress has yet to resolve into a complete
(NewReplicaSetAvailable) or TimedOut condition.
Automatic merge from submit-queue
Make controller-manager resilient to stale serviceaccount tokens
Now that the controller manager is spinning up controller loops using service accounts, we need to be more proactive in making sure the clients will actually work.
Future additional work:
* make a controller that reaps invalid service account tokens (c.f. https://github.com/kubernetes/kubernetes/issues/20165)
* allow updating the client held by a controller with a new token while the controller is running (c.f. https://github.com/kubernetes/kubernetes/issues/4672)
Automatic merge from submit-queue
Convert HPA controller to support HPA v2 mechanics
This PR converts the HPA controller to support the mechanics from HPA v2.
The HPA controller continues to make use of the HPA v1 client, but utilizes
the conversion logic to work with autoscaling/v2alpha1 objects internally.
It is the follow-up PR to #36033 and part of kubernetes/features#117.
**Release note**:
```release-note
NONE
```
There was a bug in the HPA v1 conversion logic that would occur when
a custom metric and a metric that was encoded in v1 as
targetCPUUtilizationPercentage were used at the same time. In this
case, the custom metric could overwrite the CPU metric, or vice versa.
This fixes that bug, and ensures that the fuzzer tests round-tripping
with multiple metrics.
This change migrates the 'openstack' provider and 'keystone'
authenticator plugin to the newer gophercloud/gophercloud library.
Note the 'rackspace' provider still uses rackspace/gophercloud.
Fixes#30404
Automatic merge from submit-queue (batch tested with PRs 39373, 41585, 41617, 41707, 39958)
Feature-Gate affinity in annotations
**What this PR does / why we need it**:
Adds back basic flaggated support for alpha Affinity annotations
**Special notes for your reviewer**:
Reconcile function is placed in the lowest common denominator, which in this case is schedulercache, because you can't place flag-gated functions in apimachinery.
**Release note**:
```
NONE
```
/cc @davidopp
Automatic merge from submit-queue (batch tested with PRs 39373, 41585, 41617, 41707, 39958)
Fix ConfigMaps for Windows
**What this PR does / why we need it**: ConfigMaps were broken for Windows as the existing code used linux specific file paths. Updated the code in `kubelet_getters.go` to use `path/filepath` to get the directories. Also reverted back the code in `secret.go` as updating `kubelet_getters.go` to use `path/filepath` also fixes `secrets`
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/39372
```release-note
Fix ConfigMap for Windows Containers.
```
cc: @pires
Automatic merge from submit-queue (batch tested with PRs 41043, 39058, 41021, 41603, 41414)
add defaultTolerationSeconds admission controller
**What this PR does / why we need it**:
Splited from #34825, add a new admission-controller that
1. adds toleration (with tolerationSeconds = 300) for taint `notReady:NoExecute` to every pod that does not already have a toleration for that taint, and
2. adds toleration (with tolerationSeconds = 300) for taint `unreachable:NoExecute` to every pod that does not already have a toleration for that taint.
**Which issue this PR fixes**:
Related issue: #1574
Related PR: #34825
**Special notes for your reviewer**:
**Release note**:
```release-note
add defaultTolerationSeconds admission controller
```
Automatic merge from submit-queue (batch tested with PRs 41043, 39058, 41021, 41603, 41414)
Rotate the kubelet certificate when about to expire.
Changes the kubelet so it doesn't use the cert/key files directly for
starting the TLS server. Instead the TLS server reads the cert/key from
the new CertificateManager component, which is responsible for
requesting new certificates from the Certificate Signing Request API on
the API Server.
We have some heuristics that ensure that volumes (and hence stateful set
pods) are spread out across zones. Sadly they forgot to account for
multiple mounts. This PR updates the heuristic to ignore the mount name
when we see something that looks like a statefulset volume, thus
ensuring that multiple mounts end up in the same AZ.
Fix#35695
We recognize a few AWS tags, and skip over masters when finding zones
for dynamic volumes. This will fix#34583.
This is not perfect, in that really the scheduler is the only component
that can correctly choose the zone, but should address the common
problem.
Automatic merge from submit-queue
nodeports usage should be part of LoadBalancer service type
Since a creation of Service of type LoadBalancer will allocate NodePorts as well, so it makes more sense to account for the NodePort usage in the LoadBalancer switch case.
check here: https://github.com/kubernetes/kubernetes/blob/master/pkg/registry/core/service/rest.go#L553 for the logic on whether it should assign a nodeport for the service.
Automatic merge from submit-queue (batch tested with PRs 41401, 41195, 41664, 41521, 41651)
Allow `make test` to pass on OSX
**What this PR does / why we need it**: `make test` doesn't pass on my OSX setup (10.11.6, go1.7, docker 1.13.1) on `master`, `release-1.5`, nor `release-1.4`. Our [docs on unit tests](https://github.com/kubernetes/community/blob/master/contributors/devel/testing.md#unit-tests) say they should always pass on OS X. This PR allows them to pass.
**Release note**:
```release-note
NONE
```
ref: #24717 for the motivation behind dereferencing mount symlinks
/cc @kubernetes/sig-testing-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41401, 41195, 41664, 41521, 41651)
Remove default failure domains from anti-affinity feature
Removing it is necessary to make performance of this feature acceptable at some point.
With default failure domains (or in general when multiple topology keys are possible), we don't have transitivity between node belonging to a topology. And without this, it's pretty much impossible to solve this effectively.
@timothysc
Automatic merge from submit-queue
Generate valid container id in fake docker client.
Fixes https://github.com/kubernetes/kubernetes/issues/41488.
This PR generates valid container id by hashing container name. So that dockershim checkpoint won't report `checkpoint key XXX is not valid` error.
This PR also reverts #41460.
@yujuhong @freehan /cc @kubernetes/sig-node-pr-reviews
Changes the kubelet so it doesn't use the cert/key files directly for
starting the TLS server. Instead the TLS server reads the cert/key from
the new CertificateManager component, which is responsible for
requesting new certificates from the Certificate Signing Request API on
the API Server.
Automatic merge from submit-queue (batch tested with PRs 41649, 41658, 41266, 41371, 41626)
Understand why kubelet cannot cleanup orphaned pod dirs
**What this PR does / why we need it**:
Understand if we are unable to clean up orphaned pod directories due to a failure to read the directory versus paths still existing to improve ability to debug error situations.
Automatic merge from submit-queue (batch tested with PRs 41649, 41658, 41266, 41371, 41626)
Split request node certificate
Split the `RequestNodeCertificate` function so the behavior can also be used by non-node callers.
Automatic merge from submit-queue
fix kubectl create configmap help messages
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
**What this PR does / why we need it**:
1. the first example command actually doesn't specify any keys. i think `with keys for each file` is incorrect.
2. `names on disk` is confusing. in usage the word `NAME` means configmap, so i think it's better saying `file basenames on disk`
3. in this context, `--from-file` can be specified multiple times, but in each occurance only one key file can be specified, besides we should keep consistent with the later case (key is specified). so i changed words in this sentence to singular. and to avoid confusing, i use `key` instead of `name`
Automatic merge from submit-queue
Switch service controller to shared informers
Originally part of #40097
cc @deads2k @smarterclayton @gmarek @wojtek-t @timothysc @sttts @liggitt @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41604, 41273, 41547)
remove validNonResourceVerbs in create role
non-resource-url is only reasonable for clusterroles
Automatic merge from submit-queue (batch tested with PRs 41604, 41273, 41547)
Switch pv controller to shared informer
This is WIP because I still need to do something with bazel? and add 'get storageclasses' to the controller-manager rbac role
@jsafrane PTAL and make sure I did not break anything in the PV controller. Do we need to clone the volumes/claims we get from the shared informer before we use them? I could not find a place where we modify them but you would know for certain.
cc @ncdc because I copied what you did in your other PRs.
Automatic merge from submit-queue
Improve code coverage for pkg/api/util
**What this PR does / why we need it**:
Improve code coverage for pkg/api/util .
Thanks.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 41517, 41494, 41163)
Deployment: filter out old RSes that are deleted or with non-zero replicas before cleanup
Fixes#36379
cc @zmerlynn @yujuhong @kargakis @kubernetes/sig-apps-bugs
Automatic merge from submit-queue
only construct shortcutmapper when we have the discovery client
Resource shortnames come from the discoveryclient, so we should only wrap with that mapper when we have the information we need.
Automatic merge from submit-queue (batch tested with PRs 40505, 34664, 37036, 40726, 41595)
dockertools: call TearDownPod when GC-ing infra pods
The docker runtime doesn't tear down networking when GC-ing pods.
rkt already does so make docker do it too. To ensure this happens,
infra pods are now always GC-ed rather than gating them by
containersToKeep.
This prevents IPAM from leaking when the pod gets killed for
some reason outside kubelet (like docker restart) or when pods
are killed while kubelet isn't running.
Fixes: https://github.com/kubernetes/kubernetes/issues/14940
Related: https://github.com/kubernetes/kubernetes/pull/35572
Automatic merge from submit-queue (batch tested with PRs 40505, 34664, 37036, 40726, 41595)
filter lastAppliedConfig annotation for describe secret
Temporarily addresses: #23564.
This patch filters out the lastAppliedConfig annotation when describing a secret.
```release-note
kubectl describe no longer prints the last-applied-configuration annotation for secrets.
```
Automatic merge from submit-queue (batch tested with PRs 38101, 41431, 39606, 41569, 41509)
Report node not ready on failed PLEG health check
Report node not ready if PLEG health check fails.
Automatic merge from submit-queue (batch tested with PRs 38101, 41431, 39606, 41569, 41509)
optimize killPod() and syncPod() functions
make sure that one of the two arguments must be non-nil: runningPod, status ,just like the function note says
and judge the return value in syncPod() function before setting podKilled
Automatic merge from submit-queue (batch tested with PRs 38101, 41431, 39606, 41569, 41509)
[hairpin] fix argument of nsenter
**Release note**:
```release-note
None
```
We should use:
nsenter --net=netnsPath -- -F some_command
instend of:
nsenter -n netnsPath -- -F some_command
Because "nsenter -n netnsPath" get an error output:
# nsenter -n /proc/67197/ns/net ip addr
nsenter: neither filename nor target pid supplied for ns/net
If we really want use -n, we need to use -n in such format:
# sudo nsenter -n/proc/67197/ns/net ip addr
To prepare for implementing ControllerRef across all controllers,
this pushes the common adopt/orphan logic into ControllerRefManager
so each controller doesn't have to duplicate it.
This also shares the adopt/orphan logic between Pods and ReplicaSets,
so it lives in only one place.
Automatic merge from submit-queue (batch tested with PRs 41505, 41484, 41544, 41514, 41022)
Proxy defer on update events
This PR is a series of discrete movements in refactoring some of kube-proxy's twistier code in prep to be more async. It should be reviewed one commit at a time. Each commit is a smallish movement, which should be easier to examine. I added significant tests along the way, which, unsurprisingly, found some bugs.
Automatic merge from submit-queue (batch tested with PRs 41505, 41484, 41544, 41514, 41022)
pkg/api/install: use apimachinery/announce+registered
Make core group a little bit less special.
This commit converts the HPA controller over to using the new version of
the HorizontalPodAutoscaler object found in autoscaling/v2alpha1. Note
that while the autoscaler will accept requests for object metrics, the
scale client will return an error on attempts to get object metrics
(since that requires the new custom metrics API, which is not yet
implemented).
This also enables the HPA object in v2alpha1 as a retrievable API
version by default.
Dead infra containers may still have network resources allocated to
them and may not be GC-ed for a long time. But allowing SyncPod()
to restart an infra container before the old one is destroyed
prevents network plugins from carrying the old network details
(eg IPAM) over to the new infra container.
The docker runtime doesn't tear down networking when GC-ing pods.
rkt already does so make docker do it too. To ensure this happens,
networking is always torn down for the container even if the
container itself is not deleted.
This prevents IPAM from leaking when the pod gets killed for
some reason outside kubelet (like docker restart) or when pods
are killed while kubelet isn't running.
Fixes: https://github.com/kubernetes/kubernetes/issues/14940
Related: https://github.com/kubernetes/kubernetes/pull/35572
We need to tear down networking when garbage collecting containers too,
and GC is run from a different goroutine in kubelet. We don't want
container network operations running for the same pod concurrently.
The PluginManager almost duplicates the network plugin interface, but
not quite since the Init() function should be called by whatever
actually finds and creates the network plugin instance. Only then
does it get passed off to the PluginManager.
The Manager synchronizes pod-specific network operations like setup,
teardown, and pod network status. It passes through all other
operations so that runtimes don't have to cache the network plugin
directly, but can use the PluginManager as a wrapper.
Automatic merge from submit-queue (batch tested with PRs 41466, 41456, 41550, 41238, 41416)
Delay Deletion of a Pod until volumes are cleaned up
#41436 fixed the bug that caused #41095 and #40239 to have to be reverted. Now that the bug is fixed, this shouldn't cause problems.
@vishh @derekwaynecarr @sjenning @jingxu97 @kubernetes/sig-storage-misc
Automatic merge from submit-queue
Remove alpha provisioning
This is the first part of https://github.com/kubernetes/features/issues/36
@kubernetes/sig-storage-misc
**Release note**:
```release-note
Alpha version of dynamic volume provisioning is removed in this release. Annotation
"volume.alpha.kubernetes.io/storage-class" does not have any special meaning. A default storage class
and DefaultStorageClass admission plugin can be used to preserve similar behavior of Kubernetes cluster,
see https://kubernetes.io/docs/user-guide/persistent-volumes/#class-1 for details.
```
Automatic merge from submit-queue (batch tested with PRs 41531, 40417, 41434)
Always detach volumes in operator executor
**What this PR does / why we need it**:
Instead of marking a volume as detached immediately in Kubelet's
reconciler, delegate the marking asynchronously to the operator
executor. This is necessary to prevent race conditions with other
operations mutating the same volume state.
An example of one such problem:
1. pod is created, volume is added to desired state of the world
2. reconciler process starts
3. reconciler starts MountVolume, which is kicked off asynchronously via
operation_executor.go
4. MountVolume mounts the volume, but hasn't yet marked it as mounted
5. pod is deleted, volume is removed from desired state of the world
6. reconciler reaches detach volume section, detects volume is no longer in desired state of world,
removes it from volumes in use
7. MountVolume tries to mark mount, throws an error because
volume is no longer in actual state of world list. After this, kubelet isn't aware of the mount
so doesn't try to unmount again.
8. controller-manager tries to detach the volume, this fails because it
is still mounted to the OS.
9. EBS gets stuck indefinitely in busy state trying to detach.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#32881, fixes ##37854 (maybe)
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Switch serviceaccounts controller to generated shared informers
Originally part of #40097
cc @deads2k @sttts @liggitt @smarterclayton @gmarek @wojtek-t @timothysc @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 37137, 41506, 41239, 41511, 37953)
Add field to control service account token automounting
Fixes https://github.com/kubernetes/kubernetes/issues/16779
* adds an `automountServiceAccountToken *bool` field to `ServiceAccount` and `PodSpec`
* if set in both the service account and pod, the pod wins
* if unset in both the service account and pod, we automount for backwards compatibility
```release-note
An `automountServiceAccountToken *bool` field was added to ServiceAccount and PodSpec objects. If set to `false` on a pod spec, no service account token is automounted in the pod. If set to `false` on a service account, no service account token is automounted for that service account unless explicitly overridden in the pod spec.
```
Automatic merge from submit-queue (batch tested with PRs 37137, 41506, 41239, 41511, 37953)
e2e test for storage class diskformat verification for vsphere cloud provider
**What this PR does / why we need it**:
This PR adds a new e2e test for vsphere cloud provider.
Test is to verify diskformat specified in storage-class is being honored while volume creation.
Steps:
1. Create StorageClass with diskformat set to valid type (supported options are `eagerzeroedthick`, `zeroedthick` and `thin`)
2. Create PVC which uses the StorageClass created in step 1.
3. Wait for PV to be provisioned.
4. Wait for PVC's status to become Bound
5. Create POD using PVC on specific node.
6. Wait for Disk to be attached to the node.
7. Get node VM's devices and find PV's Volume Disk.
8. Get Backing Info of the Volume Disk and obtain Property of `VirtualDiskFlatVer2BackingInfo` - `EagerlyScrub` and `ThinProvisioned`
9. Based on the value of `EagerlyScrub` and `ThinProvisioned`, verify if diskformat is correct.
10. Delete POD and Wait for Volume Disk to be detached from the Node.
11. Delete PVC, PV and Storage Class
**Which issue this PR fixes** *
fixes #
**Special notes for your reviewer**:
Test is executed against v1.6.0-alpha.1
Test is failing on v1.4.8
**Release Note**
```release-note
NONE
```
@kerneltime @BaluDontu @abrarshivani please review this PR
make sure that one of the two arguments must be non-nil: runningPod, status ,just like the function note says
and judge the return value in syncPod() function before setting podKilled
Automatic merge from submit-queue (batch tested with PRs 41104, 41245, 40722, 41439, 41502)
add sample fuzzing tests
Make fuzzing tests as simple as possible from both the API installer and the scheme, so its easy to add for api groups and so that I can build a scheme and then make sure I got it right.
@kubernetes/sig-api-machinery-pr-reviews @sttts @mikedanese
OSX 10.11.x has `/var` symlinked to `/private/var`, which was tripping
up logic in `mount.GetMountRefs`
This fixes unit tests for pkg/volume/fc and pkg/volume/iscsi
Automatic merge from submit-queue
kubeadm: Migrate to client-go
**What this PR does / why we need it**: Finish the migration for kubeadm to use client-go wherever possible
**Which issue this PR fixes**: fixes #https://github.com/kubernetes/kubeadm/issues/52
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Move private key parsing from serviceaccount/jwt.go to client-go/util/cert
**What this PR does / why we need it**:
Unify private key parsing from serviceaccount/jwt.go into the client-go library.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Partial fix to #40807 - only private key functions.
**Special notes for your reviewer**:
**Release note**:
```release-note
Move private key parsing from serviceaccount/jwt.go to client-go/util/cert
```
Some imports dont exist yet (or so it seems) in client-go (examples
being:
- "k8s.io/kubernetes/pkg/api/validation"
- "k8s.io/kubernetes/pkg/util/initsystem"
- "k8s.io/kubernetes/pkg/util/node"
one change in kubelet to import to client-go
Automatic merge from submit-queue
Allow multipe DNS servers as comma-seperated argument for kubelet --dns
This PR explores how kubectls "--dns" could be extended to specify multiple DNS servers for in-cluster PODs. Testing on the local libvirt-coreos cluster shows that multiple DNS server are injected without issues.
Specifying multiple DNS servers increases resilience against
- Packet drops
- Single server failure
I am debugging services that do 50+ DNS requests for a single incoming interactive request, thus highly increase the chance of a slowdown (+5s) due to a single packet drop. Switching to two DNS servers will reduce the impact of the issues (roughly +1s on glibc, 0s on musl, error-rate goes down to error-rate^2).
Note that there is no need to change any runtime related code as far as I know. In the case of "default" dns the /etc/resolv.conf is parsed and multiple DNS server are send to the backend anyway. This only adds the same capability for the clusterFirst case.
I've heard from @thockin that multiple DNS entries are somehow considered. I've no idea what was considered, though. This is what I would like to see for our production use, though.
```release-note
NONE
```
Automatic merge from submit-queue
Switch resourcequota controller to shared informers
Originally part of #40097
I have had some issues with this change in the past, when I updated `pkg/quota` to use the new informers while `pkg/controller/resourcequota` remained on the old informers. In this PR, both are switched to using the new informers. The issues in the past were lots of flakey test failures in the ResourceQuota e2es, where it would randomly fail to see deletions and handle replenishment. I am hoping that now that everything here is consistently using the new informers, there won't be any more of these flakes, but it's something to keep an eye out for.
I also think `pkg/controller/resourcequota` could be cleaned up. I don't think there's really any need for `replenishment_controller.go` any more since it's no longer running individual controllers per kind to replenish. It instead just uses the shared informer and adds event handlers to it. But maybe we do that in a follow up.
cc @derekwaynecarr @smarterclayton @wojtek-t @deads2k @sttts @liggitt @timothysc @kubernetes/sig-scalability-pr-reviews
This makes it more obvious that they run together and makes the upcoming
rate-limited syncs easier.
Also make test use ints for ports, so it's easier to see when a port is
a literal value vs a name.
This is a weird function, but I didn't want to change any semantics
until the tests are in place. Testing exposed one bug where stale
connections of renamed ports were not marked stale.
There are other things that seem wrong here, more will follow.
Move the feature test to where we are activating the feature, rather
than where we detect locality. This is in service of better tests,
which is in service of less-frequent resyncing, which is going to
require refactoring.
Automatic merge from submit-queue
shortcut expander will take the list of short names from the api ser…
**What this PR does / why we need it**: the shortcut expander will take the list of short names for resources from the API server during the discovery. For backward compatibility a hardcoded list of short names will always be appended while evaluating a short name.
Automatic merge from submit-queue
Fix AWS device allocator to only use valid device names
According to
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html
we can only use /dev/xvd[b-c][a-z] as device names - so we can only
allocate upto 52 ebs volumes on a node.
fixes#41453
cc @justinsb @kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)
apiserver: further cleanup of apiserver storage plumbing
- move kubeapiserver`s `RESTOptionsFactory` back to EtcdOptions by adding a `AddWithStorageFactoryTo`
- factor out storage backend `Config` construction from EtcdOptions
- move all `StorageFactory` related code into server/storage subpackage.
In short: remove my stomach ache about `kubeapiserver.RESTOptionsFactory`.
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)
move kubeadm api group testing to kubeadm package
I think this is sufficient to at least preserve round trip testing.
Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)
kubelet: reduce extraneous logging for pods using host network
For pods using the host network, kubelet/shim should not log
error/warning messages when determining the pod IP address.
Automatic merge from submit-queue (batch tested with PRs 41196, 41252, 41300, 39179, 41449)
record ReduceCPULimits result err info if err returned
record ReduceCPULimits result err info if err returned for debug
Automatic merge from submit-queue (batch tested with PRs 41196, 41252, 41300, 39179, 41449)
controller: cleanup workload controllers a bit
* Switches glog.Errorf to utilruntime.HandleError in DS and RC controllers
* Drops a couple of unused variables in the DS, SS, and Deployment controllers
* Updates some comments
@kubernetes/sig-apps-misc
Automatic merge from submit-queue
Fix for Premature iSCSI logout #39202.
**What this PR does / why we need it**:
Modifies the iSCSI volume plugin code to prevent premature iSCSI logouts and the establishment of multiple iSCSI connections to the same target in certain cases.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39202, fixes#41041, fixes#40941
**Special notes for your reviewer**:
The existing iSCSI connections are now rescanned on every AttachDisk call to discover newly created LUNs.
The disk mount points now contain an additional directory in the path corresponding to the disk iface that is later used for iSCSI logout.
The device prefixes that are used to count the existing references to the portal-target pair now contain the whole path including the mount point until the lun index.
**Release note**:
```release-note
Fixed issues #39202, #41041 and #40941 that caused the iSCSI connections to be prematurely closed when deleting a pod with an iSCSI persistent volume attached and that prevented the use of newly created LUNs on targets with preestablished connections.
```
Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)
fix variables in openstack.go to keep camel casing and remove unused var
Automatic merge from submit-queue
Fix bug in status manager TerminatePod
In TerminatePod, we previously pass pod.Status to updateStatusInternal. This is a bug, since it is the original status that we are given. Not only does it skip updates made to container statuses, but in some cases it reverted the pod's status to an earlier version, since it was being passed a stale status initially.
This was the case in #40239 and #41095. As shown in #40239, the pod's status is set to running after it is set to failed, occasionally causing very long delays in pod deletion since we have to wait for this to be corrected.
This PR fixes the bug, adds some helpful debugging statements, and adds a unit test for TerminatePod (which for some reason didnt exist before?).
@kubernetes/sig-node-bugs @vish @Random-Liu
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
Expose the constants in pkg/controller/bootstrap and add a validate token function
**What this PR does / why we need it**: In order to hook up #36101 against kubeadm, we have to expose the constants and add a function to validate the token
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc @jbeda @mikedanese @pires @dmmcquay
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
Reconcile bootstrap clusterroles on server start
Currently, on server start, bootstrap roles and bindings are only created if there are no existing roles or rolebindings.
Instead, we should look at each bootstrap role and rolebinding, and ensure it exists and has required permissions and subjects at server start. This allows seamless upgrades to new versions that define roles for new controllers, or add permissions to existing roles.
```release-note
Default RBAC ClusterRole and ClusterRoleBinding objects are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, annotate it with `rbac.authorization.kubernetes.io/autoupdate=false`.
```
Automatic merge from submit-queue
Make EnableCRI default to true
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release. If both flags are specified,
the --enable-cri flag overrides the --experimental-cri flag.
Automatic merge from submit-queue
Make kubectl edit work with unstructured objects
Fixes https://github.com/kubernetes/kubernetes/issues/35993
1. First (before any other changes), added several test cases for complex edit scenarios:
- [x] ensure the edit loop bails out if given the same result that already caused errors
- [x] ensure an edited file with a syntax error is reopened preserving the input
- [x] ensure objects with existing "caused-by" annotations get updated with the current command
2. Refactored the edit code to prep for switching to unstructured:
- [x] made editFn operate on a slice of resource.Info objects passed as an arg, regardless of edit mode
- [x] simplified short-circuiting logic when re-editing a file containing an error
- [x] refactored how we build the various visitors (namespace enforcement, annotation application, patching, creating) so we could easily switch to just using a single visitor over a set of resource infos read from the updated input for all of them
3. Switched to using a resource builder to parse the stream of the user's edited output
- [x] improve the error message you get on syntax errors
- [x] preserve the user's input more faithfully (see how the captured testcase requests to the server changed to reflect exactly what the user edited)
- [x] stopped doing client-side conversion (means deprecating `--output-version`)
4. Switched edit to work with generic objects
- [x] use unstructured objects
- [x] fall back to generic json merge patch for unrecognized group/version/kinds
5. Added new test cases
- [x] schemaless objects falls back to generic json merge (covers TPR scenario)
- [x] edit unknown version of known kind (version "v0" of storageclass) falls back to generic json merge
```release-note
`kubectl edit` now edits objects exactly as they were retrieved from the API. This allows using `kubectl edit` with third-party resources and extension API servers. Because client-side conversion is no longer done, the `--output-version` option is deprecated for `kubectl edit`. To edit using a particular API version, fully-qualify the resource, version, and group used to fetch the object (for example, `job.v1.batch/myjob`)
```
Automatic merge from submit-queue
fix comment
**What this PR does / why we need it**:
fix comment
Thanks.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release.
Automatic merge from submit-queue (batch tested with PRs 41382, 41407, 41409, 41296, 39636)
Update to use proxy subresource consistently
Proxy subresources have been in place since 1.2.0 and improve the ability to put policy in place around proxy access.
This PR updates the last few clients to use proxy subresources rather than the root proxy
mark --output-version as deprecated, add example for fully-qualifying version to edit
Add 'kubectl edit' testcase for editing schemaed and schemaless data together
Add 'kubectl edit' testcase for editing unknown version of known group/kind
edit: make editFn operate on arguments regardless of mode
edit: simplify short-circuiting logic when re-editing a file containing an error
edit: factor out visitor building
edit: use resource builder to get results from edited file
Add 'kubectl edit' testcase for saving a repeated error
Add 'kubectl edit' testcase for preserving an edited file with a syntax error
Add 'kubectl edit' testcase for recording command on list of objects
Automatic merge from submit-queue (batch tested with PRs 41299, 41325, 41386, 41329, 41418)
move metav1 conversions to metav1
Conversions for `metav1` types belong in metav1 and should be registered when you register the types.
@mikedanese @luxas I think this is what you just hit in your fresh scheme.
@smarterclayton @lavalamp double check the sanity, but I think this does what people expect.
Automatic merge from submit-queue (batch tested with PRs 41299, 41325, 41386, 41329, 41418)
stop senseless negotiation
Most client commands don't respect a negotiated version at all. If you request a particular version, then of course it should be respected, but if you have none to request, then the current negotiation step doesn't return anything useful so we may as well have nothing so we can at least detect the situation.
@jwforres @kubernetes/sig-cli-pr-reviews
Added a TODO to make the negotiate function useful. I think I'm inclined to remove it entirely unless someone can come up with a useful reason to have it.
Automatic merge from submit-queue (batch tested with PRs 41337, 41375, 41363, 41034, 41350)
use instance's Name to attach gce disk
**What this PR does / why we need it**:
fix#40427
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40427
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
It's not an error when recycle/delete/provision operation cannot be started
because it has failed recently. It will be restarted automatically when
backoff expires.
Automatic merge from submit-queue (batch tested with PRs 41357, 41178, 41280, 41184, 41278)
Switch RBAC subject apiVersion to apiGroup in v1beta1
Referencing a subject from an RBAC role binding, the API group and kind of the subject is needed to fully-qualify the reference.
The version is not, and adds complexity around re-writing the reference when returning the binding from different versions of the API, and when reconciling subjects.
This PR:
* v1beta1: change the subject `apiVersion` field to `apiGroup` (to match roleRef)
* v1alpha1: convert apiVersion to apiGroup for backwards compatibility
* all versions: add defaulting for the three allowed subject kinds
* all versions: add validation to the field so we can count on the data in etcd being good until we decide to relax the apiGroup restriction
```release-note
RBAC `v1beta1` RoleBinding/ClusterRoleBinding subjects changed `apiVersion` to `apiGroup` to fully-qualify a subject. ServiceAccount subjects default to an apiGroup of `""`, User and Group subjects default to an apiGroup of `"rbac.authorization.k8s.io"`.
```
@deads2k @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41115, 41212, 41346, 41340, 41172)
Enable PodTolerateNodeTaints predicate in DaemonSet controller
Ref #28687, this enables the PodTolerateNodeTaints predicate to the daemonset controller
cc @Random-Liu @dchen1107 @davidopp @mikedanese @kubernetes/sig-apps-pr-reviews @kubernetes/sig-node-pr-reviews @kargakis @lukaszo
```release-note
Make DaemonSet controller respect node taints and pod tolerations.
```
Automatic merge from submit-queue
fix service spec for kube api server
For the auto generated kube api-server service, the service spec re-uses the service port itself. The endpoint is created correctly using public port. Fix the service also because there are some plugin controllers that react to service spec itself.
Before fix:
```
sh-4.2# kubectl get endpoints
NAME ENDPOINTS AGE
kubernetes 172.17.0.2:8443,172.17.0.2:8053,172.17.0.2:8053 20h
sh-4.2# kubectl get services kubernetes -o json
...
...
"spec": {
"clusterIP": "172.30.0.1",
"ports": [
{
"name": "https",
"port": 443,
"protocol": "TCP",
"targetPort": 443 ## <--- same as port, even if the endpoint really means 8443
},
{
"name": "dns",
"port": 53,
"protocol": "UDP",
"targetPort": 8053
},
{
"name": "dns-tcp",
...
```
After fix:
```
"spec": {
"clusterIP": "172.30.0.1",
"ports": [
{
"name": "https",
"port": 443,
"protocol": "TCP",
"targetPort": 8443 # <-- fixed, now matches the endpoint object
},
{
"name": "dns",
"port": 53,
"protocol": "UDP",
"targetPort": 8053
},
{
"name": "dns-tcp",
``
In cases where insecure OpenStack endpoint is to be used
(e.g., when testing), gophercloud will fail to connect
to such endpoints. This patch adds support for custom CA
file configuration option, which, when provided, will
make gophercloud validate OpenStack endpoint against
certificate(s) read from file specified in that option.
Automatic merge from submit-queue
Added kubectl create role command
Added `kubectl create role` command.
Fixed part of #39596
**Release note**:
```
Added one new command `kubectl create role` to help user create a single role from command line.
```
Automatic merge from submit-queue (batch tested with PRs 41312, 41289)
resolve udevadm from PATH in cinder_util.go
**What this PR does / why we need it**:
When a cinder volume gets attached to a node, the cinder volume plugin calls `udevadm` with an absolute path `/usr/bin/udevadm`. This path is incorrect for recent versions of debian, ubuntu or the hyperkube image on gcr.io where `udevadm` is located at `/bin/udevadm` or `/sbin/udevadm`. A variant of the hyperkube image is used on CoreOS to run kubelet with rkt fly stage 1.
As a result of the failed `udevadm` exec, the `AttachDisk` function in `cinder_util.go` returns an error.
This PR removes the absolute path from the `udevadm` exec. As a result, `udevadm` is resolved by looking it up in `PATH`.
This is consistent with the gce volume plugin, which executes `udevadm` the same way.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#29832
**Special notes for your reviewer**:
**Release note**:
```release-note
```
To safely mark a volume detached when the volume controller manager is used.
An example of one such problem:
1. pod is created, volume is added to desired state of the world
2. reconciler process starts
3. reconciler starts MountVolume, which is kicked off asynchronously via
operation_executor.go
4. MountVolume mounts the volume, but hasn't yet marked it as mounted
5. pod is deleted, volume is removed from desired state of the world
6. reconciler detects volume is no longer in desired state of world,
removes it from volumes in use
7. MountVolume tries to mark volume in use, throws an error because
volume is no longer in actual state of world list.
8. controller-manager tries to detach the volume, this fails because it
is still mounted to the OS.
9. EBS gets stuck indefinitely in busy state trying to detach.
kubelet now detects that e.g. xfs volume is being mounted as ext3 because of
wrong volume.Spec.
Mount error is left in the error message to diagnose issues with mounting e.g.
'ext3' volume as 'ext4' - they are different filesystems, however kernel should
mount ext3 as ext4 without errors.
* Switches glog.Errorf to utilruntime.HandleError in DS and RC controllers
* Drops a couple of unused variables in the DS, SS, and Deployment controllers
* Updates some comments
Automatic merge from submit-queue (batch tested with PRs 41137, 41268)
Allow the CertificateController to use any Signer implementation.
**What this PR does / why we need it**:
This will allow developers to create `CertificateController`s with arbitrary `Signer`s, instead of forcing the use of `CFSSLSigner`. It matches the behavior of allowing an arbitrary `AutoApprover` to be passed in the constructor.
**Release note**:
```release-note
NONE
```
CC @mikedanese
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
BootstrapSigner and TokenCleaner controllers
This is part of https://github.com/kubernetes/features/issues/130 and is an implementation of https://github.com/kubernetes/community/pull/189.
Work that needs to be done yet in this PR:
* [ ] ~~e2e tests~~ Will come in new PR.
* [x] flag to disable this by default
```release-note
Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the `kube-public` namespace and cleaning out expired tokens.
```
@kubernetes/sig-cluster-lifecycle @dgoodwin @roberthbailey @mikedanese
Automatic merge from submit-queue (batch tested with PRs 41223, 40892, 41220, 41207, 41242)
Fixes#40819 and Fixes#33114
**What this PR does / why we need it**:
Start looking up the virtual machine by it's UUID in vSphere again. Looking up by IP address is problematic and can either not return a VM entirely, or could return the wrong VM.
Retrieves the VM's UUID in one of two methods - either by a `vm-uuid` entry in the cloud config file on the VM, or via sysfs. The sysfs route requires root access, but restores the previous functionality.
Multiple VMs in a vCenter cluster can share an IP address - for example, if you have multiple VM networks, but they're all isolated and use the same address range. Additionally, flannel network address ranges can overlap.
vSphere seems to have a limitation of reporting no more than 16 interfaces from a virtual machine, so it's possible that the IP address list on a VM is completely untrustworthy anyhow - it can either be empty (because the 16 interfaces it found were veth interfaces with no IP address), or it can report the flannel IP.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#40819Fixes#33114
**Special notes for your reviewer**:
**Release note**:
```release-note
Reverts to looking up the current VM in vSphere using the machine's UUID, either obtained via sysfs or via the `vm-uuid` parameter in the cloud configuration file.
```
Automatic merge from submit-queue (batch tested with PRs 41223, 40892, 41220, 41207, 41242)
skip iptables sync if no endpoint changes
Alternative to https://github.com/kubernetes/kubernetes/pull/41173fixes: #26637
No need to checksum. Just compare endpoint maps.
Automatic merge from submit-queue (batch tested with PRs 41248, 41214)
Switch hpa controller to shared informer
**What this PR does / why we need it**: switch the hpa controller to use a shared informer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: Only the last commit is relevant. The others are from #40759, #41114, #41148
**Release note**:
```release-note
```
cc @smarterclayton @deads2k @sttts @liggitt @DirectXMan12 @timothysc @kubernetes/sig-scalability-pr-reviews @jszczepkowski @mwielgus @piosz
Automatic merge from submit-queue (batch tested with PRs 41246, 39998)
Cinder volume attacher: use instanceID instead of NodeID when verifying attachment
**What this PR does / why we need it**: Cinder volume attacher incorrectly uses NodeID instead of openstack instance id, so that reconciliation fails.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39978
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
TaintController
```release-note
This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag.
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
Promote TokenReview to v1
Peer to https://github.com/kubernetes/kubernetes/pull/40709
We have multiple features that depend on this API:
- [webhook authentication](https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication)
- [kubelet delegated authentication](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authentication)
- add-on API server delegated authentication
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating the token
This PR promotes the existing v1beta1 API to v1 with no changes
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authentication.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
make round trip testing generic
RoundTrip testing is something associated with a scheme and everyone who writes an API will want to do it. In the end, we should wire each API group separately in a test scheme and have them all call this general function. Once `kubeadm` is out of the main scheme, we'll be able to remove the one really ugly hack.
@luxas @sttts @kubernetes/sig-apimachinery-pr-reviews @smarterclayton
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
StatefulSet hardening
**What this PR does / why we need it**:
This PR contains the following changes to StatefulSet. Only one change effects the semantics of how the controller operates (This is described in #38418), and this change only brings the controller into conformance with its documented behavior.
1. pcb and pcb controller are removed and their functionality is encapsulated in StatefulPodControlInterface. This class modules the design contoller.PodControlInterface and provides an abstraction to clientset.Interface which is useful for testing purposes.
2. IdentityMappers has been removed to clarify what properties of a Pod are mutated by the controller. All mutations are performed in the UpdateStatefulPod method of the StatefulPodControlInterface.
3. The statefulSetIterator and petQueue classes are removed. These classes sorted Pods by CreationTimestamp. This is brittle and not resilient to clock skew. The current control loop, which implements the same logic, is in stateful_set_control.go. The Pods are now sorted and considered by their ordinal indices, as is outlined in the documentation.
4. StatefulSetController now checks to see if the Pods matching a StatefulSet's Selector also match the Name of the StatefulSet. This will make the controller resilient to overlapping, and will be enhanced by the addition of ControllerRefs.
5. The total lines of production code have been reduced, and the total number of unit tests has been increased. All new code has 100% unit coverage giving the module 83% coverage. Tests for StatefulSetController have been added, but it is not practical to achieve greater coverage in unit testing for this code (the e2e tests for StatefulSet cover these areas).
6. Issue #38418 is fixed in that StaefulSet will ensure that all Pods that are predecessors of another Pod are Running and Ready prior to launching a new Pod. This removes the potential for deadlock when a Pod needs to be rescheduled while its predecessor is hung in Pending or Initializing.
7. All reference to pet have been removed from the code and comments.
**Which issue this PR fixes**
fixes #38418,#36859
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue #38418 which, under circumstance, could cause StatefulSet to deadlock.
Mediates issue #36859. StatefulSet only acts on Pods whose identity matches the StatefulSet, providing a partial mediation for overlapping controllers.
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
HPA v2 (API Changes)
**Release note**:
```release-note
Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics.
```
Implements the API changes for kubernetes/features#117.
This implements #34754, which is the new design for the Horizontal Pod Autoscaler. It includes improved support for custom metrics (and/or arbitrary metrics) as well as expanded support for resource metrics. The new HPA object is introduces in the API group "autoscaling/v1alpha1".
Note that the improved custom metric support currently is limited to per pod metrics from Heapster -- attempting to use the new "object metrics" will simply result in an error. This will change once #34586 is merged and implemented.
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
Implement TTL controller and use the ttl annotation attached to node in secret manager
For every secret attached to a pod as volume, Kubelet is trying to refresh it every sync period. Currently Kubelet has a ttl-cache of secrets of its pods and the ttl is set to 1 minute. That means that in large clusters we are targetting (5k nodes, 30pods/node), given that each pod has a secret associated with ServiceAccount from its namespaces, and with large enough number of namespaces (where on each node (almost) every pod is from a different namespace), that resource in ~30 GETs to refresh all secrets every minute from one node, which gives ~2500QPS for GET secrets to apiserver.
Apiserver cannot keep up with it very easily.
Desired solution would be to watch for secret changes, but because of security we don't want a node watching for all secrets, and it is not possible for now to watch only for secrets attached to pods from my node.
So as a temporary solution, we are introducing an annotation that would be a suggestion for kubelet for the TTL of secrets in the cache and a very simple controller that would be setting this annotation based on the cluster size (the large cluster is, the bigger ttl is).
That workaround mean that only very local changes are needed in Kubelet, we are creating a well separated very simple controller, and once watching "my secrets" will be possible it will be easy to remove it and switch to that. And it will allow us to reach scalability goals.
@dchen1107 @thockin @liggitt
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
Set all node conditions to Unknown when node is unreachable
**What this PR does / why we need it**:
Sets all node conditions to Unknown when node does not report status/unreachable
**Which issue this PR fixes**
fixes https://github.com/kubernetes/kubernetes/issues/36273
Start looking up the virtual machine by it's UUID in vSphere again. Looking up by IP address is problematic and can either not return a VM entirely, or could return the wrong VM.
Retrieves the VM's UUID in one of two methods - either by a `vm-uuid` entry in the cloud config file on the VM, or via sysfs. The sysfs route requires root access, but restores the previous functionality.
Multiple VMs in a vCenter cluster can share an IP address - for example, if you have multiple VM networks, but they're all isolated and use the same address range. Additionally, flannel network address ranges can overlap.
vSphere seems to have a limitation of reporting no more than 16 interfaces from a virtual machine, so it's possible that the IP address list on a VM is completely untrustworthy anyhow - it can either be empty (because the 16 interfaces it found were veth interfaces with no IP address), or it can report the flannel IP.
Automatic merge from submit-queue (batch tested with PRs 41074, 41147, 40854, 41167, 40045)
Add debug logging to eviction manager
**What this PR does / why we need it**:
This PR adds debug logging to eviction manager.
We need it to help users understand when/why eviction manager is/is not making decisions to support information gathering during support.
Automatic merge from submit-queue (batch tested with PRs 41037, 40118, 40959, 41084, 41092)
Switch CSR controller to use shared informer
Switch the CSR controller to use a shared informer. Originally part of #40097 but I'm splitting that up into multiple PRs.
I have added a test to try to ensure we don't mutate the cache. It could use some fleshing out for additional coverage but it gets the initial job done, I think.
cc @mikedanese @deads2k @liggitt @sttts @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41037, 40118, 40959, 41084, 41092)
Fix for detach volume when node is not present/ powered off
Fixes#33061
When a vm is reported as no longer present in cloud provider and is deleted by node controller, there are no attempts to detach respective volumes. For example, if a VM is powered off or paused, and pods are migrated to other nodes. In the case of vSphere, the VM cannot be started again because the VM still holds mount points to volumes that are now mounted to other VMs.
In order to re-join this node again, you will have to manually detach these volumes from the powered off vm before starting it.
The current fix will make sure the mount points are deleted when the VM is powered off. Since all the mount points are deleted, the VM can be powered on again.
This is a workaround proposal only. I still don't see the kubernetes issuing a detach request to the vsphere cloud provider which should be the case. (Details in original issue #33061 )
@luomiao @kerneltime @pdhamdhere @jingxu97 @saad-ali
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
add k8s.io/sample-apiserver to demonstrate how to build an aggregated API server
builds on https://github.com/kubernetes/kubernetes/pull/41093
This creates a sample API server is a separate staging repo to guarantee no cheating with `k8s.io/kubernetes` dependencies. The sample is run during integration tests (simple tests on it so far) to ensure that it continues to run.
@sttts @kubernetes/sig-api-machinery-misc ptal
@pwittrock @pmorie @kris-nova an aggregated API server example that will stay up to date.
Automatic merge from submit-queue (batch tested with PRs 41145, 38771, 41003, 41089, 40365)
Add `kubectl attach` support for multiple types
To address this issue: https://github.com/kubernetes/kubernetes/issues/24857
the new `kubectl attach` will contain three scenarios depend on args:
1. `kubectl attach POD` : if only one argument provided, we assume it's a pod name
2. `kubectl attach TYPE NAME` : if two arguments provided, we assume first one is resource we [supported](4770162fd3/pkg/kubectl/cmd/util/factory_object_mapping.go (L285)), the second resource's name.
3. `kubectl attach TYPE/NAME` : one argument provided and arg[0] must contain `/`, ditto
Is there any other scenarios I haven't consider in ?
for now the first scenario is compatible with changed before, also `make test` pass ✅
will write some unit test to test second and third scenario, if you guys think i'm doing the right way.
@pwittrock @kargakis @fabianofranz @ymqytw @AdoHe
Automatic merge from submit-queue (batch tested with PRs 41145, 38771, 41003, 41089, 40365)
Remove useless param from kubectl create rolebinding
The `force` param is not used in
`kubectl create rolebinding` & `kubectl create clusterrolebinding`
commands, removed it.
Automatic merge from submit-queue
Add OWNERS file for GCE cloud provider
GCE cloud provider does not have OWNERS file and all PRs need to be approved by owner of pkg/cloudprovider, which is currently only @mikedanese. Adding more options would be helpful to speed up reviews.
Feel free to add/remove some names, this first version is just my qualified guess. It's hard to distinguish generic Kubernetes refactoring from real cloud provider work in git log.
```release-note
NONE
```
1. pcb and pcb controller are removed and their functionality is
encapsulated in StatefulPodControlInterface.
2. IdentityMappers has been removed to clarify what properties of a Pod are
mutated by the controller. All mutations are performed in the
UpdateStatefulPod method of the StatefulPodControlInterface.
3. The statefulSetIterator and petQueue classes are removed. These classes
sorted Pods by CreationTimestamp. This is brittle and not resilient to
clock skew. The current control loop, which implements the same logic,
is in stateful_set_control.go. The Pods are now sorted and considered by
their ordinal indices, as is outlined in the documentation.
4. StatefulSetController now checks to see if the Pods matching a
StatefulSet's Selector also match the Name of the StatefulSet. This will
make the controller resilient to overlapping, and will be enhanced by
the addition of ControllerRefs.
Automatic merge from submit-queue (batch tested with PRs 40873, 40948, 39580, 41065, 40815)
[CRI] Enable Hostport Feature for Dockershim
Commits:
1. Refactor common hostport util logics and add more tests
2. Add HostportManager which can ADD/DEL hostports instead of a complete sync.
3. Add Interface for retreiving portMappings information of a pod in Network Host interface.
Implement GetPodPortMappings interface in dockerService.
4. Teach kubenet to use HostportManager
Automatic merge from submit-queue
[Kubelet] Delay deletion of pod from the API server until volumes are deleted
Previous PR that was reverted: #40239.
To summarize the conclusion of the previous PR after reverting:
- The status manager has the most up-to-date status, but the volume manager uses the status from the pod manager, which only is as up-to-date as the API server.
- Because of this, the previous change required an additional round trip between the kubelet and API server.
- When few pods are being added or deleted, this is only a minor issue. However, when under heavy load, the QPS limit to the API server causes this round trip to take ~60 seconds, which is an unacceptable increase in latency. Take a look at the graphs in #40239 to see the effect of QPS changes on timing.
- To remedy this, the volume manager looks at the status from the status manager, which eliminates the round trip.
cc: @vishh @derekwaynecarr @sjenning @jingxu97 @kubernetes/sig-storage-misc
Automatic merge from submit-queue
add deads2k to approvers for controllers
I've done significant maintenance on these for a while and introduced new patterns like shared informers and rate limited work queues.
Automatic merge from submit-queue
avoid repeated length calculation and some other code improvements
**What this PR does / why we need it**:
1. in function `ParsePairs`, calculating `invalidBuf`'s length over and over again brings performance penalty. a `invalidBufNonEmpty` bool value can fix this.
2. pairArg is not a string template and also there is no other arguments for `fmt.Sprintf`, so i remove `fmt.Sprintf`
3. in function `DumpReaderToFile`, we must check nil error first before defer statement, otherwise there maybe a potential nil error on `f.Close()`
4. add nil checks into `GetWideFlag` function
5. some other minor code improvements for better readability.
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue
Removed a space in portforward.go.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
Let ReadLogs return when there is a read error.
Fixes a bug in kuberuntime log.
Today, @yujuhong found that once we cancel `kubectl logs -f` with `Ctrl+C`, kuberuntime will keep complaining:
```
27939 kuberuntime_logs.go:192] Failed with err write tcp 10.240.0.4:10250->10.240.0.2:53913: write: broken pipe when writing log for log file "/var/log/pods/5bb76510-ed71-11e6-ad02-42010af00002/busybox_0.log": &{timestamp:{sec:63622095387 nsec:625309193 loc:0x484c440} stream:stdout log:[84 117 101 32 70 101 98 32 32 55 32 50 48 58 49 54 58 50 55 32 85 84 67 32 50 48 49 55 10]}
```
This is because kuberuntime keeps writing to the connection even though it is already closed. Actually, kuberuntime should return and report error whenever there is a writing error.
Ref the [docker code](3a4ae1f661/pkg/stdcopy/stdcopy.go (L159-L167))
I'm still creating the cluster and verifying this fix. Will post the result here after that.
/cc @yujuhong @kubernetes/sig-node-bugs
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
e2e tests for vSphere cloud provider
**What this PR does / why we need it**:
This PR contains changes for existing e2e volume provisioning test cases for running on vsphere cloud provider.
**Following is the summary of changes made in existing e2e test cases**
**Added test/e2e/persistent_volumes-vsphere.go**
- This test verifies deleting a PVC before the pod does not cause pod deletion to fail on PD detach and deleting the PV before the pod does not cause pod deletion to fail on PD detach.
**test/e2e/volume_provisioning.go**
- This test creates a StorageClass and claim with dynamic provisioning and alpha dynamic provisioning annotations and verifies that required volumes are getting created. Test also verifies that created volume is readable and retaining data.
- Added vsphere as supported cloud provider. Also set pluginName to "kubernetes.io/vsphere-volume" for vsphere cloud provider.
**test/e2e/volumes.go**
- Added test spec for vsphere
- This test creates requested volume, mount it on the pod, write some random content at /opt/0/index.html and verifies file contents are perfect to make sure we don't see the content from previous test runs.
- This test also passes "1234" as fsGroup to mount volume and verifies fsGroup is set correctly.
**added test/e2e/vsphere_utils.go**
- Added function verifyVSphereDiskAttached - Verify the persistent disk attached to the node.
- Added function waitForVSphereDiskToDetach - Wait until vsphere vmdk is deteched from the given node or time out after 5 minutes
- Added getVSpherePersistentVolumeSpec - create vsphere volume spec with given VMDK volume path, Reclaim Policy and labels
- Added getVSpherePersistentVolumeClaimSpec - get vsphere persistent volume spec with given selector labels
- createVSphereVolume - function to create vmdk volume
**Following is the summary of new e2e tests added with this PR**
**test/e2e/vsphere_volume_placement.go**
- contains volume placement tests using node label selector
- Test Back-to-back pod creation/deletion with the same volume source on the same worker node
- Test Back-to-back pod creation/deletion with the same volume source attach/detach to different worker nodes
**test/e2e/pv_reclaimpolicy.go**
- contains tests for PV/PVC - Reclaiming Policy
- Test verifies persistent volume should be deleted when reclaimPolicy on the PV is set to delete and associated claim is deleted
- Test also verified that persistent volume should be retained when reclaimPolicy on the PV is set to retain and associated claim is deleted
**test/e2e/pvc_label_selector.go**
- This is function test for Selector-Label Volume Binding Feature.
- Verify volume with the matching label is bounded with the PVC.
Other changes
Updated pkg/cloudprovider/providers/vsphere/BUILD and test/e2e/BUILD
**Which issue this PR fixes** *
fixes # 41087
**Special notes for your reviewer**:
Updated tests were executed on kubernetes v1.4.8 release on vsphere.
Test steps are provided in comments
@kerneltime @BaluDontu
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
Add unit tests for interactive edit command
Before updating edit to use unstructured objects and use generic JSON patching, we need better test coverage of the existing paths. This adds unit tests for the interactive edit scenarios.
This PR adds:
* Simple framework for recording tests for interactive edit:
* record.go is a tiny test server that records editor and API inputs as test expectations, and editor and API outputs as playback stubs
* record_editor.sh is a shell script that sends the before/after of an interactive `vi` edit to the test server
* record_testcase.sh (see README) starts up the test server, sets up a kubeconfig to proxy to the test server, sets EDITOR to invoke record_editor.sh, then opens a shell that lets you use `kubectl edit` normally
* Adds test cases for the following scenarios:
- [x] no-op edit (open and close without making changes)
- [x] try to edit a missing object
- [x] edit single item successfully
- [x] edit list of items successfully
- [x] edit a single item, submit with an error, re-edit, submit fixed successfully
- [x] edit list of items, submit some with errors and some good, re-edit errors, submit fixed
- [x] edit trying to change immutable things like name/version/kind, ensure preconditions prevent submission
- [x] edit in "create mode" successfully (`kubectl create -f ... --edit`)
- [x] edit in "create mode" introducing errors (`kubectl create -f ... --edit`)
* Fixes a bug with edit printing errors to stdout (caught when testing stdout/stderr against expected output)
Follow-ups:
- [ ] clean up edit code path
- [ ] switch edit to use unstructured objects
- [ ] make edit fall back to jsonmerge for objects without registered go structs (TPR, unknown versions of pods, etc)
- [ ] add tests:
- [ ] edit TPR
- [ ] edit mix of TPR and known objects
- [ ] edit known object with extra field from server
- [ ] edit known object with new version from server
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
kubelet/network-cni-plugin: modify the log's info
**What this PR does / why we need it**:
Checking the startup logs of kubelet, i can always find a error like this:
"E1215 10:19:24.891724 2752 cni.go:163] error updating cni config: No networks found in /etc/cni/net.d"
It will appears, neither i use cni network-plugin or not.
After analysis codes, i thought it should be a warn log, because it will not produce any actions like as exit or abort, and just ignored when not any valid plugins exit.
thank you!
hot fix
add unit test and statefulSet
update example
remove package
change to ResourceNames
remove some code
remove strings
add fake testing func for AttachablePodForObject
minor change
add test.obj nil check
update testfile
gofmt
update
add fallthough
revert back
Automatic merge from submit-queue (batch tested with PRs 41061, 40888, 40664, 41020, 41085)
move --runtime-config to kubeapiserver
`--runtime-config` is only useful if you have a lot of API groups in one server. If you have a single API group in your server (the vast majority of aggregated API servers), then the flag is unneeded and relatively complex. This moves it to closer to point of use.
@sttts
Automatic merge from submit-queue (batch tested with PRs 41103, 41042, 41097, 40946, 40770)
Use Clientset interface in KubeletDeps
**What this PR does / why we need it**:
This replaces the Clientset struct with the equivalent interface for the KubeClient injected via KubeletDeps. This is useful for testing and for accessing the Node and Pod status event stream without an API server.
**Special notes for your reviewer**:
Follow up to #4907
**Release note**:
`NONE`
Automatic merge from submit-queue (batch tested with PRs 41103, 41042, 41097, 40946, 40770)
dockershim: set security option separators based on the docker version
Also add a version cache to avoid hitting the docker daemon frequently.
This is part of #38164
Automatic merge from submit-queue
Add gnufied as reviewer for aws and gce volumes
Adding myself as reviewer for aws and gce volume plugins. I understand the code well enough and have helped with review in those areas already.
cc @childsb @justinsb @saad-ali
Automatic merge from submit-queue
Add OWNERS to the dockertools package
We are in the middle of switching to the CRI implementation. It's critical to minimize
the development of dockertools to avoid any more diversion. We should freeze any
non-essential changes to dockertools once CRI becomes the default. This change
adds an OWNERS file with a small group of people to ensure no unintentional changes
go through unnoticed.
Automatic merge from submit-queue
Update owners file for job and cronjob controller
I've just noticed we have outdated OWNERS files for job and cronjob controllers.
@erictune ptal
@kubernetes/sig-contributor-experience-pr-reviews fyi
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)
refactor approver and signer interfaces to be consisten w.r.t. apiserver interaction
This makes it so that only the controller loop talks to the
API server directly. The signatures for Sign and Approve also
become more consistent, while allowing the Signer to report
conditions (which it wasn't able to do before).
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)
remove the create-external-load-balancer flag in cmd/expose.go
**What this PR does / why we need it**:
In cmd/expose.go there is a todo "remove create-external-load-balancer in code on or after Aug 25, 2016.", and now it's been a long time past. So I remove this flag and modify the test cases.
Please check for this, thanks!
**Release note**:
```
remove the deprecated flag "create-external-load-balancer" and use --type="LoadBalancer" instead.
```
Automatic merge from submit-queue
Extract a number of short description strings for translation
@fabianofranz
@kubernetes/sig-cli-pr-reviews
Follow on for https://github.com/kubernetes/kubernetes/pull/39223
addressed review comments
Addressed review comment for pv_reclaimpolicy.go to verify content of the volume
addressed 2nd round of review comments
addressed 3rd round of review comments from jeffvance
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
apiserver command line options lead to config
Logically command line options lead to config, not the other way around. We're clean enough now we can actually do the inversion.
WIP because I have some test cycles to fix, but this is all the meat.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 40980, 40985)
added short names for resources which are exposed during discovery
**What this PR does / why we need it**:
The changes add short names for resources. The short names will be delivered to kubectl during discovery.
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Validate unique against HostPort/Protocol/HostIP
**What this PR does / why we need it**:
We can bind to specific IPs however validation will fail for different HostIP:HostPort combination. This is a small fix to check combination of HostPort/Protocol/HostIP rather than just HostPort/Protocol.
Sample configuration
...
"ports": [
{
"protocol": "TCP",
"containerPort": 53,
"hostPort": 55,
"hostIP": "127.0.0.1",
"name": "dns-local-tcp"
},
{
"protocol": "TCP",
"containerPort": 53,
"hostPort": 55,
"hostIP": "127.0.0.2",
"name": "dns-local-tcp2"
}
]
Before:
* spec.template.spec.containers[1].ports[2].hostPort: Duplicate value: "55/TCP"
* spec.template.spec.containers[1].ports[3].hostPort: Duplicate value: "55/TCP"
After applying the patch:
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:55 0.0.0.0:* LISTEN 3644/docker-proxy
tcp 0 0 127.0.0.2:55 0.0.0.0:* LISTEN 3629/docker-proxy
Thanks
Ashley
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Set docker opt separator correctly for SELinux options
This is based on @pmorie's commit from #40179
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Promote SubjectAccessReview to v1
We have multiple features that depend on this API:
SubjectAccessReview
- [webhook authorization](https://kubernetes.io/docs/admin/authorization/#webhook-mode)
- [kubelet delegated authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization)
- add-on API server delegated authorization
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating access
- A typo was discovered in the SubjectAccessReviewSpec Groups field name
This PR promotes the existing v1beta1 API to v1, with the only change being the typo fix to the groups field. (fixes https://github.com/kubernetes/kubernetes/issues/32709)
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authorization.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue
federation: Refactoring namespaced resources deletion code from kube ns controller and sharing it with fed ns controller
Ref https://github.com/kubernetes/kubernetes/issues/33612
Refactoring code in kube namespace controller to delete all resources in a namespace when the namespace is deleted. Refactored this code into a separate NamespacedResourcesDeleter class and calling it from federation namespace controller.
This is required for enabling cascading deletion of namespaced resources in federation apiserver.
Before this PR, we were directly deleting the namespaced resources and assuming that they go away immediately. With cascading deletion, we will have to wait for the corresponding controllers to first delete the resources from underlying clusters and then delete the resource from federation control plane. NamespacedResourcesDeleter has this waiting logic.
cc @kubernetes/sig-federation-misc @caesarxuchao @derekwaynecarr @mwielgus
Automatic merge from submit-queue (batch tested with PRs 40385, 40786, 40999, 41026, 40996)
optimize duplicate openstack serverList judgement
if len(serverList) > 1, we will return err in pager.EachPage() function,so here we do not need to judge again
Automatic merge from submit-queue (batch tested with PRs 40385, 40786, 40999, 41026, 40996)
Fixed a tiny bug on using RoleBindingGenerator
Fixed a typo bug while using RoleBindingGenerator, this
bug causes error when binding role to service accounts
through "kubectl create rolebinding" command.
Automatic merge from submit-queue
Replace hand-written informers with generated ones
Replace existing uses of hand-written informers with generated ones.
Follow-up commits will switch the use of one-off informers to shared
informers.
This is a precursor to #40097. That PR will switch one-off informers to shared informers for the majority of the code base (but not quite all of it...).
NOTE: this does create a second set of shared informers in the kube-controller-manager. This will be resolved back down to a single factory once #40097 is reviewed and merged.
There are a couple of places where I expanded the # of caches we wait for in the calls to `WaitForCacheSync` - please pay attention to those. I also added in a commented-out wait in the attach/detach controller. If @kubernetes/sig-storage-pr-reviews is ok with enabling the waiting, I'll do it (I'll just need to tweak an integration test slightly).
@deads2k @sttts @smarterclayton @liggitt @soltysh @timothysc @lavalamp @wojtek-t @gmarek @sjenning @derekwaynecarr @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue
add deads2k to registry package owners
I established the package layout and wrote a lot of the non-boilerplate code in this package.
Automatic merge from submit-queue (batch tested with PRs 40930, 40951)
Fix CRI port forwarding
Websocket support was introduced #33684, which broke the CRI
implementation. This change fixes it.
Automatic merge from submit-queue (batch tested with PRs 40943, 40967)
Switch kubectl version and api-versions to create a discovery client …
…directly.
The clientset will throw an error for aggregated apiservers because the
clientset looks for specific versions of apis that are compiled into
the client. These will be missing from aggregated apiservers.
The discoveryclient is fully dynamic and does not rely on compiled
in apiversions.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40943, 40967)
switch admission serialized config to an apiserver type
Switches the kube admission config from componentconfig to the new `apiserver.k8s.io` group so that all API servers can use the shared configuration. This switch external serialization, but it does not move the code that reads the config. I'd like to do that as a follow-on.
@kubernetes/sig-api-machinery-misc @kubernetes/api-reviewers @smarterclayton
@derekwaynecarr ptal
@sttts
1. Update the configuration file for Photon Controller cloud provider
2. Only master nodes can communicate with Photon Controller endpoint
3. Enable support for authentication-enabled Photon Controller endpoint
4. Update NodeAddresses function for query from local node
The clientset will throw an error for aggregated apiservers because the
clientset looks for specific versions of apis that are compiled into
the client. These will be missing from aggregated apiservers.
The discoveryclient is fully dynamic and does not rely on compiled
in apiversions.
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
Extract util used by jsonmergepatch and SMPatch
followup https://github.com/kubernetes/kubernetes/pull/40666#discussion_r99198931
Extract some util out of the `strategicMergePatch` to make `jsonMergePatch` doesn't depend on `strategicMergePatch`.
```release-note
None
```
cc: @liggitt
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
Rename experimental-cgroups-per-pod flag
**What this PR does / why we need it**:
1. Rename `experimental-cgroups-per-qos` to `cgroups-per-qos`
1. Update hack/local-up-cluster to match `CGROUP_DRIVER` with docker runtime if used.
**Special notes for your reviewer**:
We plan to roll this feature out in the upcoming release. Previous node e2e runs were running with this feature on by default. We will default this feature on for all e2es next week.
**Release note**:
```release-note
Rename --experiemental-cgroups-per-qos to --cgroups-per-qos
```
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
Remove the temporary fix for pre-1.0 mirror pods
The fix was introduced to fix#15960 for pre-1.0 pods. It should be safe to remove
this fix now.
Automatic merge from submit-queue
CRI: Handle cri in-place upgrade
Fixes https://github.com/kubernetes/kubernetes/issues/40051.
## How does this PR restart/remove legacy containers/sandboxes?
With this PR, dockershim will convert and return legacy containers and infra containers as regular containers/sandboxes. Then we can rely on the SyncPod logic to stop the legacy containers/sandboxes, and the garbage collector to remove the legacy containers/sandboxes.
To forcibly trigger restart:
* For infra containers, we manually set `hostNetwork` to opposite value to trigger a restart (See [here](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/kuberuntime_manager.go#L389))
* For application containers, they will be restarted with the infra container.
## How does this PR avoid extra overhead when there is no legacy container/sandbox?
For the lack of some labels, listing legacy containers needs extra `docker ps`. We should not introduce constant performance regression for legacy container cleanup. So we added the `legacyCleanupFlag`:
* In `ListContainers` and `ListPodSandbox`, only do extra `ListLegacyContainers` and `ListLegacyPodSandbox` when `legacyCleanupFlag` is `NotDone`.
* When dockershim starts, it will check whether there are legacy containers/sandboxes.
* If there are none, it will mark `legacyCleanupFlag` as `Done`.
* If there are any, it will leave `legacyCleanupFlag` as `NotDone`, and start a goroutine periodically check whether legacy cleanup is done.
This makes sure that there is overhead only when there are legacy containers/sandboxes not cleaned up yet.
## Caveats
* In-place upgrade will cause kubelet to restart all running containers.
* RestartNever container will not be restarted.
* Garbage collector sometimes keep the legacy containers for a long time if there aren't too many containers on the node. In that case, dockershim will keep performing extra `docker ps` which introduces overhead.
* Manually remove all legacy containers will fix this.
* Should we garbage collect legacy containers/sandboxes in dockershim by ourselves? /cc @yujuhong
* Host port will not be reclaimed for the lack of checkpoint for legacy sandboxes. https://github.com/kubernetes/kubernetes/pull/39903 /cc @freehan
/cc @yujuhong @feiskyer @dchen1107 @kubernetes/sig-node-api-reviews
**Release note**:
```release-note
We should mention the caveats of in-place upgrade in release note.
```
Automatic merge from submit-queue
Plumb subresource through subjectaccessreview
plumb all fields for subjectaccessreview into the resulting `authorizer.AttributesRecord`
```release-note
The SubjectAccessReview API passes subresource and resource name information to the authorizer to answer authorization queries.
```
Automatic merge from submit-queue
Optionally avoid evicting critical pods in kubelet
For #40573
```release-note
When feature gate "ExperimentalCriticalPodAnnotation" is set, Kubelet will avoid evicting pods in "kube-system" namespace that contains a special annotation - `scheduler.alpha.kubernetes.io/critical-pod`
This feature should be used in conjunction with the rescheduler to guarantee availability for critical system pods - https://kubernetes.io/docs/admin/rescheduler/
```
Automatic merge from submit-queue (batch tested with PRs 40696, 39914, 40374)
Forgiveness library changes
**What this PR does / why we need it**:
Splited from #34825, contains library changes that are needed to implement forgiveness:
1. ~~make taints-tolerations matching respect timestamps, so that one toleration can just tolerate a taint for only a period of time.~~ As TaintManager is caching taints and observing taint changes, time-based checking is now outside the library (in TaintManager). see #40355.
2. make tolerations respect wildcard key.
3. add/refresh some related functions to wrap taints-tolerations operation.
**Which issue this PR fixes**:
Related issue: #1574
Related PR: #34825, #39469
~~Please note that the first 2 commits in this PR come from #39469 .~~
**Special notes for your reviewer**:
~~Since currently we have `pkg/api/helpers.go` and `pkg/api/v1/helpers.go`, there are some duplicated periods of code laying in these two files.~~
~~Ideally we should move taints-tolerations related functions into a separate package (pkg/util/taints), and make it a unified set of implementations. But I'd just suggest to do it in a follow-up PR after Forgiveness ones done, in case of feature Forgiveness getting blocked to long.~~
**Release note**:
```release-note
make tolerations respect wildcard key
```
Automatic merge from submit-queue (batch tested with PRs 40795, 40863)
Use caching secret manager in kubelet
I just found that this is in my local branch I'm using for testing, but not in master :)
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
Promote init containers to GA
This is proposed for 1.6
PR moves beta proved concept for init containers to stable. Specification of init containers can be now stated under initContainers field in PodSpec/PodTemplateSpec. Specifying init-containers in annotation is still possible, but will be removed in future version.
```release-note
Init containers have graduated to GA and now appear as a field. The beta annotation value will still be respected and overrides the field value.
```
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
apply falls back to generic JSON patch computation if no go struct is registered for the target GVK
This PR is the master version of #40096 which is target 1.4 branch.
This PR is based on #40260
- [x] ensure subkey deletion works in CreateThreeWayJSONMergePatch
- [x] ensure type stomping works in CreateThreeWayJSONMergePatch
- [x] lots of tests for generic json patch computation
- [x] apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
- [x] prevent generic apply patch computation between different apiVersions and/or kinds
- [x] make pruner generic (apply --prune works with TPR)
```release-note
apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK
```
This makes it so that only the controller loop talks to the
API server directly. The signatures for Sign and Approve also
become more consistent, while allowing the Signer to report
conditions (which it wasn't able to do before).
Automatic merge from submit-queue
Update daemon set controller OWNERS file
Adding myself as reviewer, adding @mikedanese as approver
cc @kargakis @lukasredynk
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
genericapiserver: cut off more dependencies – episode 7
Follow-up of https://github.com/kubernetes/kubernetes/pull/40822
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
kubectl get hpa should print number of replicas
**What this PR does / why we need it**:
Prints number of replicas on deployment managed by HPA.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
None, couldn't find one related.
**Special notes for your reviewer**:
Couldn't find any tests for validating kubectl output for hpa, if there are please point me to them and I will add.
**Release note**:
```release-note
```
Automatic merge from submit-queue
Removed HPA objects from extensions api group
fix#29778
``` release-note
HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead.
```
cc @kubernetes/autoscaling
Automatic merge from submit-queue (batch tested with PRs 40855, 40859)
PV binding: send an event when there are no PVs to bind
This is similar to scheduler that says "no nodes available to schedule pods"
when it can't schedule a pod.
@kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40810, 40695)
Prevent pv controller from forcefully overwrite provisioned volume name
**What this PR does / why we need it**:
This PR adds a fix to prevents the PV controller from forcefully overwriting the provisioned volume's name with the generated PV name. Instead, it overwrites the volume's name only when it is missing. This allows dynamic provisioner implementers to set the name of the volume to a value that they choose.
**Which issue this PR fixes**
This PR does not have an issue affiliated, but it will allow PR #38924 to properly implement dynamically provisioned volume in namespaces other than default.
Automatic merge from submit-queue (batch tested with PRs 40810, 40695)
move genericvalidation to apimachinery
Moves genericvalidation to apimachinery. The first commit shows the snip I made.
@sttts
Automatic merge from submit-queue
Move post processing/backward compatibility of openapi out of generic package
Post processing step was put into generic package and was the same for both k8s api server and federation api server. They have different backward compatibility list of types. This PR move that step out of generic package and put it in each server's file.
Automatic merge from submit-queue (batch tested with PRs 39169, 40719, 38954, 40808, 40689)
add details to patch conflict
Adds conflict details to the patch message so we can debug #39471. Seems like this may have general utility since others may also wonder what they conflicted on.
@sig-api
Automatic merge from submit-queue (batch tested with PRs 39169, 40719, 38954, 40808, 40689)
genericapiserver: cut off more dependencies – episode 5
Follow-up of https://github.com/kubernetes/kubernetes/pull/40658
Left for episode 6:
- pkg/storage/etc
- pkg/storage/etc3
- pkg/storage/storagebackend/factory
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 39169, 40719, 38954, 40808, 40689)
Add StatefulSets checks at Service level
Hi!
Please let me propose some very small e2e testsuite enhancement.
This PR removed a `TODO` about checking governing service at unit test level (which is hard) and adds this to e2e testsuite.
Thanks
Sebastian
Automatic merge from submit-queue
Add websocket support for port forwarding
#32880
**Release note**:
```release-note
Port forwarding can forward over websockets or SPDY.
```
- adjust ports to int32
- CRI flows the websocket ports as query params
- Do not validate ports since the protocol is unknown
SPDY flows the ports as headers and websockets uses query params
- Only flow query params if there is at least one port query param
Automatic merge from submit-queue (batch tested with PRs 40574, 40806, 40308, 40771, 39440)
move api server no kube dep packages
Simple moves, see commit titles. I did not move the source of the generated swagger, I simply change the destination of the script. I'm ok building a little debt to complete the move.
@sttts
Automatic merge from submit-queue
securitycontext: move docker-specific logic into kubelet/dockertools
This change moves the code specific to docker to kubelet/dockertools,
while leaving the common utility functions at its current package
(pkg/securitycontext).
When we deprecate dockertools in the future, the code will be moved to
pkg/kubelet/dockershim instead.
Depending on an exact cluster setup multiple dns may make sense.
Comma-seperated lists of DNS server are quite common as DNS servers
are always plain IPs.
Automatic merge from submit-queue
Use full package path for definition name in OpenAPI spec
We were using short package name (last part of package name) plus type name for OpenAPI spec definition name. That can result in duplicate names and make the spec invalid. To be sure we will always have unique names, we are going to use full package name as definition name. Also "x-kubernetes-tag" custom field is added to definitions to list Group/Version/Kind for the definitions that has it. This will help clients to discover definitions easier.
Lastly, we've added a reference from old definition names to the new ones to keep backward compatibilities. The list of old definitions will not be updated.
**Release note**:
- Rename OpenAPI definition names to type's full package names to prevent duplicates
- Create OpenAPI extension "x-kubernetes-group-version-kind" for definitions to store Group/Version/Kind
- Deprecate old definition names and create a reference to the new definitions. Old definitions will be removed in the next release.
Prepares for allowing encryption at rest of resources as well as any
other lower level optimization we might chose to implement.
Also cleans up a bunch of ugly code.
This fix prevents the PV controller from forcefully overwriting the provisioned volume's name with the generated PV name. Instead, it allows dynamic provisioner implementers to set the name of the volume to a value that they choose.
- split out port forwarding into its own package
Allow multiple port forwarding ports
- Make it easy to determine which port is tied to which channel
- odd channels are for data
- even channels are for errors
- allow comma separated ports to specify multiple ports
Add portfowardtester 1.2 to whitelist
Automatic merge from submit-queue
remove unneeded storage options
Cleanup of some storage serialization options that only kube and federation api server (and maybe not even that one) need.
You may have called it a snip, but this moves the options out of generic entirely.
Automatic merge from submit-queue (batch tested with PRs 40529, 40630)
propagate explicit nulls in apply
Rebase of https://github.com/kubernetes/kubernetes/pull/35496 on top of https://github.com/kubernetes/kubernetes/pull/40260
The client-side propagation of the raw value is no longer needed, since the client is preserving the original object in unstructured form (explicit nulls are preserved).
Kept tests and CreateThreeWayMergePatch changes from https://github.com/kubernetes/kubernetes/pull/35496
```release-note
kubectl apply now supports explicitly clearing values not present in the config by setting them to null
```
- [x] Clean up orphaned objects in test-cmd to preserve pre- and post- conditions
- [x] improve CreateThreeWayMergePatch test to not filter based on string comparison to test name
Automatic merge from submit-queue (batch tested with PRs 40645, 40541, 40769)
Fix sorting printer when sorting by a missing field
**What this PR does / why we need it**:
When calling `kubectl get` with the `--sort-by` flag, the command will error out if the field used for sorting is not present in at least one of the objects returned in the list, *even if it is a field valid in the object's model*.
For example, taking a list of `ReplicationController` where one of them has `status: { replicas: 0 }` (so nothing in `status.availableReplicas`, even that being a valid object in the model and present in every other object of the list) :
```
$ oc get rc --sort-by=status.availableReplicas
error: availableReplicas is not found
```
This PR now traverses the entire list of objects to be sorted and, if at least one has the field provided in `--sort-by`, we sort correctly and consider the field empty in every other object where the field is not present. If none of the objects has the field, we error out (that will catch really invalid fields, and valid ones but not present in any object in the list, which is acceptable). No swagger validation here.
**Release note**:
```release-note
Fixed an issue where 'kubectl get --sort-by=' would return an error when the specified field were not present in at least one of the returned objects, even that being a valid field in the object model.
```
Automatic merge from submit-queue
Add initial french translations for kubectl
Add initial French translations, mostly as an example of how to add a new language.
@fabianofranz @kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
Reduce log noise when aws cloud cannot find public-ip4 metadata
**What this PR does / why we need it**:
The PR removes excessive log spam produced by the aws cloud provider for normal operating conditions.
**Which issue this PR fixes**
fixes https://github.com/kubernetes/kubernetes/issues/23155
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
move client/record
An attempt at moving client/record to client-go. It's proving very stubborn and needs a lot manual intervention and near as I can tell, no one actually gets any benefit from the sink and source complexity it adds.
@sttts @caesarchaoxu
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
move pkg/storage to apiserver
Mechanical move of `pkg/storage` (not sub packages) to `k8s.io/apiserver`.
@sttts
Automatic merge from submit-queue
kuberuntime: remove the kubernetesManagedLabel label
The CRI shim should be responsible for returning only those
containers/sandboxes created through CRI. Remove this label in kubelet.
Automatic merge from submit-queue (batch tested with PRs 40111, 40368, 40342, 40274, 39443)
Eliminate "Unknown service type: ExternalName"
When creating an ExternalName service, rest.go still generate the warning message "Unknown service type: ExternalName". This should be eliminated as this type of service is supported now.
Automatic merge from submit-queue
Add printer#EnsurePrintHeaders method
This patch adds a new `EnsurePrintHeaders` method to the
HumanReadablePrinter `ResourcePrinter`, which allows headers to be
printed in cases where multiple lists of the same resource are printed
consecutively, but are separated by non-printer related information.
Related downstream PR: https://github.com/openshift/origin/pull/12528
**Release note**:
```release-note
release-note-none
```
cc @fabianofranz @AdoHe
Automatic merge from submit-queue (batch tested with PRs 40527, 40738, 39366, 40609, 40748)
move portforward to client-go
Second to last one. I had to split some tests like we did for apimachinery, but they still run in kubernetes.
@sttts
Automatic merge from submit-queue (batch tested with PRs 40527, 40738, 39366, 40609, 40748)
pkg/kubelet/dockertools/docker_manager.go: removing unused stuff
This PR removes unused constants and variables. I checked that neither kubernetes nor openshift code aren't using them.
Automatic merge from submit-queue (batch tested with PRs 38772, 38797, 40732, 40740)
Prevent spurious diff in all generated clients when adding new group/version
Including custom arguments in the generated file comment means adding any new group/version results in a diff of all generated clients in all groups/versions
@ncdc PTAL
Automatic merge from submit-queue (batch tested with PRs 38772, 38797, 40732, 40740)
Synchronous spellcheck for pkg/volume/*
**What this PR does / why we need it**: Increase code readability
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: Minor contribution
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 40405, 38601, 40083, 40730)
Remove extensions/v1beta1.Jobs leftover
It looks like I missed this one file when removing `extensions/v1beta1.Jobs` last time (#38614).
@caesarxuchao ptal, since you were reviewing last time
@kubernetes/sig-api-machinery-misc fyi
Automatic merge from submit-queue
Set NetworkInterfaces and Subnets to nil before updating Azure security groups
**What this PR does / why we need it**: This is a workaround until we have an upstream fix in azure-sdk-for-go/go-autorest. Corresponding issues are https://github.com/kubernetes/kubernetes/issues/40332 and https://github.com/Azure/go-autorest/issues/112
In k8s 1.5.2, an update to azure-sdk-for-go was cherry-picked, which broke creation/updating of LBs on Azure. As we should have it back to a working state ASAP, I'd like to do a workaround for now and later when the upstream fix comes in, remove the workaround again.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40332
**Release note**:
```release-note
Fix failing load balancers in Azure
```
CC @colemickens
Automatic merge from submit-queue (batch tested with PRs 40392, 39242, 40579, 40628, 40713)
optimize podSandboxChanged() function and fix some function notes
Automatic merge from submit-queue (batch tested with PRs 40392, 39242, 40579, 40628, 40713)
[Federation][kubefed] Create a dedicated service account for federation controller manager in the host cluster and give it appropriate permissions.
Ref: Issue #39555
cc @kubernetes/sig-federation-pr-reviews @kubernetes/sig-auth-misc @kubernetes/sig-auth-pr-reviews
```release-note
kubefed init creates a service account for federation controller manager in the federation-system namespace and binds that service account to the federation-system:federation-controller-manager role that has read and list access on secrets in the federation-system namespace.
```
Automatic merge from submit-queue (batch tested with PRs 38443, 40145, 40701, 40682)
fix GetVolumeInUse() function
Since we just want to get volume name info, each volume name just need to added once. desiredStateOfWorld.GetVolumesToMount() will return volume and pod binding info,
if one volume is mounted to several pods, the volume name will be return several times. That is not what we want in this function.
We can add a new function to only get the volume name info or judge whether the volume name is added to the desiredVolumesMap array.
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)
Clean up generated comment for lister/informers
The generated comment including the command line means the addition of a new group/version touches all listers/informers, not just the ones for the new group/version
@ncdc PTAL
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)
Enable protobufs as storage format by default
**Release note**:
```release-note
Change default storage format to protobufs. With this PR apiserver will be writing objects serialized as protobufs to etcd. If the apiserver was upgraded in the existing clusters, until an object will be written, it will still be stored as JSON. Apiserver can deal with some data being in json and some in protobuf format as of 1.4 release.
```
@kubernetes/sig-api-machinery-misc @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue
Update function name for public function in commend
**What this PR does / why we need it**:
Update function name for public function in commend and fix some typos in log information.
Signed-off-by: yuexiao-wang wang.yuexiao@zte.com.cn
Automatic merge from submit-queue (batch tested with PRs 34543, 40606)
sync client-go and move util/workqueue
The vision of client-go is that it provides enough utilities to build a reasonable controller. It has been copying `util/workqueue`. This makes it authoritative.
@liggitt I'm getting really close to making client-go authoritative ptal.
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue
controller: don't run informers in unit tests when unnecessary
Fixes https://github.com/kubernetes/kubernetes/issues/39908
@mfojtik it seems that using informers makes the deployment sync for the initial relist so this races with the enqueue that these tests are testing.
Automatic merge from submit-queue
genericapiserver: cut off more dependencies – episode 3
Compare commit subjects.
This is the first step to get `apiserver_test.go` and `watch_test.go` in `pkg/genericapiserver/endpoints` cutoff from k8s.io/kubernetes dependencies.
After this we have to sync client-go and then "episode 4" can go in.
approved based on #40363
Automatic merge from submit-queue
pkg/genericapiserver: re-add generic feature gates
https://github.com/kubernetes/kubernetes/pull/40543 removed the kubernetes feature gates (which subsume the generic ones) from genericapiserver. This PR readds the generic ones again.
This is not strictly necessary for kube-apiserver. But some other downstream project without its own feature gates needs this.
Automatic merge from submit-queue
Adding vmdk file extension for vmDiskPath in vsphere DeleteVolume
**What this PR does / why we need it**:
This fix is required to handle situation when volume path is not set with vmdk file extension.
DiskAttach call does not require vmdk file extension to be explicitly set in the file path, so in PV yaml file, users have option to set such file path.
Issue happens when such PV is created with persistentVolumeReclaimPolicy set to Delete, and user deletes associated PVC. Volume deletion will fail with the message: file was not found
**Which issue this PR fixes** *
Fixes: #40560
**Special notes for your reviewer**:
Tested fix with custom image: divyen/hyperkube-fix-deletevolume:latest (Image built using base v1.5.3-beta.0)
@kerneltime @BaluDontu please review this fix.
Automatic merge from submit-queue
Update authn/authz owners and reviewers
Add myself as potential reviewer to several authn/authz-related packages.
Add @deads2k as approver to several packages he wrote.
Automatic merge from submit-queue
fix typo in logging statement
**What this PR does / why we need it**:
Typo fix in logs. I am writing an apiserver for service-catalog, and this annoys me when I see it in my logs.
**Special notes for your reviewer**:
Doc/text change only. No functional change. Feel free to combine with some existing PR.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Improve error messages for ValidateObject method.
Improved error messages for #ValidateObject method
in pkg/api/validation/schema.go.
Fixed#39110
Automatic merge from submit-queue
Remove proxy-mode annotation from kube-proxy
**What this PR does / why we need it**:
This removes the net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40582
**Special notes for your reviewer**:
**Release note**:
```release-note
Remove outdated net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy.
```
Automatic merge from submit-queue
Decrease Daemonset burst replicas due to DoS conditions.
**What this PR does / why we need it**:
We are seeing DoS conditions on our Registry if were running a large cluster with too many daemonsets bursting at once.
**Special notes for your reviewer**:
I decided not to plumb through yet another variable to the command line. Ideally such parameters could be tweaked via a configuration file.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40132, 39302, 40194, 40619, 40601)
Update translation bindata
Make on master leaves these changes in bindata.go
Automatic merge from submit-queue (batch tested with PRs 40497, 39769, 40554, 40569, 40597)
NetworkPolicy validation improvements
I noticed while implementing NetworkPolicy that I we weren't validating the "Ports" field at all.
The docs are actually completely silent about what a string-valued Port field is supposed to mean. I had guessed it meant to call `net.LookupPort()` on it (ie, map it from /etc/services) but in every other case where we have an IntOrString-valued Port field in an API struct, it refers to a named ContainerPort. But that would be extremely awkward to implement in this case; a policy specifying a named port could end up mapping to a different numeric port on every container in the namespace... Do other people actually implement string-valued ports that way? Or, for that matter, implement string-valued ports at all? (Related: I hadn't noticed until now that you can leave the Port value unspecified, allowing you to say "allow to all UDP ports, but no TCP ports" or "allow to all TCP ports, but no UDP ports". That seems like something that ended up in the spec just because it was possible, not because it was actually useful...)
@kubernetes/sig-network-misc
This change moves the code specific to docker to kubelet/dockertools,
while leaving the common utility functions at its current package
(pkg/securitycontext).
When we deprecate dockertools in the future, the code will be moved to
pkg/kubelet/dockershim instead.
Automatic merge from submit-queue (batch tested with PRs 39469, 40557)
Forgiveness api changes
**What this PR does / why we need it**:
Splited from #34825 , contains api changes that are needed to implement forgiveness:
1. update toleration api types to support forgiveness, added a new field forgivenessSeconds to indicate the duration of time it tolerates a taint.
2. update taint api types, added a new field to indicate the time the taint is added.
**Which issue this PR fixes** :
Related issue: #1574
Related PR: #34825
**Special notes for your reviewer**:
**Release note**:
```release-note
forgiveness alpha version api definition
```
Automatic merge from submit-queue (batch tested with PRs 39223, 40260, 40082, 40389)
Use metav1.*Options in genericapiserver
Treat DeleteOptions as unversioned in metainternalversion for decoding
of bodies from older clients. Use the metav1 Options structs from
generic api server and the appropriate codec.
Completes the move to using generic server side code for API objects
@sttts
Automatic merge from submit-queue (batch tested with PRs 39223, 40260, 40082, 40389)
make kubectl generic commands work with unstructured objects
part of making apply, edit, label, annotate, and patch work with third party resources
fixes#35149fixes#34413
prereq of:
https://github.com/kubernetes/kubernetes/issues/35496https://github.com/kubernetes/kubernetes/pull/40096
related to:
https://github.com/kubernetes/kubernetes/issues/39906https://github.com/kubernetes/kubernetes/issues/40119
kubectl is currently decoding any resource it doesn't have compiled-in to a ThirdPartyResourceData struct, which means it computes patches using that struct, and would try to send a ThirdPartyResourceData object to the API server when running `apply`
This PR removes the behavior that decodes unknown objects into ThirdPartyResourceData structs internally, and fixes up the following generic commands to work with unstructured objects
- [x] apply
- [x] decode into runtime.Unstructured objects
- [x] successfully use `--record` with unregistered objects
- [x] patch
- [x] decode into runtime.Unstructured objects
- [x] successfully use `--record` with unregistered objects
- [x] describe
- [x] decode into runtime.Unstructured objects
- [x] implement generic describer
- [x] fix other generic kubectl commands to work with unstructured objects
- [x] label
- [x] annotate
follow-ups for pre-existing issues:
- [ ] `explain` doesn't work with unregistered resources
- [ ] remove special casing of federation group in clientset lookups, etc
- [ ] `patch`
- [ ] doesn't honor output formats when persisting to server (`kubectl patch -f svc.json --type merge -p '{}' -o json` doesn't output json)
- [ ] --local throws exception (`kubectl patch -f svc.json --type merge -p '{}' --local`)
- [ ] `apply`
- [ ] fall back to generic JSON patch computation if no go struct is registered for the target GVK (e.g. https://github.com/kubernetes/kubernetes/pull/40096)
- [ ] ensure subkey deletion works in CreateThreeWayJSONMergePatch
- [ ] ensure type stomping works in CreateThreeWayJSONMergePatch
- [ ] lots of tests for generic json patch computation
- [ ] prevent generic apply patch computation among different versions
- [ ] reconcile treatment of nulls with https://github.com/kubernetes/kubernetes/pull/35496
- [ ] `edit`
- [ ] decode into runtime.Unstructured objects
- [ ] fall back to generic JSON patch computation if no go struct is registered for the target GVK
Automatic merge from submit-queue (batch tested with PRs 39223, 40260, 40082, 40389)
Add a script to extract strings for translation and a sample of it's output
@fabianofranz @deads2k @kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40126, 40565, 38777, 40564, 40572)
docker-CRI: Remove legacy code for non-grpc integration
A minor cleanup to remove the code that is no longer in use to simplify the logic.
Automatic merge from submit-queue (batch tested with PRs 40126, 40565, 38777, 40564, 40572)
Add IsContainerNotFound in kube_docker_client
This PR added `IsContainerNotFound` function in kube_docker_client and changed dockershim to use it.
@yujuhong @freehan
Automatic merge from submit-queue (batch tested with PRs 40126, 40565, 38777, 40564, 40572)
Do not swallow error in asw.updateNodeStatusUpdateNeeded
Ref #39056
Bubble the error up to `SetNodeUpdateStatusNeeded` and log it out.
NOTE: This does not modify interface of `SetNodeUpdateStatusNeeded`
Treat DeleteOptions as unversioned in metainternalversion for decoding
of bodies from older clients. Use the metav1 Options structs from
generic api server and the appropriate codec.
Automatic merge from submit-queue (batch tested with PRs 40543, 39999)
remove pkg/feature dependency from genericapiserver.
@sttts I think you just forgot this when you were snipping the rest of the link. If I understood the previous pull correctly, this ought to ensure proper registration of the bits we need, right?
Automatic merge from submit-queue (batch tested with PRs 40046, 40073, 40547, 40534, 40249)
fix a typo in cni log
**What this PR does / why we need it**:
fixes a typo s/unintialized/uninitialized in pkg/kubelet/network/cni/cni.go
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 40046, 40073, 40547, 40534, 40249)
Issue #13501 - printEvent Source to much more cleaner
Create a formatEventSource func and use it.
**What this PR does / why we need it**:
Adds a smarter way to print EventSource, removing "{ }" and the eventual extra ", " when Host is empty.
It was also reported in OpenShift issue https://github.com/openshift/origin/issues/6586
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#13501
**Special notes for your reviewer**:
I have used this new func in all user facing printouts, but there are other places where EventSource are printed to logs, like DumpEventsInNamespace in test/e2e/framework/util.go for instance.
Don't know if I should correct there too...
**Release note**:
```release-note
Improve formatting of EventSource in kubectl get and kubectl describe
```
Automatic merge from submit-queue (batch tested with PRs 40046, 40073, 40547, 40534, 40249)
update check for "all" resources
This patch updates the check for `all` resources to handle cases where a resource's name is "all". Rather than cycling through all given args until `all` is found, this patch makes sure that only a single argument `all` was specified at all.
**Release note**:
```release-note
release-note-none
```
@fabianofranz @pwittrock
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
CRI: Work around container create conflict.
Fixes https://github.com/kubernetes/kubernetes/issues/40443.
This PR added a random suffix in the container name when we:
* Failed to create the container because of "Conflict".
* And failed to remove the container because of "No such container".
@yujuhong @feiskyer
/cc @kubernetes/sig-node-bugs
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
move the discovery and dynamic clients
Moved the dynamic client, discovery client, testing/core, and testing/cache to `client-go`. Dependencies on api groups we don't have generated clients for have dropped out, so federation, kubeadm, and imagepolicy.
@caesarxuchao @sttts
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
CRI: use more gogoprotobuf plugins
Generate marshaler/unmarshaler code should help improve the performance.
This addresses #40098
Automatic merge from submit-queue
Delay deletion of pod from the API server until volumes are deleted
Depends on #37228, and will not pass tests until that PR is merged, and this is rebased.
Keeps all kubelet behavior the same, except the kubelet will not make the "Delete" call (kubeClient.Core().Pods(pod.Namespace).Delete(pod.Name, deleteOptions)) until the volumes associated with that pod are removed. I will perform some performance testing so that we better understand the latency impact of this change.
Is kubelet_pods.go the correct file to include the "when can I delete this pod" logic?
cc: @vishh @sjenning @derekwaynecarr
Automatic merge from submit-queue
Added shortName to meta.v1.APIResource.
The shortNames would be used to allow kubectl to have short names representations of resources from aggregated API servers.
```release-note
Adds `shortNames` to the `APIResource` from discovery which is a list of recommended shortNames for clients like `kubectl`.
```
**What this PR does / why we need it**:
The shortNames would be used to allow kubectl to have short names representations of resources from aggregated API servers.
**Special notes for your reviewer**:
I have followed the steps described here [Changing the internal structures](https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md)
I have run the following scripts:
hack/update-codegen.sh
hack/update-generated-protobuf.sh
hack/update-codecgen.sh
hack/update-openapi-spec.sh
hack/update-swagger-spec.sh
hack/update-federation-openapi-spec.sh
hack/update-api-reference-docs.sh
See also:
#[38755](https://github.com/kubernetes/kubernetes/pull/38755)
#[38835](https://github.com/kubernetes/kubernetes/pull/38835#event-914403188)
**Release note**:
Extended meta.v1.APIResource by a list of short names. The short names would be used to allow kubectl to have short names representations of resources from aggregated API servers.
Automatic merge from submit-queue
fix typo in describe.go
**What this PR does / why we need it**: fixes typo in `kubectl describe nodes` command
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: (none)
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 38739, 40480, 40495, 40172, 40393)
Rename controller pkg/registry/core/controller to pkg/registry/core/r…
…eplicationcontroller
**What this PR does / why we need it**:
Rename controller pkg/registry/core/controller to pkg/registry/core/replicationcontroller
This will clarify the purpose of the controller since intent is replicationcontroller
Please refer to
https://github.com/kubernetes/kubernetes/issues/17648
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
NONE
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 38739, 40480, 40495, 40172, 40393)
Use fnv hash in the CRI implementation
fnv is more stable than adler. This PR changes CRI implementation to
use fnv for generating container hashes, but leaving the old
implementation (dockertools/rkt). This is because hash is what kubelet
uses to identify a container -- changes to the hash will cause kubelet
to restart existing containers. This is ok for CRI implementation (which
requires a disruptive upgrade already), but not for older implementations.
#40140
Automatic merge from submit-queue
Object r repeat close
**What this PR does / why we need it**:
Object r repeat close.
each end of the for loop, the object r has been closed. so omitting defer r.close(), to avoid repeat close.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 39538, 40188, 40357, 38214, 40195)
Use SecretManager when getting secrets for EnvFrom
Merges crossed in the night which missed this needed change.
Automatic merge from submit-queue (batch tested with PRs 39538, 40188, 40357, 38214, 40195)
genericapiserver: cut off more dependencies – episode 2
Compare commit subjects.
approved based on #40363
Automatic merge from submit-queue
Add serviceaccount owners files
Poor @derekwaynecarr is the sole approver/reviewer for the entire admission package.
This adds OWNERS files for service account controller and admission packages
Leave the old implementation (dockertools/rkt) untouched so that
containers will not be restarted during kubelet upgrade. For CRI
implementation (kuberuntime), container restart is required for kubelet
upgrade.
Automatic merge from submit-queue (batch tested with PRs 40428, 40176)
Cleaup Affinity post conversion from annotations to fields
**What this PR does / why we need it**:
Cleans up leftover work from the conversion of affinity from annotations to fields.
fixes#40016
related #25319
**Special notes for your reviewer**:
There are some TODO items left for @luxas or @errordeveloper b/c they were trying to use affinity in a way that is not possible.
**Release note**:
```release-note
NONE
```
/cc @kubernetes/sig-scheduling-misc @rrati
Automatic merge from submit-queue (batch tested with PRs 38905, 40421)
Revert "Add the ability to edit fields within a config map."
Fixes#40396
This reverts commit 31eca372c9.
#38445 broke edit for all resources other than configmaps:
```
$ kubectl edit serviceaccount default
Error from server (NotFound): configmaps "serviceaccount" not found
```
also, `edit` is a generic command, we can't add subcommands that claim the `configmap` space and mess with the things resourcebuilder accepts
Automatic merge from submit-queue
update kubectl approvers
All the other owners files have names (not groups). Updated for individual users for approvers so `/approve` works.
@kubernetes/sig-cli-api-reviews
Automatic merge from submit-queue
move client/cache and client/discovery to client-go
mechanical changes to move those packages. Had to create a `k8s.io/kubernetes/pkg/client/tests` package for tests that were blacklisted from client-go. We can rewrite these tests later and move them, but for now they'll still run at least.
@caesarxuchao @sttts
Automatic merge from submit-queue (batch tested with PRs 40130, 40419, 40416)
kubectl: update deployment describer to correctly show total replicas
@kubernetes/sig-cli-misc
Automatic merge from submit-queue
Get rid of panics.go in apiserver logs
Displaying the place in which the "real" handler is was discarded as too
big change - it would cause all HTTP handlers need to handle request
logging on their own.
fixes#37706
Automatic merge from submit-queue (batch tested with PRs 39064, 40294)
Refactor persistent volume tests
This is an attempt to make the binder tests a bit more concise. The PVCs are being created by a "templating" function. There is also a handful of PVs in the tests but those vary quite more and I don't think similar approach would save us much code.
Reference:
https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29006#-KPJuVeDE0O6TvDP9jia
@jsafrane: I hope this is what you have on mind.
Displaying the place in which the "real" handler is was discarded as too
big change - it would cause all HTTP handlers need to handle request
logging on their own.
Automatic merge from submit-queue (batch tested with PRs 37055, 40142)
azure: azure container registry: fix login server
**What this PR does / why we need it**:
Fixes the Azure Container Registry integration
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
n/a
**Special notes for your reviewer**:
Before this change, if I created an ACR with name `colemicktest`, then the login server would be `colemicktest-microsoft.azurecr.io`. This code was concating to form `colemicktest.azurecr.io` which does not work.
The fix is to reach into RegistryProperties and read out the login server domain name directly.
Also, this should eliminate that existed when ACR gets to sovereign clouds.
**Release note**:
```release-note
azure: fix Azure Container Registry integration
```
Automatic merge from submit-queue
Add version to OpenAPI spec
OpenAPI is missing a version string. It should follow kubernetes version. It is a bugfix that need to be cherrypicked in 1.5 too.
Automatic merge from submit-queue
AWS: trust region if found from AWS metadata
```release-note
AWS: trust region if found from AWS metadata
```
Means we can run in newly announced regions without a code change.
We don't register the ECR provider in new regions, so we will still need
a code change for now.
Fix#35014
Automatic merge from submit-queue
Move remaining *Options to metav1
Primarily delete options, but will remove all internal references to non-metav1 options (except ListOptions).
Still working through it @sttts @deads2k
Automatic merge from submit-queue (batch tested with PRs 39275, 40327, 37264)
dockertools: remove some dead code
Remove `dockerRoot` that's not used anywhere.
Automatic merge from submit-queue
Fix bad time values in kubelet FakeRuntimeService
These values don't affect tests but they can be confusing
for developers looking at the code for reference.
Automatic merge from submit-queue (batch tested with PRs 40299, 40311)
move authoritative client-go util out of pkg
Move `client-go/pkg/util` which are authoritative to `client-go/util` to make it easier to reason about what comes from where.
Automatic merge from submit-queue
Adding rescan scsi controller for cinder
For lsilogic scsi controller, attached cinder volume does not
appear under /dev/ automatically unless do a rescan.
This approach was used in vSphere volume provider before PR #27496
dropped support for lsilogic scsi controller.
Automatic merge from submit-queue
Optional configmaps and secrets
Allow configmaps and secrets for environment variables and volume sources to be optional
Implements approved proposal c9f881b7bb
Release note:
```release-note
Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding `optional: true` to the volume or environment variable source specifications.
```
Automatic merge from submit-queue (batch tested with PRs 40328, 40057)
Allows kubectl create -f to filter by selector.
From #32544.
The underlying `seletor` infrastructrue was implemented by #32599. The test is basically copy-paste of the corresponding test for `kubectl apply -f -l`.
```release-note
kubectl create now accepts the label selector flag for filtering objects to create
```
Move over only the conversions that are needed, create a new scheme that
is private to meta and only accessible via ParameterCodec. Move half of
pkg/util/labels/.readonly to pkg/apis/meta/v1/labels.go
Automatic merge from submit-queue (batch tested with PRs 39341, 40010)
genericapiserver/api/filters: cut off pkg/api dependency and fix timeout status
The timeout status did not have a proper group version and verb. With the `RequestInfo` available we can easily create a proper status object.
As a side-product we cut off the dependency to `pkg/api` from `genericapiserver/api/filters`.
Follow-up with version negotiation is here: https://github.com/kubernetes/kubernetes/pull/40211
Automatic merge from submit-queue
Improve TerminationMessagePath to be more flexible
* Support `terminationMessagePolicy: fallbackToLogsOnError` which allows pod authors to get useful information from containers as per kubernetes/community#154
* Set an upper bound on the size of the termination message path or log output to prevent callers from DoSing the master
* Add tests for running as root, non-root, and for the new terminationMessagePolicy cases.
I set the limit to 4096 bytes, but this may be too high for large pod containers. Probably need to set an absolute bound, i.e. max message size allowed is 20k total, and we truncate if we're above that limit.
Fixes#31839, #23569
```release-note
A new field `terminationMessagePolicy` has been added to containers that allows a user to request `FallbackToLogsOnError`, which will read from the container's logs to populate the termination message if the user does not write to the termination message log file. The termination message file is now properly readable for end users and has a maximum size (4k bytes) to prevent abuse. Each pod may have up to 12k bytes of termination messages before the contents of each will be truncated.
```
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
Enforce the following limits:
12kb for total message length in container status
4kb for the termination message path file
2kb or 80 lines (whichever is shorter) from the log on error
Fallback to log output if the user requests it.
Automatic merge from submit-queue
Make generic registry easier to understand
**What this PR does / why we need it**:
This PR makes the generic registry and some areas of the api REST abstractions easier to understand by adding and clarifying comments. These comments are based on digging that was done to implement a new API server and REST storage for resources in a wholly-new API group.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
make client-go tools/auth and tools/clientcmd authoritative
These moved easily. Pure mechanical. I'll have a couple snip pulls for some of the more complicated packages. `pkg/util/homedir` was a transitive that moved with. It was only used in `kubectl` and you'd need it to be consistent with `clientcmd` loading order.
@sttts
Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
Add the ability to edit fields within a config map.
Addresses part of https://github.com/kubernetes/kubernetes/issues/36222
Example command:
```console
$ kubectl edit configmap foo --config-map-data=bar
```
Will open the data element named `bar` in the `ConfigMap` named `foo` in `$EDITOR`, the edited contents are then updated back to the config map.
@kubernetes/sig-cli
```release-note
Add a special purpose tool for editing individual fields in a ConfigMap with kubectl
```
Automatic merge from submit-queue
separate announce factories
We caught this part way through Daniel's refactor from 1.5. Have to separate announce factories to be able to double register types for client-go.
@kubernetes/sig-apimachinery-misc
Automatic merge from submit-queue (batch tested with PRs 40196, 40143, 40277)
avoid incorrect short-circuit of client-ca setup
PasswordFile options should only affect computation of the SupportsBasicAuth field, not short-circuit setting up the client ca certpool
Automatic merge from submit-queue (batch tested with PRs 40196, 40143, 40277)
Emit warning event when CronJob cannot determine starting time
**What this PR does / why we need it**:
In #39608, we've modified the error message for when a CronJob has too many unmet starting times to enumerate to figure out the next starting time. This makes it more "actionable", and the user can now set a deadline to avoid running into this. However, the error message is still only controller level AFAIK and thus not exposed to the user. From his perspective, there is no way to tell why the CronJob is not scheduling the next instance.
The PR adds a warning event in addition to the error in the controller manager's log.
**Which issue this PR fixes**: This is an addition to PR #39608 regarding #36311.
**Special notes for your reviewer**: cc @soltysh
**Release note**:
```release-note
```
Automatic merge from submit-queue
Remove TODOs to refactor kubelet labels
To address #39650 completely.
Remove label refactoring TODOs, we don't need them since CRI rollout is on the way.
Automatic merge from submit-queue (batch tested with PRs 40066, 36250, 40255, 40225, 40204)
Fix validation errors in cli are cryptic
fix issue https://github.com/kubernetes/kubernetes/issues/39811
now the error message is like:
`error validating "/home/shiywang/template/test.yaml": error validating data: field ports for v1.ServiceSpec: is required; if you choose to ignore these errors, turn validation off with --validate=false`
still need add some unit test, plz don't test now
Automatic merge from submit-queue
Set custom PollingDelay of 5 seconds for Azure VirtualMachinesClient
The default polling delay of 1 minute results in very long delays when
an Azure Disk is attached to a node. It gets worse as go-autorest
doubles the default delay to 2 minutes.
Please see: https://github.com/kubernetes/kubernetes/issues/35180#issuecomment-273085063
Only the PollingDelay for VirtualMachinesClient is modified here to
avoid too much pressure on Azure quotas.
Release Nodes:
```release-note
Reduce time needed to attach Azure disks
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
declare it, but no use it
**What this PR does / why we need it**:
declare it, but no use it
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Invalidate updates to non-convertible selectors
Follow-up to https://github.com/kubernetes/kubernetes/pull/24733
@deads2k @lavalamp @smarterclayton @bgrant0607 @liggitt @mfojtik
First commit contains the necessary validation for replication controllers with non-convertible selectors.
Second commit updates the name for the annotation added during conversion since it is invalid currently:
```
+++ [0719 11:19:54] Running tests without code coverage
--- FAIL: TestValidateUpdate (0.00s)
strategy_test.go:191: unexpected error: [metadata.annotations: Invalid value: "kubernetes.io/non-convertible/spec.selector": must match the regex ([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9] (e.g. 'MyName' or 'my.name' or '123-abc') with an optional DNS subdomain prefix and '/' (e.g. 'example.com/MyName', metadata.annotations: Invalid value: "kubernetes.io/non-convertible/spec.selector": must match the regex ([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9] (e.g. 'MyName' or 'my.name' or '123-abc') with an optional DNS subdomain prefix and '/' (e.g. 'example.com/MyName']
FAIL
FAIL k8s.io/kubernetes/pkg/registry/controller 0.015s
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Add a log at the start of a watch request.
Fixes https://github.com/kubernetes/kubernetes/issues/16899 (> than a year old!)
@lavalamp
Automatic merge from submit-queue (batch tested with PRs 40250, 40134, 40210)
Typo fix: Change logging function to formatting version
**What this PR does / why we need it**:
Slightly broken logging message:
```
I0120 10:56:08.555712 7575 kubelet_node_status.go:135] Deleted old node object %qkubernetes-cit-kubernetes-cr0-0
```
Automatic merge from submit-queue (batch tested with PRs 40250, 40134, 40210)
Remove conversion package
This package is empty, it would be nice if we could remove it.
Automatic merge from submit-queue (batch tested with PRs 40232, 40235, 40237, 40240)
Fixup pet terminology in log and user-facing events
**What this PR does / why we need it**:
Removes some user-facing strings for pet terminology.
Automatic merge from submit-queue (batch tested with PRs 40232, 40235, 40237, 40240)
move listers out of cache to reduce import tree
Moving the listers from `pkg/client/cache` snips links to all the different API groups from `pkg/storage`, but the dreaded `ListOptions` remains.
@sttts
Automatic merge from submit-queue (batch tested with PRs 40187, 40231, 40152)
remove the delete-pod-uid spec
**What this PR does / why we need it**:
remove the specifications about "kubectl delete pod uid" cause it's no longer supported.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40121
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
update short help message in set resource
To keep the help message well-formed.
```
#kubectl set -h
Configure application resources
These commands help you make changes to existing application resources.
Available Commands:
image Update image of a pod template
resources update resource requests/limits on objects with pod templates
selector Set the selector on a resource
Usage:
kubectl set SUBCOMMAND [options]
```
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
Cleanup temp dirs
So funny story my /tmp ran out of space running the unit tests so I am cleaning up all the temp dirs we create.
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
move apimachinery/announced to apimachinery
This also rewires the announce package to allow installation into multiple groups via multiple calls to `install.Install`. At some point, we'll want to unwire the hardcoded, "install to this scheme" call.
@lavalamp something we've wanted for a while and need for genericapiserver
@seh you've asked related questions
@sttts ptal. First commit is interesting, second commit is a straight move.
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubelet: storage: teardown terminated pod volumes
This is a continuation of the work done in https://github.com/kubernetes/kubernetes/pull/36779
There really is no reason to keep volumes for terminated pods attached on the node. This PR extends the removal of volumes on the node from memory-backed (the current policy) to all volumes.
@pmorie raised a concern an impact debugging volume related issues if terminated pod volumes are removed. To address this issue, the PR adds a `--keep-terminated-pod-volumes` flag the kubelet and sets it for `hack/local-up-cluster.sh`.
For consideration in 1.6.
Fixes#35406
@derekwaynecarr @vishh @dashpole
```release-note
kubelet tears down pod volumes on pod termination rather than pod deletion
```
Automatic merge from submit-queue (batch tested with PRs 40011, 40159)
dockertools/nsenterexec: fix err shadow
The shadow of err meant the combination of `exec-handler=nsenter` +
`tty` + a non-zero exit code meant that the exit code would be LOST
FOREVER 👻
This isn't all that important since no one really used the nsenter exec
handler as I understand it
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 36693, 40154, 40170, 39033)
make client-go authoritative for pkg/client/restclient
Moves client/restclient to client-go and a util/certs, util/testing as transitives.
Automatic merge from submit-queue (batch tested with PRs 40168, 40165, 39158, 39966, 40190)
dockershim: add support for the 'nsenter' exec handler
This change simply plumbs the kubelet configuration
(--docker-exec-handler) to DockerService.
This fixes#35747.
Automatic merge from submit-queue (batch tested with PRs 40168, 40165, 39158, 39966, 40190)
CRI: upgrade protobuf to v3
For #38854, this PR upgrades CRI protobuf version to v3, and also updated related packages for confirming to new api.
**Release note**:
```
CRI: upgrade protobuf version to v3.
```
The shadow of err meant the combination of `exec-handler=nsenter` +
`tty` + a non-zero exit code meant that the exit code would be LOST
FOREVER 👻
This isn't all that important since no one really used the nsenter exec
handler as I understand it
Automatic merge from submit-queue (batch tested with PRs 39446, 40023, 36853)
Create environment variables from secrets
Allow environment variables to be populated from entire secrets.
**Release note**:
```release-note
Populate environment variables from a secrets.
```
Automatic merge from submit-queue (batch tested with PRs 39772, 39831, 39481, 40167, 40149)
Check if error is Status in result.Stream()
Fix#38774
This adds the same functionality to `.Stream()` that was added to `.Error()`, `.Into()`, and `.Get()` in ce187f9c6a to try decoding the body as a Status.
This broke `.Stream()` because the decoding of the body as `Status` was removed from `transformResponse` in ce187f9c6a (diff-de85e3effc36b7bbe3fb9eae6c833cf3L933)
Automatic merge from submit-queue
promote certificates api to beta
Mostly posting to see what breaks but also this API is ready to be promoted.
```release-note
Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading.
```
@kubernetes/api-approvers @jcbsmpsn @pipejakob
Automatic merge from submit-queue
Update root approvers files
Replaces #40040
Update top level OWNERS files mostly to set assignees to approvers. Also remove @bgrant0607 from everywhere but the very top level OWNERS file.
Automatic merge from submit-queue
Create SecretManager interface and switch to caching secrets in kubelet
Ref #19188
Obviously we would need to extend the interface to solve #19188 but this is good first step anyway.
Automatic merge from submit-queue
Prune reviewers from pkg/cloudprovider
**What this PR does / why we need it**
Per discussion in https://github.com/kubernetes/kubernetes/pull/36530 the `OWNERS` file for `pkg/cloudprovider` should not contain additional reviewers at this time.
**Special notes for your reviewer**:
Sorry for the extra work in review
**Release note**:
`NONE`
Automatic merge from submit-queue
move pkg/fields to apimachinery
Purely mechanical move of `pkg/fields` to apimachinery.
Discussed with @lavalamp on slack. Moving this an `labels` to apimachinery.
@liggitt any concerns? I think the idea of field selection should become generic and this ends up shared between client and server, so this is a more logical location.
This patch adds a new `EnsurePrintHeaders` method to the
HumanReadablePrinter `ResourcePrinter`, which allows headers to be
printed in cases where multiple lists of the same resource are printed
consecutively, but are separated by non-printer related information.
Automatic merge from submit-queue
make client-go more authoritative
Builds on https://github.com/kubernetes/kubernetes/pull/40103
This moves a few more support package to client-go for origination.
1. restclient/watch - nodep
1. util/flowcontrol - used interface
1. util/integer, util/clock - used in controllers and in support of util/flowcontrol
Automatic merge from submit-queue
controller: decouple cleanup policy from deployment strategies
Deployments get cleaned up only when they are paused, they get scaled up/down,
or when the strategy that drives rollouts completes. This means that stuck
deployments that fall into none of the above categories will not get cleaned
up. Since cleanup is already safe by itself (we only delete old replica sets
that are synced by the replica set controller and have no replicas) we can
execute it for every deployment when there is no intention to rollback.
Fixes https://github.com/kubernetes/kubernetes/issues/40068
Automatic merge from submit-queue
Fix resttest Update action when AllowUnconditionalUpdate is false
Currently our storage Update test assumes that AllowUncoditionalUpdate returns true, and in testUpdateRejectsMismatchedNamespace updates the same object it's passing to create. This results in errors when trying to update that object, due to resourceVersion not being set to a proper value. This patch modifes this so that the update is executed on a stored object, which will have correct values set.
@deads2k ptal
@kubernetes/sig-api-machinery-misc fyi
Automatic merge from submit-queue
Fixed merging of host's and dns' search lines
Fixed forming of pod's Search line in resolv.conf:
- exclude duplicates while merging of host's and dns' search lines to form pod's one
- truncate pod's search line if it exceeds resolver limits: is > 255 chars and containes > 6 searches
- monitoring the resolv.conf file which is used by kubelet (set thru --resolv-conf="") and logging and eventing if search line in it consists of more than 3 entries (or 6 if Cluster Domain is set) or its lenght is > 255 chars
- logging and eventing when a pod's search line is > 255 chars or containes > 6 searches during forming
Fixes#29270
**Release note**:
```release-note
Fixed forming resolver search line for pods: exclude duplicates, obey libc limitations, logging and eventing appropriately.
```
Deployments get cleaned up only when they are paused, they get scaled up/down,
or when the strategy that drives rollouts completes. This means that stuck
deployments that fall into none of the above categories will not get cleaned
up. Since cleanup is already safe by itself (we only delete old replica sets
that are synced by the replica set controller and have no replicas) we can
execute it for every deployment when there is no intention to rollback.
Automatic merge from submit-queue
Add authorization mode to kubeadm
This PR adds an option in `kubeadm` to allow a user to specify an [authorization plugin](https://kubernetes.io/docs/admin/authorization/). It defaults to RBAC.
Automatic merge from submit-queue
Curating Owners: pkg/cloudprovider
cc @runseb @justinsb @kerneltime @mikedanese @svanharmelen @anguslees @brendandburns @abrarshivani @imkin @luomiao @colemickens @ngtuna @dagnello @abithap
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue (batch tested with PRs 36467, 36528, 39568, 40094, 39042)
do not filter kubectl get pods if -o json or yaml
Fixes: https://github.com/kubernetes/kubernetes/issues/38327
This patch sets the value of --show-all to true if the output format
specified is 'json' or 'yaml'.
**Release note**:
```release-note
release-note-none
```
@smarterclayton
Automatic merge from submit-queue
Curating Owners: pkg/kubelet
cc @euank @vishh @dchen1107 @feiskyer @yujuhong @yifan-gu @derekwaynecarr @saad-ali
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Use versioned Taint/Toleration/AvoidPods objects when marshalling
fixes#39847
`kubectl taint`, the kubelet's `--register-with-taints` option, and several Taint/Toleration/AllowPod annotation helpers were marshaling/unmarshaling using internal structs
Automatic merge from submit-queue (batch tested with PRs 39625, 39842)
AWS: Remove duplicate calls to DescribeInstance during volume operations
This change removes all duplicate calls to describeInstance
from aws volume code path.
**What this PR does / why we need it**:
This PR removes the duplicate calls present in disk check code paths in AWS. I can confirm that `getAWSInstance` actually returns all instance information already and hence there is no need of making separate `describeInstance` call.
Related to - https://github.com/kubernetes/kubernetes/issues/39526
cc @justinsb @jsafrane
Automatic merge from submit-queue (batch tested with PRs 39625, 39842)
Add RBAC v1beta1
Add `rbac.authorization.k8s.io/v1beta1`. This scrubs `v1alpha1` to remove cruft, then add `v1beta1`. We'll update other bits of infrastructure to code to `v1beta1` as a separate step.
```release-note
The `attributeRestrictions` field has been removed from the PolicyRule type in the rbac.authorization.k8s.io/v1alpha1 API. The field was not used by the RBAC authorizer.
```
@kubernetes/sig-auth-misc @liggitt @erictune
Automatic merge from submit-queue
Enable lazy initialization of ext3/ext4 filesystems
**What this PR does / why we need it**: It enables lazy inode table and journal initialization in ext3 and ext4.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#30752, fixes#30240
**Release note**:
```release-note
Enable lazy inode table and journal initialization for ext3 and ext4
```
**Special notes for your reviewer**:
This PR removes the extended options to mkfs.ext3/mkfs.ext4, so that the defaults (enabled) for lazy initialization are used.
These extended options come from a script that was historically located at */usr/share/google/safe_format_and_mount* and later ported to GO so this dependency to the script could be removed. After some search, I found the original script here: https://github.com/GoogleCloudPlatform/compute-image-packages/blob/legacy/google-startup-scripts/usr/share/google/safe_format_and_mount
Checking the history of this script, I found the commit [Disable lazy init of inode table and journal.](4d7346f7f5). This one introduces the extended flags with this description:
```
Now that discard with guaranteed zeroing is supported by PD,
initializing them is really fast and prevents perf from being affected
when the filesystem is first mounted.
```
The problem is, that this is not true for all cloud providers and all disk types, e.g. Azure and AWS. I only tested with magnetic disks on Azure and AWS, so maybe it's different for SSDs on these cloud providers. The result is that this performance optimization dramatically increases the time needed to format a disk in such cases.
When mkfs.ext4 is told to not lazily initialize the inode tables and the check for guaranteed zeroing on discard fails, it falls back to a very naive implementation that simply loops and writes zeroed buffers to the disk. Performance on this highly depends on free memory and also uses up all this free memory for write caching, reducing performance of everything else in the system.
As of https://github.com/kubernetes/kubernetes/issues/30752, there is also something inside kubelet that somehow degrades performance of all this. It's however not exactly known what it is but I'd assume it has something to do with cgroups throttling IO or memory.
I checked the kernel code for lazy inode table initialization. The nice thing is, that the kernel also does the guaranteed zeroing on discard check. If it is guaranteed, the kernel uses discard for the lazy initialization, which should finish in a just few seconds. If it is not guaranteed, it falls back to using *bio*s, which does not require the use of the write cache. The result is, that free memory is not required and not touched, thus performance is maxed and the system does not suffer.
As the original reason for disabling lazy init was a performance optimization and the kernel already does this optimization by default (and in a much better way), I'd suggest to completely remove these flags and rely on the kernel to do it in the best way.
Automatic merge from submit-queue (batch tested with PRs 40038, 40041, 39036)
don't show deleted pull secrets - kubectl describe
This patch filters out any image pull secrets that have been deleted
when printing the describer output for a service account.
Related downstream bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1403376
**Release note**:
```release-note
release-note-none
```
@fabianofranz @AdoHe
Automatic merge from submit-queue (batch tested with PRs 40038, 40041, 39036)
move admission to genericapiserver
I disconnected the initialization that was type specific for later assessment.
@sttts
Means we can run in newly announced regions without a code change.
We don't register the ECR provider in new regions, so we will still need
a code change for now.
This also means we do trust config / instance metadata, and don't reject
incorrectly configured zones.
Fix#35014
The default polling delay of 1 minute results in very long delays when
an Azure Disk is attached to a node. It gets worse as go-autorest
doubles the default delay to 2 minutes.
Please see: https://github.com/kubernetes/kubernetes/issues/35180#issuecomment-273085063
Only the PollingDelay for VirtualMachinesClient is modified here to
avoid too much pressure on Azure quotas.
Automatic merge from submit-queue (batch tested with PRs 39826, 40030)
azure disk: restrict length of name
**What this PR does / why we need it**:
Fixes dynamic disk provisioning on Azure by properly truncating the disk name to conform to the Azure API spec.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
n/a
**Special notes for your reviewer**:
n/a
**Release note**:
```release-note
azure disk: restrict name length for Azure specifications
```
cc: @rootfs
Automatic merge from submit-queue
Made tracing of calls and container lifecycle steps in FakeDockerClient optional
Fixes#39717
Slightly refactored the FakeDockerClient code and made tracing optional (but enabled by default).
@yujuhong @Random-Liu
Automatic merge from submit-queue
Curating Owners: pkg/volume
cc @jsafrane @spothanis @agonzalezro @justinsb @johscheuer @simonswine @nelcy @pmorie @quofelix @sdminonne @thockin @saad-ali @rootfs
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Enable streaming proxy redirects by default (beta)
Prerequisite to moving CRI to Beta.
I'd like to enable this early in our 1.6 cycle to get plenty of test coverage before release.
@yujuhong @liggitt
```release-note
Follow redirects for streaming requests (exec/attach/port-forward) in the apiserver by default (alpha -> beta).
```
Automatic merge from submit-queue
Remove packages which are now apimachinery
Removes all the content from the packages that were moved to `apimachinery`. This will force all vendoring projects to figure out what's wrong. I had to leave many empty marker packages behind to have verify-godep succeed on vendoring heapster.
@sttts straight deletes and simple adds
Automatic merge from submit-queue (batch tested with PRs 34763, 38706, 39939, 40020)
Use Statefulset instead in e2e and controller
Quick fix ref: #35534
We should finish the issue to meet v1.6 milestone.
Automatic merge from submit-queue (batch tested with PRs 34763, 38706, 39939, 40020)
prevent anonymous auth and allow all
https://github.com/kubernetes/kubernetes/pull/38696 for master
@kubernetes/sig-auth
```release-note
Anonymous authentication is now automatically disabled if the API server is started with the AlwaysAllow authorizer.
```
Automatic merge from submit-queue
log info on invalid --output-version
**Release note**:
``` release-note
release-note-none
```
Object versions default to the current version (v1) when a specified
`--output-version` is invalid. This patch logs a warning when this is
the case. Cases affected are all commands with the `--output-version`
option, and anywhere runtime objects are converted to versioned objects.
**Example**
```
$ kubectl get pod <mypod> -o json --output-version=invalid
W1013 17:24:16.810278 26719 result.go:238] info: the output version
specified (invalid) is invalid, defaulting to v1
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "mypod",
"namespace": "test",
...
```
Automatic merge from submit-queue
Move PatchType to apimachinery/pkg/types
Fixes https://github.com/kubernetes/kubernetes/issues/39970
`PatchType` is shared by the client and server, they have to agree, and its critical for our API to function.
@smarterclayton @kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 39911, 40002, 39969, 40012, 40009)
kubectl: fix rollback dryrun when version is not specified
@kubernetes/sig-cli-misc
- exclude duplicates while merging of host's and dns' search lines to form pod's one
- truncate pod's search line if it exceeds resolver limits: is > 255 chars and containes > 6 searches
- monitoring the resolv.conf file which is used by kubelet (set thru --resolv-conf="") and logging and eventing if search line in it consists of more than 3 entries
(or 6 if Cluster Domain is set) or its lenght is > 255 chars
- logging and eventing when a pod's search line is > 255 chars or containes > 6 searches during forming
Fixes#29270
Automatic merge from submit-queue
genericapiserver: cut off pkg/serviceaccount dependency
**Blocked** by pkg/api/validation/genericvalidation to be split up and moved into apimachinery.
Automatic merge from submit-queue (batch tested with PRs 39948, 39997)
Fix ScheduledJob -> CronJob rename leftovers
I found a few leftovers from the rename I did some time ago.
@kubernetes/sig-apps-misc ptal
Automatic merge from submit-queue
Move pkg/api/rest into genericapiserver
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Do not list CronJob unmet starting times beyond deadline
**What this PR does / why we need it**:
See #36311. `getRecentUnmetScheduleTimes` gives up after 100 unmet times to avoid wasting too much CPU or memory generating all the times, as it generates them sequentially.
When concurrency is forbidden, this is conceptually un-necessary: we only need the last unmet start time. This suggests that when concurrency is forbidden, we could generate times by going backward in time from now. This is not very practical as CronJob currently relies on a package that only provides `Next` and no `Prev`. Hand-cooking a `Prev` does not seem like a good idea. I could submit a PR to the cron library to add a `Prev` method, and use that when concurrency is forbidden through something like `getLastUnmetScheduleTime`. This would be `O(1)` and there would be no limit involved.
(edit: actually, even for the other concurrency settings, we only start the last unmet start times -- there is a `TODO` in the controller to actually start all of them, but that is not implemented at the moment. This means the solution would apply, at least temporarily, to all concurrency settings).
cc @soltysh what do you think?
In the meantime, I would suggest to do something simple. Currently, the user has no way to configure anything to ensure that his CronJob will not get stuck if one job takes more that 100 unmet times.
`getRecentUnmetScheduleTimes` starts with an initial time corresponding to the last start (or to the creation of the CronJob, if nothing has started yet). However, when `StartingDeadlineSeconds` is set, the controller will not start anything that is older than the deadline, so if the last start is way beyond the deadline, we are generating potentially lots of unmet start times that will not be considered by the scheduler for scheduling anyway.
Consider a job running every minute, where the last instance has taken 120 minutes. This means there are more than 100 unmet times when we start counting from the last start time.
**The PR makes `getRecentUnmetScheduleTimes` only consider times that do not fall beyond the deadline.** Here, the CronJob can be configured with a `StartingDeadlineSeconds` of, say, 10 minutes. After the 120min job has run, `getRecentUnmetScheduleTimes` will only consider the times in the last 10 minutes from now, and will not get stuck.
As a side note on the max. number of unmet times to use as limits in terms of CPU used by the controller: I have run a quick benchmark on my i7 mac. Schedules corresponding to "once a week" tend to be more expensive to generate unmet times for. Just FYI.
```
+--------------+---------------+--------------+
| SCHEDULE | MISSED STARTS | TIMING |
+--------------+---------------+--------------+
| */1 * * * ? | 100 | 383.645µs |
| */30 * * * ? | 100 | 354.765µs |
| 30 1 * * ? | 100 | 1.065124ms |
| 30 1 * * 0 | 100 | 1.80034ms |
| */1 * * * ? | 500 | 1.341365ms |
| */30 * * * ? | 500 | 1.814441ms |
| 30 1 * * ? | 500 | 8.475012ms |
| 30 1 * * 0 | 500 | 10.020613ms |
| */1 * * * ? | 1000 | 2.551697ms |
| */30 * * * ? | 1000 | 4.075813ms |
| 30 1 * * ? | 1000 | 17.674945ms |
| 30 1 * * 0 | 1000 | 19.149324ms |
| */1 * * * ? | 10000 | 25.725531ms |
| */30 * * * ? | 10000 | 87.520022ms |
| 30 1 * * ? | 10000 | 174.29216ms |
| 30 1 * * 0 | 10000 | 196.565748ms |
+--------------+---------------+--------------+
```
using
```.go
package main
import (
"fmt"
"time"
"os"
"strconv"
"github.com/robfig/cron"
"github.com/olekukonko/tablewriter"
)
func timeSchedule(schedule string, iterations int) (time.Duration) {
sched, err := cron.ParseStandard(schedule)
if err != nil {
panic(fmt.Sprintf("Unparseable schedule: %s", err))
}
start := time.Now()
t := time.Now()
for i := 1; i <= iterations; i++ {
t = sched.Next(t)
}
return time.Since(start)
}
func main() {
table := tablewriter.NewWriter(os.Stdout)
table.SetHeader([]string{"Schedule", "Missed starts", "Timing"})
schedules := []string{"*/1 * * * ?", "*/30 * * * ?", "30 1 * * ?", "30 1 * * 0"}
iteration_nums := []int{100, 500, 1000, 10000}
for _, iterations := range iteration_nums {
for _, schedule := range schedules {
table.Append([]string{schedule,
strconv.Itoa(iterations),
timeSchedule(schedule, iterations).String()})
}
}
table.Render()
}
```
**Which issue this PR fixes**: fixes#36311
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Report the Pod name and namespace when kubelet fails to sync the container
This helps debugging problems with SELinux (and other problems related to the Docker failed to run the container) as currently only the UUID of the Pod is reported:
```
Error syncing pod 670f607d-b5a8-11a4-b673-005056b7468b, skipping: failed to "StartContainer" for "deployment" with RunContainerError: "runContainer: Error response from daemon: Relabeling content in /usr is not allowed."
```
Here it would be useful to know what pod in which namespace is trying to mount the "/usr".
Automatic merge from submit-queue
move name generation to generic api server storage helpers
Move name generation to the genericapiserver since only the server needs to know about it.
@kubernetes/sig-api-machinery-misc @sttts
Automatic merge from submit-queue (batch tested with PRs 39806, 39887, 39401)
refactor delete to remove cobra dependency
FYI. As part of CLI Q1 roadmap, we would like to reduce the dependency of Cobra from actual commands implementations. In this PR, I tried to refactor delete command to achieve this. @kubernetes/sig-cli-misc a quick review is quite welcome, and I am just working on more PRs.
Automatic merge from submit-queue (batch tested with PRs 39806, 39887, 39401)
export list of user resources
This patch exports the list of "userResources" found in
`pkg/kubectl/cmd/util/shortcut_resmapper.go` to allow its use in
external packages and clients.
Related downstream PR: https://github.com/openshift/origin/pull/12147
**Release note**:
```release-note
release-note-none
```
cc @deads2k
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
move api/errors to apimachinery
`pkg/api/errors` is a set of helpers around `meta/v1.Status` that help to create and interpret various apiserver errors. Things like `.NewNotFound` and `IsNotFound` pairings. This pull moves it into apimachinery for use by the clients and servers.
@smarterclayton @lavalamp First commit is the move plus minor fitting. Second commit is straight replace and generation.
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
genericapiserver: cut off pkg/apis/extensions and pkg/storage dependencies
Move BuildDefaultStorageFactory to kubeapiserver.
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
Add optional per-request context to restclient
**What this PR does / why we need it**: It adds per-request contexts to restclient's API, and uses them to add timeouts to all proxy calls in the e2e tests. An entire e2e shouldn't hang for hours on a single API call.
**Which issue this PR fixes**: #38305
**Special notes for your reviewer**:
This adds a feature to the low-level rest client request feature that is entirely optional. It doesn't affect any requests that don't use it. The api of the generated clients does not change, and they currently don't take advantage of this.
I intend to patch this in to 1.5 as a mostly test only change since it's not going to affect any controller, generated client, or user of the generated client.
cc @kubernetes/sig-api-machinery
cc @saad-ali
Automatic merge from submit-queue (batch tested with PRs 39947, 39936, 39902, 39859, 39915)
Make api serialization test easier to follow
**What this PR does / why we need it**:
This PR makes the serialization test easier to understand and is informed by the recent experience of writing a new serialization test in SIG service catalog.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 39417, 39679)
Fix 2 `sucessfully` typos
**What this PR does / why we need it**: Only fixes two typos in comments/logging
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Don't blame DNS spec on Kubernetes requirement for lower-case DNS labels.
**What this PR does / why we need it**: #39635 was rejected because it wasn't clear to the author (me) that lower-case DNS labels are in fact a [Kubernetes requirement](https://github.com/kubernetes/kubernetes/pull/39635#issuecomment-271404975) rather than from the [DNS RFC 1035](https://www.ietf.org/rfc/rfc1035.txt) or/and [DNS RFC 1123](https://www.ietf.org/rfc/rfc1123.txt).
**Special notes for your reviewer**: @thockin this is a first pass to make the error messages clearer about the fact that DNS specs are not to _blame_.
Automatic merge from submit-queue
Updated unit tests
@janetkuo updated the flaky unit test to have the same structure with regard to uncasting as the rest of the tests. ptal
Automatic merge from submit-queue
Curating Owners: pkg/conversion
cc @lavalamp @smarterclayton @wojtek-t @derekwaynecarr
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Only set empty list for list types
If List() impls return non-list objects (like Status objects), we shouldn't try to set them to an empty list
follow up to #39834
Automatic merge from submit-queue
Curating Owners: pkg/master
cc @lavalamp @nikhiljindal @mikedanese @derekwaynecarr
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Curating Owners: pkg/storage
cc @lavalamp @timothysc @liggitt @xiang90 @wojtek-t
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Curating Owners: pkg/auth/
cc @liggitt @erictune
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the future in
the **reviewers** section. You probably do NOT need to modify the **approvers**
section. Names are sorted by relevance, using some secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Curating Owners: pkg/credentialprovider
cc @liggitt @erictune
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)
Admission control support for versioned configuration files
**What this PR does / why we need it**:
Today, the `--admission-control-config-file=` argument takes an opaque file that is shared across all admission controllers to provide configuration. This file is not well-versioned and it's shared across multiple plug-ins. Some plugins take file based configuration (`ImagePolicyWebhook`) and others abuse flags to provide configuration because we lacked a good example (`InitialResources`). This PR defines a versioned configuration format that we can use moving forward to provide configuration input to admission controllers that is well-versioned, and does not require the addition of new flags.
The sample configuration file would look as follows:
```
apiVersion: componentconfig/v1alpha1
kind: AdmissionConfiguration
plugins:
- name: "ImagePolicyWebhook"
path: "image-policy-webhook.json"
```
The general behavior is each plugin that requires additional configuration is enumerated by name. An alternate file location is provided for its specific configuration, or the configuration can be embedded as a raw extension via the configuration section.
**Special notes for your reviewer**:
A follow-on PR will be needed to make `ImagePolicyWebhook` to use versioned configuration. This PR maintains backwards compatibility by ignoring configuration it cannot understand and therefore treating the file as opaque. I plan to make use of this PR to complete https://github.com/kubernetes/kubernetes/pull/36765 which attempts to allow more configuration parameters to the `ResourceQuota` admission plugin.
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)
Update deployment equality helper
@mfojtik @janetkuo this is split out of https://github.com/kubernetes/kubernetes/pull/38714 to reduce the size of that PR, ptal
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)
fix bug not using volumetype config in create volume
fixes#39843
@humblec
we are building the volumetype config but I don't see where we are using it in the CreateVolume for dyn provisioning, this is why volumetype parameter from the Storage Class was being overlooked because we are hard coding constants like replicaCount which is always 3.
unless I'm missing something?
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)
Made cache.Controller to be interface.
**What this PR does / why we need it**:
#37504
Automatic merge from submit-queue
run staging client-go update
Chasing to see what real problems we have in staging-client-go.
@sttts you get similar results?
Automatic merge from submit-queue
Curating Owners: pkg/quota
cc @vishh @derekwaynecarr
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
replace global registry in apimachinery with global registry in k8s.io/kubernetes
We'd like to remove all globals, but our immediate problem is that a shared registry between k8s.io/kubernetes and k8s.io/client-go doesn't work. Since client-go makes a copy, we can actually keep a global registry with other globals in pkg/api for now.
@kubernetes/sig-api-machinery-misc @lavalamp @smarterclayton @sttts
This patch exports the list of "userResources" found in
`pkg/kubectl/cmd/util/shortcut_resmapper.go` to allow its use in
external packages and clients.
Automatic merge from submit-queue
break from the for loop
**What this PR does / why we need it**:
exit loop, because the following actions will not affect the result
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Fix cadvisor_unsupported.go build tags
Make it so cadvisor_unsupported.go is used for linux without cgo or
non-linux/windows OSes.
Automatic merge from submit-queue (batch tested with PRs 39834, 38665)
Use parallel list for deleting items from a primitive list with merge strategy
Implemented parallel list for deleting items from a primitive list with merge strategy. Ref: [design doc](https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#list-of-primitives)
fixes#35163 and #32398
When using parallel list, we don't need to worry about version skew.
When an old APIServer gets a new patch like:
```yaml
metadata:
$deleteFromPrimitiveList/finalizers:
- b
finalizers:
- c
```
It won't fail and work as before, because the parallel list will be dropped during json decoding.
Remaining issue: There is no check when creating a set (primitive list with merge strategy). Duplicates may get in.
It happens in two cases:
1) Creation using POST
2) Creating a list that doesn't exist before using PATCH
Fixing the first case is the beyond the scope of this PR.
The second case can be fixed in this PR if we need that.
cc: @pwittrock @kubernetes/kubectl @kubernetes/sig-api-machinery
```release-note
Fix issue around merging lists of primitives when using PATCH or kubectl apply.
```
Automatic merge from submit-queue
Ensure empty lists don't return nil items fields
Fixes#39822
```release-note
Fixes API compatibility issue with empty lists incorrectly returning a null `items` field instead of an empty array.
```
Automatic merge from submit-queue
kubelet: remove the pleg health check from healthz
This prevents kubelet from being killed when docker hangs.
Also, kubelet will report node not ready if PLEG hangs (`docker ps` + `docker inspect`).
Automatic merge from submit-queue
documentation string for DefaultExternalAddress
**What this PR does / why we need it**:
docs for a public function
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
I encountered this function that did not have a doc string. The function was easy to read, so I wrote a doc string for the function.
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)
[scheduling] Moved pod affinity and anti-affinity from annotations to api fields #25319
Converted pod affinity and anti-affinity from annotations to api fields
Related: #25319
Related: #34508
**Release note**:
```Pod affinity and anti-affinity has moved from annotations to api fields in the pod spec. Pod affinity or anti-affinity that is defined in the annotations will be ignored.```
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)
default a CSR's allowed usage to key encipherment and digital signing
Some pretty safe and sane defaults.
@liggitt
Automatic merge from submit-queue (batch tested with PRs 39661, 39740, 39801, 39468, 39743)
enable psp by default
Enable the extensions/psp resource by default
**Release note**:
```release-note
PodSecurityPolicy resource is now enabled by default in the extensions API group.
```
Automatic merge from submit-queue (batch tested with PRs 39661, 39740, 39801, 39468, 39743)
fix nodeStatusUpdateRetry count exceeding condition judgement
When tryUpdateNodeStatus() return err,err!=nil, but nc.kubeClient.Core().Nodes().Get() return no err, err==nil,
And we run nodeStatusUpdateRetry times, when for loop ends, err == nil, we can not print error info and run continue, so maybe the condition judgement is not right
Maybe caused #38671
Automatic merge from submit-queue (batch tested with PRs 39661, 39740, 39801, 39468, 39743)
add --controllers to controller manager
Adds a `--controllers` flag to the `kube-controller-manager` to indicate which controllers are enabled and disabled. From the help:
```
--controllers stringSlice A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'.
All controllers: certificatesigningrequests, cronjob, daemonset, deployment, disruption, endpoint, garbagecollector, horizontalpodautoscaling, job, namespace, podgc, replicaset, replicationcontroller, resourcequota, serviceaccount, statefuleset
```
Automatic merge from submit-queue
Refactor registry etcd to storage
Fixes#17546
Simple shuffle on naming so any sane new person entering the code base can understand what the actual etcd dependencies are.
When tryUpdateNodeStatus() return err,err!=nil, but nc.kubeClient.Core().Nodes().Get() return no err, err==nil,
And we run nodeStatusUpdateRetry times, when for loop ends, err == nil, we can not print error info and run continue, so the condition judgement is wrong.
Automatic merge from submit-queue
proxy/iptables: don't sync proxy rules if services map didn't change
Build the service map in a separate testable function. Return that map instead of changing proxier.serviceMap directly. Use reflect.DeepEqual() to skip syncing proxy rules if nothing actually changed.
@thockin @kubernetes/rh-networking @kubernetes/sig-network-misc @timothysc @wojtek-t @jeremyeder @caseydavenport
Automatic merge from submit-queue (batch tested with PRs 39768, 39463)
modify Umask() and delete ErrorTimeout function
ErrorTimeout() is never used,remove it. And modify Umask() args
Automatic merge from submit-queue (batch tested with PRs 39768, 39463)
Check if path exists before performing unmount
This is part 3 of an effort to check if path exists before performing an unmount operation.
[Part 1](https://github.com/kubernetes/kubernetes/pull/38547) and [part 2](https://github.com/kubernetes/kubernetes/pull/39311) involved auditing the different volume plugins and refactoring their `TearDownAt()s` to use the common util function/or create one if absent.
The ideal way to do this change would involve refactoring of the `TearDownAt()s` of these plugins and make a common util function that checks path. (The plugins involved in this PR use someway of unmounting a bind mount and unmounting a global path, there is also refactoring needed to consolidate disk_manager of fc, rbd and iscsi). A non-goal part of this effort can also involve refactoring all the `SetupAt()s`
In the interest of time and considering other higher priority issues that I am caught up with, I am unable to give the time the refactoring needs. Hence I've made the minimum change that would give the desired output.
I am tracking the work pending in this issue: https://github.com/kubernetes/kubernetes/issues/39251
```release-note
NONE
```
Automatic merge from submit-queue
Fix bug when printing replicas in Deployment Describer
Deployment describer is printing replicas using pointer instead of integer. This bug is introduced by #36673 which updates the describer to use versioned API. cc @caesarxuchao
Fixes#38697
Automatic merge from submit-queue
[CRI] Don't include user data in CRI streaming redirect URLs
Fixes: https://github.com/kubernetes/kubernetes/issues/36187
Avoid userdata in the redirect URLs by caching the {Exec,Attach,PortForward}Requests with a unique token. When the redirect URL is created, the token is substituted for the request params. When the streaming server receives the token request, the token is used to fetch the actual request parameters out of the cache.
For additional security, the token is generated using the secure random function, is single use (i.e. the first request with the token consumes it), and has a short expiration time.
/cc @kubernetes/sig-node
Automatic merge from submit-queue (batch tested with PRs 39483, 39088, 38787)
daemonset: differentiate between cases in nodeShouldRun
specifically we need to differentiate between wanting to run,
should run and should continue running. This is required to
support all taint effects and will improve reporting and end
user debuggability.
fixes https://github.com/kubernetes/kubernetes/issues/28839 among other things
Automatic merge from submit-queue
delete unused variable in etcd_help_test.go
**What this PR does / why we need it**:
delete unused variable validEtcdVersion ,for keep the code clean
**Special notes for your reviewer**: none
**Release note**: none
secifically we need to differentiate between wanting to run,
should run and should continue running. This is required to
support all taint effects and will improve reporting and end
user debuggability.
Instead of copying the map, like OnServicesUpdate() used to do and which
was copied into buildServiceMap() to preserve semantics while creating
testcases, start with a new empty map and do deletion checking later.
The API docs say:
// ServiceTypeExternalName means a service consists of only a reference to
// an external name that kubedns or equivalent will return as a CNAME
// record, with no exposing or proxying of any pods involved.
which implies that ExternalName services should be ignored for proxy
purposes.
Automatic merge from submit-queue
Fix kubelet cross build
**What this PR does / why we need it**: Cross builds are not passing for MacOS and Windows. We are expecting Windows binaries for `kubelet` and `kube-proxy` to be released by the first time with 1.5.2 to be released later today.
**Which issue this PR fixes**:
fixes#39005fixes#39714
**Special notes for your reviewer**: /cc @feiskyer @smarterclayton @vishh this should be P0 in order to be merged before 1.5.2 and obviously fix the cross build.
Object versions default to the current version (v1) when a specified
`--output-version` is invalid. This patch logs a warning when this is
the case. Cases affected are all commands with the `--output-version`
option, and anywhere runtime objects are converted to versioned objects.
**Example**
```
$ kubectl get pod <mypod> -o json --output-version=invalid
W1013 17:24:16.810278 26719 result.go:238] info: the output version
specified (invalid) is invalid, defaulting to v1
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "mypod",
"namespace": "test",
...
```
Automatic merge from submit-queue
Fix kubectl get -f <file> -o <nondefault printer> so it prints all items in the file
**What this PR does / why we need it**: Fix kubectl get -f <file> -o <nondefault printer> so it prints all the objects in the file, instead of just the first one. Also add a test for this feature.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#38907
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @AdoHe @deads2k @liggitt @fabianofranz @kubernetes/kubectl @kubernetes/sig-cli-misc
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
Add checking the nil value
**What this PR does / why we need it**:
Add checking the nil value.
Thanks.
**Special notes for your reviewer**:
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
Create k8s.io/apimachinery repo
Don't panic.
The diff is quite large, but its all generated change. The first few commits are where are all the action is. I built a script to find the fanout from
```
k8s.io/kubernetes/pkg/apimachinery/registered
k8s.io/kubernetes/pkg/runtime/serializer
k8s.io/kubernetes/pkg/runtime/serializer/yaml
k8s.io/kubernetes/pkg/runtime/serializer/streaming
k8s.io/kubernetes/pkg/runtime/serializer/recognizer/testing
```
It copied
```
k8s.io/kubernetes/pkg/api/meta
k8s.io/kubernetes/pkg/apimachinery
k8s.io/kubernetes/pkg/apimachinery/registered
k8s.io/kubernetes/pkg/apis/meta/v1
k8s.io/kubernetes/pkg/apis/meta/v1/unstructured
k8s.io/kubernetes/pkg/conversion
k8s.io/kubernetes/pkg/conversion/queryparams
k8s.io/kubernetes/pkg/genericapiserver/openapi/common - this needs to renamed post-merge. It's just types
k8s.io/kubernetes/pkg/labels
k8s.io/kubernetes/pkg/runtime
k8s.io/kubernetes/pkg/runtime/schema
k8s.io/kubernetes/pkg/runtime/serializer
k8s.io/kubernetes/pkg/runtime/serializer/json
k8s.io/kubernetes/pkg/runtime/serializer/protobuf
k8s.io/kubernetes/pkg/runtime/serializer/recognizer
k8s.io/kubernetes/pkg/runtime/serializer/recognizer/testing
k8s.io/kubernetes/pkg/runtime/serializer/streaming
k8s.io/kubernetes/pkg/runtime/serializer/versioning
k8s.io/kubernetes/pkg/runtime/serializer/yaml
k8s.io/kubernetes/pkg/selection
k8s.io/kubernetes/pkg/types
k8s.io/kubernetes/pkg/util/diff
k8s.io/kubernetes/pkg/util/errors
k8s.io/kubernetes/pkg/util/framer
k8s.io/kubernetes/pkg/util/json
k8s.io/kubernetes/pkg/util/net
k8s.io/kubernetes/pkg/util/runtime
k8s.io/kubernetes/pkg/util/sets
k8s.io/kubernetes/pkg/util/validation
k8s.io/kubernetes/pkg/util/validation/field
k8s.io/kubernetes/pkg/util/wait
k8s.io/kubernetes/pkg/util/yaml
k8s.io/kubernetes/pkg/watch
k8s.io/kubernetes/third_party/forked/golang/reflect
```
The script does the import rewriting and gofmt. Then you do a build, codegen, bazel update, and it produces all the updates.
If we agree this is the correct approach. I'll create a verify script to make sure that no one messes with any files in the "dead" packages above.
@kubernetes/sig-api-machinery-misc @smarterclayton @sttts @lavalamp @caesarxuchao
`staging/prime-apimachinery.sh && hack/update-codegen.sh && nice make WHAT="federation/cmd/federation-apiserver/ cmd/kube-apiserver" && hack/update-openapi-spec.sh && hack/update-federation-openapi-spec.sh && hack/update-codecgen.sh && hack/update-codegen.sh && hack/update-generated-protobuf.sh && hack/update-bazel.sh`
Fix kubectl get -f <file> -o <nondefault printer> so it prints all the
objects in the file, instead of just the first one. Also add a test for
this feature.
Automatic merge from submit-queue (batch tested with PRs 39684, 39577, 38989, 39534, 39702)
kubelet: request client auth certificates from certificate API.
This fixes kubeadm and --experiment-kubelet-bootstrap.
cc @liggitt
Automatic merge from submit-queue (batch tested with PRs 39684, 39577, 38989, 39534, 39702)
Set PodStatus QOSClass field
This PR continues the work for https://github.com/kubernetes/kubernetes/pull/37968
It converts all local usage of the `qos` package class types to the new API level types (first commit) and sets the pod status QOSClass field in the at pod creation time on the API server in `PrepareForCreate` and in the kubelet in the pod status update path (second commit). This way the pod QOS class is set even if the pod isn't scheduled yet.
Fixes#33255
@ConnorDoyle @derekwaynecarr @vishh
