Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)
apiserver: further cleanup of apiserver storage plumbing
- move kubeapiserver`s `RESTOptionsFactory` back to EtcdOptions by adding a `AddWithStorageFactoryTo`
- factor out storage backend `Config` construction from EtcdOptions
- move all `StorageFactory` related code into server/storage subpackage.
In short: remove my stomach ache about `kubeapiserver.RESTOptionsFactory`.
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)
PV E2E: dynamically provisioned volume should not create in unmanaged zone
**What this PR does / why we need it**:
Adds e2e test to that attempts to provision a volume in an unmanaged zone and fails on success. This is to catch regressions of #31948.
cc @jeffvance
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)
Isolate recycler behavior in PV E2E
**What this PR does / why we need it**:
Sets the default `reclaimPolicy` for PV E2E to `Retain` and isolates `Recycle` tests to their own context. The purpose of this is to future proof the PV test suite against the possible deprecation of the `Recycle` behavior. This is done by consolidating recycling test code into a single Context block that can be removed en masse without affecting the test suite.
Secondly, adds a liveliness check for the NFS server pod prior to each test to avoid maxing out timeouts if the NFS server becomes unavailable.
cc @saad-ali @jeffvance
Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)
Refactored kubemark code into provider-specific and provider-independent parts [Part-3]
Fixes#38967
Applying final part of the changes in PR #39033 (which refactored kubemark code completely). The changes included in this PR are:
- Removed `test/kubemark/common.sh` and moved relevant parts of its code to the right places in start-kubemark/stop-kubemark scripts.
- Added DOCKER_REGISTRY, PROJECT, KUBEMARK_IMAGE_MAKE_TARGET variables to `/test/kubemark/cloud-provider-config.sh` to make the kubemark image push location variable wrt provider.
- Removed get-real-pod-for-hollow-node.sh as it doesn't seem to do anything useful.
@kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)
Secure kube-scheduler
This PR:
* Adds a bootstrap `system:kube-scheduler` clusterrole
* Adds a bootstrap clusterrolebinding to the `system:kube-scheduler` user
* Sets up a kubeconfig for kube-scheduler on GCE (following the controller-manager pattern)
* Switches kube-scheduler to running with kubeconfig against secured port (salt changes, beware)
* Removes superuser permissions from kube-scheduler in local-up-cluster.sh
* Adds detailed RBAC deny logging
```release-note
On kube-up.sh clusters on GCE, kube-scheduler now contacts the API on the secured port.
```
Automatic merge from submit-queue
delete cadvisor pod after test
tracing looks at events for pod deletion and volume teardown. SInce the cadvisor pod has more than 1 volume, this can make results harder to analyze.
This PR moves the deletion of the cadvisor pod to after the logPodCreateThroughput call, since that marks the "end" of the test.
cc: @dchen1107 @Random-Liu
Automatic merge from submit-queue (batch tested with PRs 41216, 41362, 41275, 41277, 41412)
Fix statefulset e2e test
...by removing the liveness/readiness probes from the cockroachdb
manifests, as explained in
https://github.com/kubernetes/test-infra/issues/1740#issuecomment-279555187
@kow3ns @spxtr
Added nfs-server verification between tests; prototyped recycler test
recycler test in working state, needs clean up/comments
comments added; gets mount path from pod instead of hard code
typos
removed NFS server checker, will be handled in separate PR
catch err; inaccurate comment
gofmt; typo
Automatic merge from submit-queue (batch tested with PRs 41382, 41407, 41409, 41296, 39636)
Update to use proxy subresource consistently
Proxy subresources have been in place since 1.2.0 and improve the ability to put policy in place around proxy access.
This PR updates the last few clients to use proxy subresources rather than the root proxy
Automatic merge from submit-queue (batch tested with PRs 41382, 41407, 41409, 41296, 39636)
implement configmap upgrade test
**What this PR does / why we need it**: Add an automated ConfigMap upgrade test to the e2e upgrade test suite. The test creates a ConfigMap and verifies it can be consumed by pods before/after upgrade. See PR #39953 and #40747 for context.
**Special notes for your reviewer**:
@krousey please review for consistency with new upgrade test interface. I copied heavily from secrets test for this. I will implement similar tests for DaemonSets and Jobs next. Note that I have not run this test locally.
**Release note**:
`NONE`
Automatic merge from submit-queue (batch tested with PRs 41299, 41325, 41386, 41329, 41418)
Fix resource leak in federation e2e tests and another issue
**What this PR does / why we need it**:
The cleanup after federation service e2e tests is not effective as this function cleanupServiceShardsAndProviderResources is getting called with empty string for namespace ("nsName") because the nsName variable is getting redefined.
Another issue is we are prematurely exiting the Poll in waitForServiceOrFail and the error check is incorrect.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixing the 2 issues mentioned above.
**Special notes for your reviewer**:
**Release note**:
`NONE`
cc @madhusudancs @kubernetes/sig-federation-bugs
...by removing the liveness/readiness probes from the cockroachdb
manifests, as explained in
github.com/kubernetes/test-infra/issues/1740#issuecomment-279555187
Automatic merge from submit-queue (batch tested with PRs 41342, 41257)
Move two flaky e2e tests to the flaky suite.
cc @kargakis @davidopp
We should have moved these a long time ago.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41357, 41178, 41280, 41184, 41278)
[Federation] Add an end-to-end test verifying that deleting a federated namespace deletes child replicasets.
Verifies #38225.
Also, remove a few custom package aliases.
Automatic merge from submit-queue (batch tested with PRs 41357, 41178, 41280, 41184, 41278)
Switch RBAC subject apiVersion to apiGroup in v1beta1
Referencing a subject from an RBAC role binding, the API group and kind of the subject is needed to fully-qualify the reference.
The version is not, and adds complexity around re-writing the reference when returning the binding from different versions of the API, and when reconciling subjects.
This PR:
* v1beta1: change the subject `apiVersion` field to `apiGroup` (to match roleRef)
* v1alpha1: convert apiVersion to apiGroup for backwards compatibility
* all versions: add defaulting for the three allowed subject kinds
* all versions: add validation to the field so we can count on the data in etcd being good until we decide to relax the apiGroup restriction
```release-note
RBAC `v1beta1` RoleBinding/ClusterRoleBinding subjects changed `apiVersion` to `apiGroup` to fully-qualify a subject. ServiceAccount subjects default to an apiGroup of `""`, User and Group subjects default to an apiGroup of `"rbac.authorization.k8s.io"`.
```
@deads2k @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41357, 41178, 41280, 41184, 41278)
fix flaky host cleanup test
**What this PR does / why we need it**:
Fixes 2 flakes in the "HostCleanup tests in e2e/_kubelet.go_
Also does some very minor refactoring.
**Which issue this PR fixes**
This is an improved fix for issue [31272](https://github.com/kubernetes/kubernetes/issues/31272)
**Special notes for your reviewer**:
```release-note
NONE
```
Automatic merge from submit-queue
Fixes Hazelcast example e2e test
**What this PR does / why we need it**:
This PR fixes the Hazelcast example e2e test
**Special notes for your reviewer**:
It is related to this PR https://github.com/kubernetes/kubernetes/pull/39580
Automatic merge from submit-queue (batch tested with PRs 41274, 41241)
[Federation] Make federation namespace e2e tests parallelizable.
Because deleteAllTestNamespaces deleted all the e2e namespaces it interefered with other federation namespace tests running in parallel. This change should mitigate the problem and make the tests runnable in parallel.
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
Add e2e test for external provisioners
this is a retry of https://github.com/kubernetes/kubernetes/pull/39545
This time around:
* take advantage of the system:persistent-volume-provisioner bootstrap cluster role to grant the external provisioner pod serviceaccount permissions
* add storageclass suffix so that the first and third test don't conflict when one creates the storageclass with the same name before the other
also tested more thoroughly myself on gce :)
@jsafrane if you would like to re-review
Automatic merge from submit-queue (batch tested with PRs 41248, 41214)
Switch hpa controller to shared informer
**What this PR does / why we need it**: switch the hpa controller to use a shared informer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: Only the last commit is relevant. The others are from #40759, #41114, #41148
**Release note**:
```release-note
```
cc @smarterclayton @deads2k @sttts @liggitt @DirectXMan12 @timothysc @kubernetes/sig-scalability-pr-reviews @jszczepkowski @mwielgus @piosz
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
add e2e tests for replicasets with weight, min and max replicas
e2e test with weight, min and max replicas set
#31904#32014
@quinton-hoole @nikhiljindal @deepak-vij @kshafiee @mwielgus
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
TaintController
```release-note
This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag.
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
[Federation][e2e] Fix few flakes in federation e2e tests
**What this PR does / why we need it**:
Fixes few flakes in #37105
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # partly fixes few test cases in the above mentioned issue.
**Special notes for your reviewer**:
While cleaning up in AfterEach Block some objects are returned while listing, but by the time the object is delete is issued the object is disappearing resulting in this flake occasionally.
To fix this, we need to check if the err is NotFound while deleting, its ok and need not fail the test.
**Release note**: `NONE`
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
Promote TokenReview to v1
Peer to https://github.com/kubernetes/kubernetes/pull/40709
We have multiple features that depend on this API:
- [webhook authentication](https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication)
- [kubelet delegated authentication](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authentication)
- add-on API server delegated authentication
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating the token
This PR promotes the existing v1beta1 API to v1 with no changes
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authentication.k8s.io API group was promoted to v1
```
Because deleteAllTestNamespaces deleted all the e2e namespaces
it interefered with other federation namespace tests running in
parallel. This change should mitigate the problem and make the
tests runnable in parallel.
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
StatefulSet hardening
**What this PR does / why we need it**:
This PR contains the following changes to StatefulSet. Only one change effects the semantics of how the controller operates (This is described in #38418), and this change only brings the controller into conformance with its documented behavior.
1. pcb and pcb controller are removed and their functionality is encapsulated in StatefulPodControlInterface. This class modules the design contoller.PodControlInterface and provides an abstraction to clientset.Interface which is useful for testing purposes.
2. IdentityMappers has been removed to clarify what properties of a Pod are mutated by the controller. All mutations are performed in the UpdateStatefulPod method of the StatefulPodControlInterface.
3. The statefulSetIterator and petQueue classes are removed. These classes sorted Pods by CreationTimestamp. This is brittle and not resilient to clock skew. The current control loop, which implements the same logic, is in stateful_set_control.go. The Pods are now sorted and considered by their ordinal indices, as is outlined in the documentation.
4. StatefulSetController now checks to see if the Pods matching a StatefulSet's Selector also match the Name of the StatefulSet. This will make the controller resilient to overlapping, and will be enhanced by the addition of ControllerRefs.
5. The total lines of production code have been reduced, and the total number of unit tests has been increased. All new code has 100% unit coverage giving the module 83% coverage. Tests for StatefulSetController have been added, but it is not practical to achieve greater coverage in unit testing for this code (the e2e tests for StatefulSet cover these areas).
6. Issue #38418 is fixed in that StaefulSet will ensure that all Pods that are predecessors of another Pod are Running and Ready prior to launching a new Pod. This removes the potential for deadlock when a Pod needs to be rescheduled while its predecessor is hung in Pending or Initializing.
7. All reference to pet have been removed from the code and comments.
**Which issue this PR fixes**
fixes #38418,#36859
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue #38418 which, under circumstance, could cause StatefulSet to deadlock.
Mediates issue #36859. StatefulSet only acts on Pods whose identity matches the StatefulSet, providing a partial mediation for overlapping controllers.
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
Implement TTL controller and use the ttl annotation attached to node in secret manager
For every secret attached to a pod as volume, Kubelet is trying to refresh it every sync period. Currently Kubelet has a ttl-cache of secrets of its pods and the ttl is set to 1 minute. That means that in large clusters we are targetting (5k nodes, 30pods/node), given that each pod has a secret associated with ServiceAccount from its namespaces, and with large enough number of namespaces (where on each node (almost) every pod is from a different namespace), that resource in ~30 GETs to refresh all secrets every minute from one node, which gives ~2500QPS for GET secrets to apiserver.
Apiserver cannot keep up with it very easily.
Desired solution would be to watch for secret changes, but because of security we don't want a node watching for all secrets, and it is not possible for now to watch only for secrets attached to pods from my node.
So as a temporary solution, we are introducing an annotation that would be a suggestion for kubelet for the TTL of secrets in the cache and a very simple controller that would be setting this annotation based on the cluster size (the large cluster is, the bigger ttl is).
That workaround mean that only very local changes are needed in Kubelet, we are creating a well separated very simple controller, and once watching "my secrets" will be possible it will be easy to remove it and switch to that. And it will allow us to reach scalability goals.
@dchen1107 @thockin @liggitt
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
fix scheduler performance test script
**What this PR does / why we need it**:
`test-performance.sh` is in dir `kubernetes/test/integration/scheduler_perf`
the dir `kubernetes/test/component/scheduler/perf` does not exist
Thanks.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 41074, 41147, 40854, 41167, 40045)
Fix some funky funcs.
This is code cleanup. Fix function declarations and remove stale comment.
Automatic merge from submit-queue (batch tested with PRs 41074, 41147, 40854, 41167, 40045)
Upgrade test for deployments
Upgrade test for Deployments. Should prevent issues like https://github.com/kubernetes/kubernetes/issues/40415 in the future.
@krousey @janetkuo @soltysh
Haven't managed to run it locally...
```
$ go run hack/e2e.go --up --test --test_args="--ginkgo.focus=\[Feature:MasterUpgrade\] --upgrade-target=ci/latest --upgrade-image=gci"
2017/02/02 11:43:22 e2e.go:946: Running: ./hack/e2e-internal/e2e-down.sh
2017/02/02 11:43:22 e2e.go:948: Step './hack/e2e-internal/e2e-down.sh' finished in 7.278236ms
2017/02/02 11:43:22 e2e.go:946: Running: ./hack/e2e-internal/e2e-up.sh
2017/02/02 11:43:22 e2e.go:948: Step './hack/e2e-internal/e2e-up.sh' finished in 5.286328ms
2017/02/02 11:43:22 e2e.go:946: Running: ./cluster/kubectl.sh version --match-server-version=false
2017/02/02 11:43:22 e2e.go:948: Step './cluster/kubectl.sh version --match-server-version=false' finished in 213.847259ms
2017/02/02 11:43:22 e2e.go:946: Running: ./hack/e2e-internal/e2e-status.sh
2017/02/02 11:43:22 e2e.go:948: Step './hack/e2e-internal/e2e-status.sh' finished in 103.253183ms
2017/02/02 11:43:22 e2e.go:230: Something went wrong: encountered 2 errors: [exit status 1 exit status 1]
exit status 1
```
@krousey any eta for when the upgrade framework will be integrated in the pr builder?
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
add k8s.io/sample-apiserver to demonstrate how to build an aggregated API server
builds on https://github.com/kubernetes/kubernetes/pull/41093
This creates a sample API server is a separate staging repo to guarantee no cheating with `k8s.io/kubernetes` dependencies. The sample is run during integration tests (simple tests on it so far) to ensure that it continues to run.
@sttts @kubernetes/sig-api-machinery-misc ptal
@pwittrock @pmorie @kris-nova an aggregated API server example that will stay up to date.
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Remove deprecated kubelet flags that look safe to remove
Removes:
```
--config
--auth-path
--resource-container
--system-container
```
which have all been marked deprecated since at least 1.4 and look safe to remove.
```release-note
The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed.
```
Automatic merge from submit-queue (batch tested with PRs 41145, 38771, 41003, 41089, 40365)
Use privileged containers for statefulset e2e tests
Test containers need to run as spc_t in order to interact with the host
filesystem under /tmp, as the tests for StatefulSet are doing. Docker
will transition the container into this domain when running the container
as privileged.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
**Release note**:
```release-note
NONE
```
/cc @ncdc @soltysh @pmorie
1. pcb and pcb controller are removed and their functionality is
encapsulated in StatefulPodControlInterface.
2. IdentityMappers has been removed to clarify what properties of a Pod are
mutated by the controller. All mutations are performed in the
UpdateStatefulPod method of the StatefulPodControlInterface.
3. The statefulSetIterator and petQueue classes are removed. These classes
sorted Pods by CreationTimestamp. This is brittle and not resilient to
clock skew. The current control loop, which implements the same logic,
is in stateful_set_control.go. The Pods are now sorted and considered by
their ordinal indices, as is outlined in the documentation.
4. StatefulSetController now checks to see if the Pods matching a
StatefulSet's Selector also match the Name of the StatefulSet. This will
make the controller resilient to overlapping, and will be enhanced by
the addition of ControllerRefs.
Automatic merge from submit-queue (batch tested with PRs 40175, 41107, 41111, 40893, 40919)
[Federation][e2e] Move Cluster Registration to federation-up.sh
**What this PR does / why we need it**:
Remove cluster register/unregister calls from test case BeforeEach/AfterEach blocks.
Register clusters once in federation-up.sh
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40768
**Special notes for your reviewer**:
**Release note**: `NONE`
cc: @madhusudancs @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
e2e tests for vSphere cloud provider
**What this PR does / why we need it**:
This PR contains changes for existing e2e volume provisioning test cases for running on vsphere cloud provider.
**Following is the summary of changes made in existing e2e test cases**
**Added test/e2e/persistent_volumes-vsphere.go**
- This test verifies deleting a PVC before the pod does not cause pod deletion to fail on PD detach and deleting the PV before the pod does not cause pod deletion to fail on PD detach.
**test/e2e/volume_provisioning.go**
- This test creates a StorageClass and claim with dynamic provisioning and alpha dynamic provisioning annotations and verifies that required volumes are getting created. Test also verifies that created volume is readable and retaining data.
- Added vsphere as supported cloud provider. Also set pluginName to "kubernetes.io/vsphere-volume" for vsphere cloud provider.
**test/e2e/volumes.go**
- Added test spec for vsphere
- This test creates requested volume, mount it on the pod, write some random content at /opt/0/index.html and verifies file contents are perfect to make sure we don't see the content from previous test runs.
- This test also passes "1234" as fsGroup to mount volume and verifies fsGroup is set correctly.
**added test/e2e/vsphere_utils.go**
- Added function verifyVSphereDiskAttached - Verify the persistent disk attached to the node.
- Added function waitForVSphereDiskToDetach - Wait until vsphere vmdk is deteched from the given node or time out after 5 minutes
- Added getVSpherePersistentVolumeSpec - create vsphere volume spec with given VMDK volume path, Reclaim Policy and labels
- Added getVSpherePersistentVolumeClaimSpec - get vsphere persistent volume spec with given selector labels
- createVSphereVolume - function to create vmdk volume
**Following is the summary of new e2e tests added with this PR**
**test/e2e/vsphere_volume_placement.go**
- contains volume placement tests using node label selector
- Test Back-to-back pod creation/deletion with the same volume source on the same worker node
- Test Back-to-back pod creation/deletion with the same volume source attach/detach to different worker nodes
**test/e2e/pv_reclaimpolicy.go**
- contains tests for PV/PVC - Reclaiming Policy
- Test verifies persistent volume should be deleted when reclaimPolicy on the PV is set to delete and associated claim is deleted
- Test also verified that persistent volume should be retained when reclaimPolicy on the PV is set to retain and associated claim is deleted
**test/e2e/pvc_label_selector.go**
- This is function test for Selector-Label Volume Binding Feature.
- Verify volume with the matching label is bounded with the PVC.
Other changes
Updated pkg/cloudprovider/providers/vsphere/BUILD and test/e2e/BUILD
**Which issue this PR fixes** *
fixes # 41087
**Special notes for your reviewer**:
Updated tests were executed on kubernetes v1.4.8 release on vsphere.
Test steps are provided in comments
@kerneltime @BaluDontu
Test containers need to run as spc_t in order to interact with the host
filesystem under /tmp, as the tests for StatefulSet are doing. Docker
will transition the container into this domain when running the container
as privileged.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)
Remove checks for pods responding in deployment e2e tests
Fixes#39879
Remove it because it caused deployment e2e tests sometimes timed out waiting for pods responding, and pods responding isn't related to deployment controller and is not a prerequisite of deployment e2e tests.
@kargakis
addressed review comments
Addressed review comment for pv_reclaimpolicy.go to verify content of the volume
addressed 2nd round of review comments
addressed 3rd round of review comments from jeffvance
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
scrub aggregator names to eliminate discovery
Cleanup old uses of `discovery`. Also removes the legacy functionality.
@kubernetes/sig-api-machinery-misc @sttts
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Promote SubjectAccessReview to v1
We have multiple features that depend on this API:
SubjectAccessReview
- [webhook authorization](https://kubernetes.io/docs/admin/authorization/#webhook-mode)
- [kubelet delegated authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization)
- add-on API server delegated authorization
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating access
- A typo was discovered in the SubjectAccessReviewSpec Groups field name
This PR promotes the existing v1beta1 API to v1, with the only change being the typo fix to the groups field. (fixes https://github.com/kubernetes/kubernetes/issues/32709)
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authorization.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Bump GCI to gci-beta-56-9000-80-0
cc/ @Random-Liu @adityakali
Changelogs since gci-dev-56-8977-0-0 (currently used in Kubernetes):
- "net.ipv4.conf.eth0.forwarding" and "net.ipv4.ip_forward" may get reset to 0
- Track CVE-2016-9962 in Docker in GCI
- Linux kernel CVE-2016-7097
- Linux kernel CVE-2015-8964
- Linux kernel CVE-2016-6828
- Linux kernel CVE-2016-7917
- Linux kernel CVE-2016-7042
- Linux kernel CVE-2016-9793
- Linux kernel CVE-2016-7039 and CVE-2016-8666
- Linux kernel CVE-2016-8655
- Toolbox: allow docker image to be loaded from local tarball
- Update compute-image-package in GCI
- Change the product name on /etc/os-release (to COS)
- Remove 'dogfood' from HWID_OVERRIDE in /etc/lsb-release
- Include Google NVME extensions to optimize LocalSSD performance.
- /proc/<pid>/io missing on GCI (enables process stats accounting)
- Enable BLK_DEV_THROTTLING
cc/ @roberthbailey @fabioy for GKE cluster update
Automatic merge from submit-queue
federation: Refactoring namespaced resources deletion code from kube ns controller and sharing it with fed ns controller
Ref https://github.com/kubernetes/kubernetes/issues/33612
Refactoring code in kube namespace controller to delete all resources in a namespace when the namespace is deleted. Refactored this code into a separate NamespacedResourcesDeleter class and calling it from federation namespace controller.
This is required for enabling cascading deletion of namespaced resources in federation apiserver.
Before this PR, we were directly deleting the namespaced resources and assuming that they go away immediately. With cascading deletion, we will have to wait for the corresponding controllers to first delete the resources from underlying clusters and then delete the resource from federation control plane. NamespacedResourcesDeleter has this waiting logic.
cc @kubernetes/sig-federation-misc @caesarxuchao @derekwaynecarr @mwielgus
Automatic merge from submit-queue (batch tested with PRs 40385, 40786, 40999, 41026, 40996)
Refactor federated services tests a bit to move a test that requires no cluster creation to a separate block.
Follow up to PR #40769.
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
Replace hand-written informers with generated ones
Replace existing uses of hand-written informers with generated ones.
Follow-up commits will switch the use of one-off informers to shared
informers.
This is a precursor to #40097. That PR will switch one-off informers to shared informers for the majority of the code base (but not quite all of it...).
NOTE: this does create a second set of shared informers in the kube-controller-manager. This will be resolved back down to a single factory once #40097 is reviewed and merged.
There are a couple of places where I expanded the # of caches we wait for in the calls to `WaitForCacheSync` - please pay attention to those. I also added in a commented-out wait in the attach/detach controller. If @kubernetes/sig-storage-pr-reviews is ok with enabling the waiting, I'll do it (I'll just need to tweak an integration test slightly).
@deads2k @sttts @smarterclayton @liggitt @soltysh @timothysc @lavalamp @wojtek-t @gmarek @sjenning @derekwaynecarr @kubernetes/sig-scalability-pr-reviews
Remove it because it caused deployment e2e tests sometimes timed out
waiting for pods responding, and pods responding isn't related to
deployment controller and is not a prerequisite of deployment e2e tests.
Automatic merge from submit-queue (batch tested with PRs 40978, 40994, 41008, 40622)
Refactored kubemark code into provider-specific and provider-independent parts [Part-2]
Applying part of the changes of PR https://github.com/kubernetes/kubernetes/pull/39033 (which refactored kubemark code completely). The changes included in this PR are:
- Added test/kubemark/skeleton/util.sh which defines a well-commented interface that any cloud-provider should implement to run kubemark.
This includes functions like creating the master machine instance along with its resources, remotely executing a given command on the master (like ssh), scp, deleting the master instance and its resources.
All these functions have to be over-ridden by each cloud provider inside the file /test/kubemark/$CLOUD_PROVIDER/util.sh
- Implemented the above mentioned interface for gce in /test/kubemark/$CLOUD_PROVIDER/util.sh
- Made start- and stop- kubemark scripts (almost) provider independent by making them source the interface based on cloud provider.
@kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue
Add SIG to test owners
**What this PR does / why we need it**:
This PR adds a `sig` column to the test owners file generation script.
A problem experienced with the current owners file is that since members are auto-assigned there are times where tests are assigned to non-active users who don't follow up to notifications to fix flakes. By assigning a SIG to each test we can hold a group we know is active responsible for taking care of flakes it's less likely that flakes will fall through the cracks.
**Special notes for your reviewer**:
* A companion PR will go into *kubernetes/contrib* adding support for mungers parsing this new column.
* Another PR in contrib will add labeling GitHub flake issues with the appropriate SIG
* Currently SIGs are not labeled, this will be added in another PR where SIG determinations can be discussed
@saad-ali @pwittrock
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
Rename experimental-cgroups-per-pod flag
**What this PR does / why we need it**:
1. Rename `experimental-cgroups-per-qos` to `cgroups-per-qos`
1. Update hack/local-up-cluster to match `CGROUP_DRIVER` with docker runtime if used.
**Special notes for your reviewer**:
We plan to roll this feature out in the upcoming release. Previous node e2e runs were running with this feature on by default. We will default this feature on for all e2es next week.
**Release note**:
```release-note
Rename --experiemental-cgroups-per-qos to --cgroups-per-qos
```
Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)
PV E2E: provide each spec with a fresh nfs host
**What this PR does / why we need it**:
PersistentVolume e2e currently reuses an NFS host pod created at the start of the suite and accessed by each test. This is far less favorable than using a fresh volume per test. Additionally, this guards against the volume host pod or it's kubelet being disrupted, which has led to flakes.
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)
Add [Flaky] tag to persistent volumes tests
**What this PR does / why we need it**:
Persistent Volume tests continue to flake in CI.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
```release-note
NONE
```
Automatic merge from submit-queue
Add an upgrade test for secrets.
**What this PR does / why we need it**: This PR adds an upgrade test for secrets. It creates a secret and makes sure that pods can consume it before an after an upgrade.
Automatic merge from submit-queue (batch tested with PRs 40696, 39914, 40374)
Forgiveness library changes
**What this PR does / why we need it**:
Splited from #34825, contains library changes that are needed to implement forgiveness:
1. ~~make taints-tolerations matching respect timestamps, so that one toleration can just tolerate a taint for only a period of time.~~ As TaintManager is caching taints and observing taint changes, time-based checking is now outside the library (in TaintManager). see #40355.
2. make tolerations respect wildcard key.
3. add/refresh some related functions to wrap taints-tolerations operation.
**Which issue this PR fixes**:
Related issue: #1574
Related PR: #34825, #39469
~~Please note that the first 2 commits in this PR come from #39469 .~~
**Special notes for your reviewer**:
~~Since currently we have `pkg/api/helpers.go` and `pkg/api/v1/helpers.go`, there are some duplicated periods of code laying in these two files.~~
~~Ideally we should move taints-tolerations related functions into a separate package (pkg/util/taints), and make it a unified set of implementations. But I'd just suggest to do it in a follow-up PR after Forgiveness ones done, in case of feature Forgiveness getting blocked to long.~~
**Release note**:
```release-note
make tolerations respect wildcard key
```
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
Density Test includes deletion and volumes
Moved the calls to deletePodSync to BEFORE logDensityTimeSeries. This is because the parser considers a line printed in logDensityTimeSeries to be the "end" of the test. This change includes deletion in the "test window", but makes no other changes.
I also added volumes to the test, so that we can make sure that mounting and unmounting volumes are also taken into account for performance profiling.
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
Promote init containers to GA
This is proposed for 1.6
PR moves beta proved concept for init containers to stable. Specification of init containers can be now stated under initContainers field in PodSpec/PodTemplateSpec. Specifying init-containers in annotation is still possible, but will be removed in future version.
```release-note
Init containers have graduated to GA and now appear as a field. The beta annotation value will still be respected and overrides the field value.
```
Automatic merge from submit-queue (batch tested with PRs 40864, 40666, 38382, 40874)
test: reduce deployment progress deadline, ensure its rs is up
Fixes https://github.com/kubernetes/kubernetes/issues/39785 by reducing the deadline of the expected progress and making sure the new replica set is up before checking the deployment condition.
@kubernetes/sig-apps-misc
Automatic merge from submit-queue (batch tested with PRs 40884, 40809, 40845, 40866, 40875)
Remove many job e2e tests.
These tests have equivalent unit test coverage as far as I can tell, in [pkg/controller/job/jobcontroller_test.go](https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/job/jobcontroller_test.go). See #40839 for context.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40884, 40809, 40845, 40866, 40875)
Node E2E: Create new ubuntu image with docker 1.12.6.
We should test the newest docker 1.12 version - 1.12.6.
/cc @dchen1107 @yujuhong @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40812, 39903, 40525, 40729)
test/node_e2e: wire-in cri-enabled local testing
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate argument to
the kubelet.
Automatic merge from submit-queue
test: move deployment helper in testing framework
Wanted to get this out of the way before submitting an upgrade test for Deployments and I need the helper in the framework utility
@janetkuo @soltysh
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
genericapiserver: cut off more dependencies – episode 7
Follow-up of https://github.com/kubernetes/kubernetes/pull/40822
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
Test GCE PD unmounts and detaches when the namespace of the pvc&pod is deleted.
Addition to Persistent Volume E2E testing. On a GCE cluster, create a pv, pvc, and client pod. Delete the namespace and check that the disk detaches successfully.
@jeffvance
~~DEPENDENT ON~~ #34353 merged. No dependencies.
Automatic merge from submit-queue
Removed HPA objects from extensions api group
fix#29778
``` release-note
HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead.
```
cc @kubernetes/autoscaling
Automatic merge from submit-queue (batch tested with PRs 39169, 40719, 38954, 40808, 40689)
Add StatefulSets checks at Service level
Hi!
Please let me propose some very small e2e testsuite enhancement.
This PR removed a `TODO` about checking governing service at unit test level (which is hard) and adds this to e2e testsuite.
Thanks
Sebastian
Automatic merge from submit-queue
Add websocket support for port forwarding
#32880
**Release note**:
```release-note
Port forwarding can forward over websockets or SPDY.
```
This commit wires-in the pre-existing `--container-runtime` flag for
local node_e2e testing.
This is needed in order to further skip docker specific testing
and validation.
Local CRI node_e2e can now be performed via
`make test-e2e-node RUNTIME=remote REMOTE=false`
which will also take care of passing the appropriate arguments to
the kubelet.
Automatic merge from submit-queue
Use full package path for definition name in OpenAPI spec
We were using short package name (last part of package name) plus type name for OpenAPI spec definition name. That can result in duplicate names and make the spec invalid. To be sure we will always have unique names, we are going to use full package name as definition name. Also "x-kubernetes-tag" custom field is added to definitions to list Group/Version/Kind for the definitions that has it. This will help clients to discover definitions easier.
Lastly, we've added a reference from old definition names to the new ones to keep backward compatibilities. The list of old definitions will not be updated.
**Release note**:
- Rename OpenAPI definition names to type's full package names to prevent duplicates
- Create OpenAPI extension "x-kubernetes-group-version-kind" for definitions to store Group/Version/Kind
- Deprecate old definition names and create a reference to the new definitions. Old definitions will be removed in the next release.
- split out port forwarding into its own package
Allow multiple port forwarding ports
- Make it easy to determine which port is tied to which channel
- odd channels are for data
- even channels are for errors
- allow comma separated ports to specify multiple ports
Add portfowardtester 1.2 to whitelist
Automatic merge from submit-queue (batch tested with PRs 40529, 40630)
propagate explicit nulls in apply
Rebase of https://github.com/kubernetes/kubernetes/pull/35496 on top of https://github.com/kubernetes/kubernetes/pull/40260
The client-side propagation of the raw value is no longer needed, since the client is preserving the original object in unstructured form (explicit nulls are preserved).
Kept tests and CreateThreeWayMergePatch changes from https://github.com/kubernetes/kubernetes/pull/35496
```release-note
kubectl apply now supports explicitly clearing values not present in the config by setting them to null
```
- [x] Clean up orphaned objects in test-cmd to preserve pre- and post- conditions
- [x] improve CreateThreeWayMergePatch test to not filter based on string comparison to test name
Automatic merge from submit-queue (batch tested with PRs 40529, 40630)
test/e2e_node: tie together expected string and exec
This commit ties together busybox-sh invocation and test expectation
to avoid subtle mismatches between exec command and output string.
Automatic merge from submit-queue (batch tested with PRs 40645, 40541, 40769)
[Federation] Marked the tests that don't need registered clusters so.
Somewhat related to issue #40766.
cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 40527, 40738, 39366, 40609, 40748)
Removed Flaky tag from PV e2e, added [Volume] to disruptive PV e2e
**What this PR does / why we need it**:
Removes `[Flaky]` from PV e2e testing. Flakes were due to interference from an external test disrupting a cluster node. The test has been [removed](9f36c032de) and the flakes have [cleared](https://k8s-testgrid.appspot.com/google-gce#gce-flaky).
Secondly, added `[Volume]` tag to PV disruptive tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39119
**Release note**:
NONE
```release-note
```
Automatic merge from submit-queue
Add flag to node e2e test specifying location of ssh privkey
**What this PR does / why we need it**: in CI, the ssh private key is not always located at `$HOME/.ssh`, so it's helpful to be able to override it.
@krzyzacy here's my resurrected change. I'm not sure why I neglected to follow-through on it originally.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
test/images/mount-tester: ensure exec binary is o+rx
The `mount-tester` image is currently used as a base layer for other
test images (like `mounttest-user`) which perform uid/gid changes.
However, the binary built in here just follows local umask, and currently is
```
-rwxr-x--- 1 root root 2052704 May 19 2016 mt
```
This commit adds an explicit chmod on the binary to make sure uid/gid
tests can still run it as "others".
Automatic merge from submit-queue
refactor pv e2e code to improve readability
**What this PR does / why we need it**:
Moved the helper functions out of _persistent_volumes.go_ to a new file, _pvutil.go_, in order to improve readability and make it easier to add new tests.
Also, all pod delete code now calls the same helper function `deletePodWithWait`.
**Release note**:
```
NONE
```
Automatic merge from submit-queue
e2e test should fail if err==timeout
If err==timeout, it means the replicaset (the owner) is not deleted after 30s, which indicates a bug, so the e2e test should fail.
Automatic merge from submit-queue
Remove jsafrane from some tests
I do not know anything about ESIPP nor clouddns and I have never touched these tests. It would be better to assign flakes to someone else.
@bprashanth @quinton-hoole, PTAL. I took your names as authors of these two tests.
Automatic merge from submit-queue
Improve the multiarch situation; armel => armhf; reenable pcc64le; remove the patched golang
**What this PR does / why we need it**:
- Improves the multiarch situation as described in #38067
- Tries to bump to go1.8 for arm (and later enable ppc64le)
- GOARM 6 => GOARM 7
- Remove the golang 1.7 patch
- armel => armhf
- Bump QEMU version to v2.7.0
**Release note**:
```release-note
Improve the ARM builds and make hyperkube on ARM working again by upgrading the Go version for ARM to go1.8beta2
```
@kubernetes/sig-testing-misc @jessfraz @ixdy @jbeda @david-mcmahon @pwittrock
Automatic merge from submit-queue (batch tested with PRs 40584, 40319)
ssh support for local
**What this PR does / why we need it**: adds local deployment support for e2e tests. Useful for non-cloud, simple testing.
**Special notes for your reviewer**: Formerly this pr was part of #38214
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40132, 39302, 40194, 40619, 40601)
Update NPD version to v0.3.0-alpha.0 in kubemark.
@wojtek-t @shyamjvs Update the NPD version in kubemark.
I just built the alpha release https://github.com/kubernetes/node-problem-detector/releases/tag/v0.3.0-alpha.0.
And the PR https://github.com/kubernetes/node-problem-detector/pull/79 is included.
However, I'm not sure whether 1 minute period is longer enough.
If it's still not longer enough, in fact we can extend it by split the resync and heartbeat:
* Every 1 minute, check whether there is inconsistency between apiserver and npd, and only update when there is inconsistency. (1 GET/m)
* Every > 2 minute, do forcibly update as heartbeat. (<0.5 PATCH/m)
And I can also make the sync period configurable after we finalize the sync mechanism.
Automatic merge from submit-queue (batch tested with PRs 39469, 40557)
Refactored kubemark code into provider-specific and provider-independent parts [Part-1]
Applying part of the changes of PR https://github.com/kubernetes/kubernetes/pull/39033 (which refactored kubemark code completely). The changes included in this PR are:
The following are the major changes as part of this refactoring:
- Moved cluster-kubemark/config-default.sh -> cluster-kubemark/gce/config-default.sh (as the config is gce-specific)
- Changed kubernetes/cluster/kubemark/util.sh to source the right scripts based on the cloud-provider
- Added the file test/kubemark/cloud-provider-config.sh which sets the variable CLOUD_PROVIDER that is later picked up by various scripts (run-e2e-tests.sh, common.sh)
- Removed useless code and restructured start-kubemark.sh and stop-kubemark.sh scripts.
@kubernetes/sig-scalability-misc @wojtek-t @gmarek
The `mount-tester` image is currently used as a base layer for other
test images (like `mounttest-user`) which perform uid/gid changes.
However, the binary built in here just follows local umask, and currently is
```
-rwxr-x--- 1 root root 2052704 May 19 2016 mt
```
This commit adds an explicit chmod on the binary to make sure uid/gid
tests can still run it as "others".
Automatic merge from submit-queue (batch tested with PRs 40046, 40073, 40547, 40534, 40249)
Fix e2e: validates that InterPodAntiAffinity is respected if matching 2
**What this PR does / why we need it**:
Fixed e2e: validates that InterPodAntiAffinity is respected if matching 2
```
• Failure [120.255 seconds]
[k8s.io] SchedulerPredicates [Serial]
/root/code/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:656
validates that InterPodAntiAffinity is respected if matching 2 [It]
/root/code/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:561
Not scheduled Pods: []v1.Pod(nil)
Expected
<int>: 0
to equal
<int>: 1
/root/code/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/30142
**Special notes for your reviewer**:
xref: https://bugzilla.redhat.com/show_bug.cgi?id=1413748
While looking into the above bug, I found that the e2e was failing.
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
move the discovery and dynamic clients
Moved the dynamic client, discovery client, testing/core, and testing/cache to `client-go`. Dependencies on api groups we don't have generated clients for have dropped out, so federation, kubeadm, and imagepolicy.
@caesarxuchao @sttts
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue (batch tested with PRs 38739, 40480, 40495, 40172, 40393)
Rename controller pkg/registry/core/controller to pkg/registry/core/r…
…eplicationcontroller
**What this PR does / why we need it**:
Rename controller pkg/registry/core/controller to pkg/registry/core/replicationcontroller
This will clarify the purpose of the controller since intent is replicationcontroller
Please refer to
https://github.com/kubernetes/kubernetes/issues/17648
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
NONE
**Release note**:
```NONE
```
Automatic merge from submit-queue
Prep node_e2e for GCI to COS name change
GCI will soon change name in etc/os-release from "gci" to "cos".
This prepares the node_e2e tests to deal with that change and also updates some comments/log messages/var names in anticipation.
Automatic merge from submit-queue (batch tested with PRs 39538, 40188, 40357, 38214, 40195)
test for host cleanup in unfavorable pod deletes
addresses issue #31272 with a new e2e test in _kubelet.go_
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 39538, 40188, 40357, 38214, 40195)
genericapiserver: cut off more dependencies – episode 2
Compare commit subjects.
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 39538, 40188, 40357, 38214, 40195)
Decoupling scheduler creation from creation of scheduler.Config struc…
**What this PR does / why we need it**:
Adds functionality to the scheduler to initialize from an Configurator interface, rather then via a Config struct.
**Which issue this PR fixes**
Reduces coupling to `scheduler.Config` data structure format so that we can proliferate more interface driven composition of scheduler components.
Automatic merge from submit-queue (batch tested with PRs 40428, 40176)
Cleaup Affinity post conversion from annotations to fields
**What this PR does / why we need it**:
Cleans up leftover work from the conversion of affinity from annotations to fields.
fixes#40016
related #25319
**Special notes for your reviewer**:
There are some TODO items left for @luxas or @errordeveloper b/c they were trying to use affinity in a way that is not possible.
**Release note**:
```release-note
NONE
```
/cc @kubernetes/sig-scheduling-misc @rrati
Automatic merge from submit-queue
move client/cache and client/discovery to client-go
mechanical changes to move those packages. Had to create a `k8s.io/kubernetes/pkg/client/tests` package for tests that were blacklisted from client-go. We can rewrite these tests later and move them, but for now they'll still run at least.
@caesarxuchao @sttts
Automatic merge from submit-queue (batch tested with PRs 40130, 40419, 40416)
fixing source for heapster eventer in kubemark
Fixing the out of place heapster eventer source IP.
cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 40215, 40340, 39523)
Retry resource quota lookup until count stabilizes
On contended servers the service account controller can slow down,
leading to the count changing during a run. Wait up to 5s for the count
to stabilize, assuming that updates come at a consistent rate, and are
not held indefinitely.
Upstream of openshift/origin#12605 (we create more secrets and flake more often)
Automatic merge from submit-queue
Fix federation component logging when e2e test case fails
When a federation e2e test case fails, federation component logs (esp. controller-manager) were very useful in debugging the failure cause. Due to recent updates in framework, the logs were not captured. This PR will fix those issues.
cc @kubernetes/sig-federation-misc @nikhiljindal @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 40335, 40320, 40324, 39103, 40315)
add some resillency to the new volume-server func
**What this PR does / why we need it**: server-pod `Create` won't fail the test if the server pod already exists.
**Special notes for your reviewer**: Formerly this pr was part of #38214
Automatic merge from submit-queue
Move b.gcr.io/k8s_authenticated_test to gcr.io/k8s-authenticated-test
**What this PR does / why we need it**:
Per https://cloud.google.com/container-registry/docs/support/deprecation-notices, b.gcr.io access will be deprecated soon.
I've already mirrored the repo to the location specified in this PR.
Automatic merge from submit-queue (batch tested with PRs 39260, 40216, 40213, 40325, 40333)
Adding framework to allow multiple upgrade tests
**What this PR does / why we need it**: This adds a framework for multiple tests to run during an upgrade. This also moves the existing services test to that framework.
On contended servers the service account controller can slow down,
leading to the count changing during a run. Wait up to 5s for the count
to stabilize, assuming that updates come at a consistent rate, and are
not held indefinitely.
Automatic merge from submit-queue
Move remaining *Options to metav1
Primarily delete options, but will remove all internal references to non-metav1 options (except ListOptions).
Still working through it @sttts @deads2k
Automatic merge from submit-queue
Adding OWNERS file for federation e2e tests
Now that we have a separate `test/e2e_federation` dir for federation tests (thanks to @shashidharatd), we can have our own OWNERS file.
OWNERS file copied from https://github.com/kubernetes/kubernetes/pull/40328.
cc @kubernetes/sig-federation-misc
Automatic merge from submit-queue
Optional configmaps and secrets
Allow configmaps and secrets for environment variables and volume sources to be optional
Implements approved proposal c9f881b7bb
Release note:
```release-note
Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding `optional: true` to the volume or environment variable source specifications.
```
Automatic merge from submit-queue (batch tested with PRs 36693, 40154, 40170, 39033)
Refactored kubemark into cloud-provider independent code and GCE specific code
Ref issue #38967
The following are the major changes as part of this refactoring:
- Moved cluster-kubemark/config-default.sh -> cluster-kubemark/gce/config-default.sh (as the config is gce-specific)
- Changed kubernetes/cluster/kubemark/util.sh to source the right scripts based on the cloud-provider
- Added test/kubemark/skeleton/util.sh which defines a well-commented interface that any cloud-provider should implement to run kubemark. (We have this interface defined only for gce currently)
This includes functions like creating the master machine instance along with its resources, executing a given command on the master (like ssh), scp, deleting the master instance and its resources.
All these functions have to be overrided by each cloud provider inside the file /test/kubemark/$CLOUD_PROVIDER/util.sh
- Added the file test/kubemark/cloud-provider-config.sh which sets the variable CLOUD_PROVIDER that is later picked up by various scripts (start-kubemark.sh, stop-kubemark.sh, run-e2e-tests.sh)
- Removed test/kubemark/common.sh and moved whatever provider-independent code it had into start-kubemark.sh (the only place where the scipt is called) and moved the little gce-specific code
into test/kubemark/gce/util.sh.
- Finally, removed useless code and restructured start-kubemark.sh and stop-kubemark.sh scripts.
@kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 39446, 40023, 36853)
Create environment variables from secrets
Allow environment variables to be populated from entire secrets.
**Release note**:
```release-note
Populate environment variables from a secrets.
```
Automatic merge from submit-queue
promote certificates api to beta
Mostly posting to see what breaks but also this API is ready to be promoted.
```release-note
Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading.
```
@kubernetes/api-approvers @jcbsmpsn @pipejakob
Automatic merge from submit-queue
Update root approvers files
Replaces #40040
Update top level OWNERS files mostly to set assignees to approvers. Also remove @bgrant0607 from everywhere but the very top level OWNERS file.
Automatic merge from submit-queue
move pkg/fields to apimachinery
Purely mechanical move of `pkg/fields` to apimachinery.
Discussed with @lavalamp on slack. Moving this an `labels` to apimachinery.
@liggitt any concerns? I think the idea of field selection should become generic and this ends up shared between client and server, so this is a more logical location.
Automatic merge from submit-queue
make client-go more authoritative
Builds on https://github.com/kubernetes/kubernetes/pull/40103
This moves a few more support package to client-go for origination.
1. restclient/watch - nodep
1. util/flowcontrol - used interface
1. util/integer, util/clock - used in controllers and in support of util/flowcontrol
Automatic merge from submit-queue (batch tested with PRs 40081, 39951)
Passing correct master address to kubemark NPD & authenticating+authorizing it with apiserver
Fixes#39245
Fixes https://github.com/kubernetes/node-problem-detector/issues/50
Added RBAC for npd and fixed issue with the npd falling back to inClusterConfig.
cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue
Move Federation e2e test code to independent package
**What this PR does / why we need it**: Move federation e2e test code to an independent package called e2e_federation. This will help in multiple ways.
- easy to move the federation related code to a separate repo from core.
- one step closer to register/unregister clusters to federation only once during e2e instead of every test case. we need to introduce singleton to register cluster during framework creation which will be handled in subsequent PR.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Optimize federation e2e suite which takes long time to execute currently.
**Special notes for your reviewer**: I have tried to segregate into multiple commits. request to review commit by commit. also mostly the change is about moving the functions to a new location/package.
**Release note**:
```release-note
```
@madhusudancs @nikhiljindal @colhom
Automatic merge from submit-queue
turn on dynamic config for flaky tests
Added dynamic config to inode eviction node e2e tests in #39546, but did not enable it for flaky tests. This PR enables this feature for the flaky test suite
Automatic merge from submit-queue (batch tested with PRs 39898, 39904)
[scheduler] interface for config
**What this PR fixes**
This PR converts the Scheduler configuration factory into an interface, so that
- the scheduler_perf and scheduler integration tests dont rely on the struct for their implementation
- the exported functionality of the factory (i.e. what it needs to provide to create a scheduler configuration) is completely explicit, rather then completely coupled to a struct.
- makes some parts of the factory immutable, again to minimize possible coupling.
This makes it easier to make a custom factory in instances where we might specifically want to import scheduler logic without actually reusing the entire scheduler codebase.
Automatic merge from submit-queue
Build release tars using bazel
**What this PR does / why we need it**: builds equivalents of the various kubernetes release tarballs, solely using bazel.
For example, you can now do
```console
$ make bazel-release
$ hack/e2e.go -v -up -test -down
```
**Special notes for your reviewer**: this is currently dependent on 3b29803eb5, which I have yet to turn into a pull request, since I'm still trying to figure out if this is the best approach.
Basically, the issue comes up with the way we generate the various server docker image tarfiles and load them on nodes:
* we `md5sum` the binary being encapsulated (e.g. kube-proxy) and save that to `$binary.docker_tag` in the server tarball
* we then build the docker image and tag using that md5sum (e.g. `gcr.io/google_containers/kube-proxy:$MD5SUM`)
* we `docker save` this image, which embeds the full tag in the `$binary.tar` file.
* on cluster startup, we `docker load` these tarballs, which are loaded with the tag that we'd created at build time. the nodes then use the `$binary.docker_tag` file to find the right image.
With the current bazel `docker_build` rule, the tag isn't saved in the docker image tar, so the node is unable to find the image after `docker load`ing it.
My changes to the rule save the tag in the docker image tar, though I don't know if there are subtle issues with it. (Maybe we want to only tag when `--stamp` is given?)
Also, the docker images produced by bazel have the timestamp set to the unix epoch, which is not great for debugging. Might be another thing to change with a `--stamp`.
Long story short, we probably need to follow up with bazel folks on the best way to solve this problem.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Curating Owners: test/e2e_node
cc @random-liu @timstclair @vishh
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue (batch tested with PRs 39625, 39842)
Add RBAC v1beta1
Add `rbac.authorization.k8s.io/v1beta1`. This scrubs `v1alpha1` to remove cruft, then add `v1beta1`. We'll update other bits of infrastructure to code to `v1beta1` as a separate step.
```release-note
The `attributeRestrictions` field has been removed from the PolicyRule type in the rbac.authorization.k8s.io/v1alpha1 API. The field was not used by the RBAC authorizer.
```
@kubernetes/sig-auth-misc @liggitt @erictune
Automatic merge from submit-queue
Enable lazy initialization of ext3/ext4 filesystems
**What this PR does / why we need it**: It enables lazy inode table and journal initialization in ext3 and ext4.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#30752, fixes#30240
**Release note**:
```release-note
Enable lazy inode table and journal initialization for ext3 and ext4
```
**Special notes for your reviewer**:
This PR removes the extended options to mkfs.ext3/mkfs.ext4, so that the defaults (enabled) for lazy initialization are used.
These extended options come from a script that was historically located at */usr/share/google/safe_format_and_mount* and later ported to GO so this dependency to the script could be removed. After some search, I found the original script here: https://github.com/GoogleCloudPlatform/compute-image-packages/blob/legacy/google-startup-scripts/usr/share/google/safe_format_and_mount
Checking the history of this script, I found the commit [Disable lazy init of inode table and journal.](4d7346f7f5). This one introduces the extended flags with this description:
```
Now that discard with guaranteed zeroing is supported by PD,
initializing them is really fast and prevents perf from being affected
when the filesystem is first mounted.
```
The problem is, that this is not true for all cloud providers and all disk types, e.g. Azure and AWS. I only tested with magnetic disks on Azure and AWS, so maybe it's different for SSDs on these cloud providers. The result is that this performance optimization dramatically increases the time needed to format a disk in such cases.
When mkfs.ext4 is told to not lazily initialize the inode tables and the check for guaranteed zeroing on discard fails, it falls back to a very naive implementation that simply loops and writes zeroed buffers to the disk. Performance on this highly depends on free memory and also uses up all this free memory for write caching, reducing performance of everything else in the system.
As of https://github.com/kubernetes/kubernetes/issues/30752, there is also something inside kubelet that somehow degrades performance of all this. It's however not exactly known what it is but I'd assume it has something to do with cgroups throttling IO or memory.
I checked the kernel code for lazy inode table initialization. The nice thing is, that the kernel also does the guaranteed zeroing on discard check. If it is guaranteed, the kernel uses discard for the lazy initialization, which should finish in a just few seconds. If it is not guaranteed, it falls back to using *bio*s, which does not require the use of the write cache. The result is, that free memory is not required and not touched, thus performance is maxed and the system does not suffer.
As the original reason for disabling lazy init was a performance optimization and the kernel already does this optimization by default (and in a much better way), I'd suggest to completely remove these flags and rely on the kernel to do it in the best way.
Automatic merge from submit-queue
Start making `k8s.io/client-go` authoritative for generic client packages
Right now, client-go has copies of various generic client packages which produces golang type incompatibilities when you want to switch between different kinds of clients. In many cases, there's no reason to have two sets of packages. This pull eliminates the copy for `pkg/client/transport` and makes `client-go` the authoritative copy.
I recommend going by commits, the first just synchronizes the client-go code again so that I could test the copy script to make sure it correctly preserves the original package.
@kubernetes/sig-api-machinery-misc @lavalamp @sttts
Automatic merge from submit-queue (batch tested with PRs 34763, 38706, 39939, 40020)
Use Statefulset instead in e2e and controller
Quick fix ref: #35534
We should finish the issue to meet v1.6 milestone.
Automatic merge from submit-queue
Move PatchType to apimachinery/pkg/types
Fixes https://github.com/kubernetes/kubernetes/issues/39970
`PatchType` is shared by the client and server, they have to agree, and its critical for our API to function.
@smarterclayton @kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 39911, 40002, 39969, 40012, 40009)
Fix RBAC role for kube-proxy in Kubemark
Ref #39959
This should ensure that kube-proxy (in Kubemark) has the required role and RBAC binding.
@deads2k PTAL
cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
move api/errors to apimachinery
`pkg/api/errors` is a set of helpers around `meta/v1.Status` that help to create and interpret various apiserver errors. Things like `.NewNotFound` and `IsNotFound` pairings. This pull moves it into apimachinery for use by the clients and servers.
@smarterclayton @lavalamp First commit is the move plus minor fitting. Second commit is straight replace and generation.
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
Add optional per-request context to restclient
**What this PR does / why we need it**: It adds per-request contexts to restclient's API, and uses them to add timeouts to all proxy calls in the e2e tests. An entire e2e shouldn't hang for hours on a single API call.
**Which issue this PR fixes**: #38305
**Special notes for your reviewer**:
This adds a feature to the low-level rest client request feature that is entirely optional. It doesn't affect any requests that don't use it. The api of the generated clients does not change, and they currently don't take advantage of this.
I intend to patch this in to 1.5 as a mostly test only change since it's not going to affect any controller, generated client, or user of the generated client.
cc @kubernetes/sig-api-machinery
cc @saad-ali
Automatic merge from submit-queue
Fix examples e2e permission check
Ref #39382
Follow-up from #39896
Permission check should be done within the e2e test namespace, not cluster-wide
Also improved RBAC audit logging to make the scope of the permission check clearer
Automatic merge from submit-queue
Remove sleep from DynamicProvisioner test.
The comment says that the sleep is there because of 10 minute PV controller
sync. The controller sync is now 15 seconds and it should be quick enough
to hide this in subsequent `WaitForPersistentVolumeDeleted(.. , 20*time.Minute)`
Automatic merge from submit-queue (batch tested with PRs 38427, 39896, 39889, 39871, 39895)
Grant permissions to e2e examples test service account
ref #39382
Automatic merge from submit-queue
Updated unit tests
@janetkuo updated the flaky unit test to have the same structure with regard to uncasting as the rest of the tests. ptal
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)
Made cache.Controller to be interface.
**What this PR does / why we need it**:
#37504
Automatic merge from submit-queue
run staging client-go update
Chasing to see what real problems we have in staging-client-go.
@sttts you get similar results?
Automatic merge from submit-queue
replace global registry in apimachinery with global registry in k8s.io/kubernetes
We'd like to remove all globals, but our immediate problem is that a shared registry between k8s.io/kubernetes and k8s.io/client-go doesn't work. Since client-go makes a copy, we can actually keep a global registry with other globals in pkg/api for now.
@kubernetes/sig-api-machinery-misc @lavalamp @smarterclayton @sttts
Automatic merge from submit-queue
Update images that use ubuntu-slim base image to :0.6
**What this PR does / why we need it**: `ubuntu-slim:0.4` is somewhat old, being based on Ubuntu 16.04, whereas `ubuntu-slim:0.6` is based on Ubuntu 16.04.1.
**Special notes for your reviewer**: I haven't pushed any of these images yet, so I expect all of the e2e builds to fail. If we're happy with the changes, I can push the images and then re-trigger tests.
**Release note**:
```release-note
NONE
```
cc @aledbf as FYI
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)
[scheduling] Moved pod affinity and anti-affinity from annotations to api fields #25319
Converted pod affinity and anti-affinity from annotations to api fields
Related: #25319
Related: #34508
**Release note**:
```Pod affinity and anti-affinity has moved from annotations to api fields in the pod spec. Pod affinity or anti-affinity that is defined in the annotations will be ignored.```
Automatic merge from submit-queue
Refactor registry etcd to storage
Fixes#17546
Simple shuffle on naming so any sane new person entering the code base can understand what the actual etcd dependencies are.
Automatic merge from submit-queue (batch tested with PRs 39768, 39463)
Check if path exists before performing unmount
This is part 3 of an effort to check if path exists before performing an unmount operation.
[Part 1](https://github.com/kubernetes/kubernetes/pull/38547) and [part 2](https://github.com/kubernetes/kubernetes/pull/39311) involved auditing the different volume plugins and refactoring their `TearDownAt()s` to use the common util function/or create one if absent.
The ideal way to do this change would involve refactoring of the `TearDownAt()s` of these plugins and make a common util function that checks path. (The plugins involved in this PR use someway of unmounting a bind mount and unmounting a global path, there is also refactoring needed to consolidate disk_manager of fc, rbd and iscsi). A non-goal part of this effort can also involve refactoring all the `SetupAt()s`
In the interest of time and considering other higher priority issues that I am caught up with, I am unable to give the time the refactoring needs. Hence I've made the minimum change that would give the desired output.
I am tracking the work pending in this issue: https://github.com/kubernetes/kubernetes/issues/39251
```release-note
NONE
```
Automatic merge from submit-queue
Add [Volume] tag to all the volume-related E2E tests.
**What this PR does / why we need it**:
Tags all the volume/storage related e2e tests to make it easier to run a volume test suite.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes#35542
**Special notes for your reviewer**:
Please let me know if there are tests that should/should not be included.
**Release note**:
NONE
```release-note
```
Automatic merge from submit-queue
[CRI] Don't include user data in CRI streaming redirect URLs
Fixes: https://github.com/kubernetes/kubernetes/issues/36187
Avoid userdata in the redirect URLs by caching the {Exec,Attach,PortForward}Requests with a unique token. When the redirect URL is created, the token is substituted for the request params. When the streaming server receives the token request, the token is used to fetch the actual request parameters out of the cache.
For additional security, the token is generated using the secure random function, is single use (i.e. the first request with the token consumes it), and has a short expiration time.
/cc @kubernetes/sig-node
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
e2e tests: new portforwardertester with another three tests for case …
PR include:
- add new e2e test cases for BIND_ADDRESS='0.0.0.0'
- add to portforwardertester.go os.Getenv("BIND_ADDRESS") and if not set, it should be localhost for backward compability with existing tests
- for existing tests pass explicity BIND_ADDRESS='localhost'
- rename existing tests
It was mention in the issue: #32128
cc @mzylowski @pskrzyns
Automatic merge from submit-queue (batch tested with PRs 39495, 39547)
Tag persistent volume PersistentVolume E2E [Volume][Serial][Flaky]
**What this PR does / why we need it**:
When run parallel with other tests that use PV(C)s, cross-test binding causes flakes. Add `[Serial]` tag.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: f
Partly addresses #39119
**Special notes for your reviewer**:
cc @saad-ali @jsafrane @jeffvance
Automatic merge from submit-queue (batch tested with PRs 39684, 39577, 38989, 39534, 39702)
Set PodStatus QOSClass field
This PR continues the work for https://github.com/kubernetes/kubernetes/pull/37968
It converts all local usage of the `qos` package class types to the new API level types (first commit) and sets the pod status QOSClass field in the at pod creation time on the API server in `PrepareForCreate` and in the kubelet in the pod status update path (second commit). This way the pod QOS class is set even if the pod isn't scheduled yet.
Fixes#33255
@ConnorDoyle @derekwaynecarr @vishh
Automatic merge from submit-queue (batch tested with PRs 39694, 39383, 39651, 39691, 39497)
Fix evictions test
**What this PR does / why we need it**:
Fixes bugs in evictions test. Make vet happy.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#39412
Ref: #39452
cc: @calebamiles
Automatic merge from submit-queue (batch tested with PRs 39694, 39383, 39651, 39691, 39497)
Allow rolebinding/clusterrolebinding with explicit bind permission check
Fixes https://github.com/kubernetes/kubernetes/issues/39176
Fixes https://github.com/kubernetes/kubernetes/issues/39258
Allows creating/updating a rolebinding/clusterrolebinding if the user has explicitly been granted permission to perform the "bind" verb against the referenced role/clusterrole (previously, they could only bind if they already had all the permissions in the referenced role via an RBAC role themselves)
```release-note
To create or update an RBAC RoleBinding or ClusterRoleBinding object, a user must:
1. Be authorized to make the create or update API request
2. Be allowed to bind the referenced role, either by already having all of the permissions contained in the referenced role, or by having the "bind" permission on the referenced role.
```
Automatic merge from submit-queue (batch tested with PRs 38212, 38792, 39641, 36390, 39005)
Updating federated service controller to support cascading deletion
Ref https://github.com/kubernetes/kubernetes/issues/33612
Service controller is special than other federation controllers because it does not use federatedinformer and updater to sync services (it was written before we had those frameworks).
Updating service controller code to instantiate these frameworks and then use deletion helper to perform cascading deletion.
Note that, I havent changed the queuing logic in this PR so we still dont use federated informer to manage the queue. Will do that in the next PR.
cc @kubernetes/sig-federation-misc @mwielgus @quinton-hoole
```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated services. Setting it to false while deleting a federated service also deletes the corresponding services from all registered clusters.
```
Automatic merge from submit-queue (batch tested with PRs 39695, 37054, 39627, 39546, 39615)
Add configs that run more advanced density and load tests
Wojtek is on vacation this week - @timothysc can you please take a look? It's rather terrible, but I don't have a better idea on how to make parametric tests.
cc @wojtek-t
Automatic merge from submit-queue (batch tested with PRs 39695, 37054, 39627, 39546, 39615)
Use Dynamic Config in e2e_node inode eviction test
Alternative solution to #39249. Similar to solution proposed by @vishh in #36828.
@Random-Liu @mtaufen
Automatic merge from submit-queue (batch tested with PRs 39648, 38167, 39591, 39415, 39612)
Add verbs to thirdparty resources in discovery
The namespace controller ignores thirdparty resources right now because verbs are not set. This PR sets a static list of verbs.
Moreover, integration tests are added for the discovery info of thirdparty resources.
/cc @zhouhaibing089
Automatic merge from submit-queue
Deleting federation-util-14.go that is not being used anywhere
We have the same code in federation-util.go
cc @mwielgus @madhusudancs
Automatic merge from submit-queue (batch tested with PRs 39628, 39551, 38746, 38352, 39607)
Increasing times on reconciling volumes fixing impact to AWS.
#**What this PR does / why we need it**:
We are currently blocked by API timeouts with PV volumes. See https://github.com/kubernetes/kubernetes/issues/39526. This is a workaround, not a fix.
**Special notes for your reviewer**:
A second PR will be dropped with CLI cobra options in it, but we are starting with increasing the reconciliation periods. I am dropping this without major testing and will test on our AWS account. Will be marked WIP until I run smoke tests.
**Release note**:
```release-note
Provide kubernetes-controller-manager flags to control volume attach/detach reconciler sync. The duration of the syncs can be controlled, and the syncs can be shut off as well.
```
Added [Volume] tag per issue #35542; added [Flaky] to GCE tests until confirmed fixed. Added [Serial] to NFS to address possible cross test contamination.
Automatic merge from submit-queue (batch tested with PRs 39394, 38270, 39473, 39516, 36243)
Modified run-gcloud-compute-with-retries and used it wherever possible in kubemark
This PR fixes#39335
Simple changes fixing flaky issues within kubemark.
@kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 39394, 38270, 39473, 39516, 36243)
Fix wrong skipf parameter
**How to reproduce**
When run e2e test, it reports `%!!(MISSING)d(MISSING)`:
```
STEP: Checking for multi-zone cluster. Zone count = 1
Dec 6 14:16:43.272: INFO: Zone count is %!!(MISSING)d(MISSING), only run for multi-zone clusters, skipping test
[AfterEach] [k8s.io] Multi-AZ Clusters
```
We need to pass a string parameter to `SkipUnlessAtLeast`
The comment says that the sleep is there because of 10 minute PV controller
sync. The controller sync is now 15 seconds and it should be quick enough
to hide this in subsequent WaitForPersistentVolumeDeleted(.. , 20*time.Minute)
Automatic merge from submit-queue
Updated kubemark with RBAC for controllers, proxy and kubelet
Fixes issue #39244
@kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 39493, 39496)
Use privileged containers for host path e2e tests
Test containers need to run as spc_t in order to interact with the host
filesystem under /tmp, as the tests for HostPath are doing. Docker will
transition the container into this domain when running the container as
privileged.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Currently, this test fails with AVC denials like:
```
time->Thu Jan 5 10:17:51 2017
type=SYSCALL msg=audit(1483629471.846:6623): arch=c000003e syscall=257 success=no exit=-13 a0=ffffffffffffff9c a1=c820010120 a2=80241 a3=1a4 items=0 ppid=4112 pid=4130 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="mt" exe="/mt" subj=system_u:system_r:svirt_lxc_net_t:s0:c123,c328 key=(null)
type=AVC msg=audit(1483629471.846:6623): avc: denied { write } for pid=4130 comm="mt" name="sub-path" dev="xvda2" ino=118491348 scontext=system_u:system_r:svirt_lxc_net_t:s0:c123,c328 tcontext=system_u:object_r:container_runtime_tmp_t:s0 tclass=dir
```
```release-note
NONE
```
/cc @ncdc @pmorie
Test containers need to run as spc_t in order to interact with the host
filesystem under /tmp, as the tests for HostPath are doing. Docker will
transition the container into this domain when running the container as
privileged.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)
Bump etcd to 3.0.14 and switch to v3 API in etcd.
Ref #20504
**Release note**:
```release-note
Switch default etcd version to 3.0.14.
Switch default storage backend flag in apiserver to `etcd3` mode.
```
Automatic merge from submit-queue (batch tested with PRs 39408, 38981)
Remove RBAC UserAll
* Removes special handling of User * subjects in rolebinding matching evaluation
* Converts v1alpha1 rolebindings to `User *` subjects to `Group system:authenticated` subjects for backwards compatibility
```release-note
RBAC's special handling of the User subject named "*" in RoleBinding and ClusterRoleBinding objects is being deprecated and will be removed in v1beta1. Existing v1alpha1 role bindings to User "*" will be converted to the group "system:authenticated". To match unauthenticated requests, RBAC role bindings must explicitly bind to the group "system:unauthenticated".
```
Automatic merge from submit-queue
Ensure invalid token returns 401 error, not 403
fixes#39267
If a user attempts to use a bearer token, and the token is rejected, the authenticator should return an error. This distinguishes requests that did not provide a bearer token (and are unauthenticated without error) from ones that attempted to, and failed.
Automatic merge from submit-queue
Remove jobs that do not exist from active list of CronJob
**What this PR does / why we need it**: This PR modifies the controller for CronJob to remove from the active job list any job that does not exist anymore, to avoid staying blocked in active state forever. See #37957.
**Which issue this PR fixes**: fixes#37957
**Special notes for your reviewer**:
**Release note**:
```
```
Automatic merge from submit-queue (batch tested with PRs 38433, 36245)
Allow pods to define multiple environment variables from a whole ConfigMap
Allow environment variables to be populated from ConfigMaps
- ConfigMaps represent an entire set of EnvVars
- EnvVars can override ConfigMaps
fixes#26299
Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)
Moves e2e service util functions into service_util.go and cleans up
Basically moves codes into a central place for service util functions.
Some other codes are touched mostly only due to this migration. Also put a bunch of network reachability utils functions into network_utils.go. They seem somehow redundant, may consider combine they later.
@bowei @freehan
