Automatic merge from submit-queue
Move .gitattributes annotation to the root, so GitHub will respect them.
This should fix the merge conflicts by letting GitHub use the simpler line-by-line algorithm for this file. Having .gitattributes in a sub-directory would work for local merging, but would show conflicts on the web UI.
Automatic merge from submit-queue
[Federation][join-01] Implement `kubefed join` command.
Supersedes PR #35155.
Please review only the last commit here. This is based on PR #35492 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @mwielgus
Automatic merge from submit-queue
Making the pod.alpha.kubernetes.io/initialized annotation optional in PetSet pods
**What this PR does / why we need it**: As of now, the absence of the annotation `pod.alpha.kubernetes.io/initialized` in PetSets causes the PetSet controller to effectively "pause". Being a debug hook, users expect that its absence has no effect on the working of a PetSet. This PR inverts the logic so that we let the PetSet controller operate as expected in the absence of the annotation.
Letting the annotation remain alpha seems ok. Renaming it to something more meaningful needs further discussion.
**Which issue this PR fixes** _(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)_: fixes https://github.com/kubernetes/kubernetes/issues/35498
**Special notes for your reviewer**:
**Release note**:
``` release-note
The annotation "pod.alpha.kubernetes.io/initialized" on StatefulSets (formerly PetSets) is now optional and only encouraged for debug use.
```
cc @erictune @smarterclayton @bprashanth @kubernetes/sig-apps
@kow3ns The examples will need to be cleaned up as well I think later on to remove them.
Automatic merge from submit-queue
Disable gci-mounter in cri node e2e tests
gci-mounter is still being validated and there are known issues. Do not enable it
for cri tests for now.
Automatic merge from submit-queue
Node controller to not force delete pods
Fixes https://github.com/kubernetes/kubernetes/issues/35145
- [x] e2e tests to test Petset, RC, Job.
- [x] Remove and cover other locations where we force-delete pods within the NodeController.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
Node controller no longer force-deletes pods from the api-server.
* For StatefulSet (previously PetSet), this change means creation of replacement pods is blocked until old pods are definitely not running (indicated either by the kubelet returning from partitioned state, or deletion of the Node object, or deletion of the instance in the cloud provider, or force deletion of the pod from the api-server). This has the desirable outcome of "fencing" to prevent "split brain" scenarios.
* For all other existing controllers except StatefulSet , this has no effect on the ability of the controller to replace pods because the controllers do not reuse pod names (they use generate-name).
* User-written controllers that reuse names of pod objects should evaluate this change.
```
Automatic merge from submit-queue
Disruption e2e: wait for running pods in the table test, too
**What this PR does / why we need it**: Makes the `DisruptionController` tests less flaky.
**Which issue this PR fixes**: fixes#34032
Automatic merge from submit-queue
e2e: Fix GetReadySchedulableNodesOrDie for taints
**What this PR does / why we need it**:
This changes framework.GetReadySchedulableNodesOrDie and
framework.GetMasterAndWorkerNodesOrDie so that nodes that can't take a
generic fake pod due to a taint/toleration mismatch aren't returned.
This is a rehash of #35210, but pulls in the scheduler code.
**Which issue this PR fixes**: c.f. #35210
**Special notes for your reviewer**: I think it's gross that we keep having to manually compute this in e2es. Maybe we need a bug for that?
Automatic merge from submit-queue
Eviction manager evicts based on inode consumption
Fixes: #32526 Integrate Cadvisor per-container inode stats into the summary api. Make the eviction manager act based on inode consumption to evict pods using the most inodes.
This PR is pending on a cadvisor godeps update which will be included in PR #35136
This changes framework.GetReadySchedulableNodesOrDie and
framework.GetMasterAndWorkerNodesOrDie so that nodes that can't take a
generic fake pod due to a taint/toleration mismatch aren't returned.
This is a rehash of #35210, but pulls in the scheduler code.
Run:
etcd &
kube-apiserver --etcd-servers=... ...
UPDATE_NODE_APISERVER go test ./test/integration/master
-test.run=TestUpdateNodeObjects -test.v -tags integration
Simulates the core update loops from nodes to the API server, allowing
baseline profiling for steady state of large clusters. May require
tweaking the http.Transport used by the client to support >N idle
connections to the master.
Automatic merge from submit-queue
remove non-reuseable bits of MasterServer
Scrub `master.go` again. I think I'm pretty happy with this shape. I may promote `InstallAPIs` since we're likely to want it downstream.
Automatic merge from submit-queue
Allow apiserver to choose preferred kubelet address type
Follow up to #33718 to stay compatible with clusters using DNS names for master->node communications. Adds the `--kubelet-preferred-address-types` apiserver flag for clusters that prefer a different node address type.
```release-note
The apiserver can now select which type of kubelet-reported address to use for master->node communications, using the --kubelet-preferred-address-types flag.
```
Automatic merge from submit-queue
Misc master and federation cleanups
- misc small cleanups
- make ServerRunOption embeddings explicit in order to make the technical debt in our plumbing code visible.
Automatic merge from submit-queue
kubeadm: added unit test for app/preflight pkg
Added unit test for kubeadm/app/preflight package testing functionality of checks.go.
This PR is part of the ongoing effort to add tests (#35025)
/cc @pires @jbeda
Automatic merge from submit-queue
Verify and update client-go staging area for every PR
We need to keep the staging area up-to-date to prevent PRs from breaking client-go.
It's marked as "WIP" because we need to decide the [versioning strategy](https://github.com/kubernetes/client-go/issues/9) for client-go first. This PR contains breaking changes for client-go.
This is blocking #29934 and potentially #34441
cc @kubernetes/sig-api-machinery
Automatic merge from submit-queue
Let release_1_5 clientset include multiple versions of a group
Fix#35237
This PR make versioned clientset to include multiple versions of a group. Currently only `batch` has `v1` and `v2alpha1`. The clientset interface now looks like:
```go
BatchV2alpha1() v2alpha1batch.BatchV2alpha1Interface
BatchV1() v1batch.BatchV1Interface
// Deprecated: please explicitly pick a version if possible.
Batch() v1batch.BatchV1Interface
```
Commit "update client-gen to say internalversion rather than unversioned" fixes https://github.com/kubernetes/kubernetes/issues/24481.
cc @kubernetes/sig-api-machinery @soltysh @deads2k @nikhiljindal
```release-note
release_1_5 clientset supports multiple versions of a group.
```
Automatic merge from submit-queue
Make overlapping deployments deletable
@kubernetes/deployment ptal
Fixes https://github.com/kubernetes/kubernetes/issues/34466 by 1) not adding the overlapping annotation in the working deployment, 2) updates observedGeneration for overlapping deployments, and 3) updates the kubectl deployment reaper to do non-cascading deletion for deployments with the overlapping annotation.
Automatic merge from submit-queue
Convert - to _ for protobuf package names
Convert - to _ for protobuf package names to allow protobuf code generation
support for go packages that have - in their names.
@smarterclayton @deads2k @liggitt @sttts @lavalamp @nikhiljindal @kubernetes/sig-api-machinery
Automatic merge from submit-queue
convert SA controller to shared informers
convert the SA controller to shared informer + workqueue.
I think one of @derekwaynecarr @ncdc or @liggitt
Automatic merge from submit-queue
allow authentication through a front-proxy
This allows a front proxy to set a request header and have that be a valid `user.Info` in the authentication chain. To secure this power, a client certificate may be used to confirm the identity of the front proxy
@kubernetes/sig-auth fyi
@erictune per-request
@liggitt you wrote the openshift one, ptal.
Automatic merge from submit-queue
Simplify negotiation in server in preparation for multi version support
This is a pre-factor for #33900 to simplify runtime.NegotiatedSerializer, tighten up a few abstractions that may break when clients can request different client versions, and pave the way for better negotiation.
View this as pure simplification.
Automatic merge from submit-queue
Fixing e2e tests which rely on network disruptions
**What this PR does / why we need it**: It fixes e2e tests on network disruptions
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/27324, https://github.com/kubernetes/kubernetes/issues/35293https://github.com/kubernetes/kubernetes/pull/33655 changed the kubelet's `--api-server` flag to use the external IP address of the APIServer in GCE. Hence, the iptables rules were failing previously. We now return the external IP for `getMaster()`. This fix is required in order to write e2e tests which test behavior in the face of network disruptions.
/cc @bprashanth @dchen1107 @kubernetes/sig-apps
Automatic merge from submit-queue
Add a retry when reading a file content from a container
To avoid temporal failure in reading the file content, add a retry
process in function verifyPDContentsViaContainer
Automatic merge from submit-queue
[PHASE 1] Opaque integer resource accounting.
## [PHASE 1] Opaque integer resource accounting.
This change provides a simple way to advertise some amount of arbitrary countable resource for a node in a Kubernetes cluster. Users can consume these resources by including them in pod specs, and the scheduler takes them into account when placing pods on nodes. See the example at the bottom of the PR description for more info.
Summary of changes:
- Defines opaque integer resources as any resource with prefix `pod.alpha.kubernetes.io/opaque-int-resource-`.
- Prevent kubelet from overwriting capacity.
- Handle opaque resources in scheduler.
- Validate integer-ness of opaque int quantities in API server.
- Tests for above.
Feature issue: https://github.com/kubernetes/features/issues/76
Design: http://goo.gl/IoKYP1
Issues:
kubernetes/kubernetes#28312kubernetes/kubernetes#19082
Related:
kubernetes/kubernetes#19080
CC @davidopp @timothysc @balajismaniam
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Added support for accounting opaque integer resources.
Allows cluster operators to advertise new node-level resources that would be
otherwise unknown to Kubernetes. Users can consume these resources in pod
specs just like CPU and memory. The scheduler takes care of the resource
accounting so that no more than the available amount is simultaneously
allocated to pods.
```
## Usage example
```sh
$ echo '[{"op": "add", "path": "pod.alpha.kubernetes.io~1opaque-int-resource-bananas", "value": "555"}]' | \
> http PATCH http://localhost:8080/api/v1/nodes/localhost.localdomain/status \
> Content-Type:application/json-patch+json
```
```http
HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 11 Aug 2016 16:44:55 GMT
Transfer-Encoding: chunked
{
"apiVersion": "v1",
"kind": "Node",
"metadata": {
"annotations": {
"volumes.kubernetes.io/controller-managed-attach-detach": "true"
},
"creationTimestamp": "2016-07-12T04:07:43Z",
"labels": {
"beta.kubernetes.io/arch": "amd64",
"beta.kubernetes.io/os": "linux",
"kubernetes.io/hostname": "localhost.localdomain"
},
"name": "localhost.localdomain",
"resourceVersion": "12837",
"selfLink": "/api/v1/nodes/localhost.localdomain/status",
"uid": "2ee9ea1c-47e6-11e6-9fb4-525400659b2e"
},
"spec": {
"externalID": "localhost.localdomain"
},
"status": {
"addresses": [
{
"address": "10.0.2.15",
"type": "LegacyHostIP"
},
{
"address": "10.0.2.15",
"type": "InternalIP"
}
],
"allocatable": {
"alpha.kubernetes.io/nvidia-gpu": "0",
"cpu": "2",
"memory": "8175808Ki",
"pods": "110"
},
"capacity": {
"alpha.kubernetes.io/nvidia-gpu": "0",
"pod.alpha.kubernetes.io/opaque-int-resource-bananas": "555",
"cpu": "2",
"memory": "8175808Ki",
"pods": "110"
},
"conditions": [
{
"lastHeartbeatTime": "2016-08-11T16:44:47Z",
"lastTransitionTime": "2016-07-12T04:07:43Z",
"message": "kubelet has sufficient disk space available",
"reason": "KubeletHasSufficientDisk",
"status": "False",
"type": "OutOfDisk"
},
{
"lastHeartbeatTime": "2016-08-11T16:44:47Z",
"lastTransitionTime": "2016-07-12T04:07:43Z",
"message": "kubelet has sufficient memory available",
"reason": "KubeletHasSufficientMemory",
"status": "False",
"type": "MemoryPressure"
},
{
"lastHeartbeatTime": "2016-08-11T16:44:47Z",
"lastTransitionTime": "2016-08-10T06:27:11Z",
"message": "kubelet is posting ready status",
"reason": "KubeletReady",
"status": "True",
"type": "Ready"
},
{
"lastHeartbeatTime": "2016-08-11T16:44:47Z",
"lastTransitionTime": "2016-08-10T06:27:01Z",
"message": "kubelet has no disk pressure",
"reason": "KubeletHasNoDiskPressure",
"status": "False",
"type": "DiskPressure"
}
],
"daemonEndpoints": {
"kubeletEndpoint": {
"Port": 10250
}
},
"images": [],
"nodeInfo": {
"architecture": "amd64",
"bootID": "1f7e95ca-a4c2-490e-8ca2-6621ae1eb5f0",
"containerRuntimeVersion": "docker://1.10.3",
"kernelVersion": "4.5.7-202.fc23.x86_64",
"kubeProxyVersion": "v1.3.0-alpha.4.4285+7e4b86c96110d3-dirty",
"kubeletVersion": "v1.3.0-alpha.4.4285+7e4b86c96110d3-dirty",
"machineID": "cac4063395254bc89d06af5d05322453",
"operatingSystem": "linux",
"osImage": "Fedora 23 (Cloud Edition)",
"systemUUID": "D6EE0782-5DEB-4465-B35D-E54190C5EE96"
}
}
}
```
After patching, the kubelet's next sync fills in allocatable:
```
$ kubectl get node localhost.localdomain -o json | jq .status.allocatable
```
```json
{
"alpha.kubernetes.io/nvidia-gpu": "0",
"pod.alpha.kubernetes.io/opaque-int-resource-bananas": "555",
"cpu": "2",
"memory": "8175808Ki",
"pods": "110"
}
```
Create two pods, one that needs a single banana and another that needs a truck load:
```
$ kubectl create -f chimp.yaml
$ kubectl create -f superchimp.yaml
```
Inspect the scheduler result and pod status:
```
$ kubectl describe pods chimp
Name: chimp
Namespace: default
Node: localhost.localdomain/10.0.2.15
Start Time: Thu, 11 Aug 2016 19:58:46 +0000
Labels: <none>
Status: Running
IP: 172.17.0.2
Controllers: <none>
Containers:
nginx:
Container ID: docker://46ff268f2f9217c59cc49f97cc4f0f085d5ac0e251f508cc08938601117c0cec
Image: nginx:1.10
Image ID: docker://sha256:82e97a2b0390a20107ab1310dea17f539ff6034438099384998fd91fc540b128
Port: 80/TCP
Limits:
cpu: 500m
memory: 64Mi
pod.alpha.kubernetes.io/opaque-int-resource-bananas: 3
Requests:
cpu: 250m
memory: 32Mi
pod.alpha.kubernetes.io/opaque-int-resource-bananas: 1
State: Running
Started: Thu, 11 Aug 2016 19:58:51 +0000
Ready: True
Restart Count: 0
Volume Mounts: <none>
Environment Variables: <none>
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
No volumes.
QoS Class: Burstable
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
9m 9m 1 {default-scheduler } Normal Scheduled Successfully assigned chimp to localhost.localdomain
9m 9m 2 {kubelet localhost.localdomain} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
9m 9m 1 {kubelet localhost.localdomain} spec.containers{nginx} Normal Pulled Container image "nginx:1.10" already present on machine
9m 9m 1 {kubelet localhost.localdomain} spec.containers{nginx} Normal Created Created container with docker id 46ff268f2f92
9m 9m 1 {kubelet localhost.localdomain} spec.containers{nginx} Normal Started Started container with docker id 46ff268f2f92
```
```
$ kubectl describe pods superchimp
Name: superchimp
Namespace: default
Node: /
Labels: <none>
Status: Pending
IP:
Controllers: <none>
Containers:
nginx:
Image: nginx:1.10
Port: 80/TCP
Requests:
cpu: 250m
memory: 32Mi
pod.alpha.kubernetes.io/opaque-int-resource-bananas: 10Ki
Volume Mounts: <none>
Environment Variables: <none>
Conditions:
Type Status
PodScheduled False
No volumes.
QoS Class: Burstable
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
3m 1s 15 {default-scheduler } Warning FailedScheduling pod (superchimp) failed to fit in any node
fit failure on node (localhost.localdomain): Insufficient pod.alpha.kubernetes.io/opaque-int-resource-bananas
```
- Prevents kubelet from overwriting capacity during sync.
- Handles opaque integer resources in the scheduler.
- Adds scheduler predicate tests for opaque resources.
- Validates opaque int resources:
- Ensures supplied opaque int quantities in node capacity,
node allocatable, pod request and pod limit are integers.
- Adds tests for new validation logic (node update and pod spec).
- Added e2e tests for opaque integer resources.
Alter how runtime.SerializeInfo is represented to simplify negotiation
and reduce the need to allocate during negotiation. Simplify the dynamic
client's logic around negotiating type. Add more tests for media type
handling where necessary.
Automatic merge from submit-queue
Update rkt version on GCI nodes to v1.18.0
v1.18.0 avoids outputting debug information by default which happens to
pollute events and kubelet logs.
Automatic merge from submit-queue
Add hack/verify-test-owners.sh to ensure tests always have owners.
This ensures that new tests or changed tests are assigned appropriate owners.
Automatic merge from submit-queue
Swap in tests ownership
To make the test ownership more closer to actual area of expertise I made the following swap. I included @mtaufen to close the cycle. Please wait with applying lgtm label for the second reviewer.
Automatic merge from submit-queue
remove the non-generated client
Removes the non-generated client from kube. The package has a few methods left, but nothing that needs updating when adding new groups.
@ingvagabund
Automatic merge from submit-queue
Fixed gcloud command in logs-generator makefile
I grepped through the code looking for `gcloud` and `push` commands and only found one Makefile missing the `--`. I added it.
fixes#33765🐛
Automatic merge from submit-queue
Set done to true & return error if RestartPolicy not Always in test framework
Found a small issue with https://github.com/kubernetes/kubernetes/pull/34632, it returns an error if the RestartPolicy is not Always, but the user will never see it because done isn't set to true & they will timeout instead.
@Random-Liu because you wrote that PR
Automatic merge from submit-queue
Adding cascading deletion support to federated namespaces
Ref https://github.com/kubernetes/kubernetes/issues/33612
With this change, whenever a federated namespace is deleted with `DeleteOptions.OrphanDependents = false`, then federation namespace controller first deletes the corresponding namespaces from all underlying clusters before deleting the federated namespace.
cc @kubernetes/sig-cluster-federation @caesarxuchao
```release-note
Adding support for DeleteOptions.OrphanDependents for federated namespaces. Setting it to false while deleting a federated namespace also deletes the corresponding namespace from all registered clusters.
```
Automatic merge from submit-queue
rename kubelet flag mounter-path to experimental-mounter-path
```release-note
* Kubelet flag '--mounter-path' renamed to '--experimental-mounter-path'
```
The feature the flag controls is an experimental feature and this renaming ensures that users do not depend on this feature just yet.
Automatic merge from submit-queue
Add multiple PV/PVC pair handling to persistent volume e2e test
Adds the framework for creating, validating, and deleting groups of PVs and PVCs.
Automatic merge from submit-queue
Speed up some networking tests in large clusters
Since we are getting towards testing larger and larger clusters (hopefully 5000-node ones soon-ish), I'm trying to limit the amount of super long tests to minimum.
This should significantly reduce amount of time used by those from test/e2e/networking.go.
@gmarek
Automatic merge from submit-queue
Remove versioned LabelSelectors
We have LabelSelectors defined in `unversioned`, `batch/v1`, `batch/v2alpha1`, and `extensions/v1beta1`. Their definitions are all the same. I kept the definition in `unversioned` and removed the others. It only makes sense to define a versioned LabelSelectors if the definition is different.
Automatic merge from submit-queue
Marked NodeOutOfDisk test with feature label
Marked NodeOutOfDisk test with feature label to temporarily remove it from flaky suite.
@madhusudancs @piosz
Automatic merge from submit-queue
always clean gce resources in service e2e
@bprashanth the previous PR was closed when I squashed my commits.
Here is the new change set, please help to review again.
1). only the following two It() create, I created a string array to persist the LB name so that they can be cleaned in AfterEach(), and the string array was reset after clean up.
```
"should be able to change the type and ports of a service [Slow]"
"should be able to create services of type LoadBalancer and externalTraffic=localOnly"
```
2). Directly call gce api to delete the resource and ignore any error returned.
Automatic merge from submit-queue
CRI: Add dockershim grpc server.
This PR adds a in-process grpc server for dockershim.
Flags change:
1. `container-runtime` will not be automatically set to remote when `container-runtime-endpoint` is set. @feiskyer
2. set kubelet flag `--experimental-runtime-integration-type=remote --container-runtime-endpoint=UNIX_SOCKET_FILE_PATH` to enable the in-process dockershim grpc server.
3. set node e2e test flag `--runtime-integration-type=remote -container-runtime-endpoint=UNIX_SOCKET_FILE_PATH` to run node e2e test against in-process dockershim grpc server.
I've run node e2e test against the remote cri integration, tests which don't rely on stream and log functions can pass.
This unblocks the following work:
1) CRI conformance test.
2) Performance comparison between in-process integration and in-process grpc integration.
@yujuhong @feiskyer
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Make hack/update_owners.py get list from local repo, add --check option.
This should become a verify step soon. The munger understands the * syntax already.
Automatic merge from submit-queue
test-images: server address is now configurable
This commit perform changes discussed in #32128
Should be merged before #35301
cc: @pskrzyns
Automatic merge from submit-queue
Add a retry when reading a file content from a container
To avoid temporal failure in reading the file content, add a retry
process in function verifyPDContentsViaContainer
Automatic merge from submit-queue
e2e node plumbing and bundling for GCI mounter
**Note:** The code in this PR only bundles the mounter and modifies `--mounter-path` if it can find `cluster/gce/gci/mounter` in the K8s source dir when building the test bundle.
This bundles the mounter script for GCI with the node e2e tests and allows the `--mounter-path` to be passed to the Kubelet via the node test framework. The node test runner will detect when we are running on a remote GCI node and add the appropriate `--mounter-path` to the `testArgs`.
It also includes a simple node test that mounts a tmpfs volume. This will exercise the Kubelet's mounter code path.
**ITEM OF NOTE:** To get the k8s root dir (in order to copy the mount script into the tarball), I changed `getK8sRootDir` -> `GetK8sRootDir` in `test/e2e_node/build/build.go`. Based on the comment above that function (and the fact that it was private to begin with), I'm not sure this is the best way to do things:
```
// TODO: Dedup / merge this with comparable utilities in e2e/util.go
```
On the other hand, the `e2e/util.go` file mentioned in that comment doesn't exist anymore. This should be resolved before this PR is merged.
Automatic merge from submit-queue
CRI: Add cri test on containervm.
As is discussed with @yujuhong, we need to validate cri on containervm.
@yujuhong @feiskyer
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Fixing a typo in federated secrets test
Realized that our secret e2e test was not running due to a typo in my last PR :)
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
service e2e: remove TODO and subtle changes in logging
Removes the stale `TODO` for external source IP preservation as the e2e test of ESIPP was added.
Changes logging in create service functions: namespace/namespace -> namespace/serviceName.
@bprashanth
Automatic merge from submit-queue
Update elasticsearch and kibana usage
```release-note
Updated default Elasticsearch and Kibana used for elasticsearch logging destination to versions 2.4.1 and 4.6.1 respectively.
```
Updated controllers for elasticsearch and kibana to use newer versions of images. Fixed e2e test because of elasticsearch backward incompatible API changes.
Fixed out of sync elasticsearch controller for coreos.
@piosz
Automatic merge from submit-queue
Loadbalanced client src ip preservation enters beta
Sounds like we're going to try out the proposal (https://github.com/kubernetes/kubernetes/issues/30819#issuecomment-249877334) for annotations -> fields on just one feature in 1.5 (scheduler). Or do we want to just convert to fields right now?
Automatic merge from submit-queue
Verify petset status.replicas in e2e test
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**: follow up #33983. PetSet status.replicas bug is fixed, so adding tests for it (especially for the `should handle healthy pet restarts during scale` case)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: cc @erictune @foxish @kubernetes/sig-apps
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Automatic merge from submit-queue
Set non-always RestartPolicy for write-pod in pv e2e
Due to https://github.com/kubernetes/kubernetes/pull/34632 the RestartPolicy can't be Always (& it shouldn't be anyway)
Automatic merge from submit-queue
Add secret e2e test for keys mapping
**What this PR does / why we need it**: Adds a basic e2e test missing in secrets
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```NONE
```
This patch adds a secret e2e test. While configmap e2e tests are far
more complete, this patch makes secret e2e tests one step closer.
Also, now is more easy to add more tests without code duplication (as I
did in earlier patches :-/), because of the functions created, and is
more easy to make it similar to confimap e2e in the future, that is
really complete.
Automatic merge from submit-queue
Adding default StorageClass annotation printout for resource_printer and describer and some refactoring
adding ISDEFAULT for _kubectl get storageclass_ output
```
[root@screeley-sc1 gce]# kubectl get storageclass
NAME TYPE ISDEFAULT
another-class kubernetes.io/gce-pd NO
generic1-slow kubernetes.io/gce-pd YES
generic2-fast kubernetes.io/gce-pd YES
```
```release-note
Add ISDEFAULT to kubectl get storageClass output
```
@kubernetes/sig-storage
Commit 53ec6e6 missed to remove this deferred call (probably due to a
rebase).
I noticied it while working with the code. I'm not sure why the original
commits removes them, but it seems the right thing to do.
This patch adds a secret e2e test. While configmap e2e tests are far
more complete, this patch makes secret e2e tests one step closer.
Also, now is more easy to add more tests without code duplication (as I
did in earlier patches :-/), because of the functions created, and is
more easy to make it similar to confimap e2e in the future, that is
really complete.
Most normal codec use should perform defaulting. DirectCodecs should not
perform defaulting. Update the defaulting_test to fuzz the list of known
defaulters. Use the new versioning.NewDefaultingCodec() method.
Automatic merge from submit-queue
Remove static kubelet client, refactor ConnectionInfoGetter
Follow up to https://github.com/kubernetes/kubernetes/pull/33718
* Collapses the multi-valued return to a `ConnectionInfo` struct
* Removes the "raw" connection info method and interface, since it was only used in a single non-test location (by the "real" connection info method)
* Disentangles the node REST object from being a ConnectionInfoProvider itself by extracting an implementation of ConnectionInfoProvider that takes a node (using a provided NodeGetter) and determines ConnectionInfo
* Plumbs the KubeletClientConfig to the point where we construct the helper object that combines the config and the node lookup. I anticipate adding a preference order for choosing an address type in https://github.com/kubernetes/kubernetes/pull/34259
Automatic merge from submit-queue
make version an explicit choice so zero config and customized work
Makes `/version` key off of setting the version. This allows composers to add a version that is correct.
Automatic merge from submit-queue
e2e: stop tracking resource usage for the "misc" container
There is e2e test checking the resource usage of "misc", and it is not
supported on GCI.
Automatic merge from submit-queue
New client-go structure
This PR is part of restructuring client-go (https://github.com/kubernetes/client-go/issues/9#issue-181545998). In short, the top-level folder for client-go versions are removed.
This PR also runs copy.sh to pick up changes in the main repository. The number of files in client-go has increase from 1361 files to 1405.
@mbohlool @mml @timoreimann
Automatic merge from submit-queue
Add test_list command, to enumerate unit and e2e tests.
This uses go/parser and go/ast to analyze all test files in ~1 second.
It only recognizes a few simple structures that the tests all have, and
modifies a few tests to fit expected structure better.
This is part of an effort to ensure all tests have owners, by having a
verify check to catch new tests being added without an owner.
This uses go/parser and go/ast to analyze all test files in ~1 second.
It only recognizes a few simple structures that the tests all have, and
modifies a few tests to fit expected structure better.
This is part of an effort to ensure all tests have owners, by having a
verify check to catch new tests being added without an owner.
Automatic merge from submit-queue
Make E2E tests easier to debug
This PR removes deferred deletion calls in E2E tests in order to make test easier to debug. Some of these tests predate namespace finalization; we should just rely on that mechanism to ensure that framework test artifacts are deleted.
Automatic merge from submit-queue
controller: set minReadySeconds in deployment's replica sets
* Estimate available pods for a deployment by using minReadySeconds on
the replica set.
* Stop requeueing deployments on pod events, superseded by following the
replica set status.
* Cleanup redundant deployment utilities
Fixes https://github.com/kubernetes/kubernetes/issues/26079
@kubernetes/deployment ptal
Automatic merge from submit-queue
default serializer
Everyone uses the same serializer. Set it as the default, but still allow someone to take control if they want.
Found while trying to use genericapiserver for composition.
Automatic merge from submit-queue
Cleanup the commented code for overriding flags with viper. For now,…
Minor cleanup for the viper configuration logic, removes commented code into a function of its own. We can decide wether or not to overwrite flag values at a later time...
Automatic merge from submit-queue
Run rbac authorizer from cache
RBAC authorization can be run very effectively out of a cache. The cache is a normal reflector backed cache (shared informer).
I've split this into three parts:
1. slim down the authorizer interfaces
1. boilerplate for adding rbac shared informers and associated listers which conform to the new interfaces
1. wiring
@liggitt @ericchiang @kubernetes/sig-auth
Automatic merge from submit-queue
Move RunRC-like functions to test/utils
Ref. #34336
cc @timothysc - the "move" part of the small refactoring. @jayunit100
Automatic merge from submit-queue
e2e: don't require minimum availability once scaling takes place
This test shouldn't care about availability at all in the first place.
@mfojtik @kubernetes/deployment ptal
Fixes https://github.com/kubernetes/kubernetes/issues/34717
Automatic merge from submit-queue
split genericapiserver configuration apart so that you can run withou…
…t flag options
@dims Take a look at this re-slicing of the `genericapiserver.Config` creation. I think this helps composers overall and resolves the chicken/egg problem you were having.
Automatic merge from submit-queue
Update scheduler perf to spin up similar client to other tests
Seems to Fix#34504 . But I'm not sure what the mechanics of the underlying client objects is all supposed to be.
Automatic merge from submit-queue
Add e2e tests for storageclass
- test pd-ssd and pd-standard on GCE,
- test all four volume types and encryption on AWS
- test just the default volume type on OpenStack (right now, there is no API
to get list of them)
These tests are quite slow, e.g. there are two tests on AWS that has to run mkfs.ext4 on 500 GB magnetic drive with low IOPS, which takes ~3-4 minutes each.
Automatic merge from submit-queue
Fix race condition in test with git server startup
Fixes https://github.com/kubernetes/kubernetes/issues/32467 (hopefully)
Previously, the test didn't ensure the git server was running before attempting to connect to it.
Automatic merge from submit-queue
Change legacy API resource registration
Updates the legacy API resource registration to emphasize its different-ness and to simplify supporting objects. The option has to remain in the genericapiserverconfig for multiple prefixes to enable cases where composers/extenders had composed additional groupless APIs. See OpenShift as an example.
However this is now transparent to "normal" composers.
@ncdc since sttts is out.
Automatic merge from submit-queue
Update `gcloud docker` commands to use `gcloud docker -- ARGS`
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```
- test pd-ssd and pd-standard on GCE,
- test all four volume types on AWS
- test just the default volume type on OpenStack (right now, there is no API
to get list of them)
* Estimate available pods for a deployment by using minReadySeconds on
the replica set.
* Stop requeueing deployments on pod events, superseded by following the
replica set status.
* Cleanup redundant deployment utilities
Automatic merge from submit-queue
Fix leaking ingress resources in federated ingress e2e test.
Originally the federated ingresses were being deleted, but due to the lack of cascading deletion, the cluster ingresses were never being deleted, leading to leaked GCE loadbalancer resources. This fixes that.
Automatic merge from submit-queue
Generate unique Operation IDs for root OpenAPI spec
This PR adds a customization method GetOperationID to OpenAPI spec generation and then use it to make sure root spec has unique operation IDs by mostly adding GroupVersion to the start of operation ID.
Automatic merge from submit-queue
Ignore mirror pods with RestartPolicy == Never in restart tests
Kubelet does not sync the mirror pods once they have terminated. If, for some
reason, that such mirror pods get deleted once they have terminated (either by
the node controller or by users), kubelet will not attempt to recreate them.
However, when kubelet restarts, it will examine the static pods, sync once,
and create a mirror pod. This has led to unexpected pod counts in disruptive
tests where kubelet gets restarted on purpose (see #34003).
This change disregard such mirror pods when totaling the pods to fix the test
flake until we have time to implement a long-term solution.
This PR addresses #34003
Automatic merge from submit-queue
Add gcl cluster logging test
This PR changes default logging destination for tests to gcp and adds test for cluster logging using google cloud logging
Fix#20760
Automatic merge from submit-queue
test: move deployment deletion in its own test
@kubernetes/deployment this PR moves the deletion of deployment in its own test (no need to delete deployments in every test since the namespace controller does that for us once the namespace gets deleted after the test finishes)
Fixes https://github.com/kubernetes/kubernetes/issues/33256
Automatic merge from submit-queue
Make logging function injectable in RCConfig
Ref. #34336
cc @timothysc - the "move" part of the small refactoring. @jayunit100
Automatic merge from submit-queue
clean api server cruft
Some cruft has developed over refactors. Remove that cruft.
@liggitt probably last in the chain so far
Automatic merge from submit-queue
remove [Conformance] flag on some e2es
Downstream distributions that absorb the upstream tests would like to give their customers a standard mechanism to validate their clusters, post setup. As of today [Conformance] works for most things, but there are a known set of tests that vary due to opinionated differences around networking, security, etc... and providing a complete skip list can be cumbersome. To address this, we've simply modified the flag on some tests to [Conformance:Variant]. All existing behavior should be maintained.
Fixes: #34105
Kubelet does not sync the mirror pods once they have terminated. If, for some
reason, that such mirror pods get deleted once they have terminated (either by
the node controller or by users), kubelet will not attempt to recreate them.
However, when kubelet restarts, it will examine the static pods, sync once,
and create a mirror pod. This has led to unexpected pod counts in disruptive
tests where kubelet gets restarted on purpose (see #34003).
This change disregard such mirror pods when totaling the pods to fix the test
flake until we have time to implement a long-term solution.
Automatic merge from submit-queue
Kubelet: Use RepoDigest for ImageID when available
```release-note
Use manifest digest (as `docker-pullable://`) as ImageID when available (exposes a canonical, pullable image ID for containers).
```
Previously, we used the docker config digest (also called "image ID"
by Docker) for the value of the `ImageID` field in the container status.
This was not particularly useful, since the config manifest is not
what's used to identify the image in a registry, which uses the manifest
digest instead. Docker 1.12+ always populates the RepoDigests field
with the manifest digests, and Docker 1.10 and 1.11 populate it when
images are pulled by digest.
This commit changes `ImageID` to point to the the manifest digest when
available, using the prefix `docker-pullable://` (instead of
`docker://`)
Related to #32159
Automatic merge from submit-queue
Run flaky tests in parallel
We should try to emulate the main CI environment in the flaky test suite so that it is clear when a test can be moved out of the flaky suite. Since a common source of flakes is unintended interactions between tests running in parallel, we should run the flaky suite in parallel to better detect such flakes.
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```
Signed-off-by: Jess Frazelle <acidburn@google.com>
Previously, we used the docker config digest (also called "image ID"
by Docker) for the value of the `ImageID` field in the container status.
This was not particularly useful, since the config manifest is not
what's used to identify the image in a registry, which uses the manifest
digest instead. Docker 1.12+ always populates the RepoDigests field
with the manifest digests, and Docker 1.10 and 1.11 populate it when
images are pulled by digest.
This commit changes `ImageID` to point to the the manifest digest when
available, using the prefix `docker-pullable://` (instead of
`docker://`)
Automatic merge from submit-queue
refactor controller as posthook when configured and enabled
builds on https://github.com/kubernetes/kubernetes/pull/33785 .
Models the bootstrap controller as a PostHook which is added when its API group is available and its configured.
@liggitt Stefan is out for a while.
Automatic merge from submit-queue
Fix confusing log messages
While debugging https://github.com/kubernetes/kubernetes/issues/33876 , I noticed following confusing message:
```
The status of Pod kibana-logging-v1-j99la is Running, waiting for it to be either Running or Failed
```
Automatic merge from submit-queue
annotate some addtional errors in e2e tests
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Adds some additional context to e2e test failures.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: some work toward #34059
**Special notes for your reviewer**: I didn't want to go through all of the offending cases so I picked off a few files and addressed those.
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
NONE
```
Automatic merge from submit-queue
remove testapi.Default.GroupVersion
I'm going to try to take this as a series of mechanicals. This removes `testapi.Default.GroupVersion()` and replaces it with `registered.GroupOrDie(api.GroupName).GroupVersion`.
@caesarxuchao I'm trying to see how much of `pkg/api/testapi` I can remove.
Automatic merge from submit-queue
Update GCI_VERSION to gci-dev-55-8866-0-0
Update GCI base image:
Change log:
* Built-in kubernetes updated to v1.4.0
* Enabled VXLAN and IP_SET config options in kernel to support some networking tools
* OpenSSL CVE fixes
```release-note
Update GCI base image:
* Enabled VXLAN and IP_SET config options in kernel to support some networking tools (ebtools)
* OpenSSL CVE fixes
```
cc/ @kubernetes/goog-image cc/ @dchen1107
Automatic merge from submit-queue
try to use ifdown/ifup if available
Tried this on both ContainerVM and GCI image.
`ip link set eth0 down` is too destructive for containerVM. It could not recover with correct network setup hence failing the test. Need to use ifdown/ifup on containerVM.
reference:
http://serverfault.com/questions/603906/ip-link-set-not-assigning-ip-address-but-ifup-does
Automatic merge from submit-queue
Revert "Add kubelet awareness to taint tolerant match caculator."
Reverts kubernetes/kubernetes#26501
Original PR was not fully reviewed by @kubernetes/sig-node
cc/ @timothysc @resouer
Automatic merge from submit-queue
Kubelet: Use RepoDigest for ImageID when available
**Release note**:
```release-note
Use manifest digest (as `docker-pullable://`) as ImageID when available (exposes a canonical, pullable image ID for containers).
```
Previously, we used the docker config digest (also called "image ID"
by Docker) for the value of the `ImageID` field in the container status.
This was not particularly useful, since the config manifest is not
what's used to identify the image in a registry, which uses the manifest
digest instead. Docker 1.12+ always populates the RepoDigests field
with the manifest digests, and Docker 1.10 and 1.11 populate it when
images are pulled by digest.
This commit changes `ImageID` to point to the the manifest digest when
available, using the prefix `docker-pullable://` (instead of
`docker://`)
Related to #32159
Automatic merge from submit-queue
Add kubelet awareness to taint tolerant match caculator.
Add kubelet awareness to taint tolerant match caculator.
Ref: #25320
This is required by `TaintEffectNoScheduleNoAdmit` & `TaintEffectNoScheduleNoAdmitNoExecute `, so that node will know if it should expect the taint&tolerant
Automatic merge from submit-queue
Don't set timeouts in clients in tests
We are not setting timeouts in production - we shouldn't do it in tests then...
Addresses point 2. of #31345
Automatic merge from submit-queue
add delete-namespace-on-failure flag
I have been doing this for a while.
Setting `--delete-namespace=false --clean-start=true` only works if you have only one e2e test running in a loop.
This PR lets someone to set `delete-namespace-on-failure=false` and run multiple tests in parallel and preserve the crime scene. It makes it easier to reproduce failures.
Let me know if this is worth it or there are some other tricks I am not aware.
Automatic merge from submit-queue
Delete federation namespace after the test completes
The code was commented because of a bug in namespace deletion which is now fixed.
Note that this deletes the namespace in federation control plane. We still need to delete the namespace from each cluster (cascading deletion)
cc @kubernetes/sig-cluster-federation
Changelog:
* Built-in kubernetes updated to v1.4.0
* Enabled VXLAN and IP_SET config options in kernel to support some networking tools
* OpenSSL CVE fixes
Automatic merge from submit-queue
Revert "Revert "move pod networking tests common""
Reverts #34011
And fix the problem causing `Granular Checks: Services [Slow] should update nodePort` tests to fail
Automatic merge from submit-queue
CRI: Add serial and benchmark test suite.
For https://github.com/kubernetes/kubernetes/issues/31459.
The serial test result will be shown on test-grid.
The benchmark test result will be shown [node-perf-dash](http://node-perf-dash.k8s.io/#/builds)
This PR also changes the cri validation test to use the same gci image with node e2e instead of the canary image. The docker version is still 1.11.2.
@yujuhong @feiskyer @yifan-gu
/cc @kubernetes/sig-node
Automatic merge from submit-queue
CRI: Add presubmit CRI validation test.
For #31459.
This PR adds a new suite for CRI presubmit validation which runs non-flaky, non-serial, non-slow test per-pr.
Except this PR, I'll also change the test-infra side. Ideally, after this is done, we should be be able to trigger CRI validation test per-pr with something like `@k8s-bot cri node e2e test this` and `@k8s-bot cri e2e test this`.
@yujuhong @feiskyer @yifan-gu @freehan
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Add a retry loop to KubeletManagerEtcHosts flake
Add a retry loop to KubeletManagerEtcHosts flake
Note: this still makes the test fail if a retry occurs, but
will give us more information regarding whether or not the
test flake could be occuring due to delay in mounting of
/etc/hosts.
Automatic merge from submit-queue
Fix#31085, include output checking in retry loop
Fix#31085.
`wget -qO-` might omit the error and `RunHostCmd()` won't be able to catch it sometimes. Verify stdout is not empty within the retry loop to fix. Test passed 1500+ times in a row.
@freehan
Note: this still makes the test fail if a retry occurs, but
will give us more information regarding whether or not the
test flake could be occuring due to delay in mounting of
/etc/hosts.
Automatic merge from submit-queue
Check for empty string post trimming
We curl in a retry loop and timeout, trimming stdout to find endpoint names. When curl hits the timeout, stdout is empty, so we insert the empty string into the received set of endpoints.
Fixes https://github.com/kubernetes/kubernetes/issues/32684
Automatic merge from submit-queue
Improve source ip preservation test, fail the test instead of panic.
From #31085.
The source IP preserve test starts to be flake again. Sending out this PR to get rid of panicing and log the unexpected output for future investigation.
@freehan
Automatic merge from submit-queue
fix loopback client config for integration tests
Wires an authenticator/authorizer pair much closer to what the actual master start does. Several pulls I have in progress require this.
@liggitt second commit wiring changes.
Automatic merge from submit-queue
[Client-gen] Let versioned client use versioned options
i.e., use v1.ListOptions, v1.DeleteOptions when possible.
Remove the extension/v1beta1.ListOptions, because it's exactly the same as v1.ListOptions, and is not referred throughout the code base. After its removal, I register v1.ListOptions during extensions/v1beta1 scheme registration.
First three commits are manual changes.
Fix#27753
cc @lavalamp
Automatic merge from submit-queue
Add a check for file size if the reading content returns empty
In order to debug the flaky tests for writing/reading files via
contains, this PR adds a check for file size if reading returns empty
content.
Automatic merge from submit-queue
Fix kubelet perf data to make it work again for perfdash.
Addresses https://github.com/kubernetes/kubernetes/pull/30333#issuecomment-248791257.
Add the "node" label back to fix kubelet perf dash. At least for now, we still need original perfdash to catch summary api performance regression.
/cc @coufon @yujuhong
Previously, we used the docker config digest (also called "image ID"
by Docker) for the value of the `ImageID` field in the container status.
This was not particularly useful, since the config manifest is not
what's used to identify the image in a registry, which uses the manifest
digest instead. Docker 1.12+ always populates the RepoDigests field
with the manifest digests, and Docker 1.10 and 1.11 populate it when
images are pulled by digest.
This commit changes `ImageID` to point to the the manifest digest when
available, using the prefix `docker-pullable://` (instead of
`docker://`)
Automatic merge from submit-queue
Heal the namespaceless ingresses in federation e2e.
For createIngressOrFail, it incorrectly returned the ingress passed in as an argument, which does not include the namespace, instead of the ingress returned from the create call (which does).
This in turn leads to errors in e2e tests like this:
INFO: Waiting for Ingress federated-ingress to acquire IP, error an empty namespace may not be set when a resource name is provided.
Self-applying LGTM label, as this is the same code that was LGTM'd by @nikhiljindal in #33502
Automatic merge from submit-queue
Delete evicted pet
If pet was evicted by kubelet - it will stuck in this state forever.
By analogy to regular pod we need to re-create pet so that it will
be re-scheduled to another node, so in order to re-create pet
and preserve consitent naming we will delete it in petset controller
and create after that.
fixes: https://github.com/kubernetes/kubernetes/issues/31098
Automatic merge from submit-queue
Deregister clusters during federated namespace e2e tear down.
This is causing other tests to leak resources.
cc @mwielgus @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Fix summary test
Issue was comparing an `unversioned.Time` rather than `time.Time`. I temporarily removed the `[Flaky]` tag so the PR builder will run the test. I will revert that change before submitting.
Automatic merge from submit-queue
fix kubectl taint e2e flake: add retries for removing taint
**What this PR does / why we need it**:
Why we need it: recent failures occurred in #29503 are caused by taints removing conflict on nodes, this PR is to fix it. (#33073 fixed taints updating conflict, but not taints removing.)
What this PR does: use `runKubectlRetryOrDie()` instead of `RunKubectlOrDie()` in all the places in "Kubectl taint" e2e tests.
**Which issue this PR fixes** : fixes part of #29503, (would like to keep this issue open for some days more to make sure no other failures occur)
**Special notes for your reviewer**: NONE
**Release note**: NONE
Automatic merge from submit-queue
Provide more detailed error message when pod fails to be success state
Return more detailed error message when it timeouts for waiting for pod to success state.
This PR extends #31895. #31895 is for running and ready state.
It is used for debugging #31561#32980
cc: @lavalamp
Automatic merge from submit-queue
pass loopback config to poststart hooks
Updates post start hooks to take a clientconfig with the new loopback credentials for bootstrapping.
@ericchiang This is a little bit of plumbing, but mainly auth I think.
Automatic merge from submit-queue
Add ECDSA support for service account tokens
Fixes#28180
```release-note
ECDSA keys can now be used for signing and verifying service account tokens.
```
Automatic merge from submit-queue
Increase timeout for federated ingress test.
Right now federated ingress e2e takes more than 1 minute, as we need to wait for the first clusters ingress to have an IP address allocated to it before creating the others. Sometimes this takes a while due to GCE loadbalancer backend delays.
Automatic merge from submit-queue
move core storage out of master.go
Moves the core resource creation out of master.go and makes it more congruent to the other storages.
WIP because I haven't run tests yet, but I figured I'd see what breaks in the morning.
Automatic merge from submit-queue
Change minion to node
Continuation of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed - just let me know.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Automatic merge from submit-queue
Rewrite summary e2e test to check metric sanity
Take two, forked from https://github.com/kubernetes/kubernetes/pull/28195
Adds a test library that extends the ginkgo matchers to check nested data structures. Then uses the new matcher library to thoroughly check the validity of every field in the summary metrics API. This approach is more flexible than the previous approach since it allows for different tests per-field, and is easier to add case-by-case exceptions. It also places the lower & upper bounds side-by-side, making the test much easier to read & reason about.
Most fields are expected to be within some bounds. This is not intended to be a performance test, so metric bounds are very loose. Rather, I'm looking to check that the values are sane to catch bugs like #27194Fixes#23411, https://github.com/kubernetes/kubernetes/issues/31989
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Make the restart test restart the nodes without a mig rolling update.
This is one approach to fix#33113. I switched from using a mig rolling-update to just pushing the reset button on the nodes and then waiting for their boot IDs to change.
Contination of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
I rolled back some of this from a previous commit because it just got
to big/messy. Will follow up with additional PRs
Signed-off-by: Doug Davis <dug@us.ibm.com>
Automatic merge from submit-queue
Fix TestCreateWithNonExistentOwner
Fix#30228
As https://github.com/kubernetes/kubernetes/issues/30228#issuecomment-248779567 described, the GC did delete the garbage, it's the test logic failed.
The test used to rely on `gc.QueuesDrained()`, which could return before the GC finished processing. It seems to be the only possible reason of the test failure. Hence, this PR changed the test to poll for the deletion of garbage.
Automatic merge from submit-queue
Use strongly-typed types.NodeName for a node name
We had another bug where we confused the hostname with the NodeName.
Also, if we want to use different values for the Node.Name (which is
an important step for making installation easier), we need to keep
better control over this.
A tedious but mechanical commit therefore, to change all uses of the
node name to use types.NodeName
Automatic merge from submit-queue
Refactor volume controller parameters into a structure
`persistentvolumecontroller.NewPersistentVolumeController` has 11 arguments now,
put them into a structure.
Also, rename `NewPersistentVolumeController` to `NewController`, `persistentvolume`
is already name of the package.
Fixes#30219
We had another bug where we confused the hostname with the NodeName.
To avoid this happening again, and to make the code more
self-documenting, we use types.NodeName (a typedef alias for string)
whenever we are referring to the Node.Name.
A tedious but mechanical commit therefore, to change all uses of the
node name to use types.NodeName
Also clean up some of the (many) places where the NodeName is referred
to as a hostname (not true on AWS), or an instanceID (not true on GCE),
etc.
Automatic merge from submit-queue
start creating controller SA roles. start with just one
This creates a clusterrole for the replicationcontroller controller. It also streamlines the rule creation code and I'll use this role as practice for wiring up RBAC rules.
@kubernetes/sig-auth
@ericchiang Jordan is ooto, mind taking a look?
persistentvolumecontroller.NewPersistentVolumeController has 11 arguments now,
put them into a structure.
Also, rename NewPersistentVolumeController to NewController, persistentvolume
is already name of the package.
Fixes#30219
Automatic merge from submit-queue
Added e2e framweork and a simple test for HA master.
Added e2e framweork for testing HA master. Added one simple e2e test for HA master that first grows and then shrinks GCE cluster.
Automatic merge from submit-queue
Corrected timeout on Downward API volume
The tests for Downward API volume were apparently created before the
default sync-frequency was set to 1 minute. As the current
implementation of Pod gives us no guarantee that the changes will be
seen in Downward API before `sync-frequency` we need the tests to
reflect this.
Fixes: #29633
Automatic merge from submit-queue
Pass the real cluster scoped service object to cleanup functions instead of passing the federation scoped object.
cc @kubernetes/sig-cluster-federation
Ref: Issue #31624
Automatic merge from submit-queue
Logging soak
Implements #24427
Needs
- #24471 so that it doesnt clog test outputs for scale
- builds on the utils function added in support of #22869
cc @timothysc @kubernetes/sig-testing
Automatic merge from submit-queue
Allow garbage collection to work against different API prefixes
The GC needs to build clients based only on Resource or Kind. Hoist the
restmapper out of the controller and the clientpool, support a new
ClientForGroupVersionKind and ClientForGroupVersionResource, and use the
appropriate one in both places.
Allows OpenShift to use the GC
Automatic merge from submit-queue
Speed up job's e2e when waiting for failure
**What this PR does / why we need it**:
Job controller synchronizes objects only when job itself or underlying pod changes. Or, when full resync is performed once 10 mins. This leads e2e test to unnecessarily wait that longer timeout, sometimes at least. I've added job modification action which triggers resync, if the job wasn't terminated within shorter period of time.
@ixdy ptal
@janetkuo @erictune fyi
Automatic merge from submit-queue
Fix node performance benchmark by using latest containervm image (docker 1.11.2)
Also add two more tests for resource tracking.
cc/ @Random-Liu @coufon
Automatic merge from submit-queue
Staging 1.5 client
Created the 1.5 folder and remove the 1.4 folder in the staging area in the master branch.
Content of kubernetes/client-go/1.4 will be pulled from the kubernetes/kubernetes 1.4 branch (https://github.com/kubernetes/contrib/pull/1719)
Automatic merge from submit-queue
remove storage related fields from genericapiserver
Removes `StorageFactory` and `StorageDecorator` from from `genericapiserver` since both constructs are related to building a `RESTStorage`, which should be provided fully formed (or via factory func) to a truly generic API server.
I found this while trying to move the creation API routes earlier.
The GC needs to build clients based only on Resource or Kind. Hoist the
restmapper out of the controller and the clientpool, support a new
ClientForGroupVersionKind and ClientForGroupVersionResource, and use the
appropriate one in both places.
Automatic merge from submit-queue
Node E2E: Change the disk eviction test to pull images again after the test.
Fixes https://github.com/kubernetes/kubernetes/issues/32022#issuecomment-248677706.
This PR changes the disk eviction test to pull test images again in `AfterEach`, because images may be evicted during the test.
@yujuhong
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Make node E2E tests more transparent
Add some logging and minor code reorg to make the node E2E tests a little more transparent and understandable.
Automatic merge from submit-queue
Integration tests: Migrate scheduler perf to the integration suite, s…
*Problem*
issues like https://github.com/kubernetes/kubernetes/pull/32384/ happen because we dont compile the integration tests as part of the build
*Solution*
This pr will make it so scheduler_perf is included in the integration tests, but it will run in short mode that only makes sure compilation happens . it also removes the over generalized test/component directory which only has one entry
```
test/integration/replicationcontroller
test/integration/scheduler
test/integration/scheduler
test/integration/scheduler_perf
test/integration/scheduler_perf
test/integration/secrets
test/integration/serviceaccount
test/integration/storageclasses
```
The tests for Downward API volume were apparently created before the
default sync-frequency was set to 1 minute. As the current
implementation of Pod gives us no guarantee that the changes will be
seen in Downward API before `sync-frequency` we need the tests to
reflect this.
Fixes: #29633
Automatic merge from submit-queue
Refactor cert utils into one pkg, add funcs from bootkube for kubeadm to use
**What this PR does / why we need it**:
We have ended-up with rather incomplete and fragmented collection of utils for handling certificates. It may be worse to consider using `cfssl` for doing all of these things, but for now there is some functionality that we need in `kubeadm` that we can borrow from bootkube. It makes sense to move the utils from bookube into core, as discussed in #31221.
**Special notes for your reviewer**: I've taken the opportunity to review names of existing funcs and tried to make some improvements in that area (with help from @peterbourgon).
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Minor Ingress tests cleanup, that includes service shard and GCE resource cleanups in underlying clusters.
Follow up for #32810.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Allow secure access to apiserver from Admission Controllers
* Allow options.InsecurePort to be set to 0 to switch off insecure access
* In NewSelfClient, Set the TLSClientConfig to the cert and key files
if InsecurePort is switched off
* Mint a bearer token that allows the client(s) created in NewSelfClient
to talk to the api server
* Add a new authenticator that checks for this specific bearer token
Fixes#13598
Automatic merge from submit-queue
Update the containervm image to the latest one (container-v1-3-v20160…
Node e2e is running with old containervm image which only has docker 1.9.1. This pr fixed such issue.
Automatic merge from submit-queue
Dumping federation events if federation e2e test failed
Updating the e2e framework to dump events in federation control plane if a federation e2e test failed.
This should help in debugging https://github.com/kubernetes/kubernetes/issues/32733
cc @kubernetes/sig-cluster-federation
Spawn pet set with single replica and simple pod. They will have
conflicting hostPort definitions, and spawned on the same node.
As the result pet set pod, it will be created after simple pod, will be
in Failed state. Pet set controller will try to re-create it. After
verifying that pet set pod failed and was recreated atleast once, we will
remove pod with conflicting hostPort and wait until pet set pod will be in
running state.
Change-Id: I5903f5881f8606c696bd390df58b06ece33be88a
Automatic merge from submit-queue
Move generated openAPI specs out of genericapiserver and make it configurable
A follow up to #31468
Generated OpenAPI Spec does not belong to genericapiserver package. A new package "generated" created in hope of all generated codes goes into it in future. Openapi folder of that package contains generated definitions and generic API server will accept the definition map as a configuration parameter.
Reference: #13414
Prior to this, we would approve eviction as long as the current state of
the pods matched the budget. The new version requires that after the
eviction, the pods would still match the budget.
Also update tests to match.
Automatic merge from submit-queue
Stop referring to loop variable inside closure.
Prior to this, I was actually running the same (last) test 5 times in a row. :-(
Fixes#33137
Automatic merge from submit-queue
Fix the properties file for node e2e cri validation.
I fixed this locally before, but accidentally missed in the PR. Sorry about that.
This time, I've tried myself, it should work.
@yujuhong
Automatic merge from submit-queue
Bump up GCI version.
```release-note
Upgrading Container-VM base image for k8s on GCE. Brief changelog as follows:
- Fixed performance regression in veth device driver
- Docker and related binaries are statically linked
- Fixed the issue of systemd being oom-killable
```
Fixes#32596
This needs a cherrypick into v1.4 release branch because it is fixing v1.4 release blocking issues. This patch is easy and safe to rollback in case of emergencies.
@vishh can you please review?
Fixes#32596 and many other issues.
cc/ @kubernetes/goog-image FYI
Automatic merge from submit-queue
Viper direct bindings to TestContext struct with hierarchichal suppor…
Part of #31453 to support hierarchichal parameters. This one does so for density, paves way for other tests as well.
Brief changelog compared to gci-dev-54-8743-3-0:
- Fixed performance regression in veth device driver
- Docker and related binaries are statically linked
- Fixed the issue of systemd being oom-killable
- Updated built-in kubelet version to 1.3.7
- add ethtool and ebtables binaries expected by kubelet
Fixes#32596
Automatic merge from submit-queue
separate RESTStorage by API group
This demonstrates how we could start separating the description of `RESTStorage` from `pkg/master`. Each API group owns constructing its own RESTStorage. Since `pkg/registry` is where we place all packages that create RESTStorage, it seems reason to split the package by `pkg/registry/<api group>/storage` to create RESTStorage and `pkg/registry/<api group>/<resource>` to support each individual resource.
The interface to construct this RESTStorage is dependent upon some `genericapiserver` types now, but we could (should?) move those items to a different location.
@kubernetes/sig-api-machinery
@lavalamp can you weigh in on how I've split these packages? I think we agree that we need better segregation, but we haven't spoken about how to spell it.
* Allow options.InsecurePort to be set to 0 to switch off insecure access
* In NewSelfClient, Set the TLSClientConfig to the cert and key files
if InsecurePort is switched off
* Mint a bearer token that allows the client(s) created in NewSelfClient
to talk to the api server
* Add a new authenticator that checks for this specific bearer token
Fixes#13598
Automatic merge from submit-queue
Node E2E: Add image white list
This is part of #29081. Fixes#29155.
As is discussed with @yujuhong in #29155, it is difficult to maintain the prepull image list if it is not enforced.
This PR added an image white list in the test framework, only images in the white list could be used in the test. If the image is not in the white list, the test will fail with reason:
```
Image "XXX" is not in the white list, consider adding it to CommonImageWhiteList in test/e2e/common/util.go or NodeImageWhiteList in test/e2e_node/image_list.go
```
Notice that if image pull policy is `PullAlways`, the image is not necessary to be in the white list or prepulled, because the test expects the image to be pulled during the test.
Currently, the image white list is only enabled in node e2e, because the image puller in e2e test is not integrated with the image white list yet.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Node E2E: Add image pull retry in image pulling test.
Fixes#29259, #28047.
This test added image pull retry in image pulling node e2e test. It will retry for 3 times until test successes.
This should be able to make the image pulling test less flaky.
@yujuhong
Automatic merge from submit-queue
Implement cleanup methods to cleanup underlying cluster resources in federated service tests.
cc @kubernetes/sig-cluster-federation @colhom
Automatic merge from submit-queue
add GenericAPIServer posthooks for initialization
Adds the concept of a `PostStartHook` to the `GenericAPIServer` to allow post-server start hooks. This gives a standard location to perform post-start bootstrapping tasks. The common case usage we have downstream are security related bootstrapping tasks that are performed on the "empty etcd" initialization cases. The RBAC authorizer is a good example of this in kube. It needs a location to create default policies to start a server which is capable of being accessed.
Kube is also likely to hit this for things like PSP and breaking the monolithic controller user into separate, scoped service accounts.
@kubernetes/sig-api-machinery for the `GenericAPIServer` bits
@kubernetes/sig-auth for the particular clusterrole bootstrapping. I've only done one to start, but I suspect we'll start making more and probably introduce a binding so that the RBAC super-user doesn't remain a special case forever.
