Commit Graph

632 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
b525becead
Merge pull request #65319 from MaciekPytel/gke_manifests
Automatic merge from submit-queue (batch tested with PRs 65319, 64513, 65474, 65601, 65634). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Allow custom manifests in GCP master setup

Add a hook in GCE setup script to allow using custom manifests on master, so we can decouple some GKE changes from k8s. Note that this PR just adds a hook there is no change in default behavior.

```release-note
```
2018-06-29 12:15:05 -07:00
Maciej Pytel
7119c8bd07 Allow custom manifests in GCP master setup 2018-06-29 15:49:54 +02:00
Kubernetes Submit Queue
5a185e0917
Merge pull request #60150 from MrHohn/kube-proxy-priority-beta
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Set pod priority on kube-proxy manifest by default

**What this PR does / why we need it**:
Follow up of https://github.com/kubernetes/kubernetes/pull/59237, set pod priority on kube-proxy by default and remove the unneeded logic in startup script.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #NONE 

**Special notes for your reviewer**:
/assign @bsalamat @bowei 
cc @tanshanshan

**Release note**:

```release-note
NONE
```
2018-06-28 18:54:52 -07:00
Kubernetes Submit Queue
c005b9d0ab
Merge pull request #65308 from kgolab/kg-cleanup-kubepath
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove unused srv_kube_path variable

**What this PR does / why we need it**:

Clean-up of an unused script variable, as discussed with @mikedanese after [a comment in PR 64503](https://github.com/kubernetes/kubernetes/pull/64503#discussion_r194505831).

**Release note**:

```release-note
NONE
```
2018-06-27 09:34:44 -07:00
David Ashpole
c8758a774e update NPD version to v0.5.0 for gci 2018-06-25 13:13:39 -07:00
Lantao Liu
2af997470f Add /home/kubernetes/bin into sudoers path, so that sudo crictl works. 2018-06-22 17:10:55 -07:00
Kubernetes Submit Queue
b48339704f
Merge pull request #65024 from jingax10/calico_custom_branch
Automatic merge from submit-queue (batch tested with PRs 65024, 65287, 65345, 64693, 64941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add a helper function to customize K8s addon yamls and use it to customize Calico addons on GKE

**What this PR does / why we need it**:

Allow customizing Calico addon in GCP. With #65022, this allows us to do a couple of things:, e.g., run Calico 3.0+ on GCP, use a non-default MTU etc.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #65045, #65067

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-06-22 06:10:10 -07:00
Jing Ai
4dce8973ce Add a helper function to customize K8s addon yamls and use it to customize Calico addons on GKE. 2018-06-21 15:07:22 -07:00
Karol Gołąb
f31d2d0bcb Removed unused srv_kube_path variable 2018-06-21 13:58:11 +02:00
Karol Wychowaniec
eefdff659d Pass cluster_location argument to Heapster 2018-06-18 13:54:22 +02:00
Kubernetes Submit Queue
ec434662bd
Merge pull request #64503 from kgolab/kg-ca-rbac
Automatic merge from submit-queue (batch tested with PRs 64503, 64903, 64643, 64987). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Create system:cluster-autoscaler account & role and introduce it to C…

**What this PR does / why we need it**:

This PR adds cluster-autoscaler ClusterRole & binding, to be used by the Cluster Autoscaler (kubernetes/autoscaler repository).
It also updates GCE scripts to make CA use the cluster-autoscaler user account.

User account instead of Service account is chosen to be more in line with kube-scheduler.

**Which issue(s) this PR fixes**:

Fixes [issue 383](https://github.com/kubernetes/autoscaler/issues/383) from kubernetes/autoscaler.

**Special notes for your reviewer**:

This PR might be treated as a security fix since prior to it CA on GCE was using system:cluster-admin account, assumed due to default handling of unsecured & unauthenticated traffic over plain HTTP.

**Release note**:

```release-note
A cluster-autoscaler ClusterRole is added to cover only the functionality required by Cluster Autoscaler and avoid abusing system:cluster-admin role.

action required: Cloud providers other than GCE might want to update their deployments or sample yaml files to reuse the role created via add-on.
```
2018-06-11 17:29:13 -07:00
Karol Gołąb
faa4dc39c4 Disambiguate a comment 2018-06-11 10:56:02 +02:00
xuzhonghu
cfcd0e4d45 stop using deprecated --etcd-quorum-read 2018-06-09 12:38:46 +08:00
Karol Gołąb
c70b554af9 Create system:cluster-autoscaler account & role and introduce it to CA start-up script 2018-06-08 14:15:52 +02:00
Kubernetes Submit Queue
898831ad9d
Merge pull request #64592 from ravisantoshgudimetla/revert-64364-remove-rescheduler
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "Remove rescheduler and corresponding tests from master"

Reverts kubernetes/kubernetes#64364

After discussing with @bsalamat on how DS controllers(ref: https://github.com/kubernetes/kubernetes/pull/63223#discussion_r192277527) cannot create pods if the cluster is at capacity and they have to rely on rescheduler for making some space, we thought it is better to 

- Bring rescheduler back.
- Make rescheduler priority aware.
- If cluster is full and if **only** DS controller is not able to create pods, let rescheduler be run and let it evict some pods which have less priority.
- The DS controller pods will be scheduled now.

So, I am reverting this PR now. Step 2, 3 above are going to be in rescheduler.

/cc @bsalamat @aveshagarwal @k82cn 

Please let me know your thoughts on this. 

```release-note
Revert #64364 to resurrect rescheduler. More info https://github.com/kubernetes/kubernetes/issues/64725 :)
```
2018-06-04 16:56:11 -07:00
Kubernetes Submit Queue
4f088e6263
Merge pull request #64591 from cadmuxe/custom_netd
Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add netd as an addon for GCP

**What this PR does / why we need it**:
Add netd as an addon for GKE.
The PR will add setup functions and var to help deploy netd daemon on GKE.
Please checkout more detail for netd at https://github.com/GoogleCloudPlatform/netd

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2018-06-04 12:26:16 -07:00
Kubernetes Submit Queue
36a3daa355
Merge pull request #61610 from rajansandeep/kubeupaddon
Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Create CoreDNS and kube-dns folders

**What this PR does / why we need it**:
Separate the CoreDNS and kube-dns manifests by creating their own folders (dns/coredns and dns/kube-dns) 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #61435 

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
cc @MrHohn
2018-06-04 12:26:12 -07:00
Koonwah Chen
37059e7efa Code clean up 2018-06-03 19:41:47 -07:00
Kubernetes Submit Queue
586e558c3b
Merge pull request #59938 from rramkumar1/gce-cluster-up-ipvs
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add ipvs module loading logic to gce scripts

**What this PR does / why we need it**:
Add ipvs module loading logic to gce scripts. 
Fixes a part of #59402.

/cc @Lion-Wei 
/assign @roberthbailey @m1093782566 

**Release note**:
```release-note
None
```
2018-05-31 20:55:44 -07:00
Koonwah Chen
d903d32856 Add netd as an addon for GKE. 2018-05-31 19:25:15 -07:00
RaviSantosh Gudimetla
872addf9e3
Revert "Remove rescheduler and corresponding tests from master" 2018-05-31 22:18:49 -04:00
Rohit Ramkumar
cc87e73dd8 Add ipvs module loading logic to gce scripts 2018-05-31 08:40:05 -07:00
ravisantoshgudimetla
aeccffc339 Phase out rescheduler in favor of priority and preemption 2018-05-29 19:52:06 -04:00
Sandeep Rajan
753632d85b create coredns and kube-dns folders 2018-05-29 11:52:57 -04:00
CJ Cullen
b3a31b28af re-reorder authorizers (RBAC before Webhook). 2018-05-22 16:48:39 -07:00
Kubernetes Submit Queue
0d815fbc27
Merge pull request #64029 from loburm/truncate-flag
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add environment variable to control truncating backend.

```release-note
NONE
```
2018-05-19 05:17:00 -07:00
Kubernetes Submit Queue
bfca0d32a5
Merge pull request #63689 from awly/gce-fix-kubelet-ca-path
Automatic merge from submit-queue (batch tested with PRs 63969, 63902, 63689, 63973, 63978). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Reuse existing CA cert path for kubelet certs

**What this PR does / why we need it**: configure-helper.sh already knows the path to CA cert, re-use that to avoid typos.

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2018-05-18 15:59:19 -07:00
Kubernetes Submit Queue
f105ae3e6d
Merge pull request #63918 from cezarygerard/sd-event-exporter
Automatic merge from submit-queue (batch tested with PRs 63569, 63918, 63980, 63295, 63989). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

New event exporter config with support for new stackdriver resources

New event exporter, with support for use new and old stackdriver resource model.

This should also be cherry-picked to release-1.10 branch, as all  fluentd-gcp components support new and stackdriver resource model.

```release-note
Update event-exporter to version v0.2.0  that supports old (gke_container/gce_instance) and new (k8s_container/k8s_node/k8s_pod) stackdriver resources.
```
2018-05-18 09:54:16 -07:00
Marian Lobur
c1d0004013 Add environment variable to control truncating backend. 2018-05-18 15:52:47 +02:00
Cezary Zawadka
d611aeac80 new event exporter config with support for new stackdriver resource types 2018-05-18 10:37:47 +02:00
Maciej Borsz
128d6d3498 Add a way to pass extra arguments to etcd. 2018-05-17 10:48:13 +02:00
Kubernetes Submit Queue
e392f5b08b
Merge pull request #63696 from grosskur/gce-advertise-addr
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

gce: Prefer MASTER_ADVERTISE_ADDRESS in apiserver setup

MASTER_ADVERTISE_ADDRESS is used to set the --advertise-address flag
for the apiserver. It's useful for running the apiserver behind a load
balancer.

However, if PROJECT_ID, TOKEN_URL, TOKEN_BODY, and NODE_NETWORK are
all set, the GCE VM's external IP address will be fetched and used
instead and MASTER_ADVERTISE_ADDRESS will be ignored.

Change this behavior so that MASTER_ADVERTISE_ADDRESS takes precedence
because it's more specific. We still fall back to using the VM's
external IP address if the other variables are set.

Also: Move the setting of --ssh-user and --ssh-keyfile based on
PROXY_SSH_USER) to a top-level block because this is common to all
codepaths.

```release-note
NONE
```
2018-05-15 23:25:22 -07:00
Kubernetes Submit Queue
7b8bb6e7d3
Merge pull request #63357 from Random-Liu/install-and-use-crictl
Automatic merge from submit-queue (batch tested with PRs 63167, 63357). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Install and use crictl in gce kube-up.sh

Download and use crictl in gce kube-up.sh.

This PR:
1. Downloads crictl `v1.0.0-beta.0` onto the node, which supports CRI v1alpha2. We'll upgrade it to `v1.0.0-beta.1` soon after the release is cut.
2. Change `kube-docker-monitor` to `kube-container-runtime-monitor`, and let it use `crictl` to do health monitoring.
3. Change `e2e-image-puller` to use `crictl`. Because of https://github.com/kubernetes/kubernetes/issues/63355, it doesn't work now. But in `crictl v1.0.0-beta.1`, we are going to statically link it, and the `e2e-image-puller` should work again.
4. Use `systemctl kill --kill-who=main` instead of `pkill`, the reason is that:
  a. `pkill docker` will send `SIGTERM` to all processes including `dockerd`, `docker-containerd`, `docker-containerd-shim`. This is not a problem for Docker 17.03 CE, because `containerd-shim` in containerd 0.2.x doesn't exit with SIGERM (see [code](https://github.com/containerd/containerd/blob/v0.2.x/containerd-shim/main.go#L123)). However, `containerd-shim` in containerd 1.0+ does exit with SIGTERM (see [code](https://github.com/containerd/containerd/blob/master/cmd/containerd-shim/main_unix.go#L200)). This means that `pkill docker` and `pkill containerd` will kill all shim processes for Docker 17.11+ and containerd 1.0+.
  b. We can use `pkill -x` instead. However, docker systemd service name is `docker`, but daemon process name is `dockerd`. We have to introduce another environment variable to specify "daemon process name". Given so, it seems easier to just use `systemctl kill` which only requires systemd service name. `systemctl kill --kill-who=main` will make sure only main process receives SIGTERM.

Signed-off-by: Lantao Liu <lantaol@google.com>

/cc @filbranden @yujuhong @feiskyer @mrunalp @kubernetes/sig-node-pr-reviews @kubernetes/sig-cluster-lifecycle-pr-reviews 

**Release note**:

```release-note
Kubernetes cluster on GCE have crictl installed now. Users can use it to help debug their node. The documentation of crictl can be found https://github.com/kubernetes-incubator/cri-tools/blob/master/docs/crictl.md.
```
2018-05-15 21:18:12 -07:00
Alan Grosskurth
3541a93f92 gce: Prefer MASTER_ADVERTISE_ADDRESS in apiserver setup
MASTER_ADVERTISE_ADDRESS is used to set the --advertise-address flag
for the apiserver. It's useful for running the apiserver behind a load
balancer.

However, if PROJECT_ID, TOKEN_URL, TOKEN_BODY, and NODE_NETWORK are
all set, the GCE VM's external IP address will be fetched and used
instead and MASTER_ADVERTISE_ADDRESS will be ignored.

Change this behavior so that MASTER_ADVERTISE_ADDRESS takes precedence
because it's more specific. We still fall back to using the VM's
external IP address if the other variables are set.

Also: Pass --ssh-user and --ssh-keyfile flags if both PROXY_SSH_USER
and MASTER_ADVERTISE_ADDRESS is set.
2018-05-15 17:00:51 -07:00
Bowei Du
2e7807a249 Enable CUSTOM_INGRESS_YAML to replace the glbc manifest
This allows for customized versions of the Ingress YAML separate from
stock Kubernetes.
2018-05-14 23:24:55 -07:00
Kubernetes Submit Queue
b617748f7b
Merge pull request #62905 from serathius/event-exporter-region
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[fluentd-gcp addon] Pass region in seperate field

This PR makes location passed to event-exporter based on `MULTIZONE` env.

Fixes https://github.com/kubernetes/kubernetes/issues/62399
```release-note
NONE
```
/cc @loburm
2018-05-11 06:00:44 -07:00
Marek Siarkowicz
f351b00a99 [fluentd-gcp addon] Pass region in seperate field 2018-05-11 09:50:07 +02:00
Andrew Lytvynov
1c94d0bd64 Reuse existing CA cert path for kubelet certs 2018-05-10 14:02:06 -07:00
Kubernetes Submit Queue
a743392937
Merge pull request #63353 from bmoyles0117/fix-stackdriver-metadata-agent-url-for-fluentd
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use the logging agent's node name as the metadata agent URL.

The Stackdriver Logging agent should use the node's hostname when it constructs the Stackdriver Metadata Agent's URL, currently, it's using the GKE Master's hostname, which is a bug.

**Release note:**
```release-note
[fluentd-gcp addon] Use the logging agent's node name as the metadata agent URL.
```
2018-05-08 16:20:43 -07:00
Kubernetes Submit Queue
940e716c06
Merge pull request #63323 from awly/gce-kubelet-ca
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

gce: plumb --kubelet-certificate-authority flag to apiserver

**What this PR does / why we need it**:
We want to start signing kubelets' serving certs with cluster CA. This
flag is required to enforce that on apiserver side.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2018-05-07 21:03:43 -07:00
Kubernetes Submit Queue
c59393e9fd
Merge pull request #63266 from awly/exec-plugin-kubeconfig
Automatic merge from submit-queue (batch tested with PRs 63340, 63266). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

gcp: allow non-bootstrap kubeconfig

**What this PR does / why we need it**:
Needed for https://github.com/kubernetes/community/pull/2022
This change lets us generate a non-bootstrap kubeconfig with exec plugin for authn.
The plugin does TLS bootstrapping internally.

**Special notes for your reviewer**:
Defaults when no new env vars are set will behave same as before this change.
`KUBELET_AUTH_TYPE` should never be `tls-auth` in practice, but leaving it there just in case.

**Release note**:
```release-note
NONE
```
2018-05-07 15:16:14 -07:00
Lantao Liu
d94a2b39d9 Install and use crictl in gce kube-up.sh
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-05-03 17:17:55 -07:00
Andrew Lytvynov
77c13d6dc7 Allow fetching bootstrap-kubeconfig from VM metadata 2018-05-03 11:32:18 -07:00
Kubernetes Submit Queue
b5f61ac129
Merge pull request #62657 from matthyx/master
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update all script shebangs to use /usr/bin/env interpreter instead of /bin/interpreter

This is required to support systems where bash doesn't reside in /bin (such as NixOS, or the *BSD family) and allow users to specify a different interpreter version through $PATH manipulation.
https://www.cyberciti.biz/tips/finding-bash-perl-python-portably-using-env.html
```release-note
Use /usr/bin/env in all script shebangs to increase portability.
```
2018-05-02 19:44:32 -07:00
Bryan Moyles
a0a7686e38 Use the logging agent's node name as the metadata agent URL. 2018-05-02 10:12:35 +02:00
Andrew Lytvynov
0a567f0990 gcp: allow non-bootstrap kubeconfig
The regular kubeconfig is fetched from metadata when
CREATE_BOOTSTRAP_KUBECONFIG==false.

We will experiment with an exec plugin that does TLS bootstrapping
internally: #61803
2018-05-01 10:40:32 -07:00
Kubernetes Submit Queue
dd1d5c74f2
Merge pull request #63152 from mikedanese/break
Automatic merge from submit-queue (batch tested with PRs 63152, 63253). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "Revert "gce: move etcd dir cleanup to manifests""

This reverts commit 2d6b4d0fa0.

```release-note
NONE
```
2018-05-01 07:36:09 -07:00
Andrew Lytvynov
e86bdf5801 gce: plumb --kubelet-certificate-authority flag to apiserver
We want to start signing kubelets' serving certs with cluster CA. This
flag is required to enforce that on apiserver side.
2018-04-30 15:16:22 -07:00
Kubernetes Submit Queue
ded95bc9f1
Merge pull request #62863 from awly/kube-controller-manager-disable-controllers
Automatic merge from submit-queue (batch tested with PRs 62718, 62863). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

gcp: add env var to configure enabled controllers in controller-manager

```release-note
NONE
```
2018-04-27 20:16:05 -07:00
Mike Danese
6817494424 Revert "Revert "gce: move etcd dir cleanup to manifests""
This reverts commit 2d6b4d0fa0.
2018-04-25 08:57:02 -07:00
Shyam Jeedigunta
2d6b4d0fa0 Revert "gce: move etcd dir cleanup to manifests"
This reverts commit ae73bed1d0.
2018-04-25 12:54:12 +02:00
Kubernetes Submit Queue
5b0df3656e
Merge pull request #63000 from kawych/versions
Automatic merge from submit-queue (batch tested with PRs 62590, 62818, 63015, 62922, 63000). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove METADATA_AGENT_VERSION config option

**What this PR does / why we need it**:
Remove METADATA_AGENT_VERSION configuration option. To keep Metadata Agent version consistent across Kubernetes deployments.

**Release note**:
```release-note
Remove METADATA_AGENT_VERSION configuration option.
```
2018-04-24 14:22:23 -07:00
Mike Danese
ae73bed1d0 gce: move etcd dir cleanup to manifests
we deploy it as a manifest, not an addon so locate it with the other
master manifests.
2018-04-24 08:02:32 -07:00
Kubernetes Submit Queue
eea406c108
Merge pull request #62669 from immutableT/deploy_helper_test
Automatic merge from submit-queue (batch tested with PRs 63007, 62919, 62669, 62860). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add unit test for configure-helper.sh.

**What this PR does / why we need it**:
Add a framework for unit-testing configure-helper.sh.
configure-helper.sh plays a critical role in initializing clusters both on GCE and GKE. It is currently, over 2K lines of code, yet it has no unit test coverage.
This PR proposes a framework/approach on how to provide test coverage for this component.
Notes: 
1. Changes to configure-helper.sh itself were necessary to enable sourcing of this script for the purposes of testing.
2. As POC api_manifest_test.go covers the logic related to the initialization of apiserver when integration with KMS was requested. The hope is that the same approach could be extended to the rest of the script.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-04-23 15:45:17 -07:00
immutablet
dc78d72f04 Add unit test for configure-helper. 2018-04-23 12:18:57 -07:00
Karol Wychowaniec
6fb42aea4a Remove METADATA_AGENT_VERSION config option 2018-04-23 12:15:48 +02:00
Kubernetes Submit Queue
77f5324223
Merge pull request #62409 from rajansandeep/corednsscaler
Automatic merge from submit-queue (batch tested with PRs 62409, 62856). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

DNS-Autoscaler support for CoreDNS

**What this PR does / why we need it**:
This PR provides the dns-horizontal autoscaler for CoreDNS in kube-up, enabling the tests to pass once CoreDNS is the default. 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #61176 

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-04-23 01:35:07 -07:00
Andrew Lytvynov
2666d73336 gcp: add env var to configure enabled controllers in controller-manager 2018-04-19 10:15:17 -07:00
Matthias Bertschy
9b15af19b2 Update all script to use /usr/bin/env bash in shebang 2018-04-19 13:20:13 +02:00
Kubernetes Submit Queue
bb8f58b6e6
Merge pull request #62195 from serathius/prometheus
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add prometheus cluster monitoring addon.

This PR adds new cluster monitoring addon based on prometheus.
It adds prometheus deployment with e2e tests.
Additional components will be added iterativly in future.
Manifests based on current Helm chart.
At current state it's not intended for production use.

cc @piosz @kawych @miekg
```release-note
Add prometheus cluster monitoring addon to kube-up
```
/sig instrumentation
/kind feature
/priority important-soon
2018-04-18 02:17:48 -07:00
wojtekt
1bcdfdbe00 Increase max requests inflight limits in gce for very large clusters 2018-04-16 20:46:41 +02:00
Michael Taufen
420edc7b50 provision Kubelet config file for GCE
This PR extends the client-side startup scripts to provision a Kubelet
config file instead of legacy flags. This PR also extends the
master/node init scripts to install this config file from the GCE
metadata server, and provide the --config argument to the Kubelet.
2018-04-13 13:08:38 -07:00
Marek Siarkowicz
113987e0db Add prometheus addon 2018-04-13 11:12:08 +02:00
Kubernetes Submit Queue
1d905bbdfc
Merge pull request #61862 from immutableT/kms-plugin-deploy-cherry-pick
Automatic merge from submit-queue (batch tested with PRs 59636, 62429, 61862). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Inject CloudKMS Plugin container into Kube-APIServer pod.

**What this PR does / why we need it**:
Inject CloudKMS Plugin container into Kube-APIServer pod when etcd level encryption via CloudKMS Plugin is requested.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE

```
2018-04-12 02:02:24 -07:00
Sandeep Rajan
8d5b9d3c36 autoscaler support for CoreDNS 2018-04-11 11:54:23 -04:00
immutablet
cbc428395c Enable CloudKMS Plugin deployment. 2018-04-10 09:47:32 -07:00
Kubernetes Submit Queue
4009cb3b8b
Merge pull request #62076 from qingling128/master
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add support to ingest log entries to Stackdriver against new "k8s_container" and "k8s_node" resources.

**What this PR does / why we need it**:

**Which issue(s) this PR fixes**
Fluentd 0.14 has some memory leak issues that caused the e2e tests to be flaky. Downgrading to v0.12.

**Special notes for your reviewer**:
We never released any previous version with Fluentd v0.14. Only upgraded it very recently. So this downgrading is not visible to users.

**Release note**:
```release-note
Add support to ingest log entries to Stackdriver against new "k8s_container" and "k8s_node" resources.
```
2018-04-06 09:51:32 -07:00
Ling Huang
cbec62ada4 Add support to ingest log entries to Stackdriver against new "k8s_container" and "k8s_node" resources. 2018-04-06 08:47:19 -04:00
Supriya Garg
e350c46116 Update the stackdriver agents yaml to include a deployment for cluster level resources 2018-04-05 10:09:11 -04:00
Kubernetes Submit Queue
839fe27dc8
Merge pull request #61567 from davidz627/fix/localSSDPermissions
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixes permissions error for Local SSD when created with NODE_LOCAL_SSDS flag

**What this PR does / why we need it**:
The PR fixes a permissions error introduced in 1.9 whereby users are unable to write to their Local SSD if it is created with the `NODE_LOCAL_SSDS` flag.

This will need to be cherrypicked to 1.9 and 1.10.

/sig storage
/kind bug
/assign @msau42 

```release-note
NONE
```
2018-03-26 05:40:18 -07:00
David Zhu
d3670856dd Added chmod a+x for local SSD when disk is created with NODE_LOCAL_SSDS 2018-03-22 16:03:03 -07:00
Zhen Wang
d5c2cdcbbb Update GCP fluentd configmap for GKE node journal logging 2018-03-22 12:04:11 -07:00
Mik Vyatskov
07905d6ee8 Make log audit backend configurable in GCE
Signed-off-by: Mik Vyatskov <vmik@google.com>
2018-03-08 14:09:32 +01:00
Kubernetes Submit Queue
cca92f67aa
Merge pull request #60639 from konryd/dashboardmesg
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Added dashboard banner passthrough to GCE kube-up.

**What this PR does / why we need it**: This PR adds GCE-specific provider flag for setting system-banner in the dashboard ui.

```release-note
NONE
```
2018-03-05 07:52:42 -08:00
Kubernetes Submit Queue
209cdd9048
Merge pull request #60237 from crassirostris/audit-use-buffered-backend
Automatic merge from submit-queue (batch tested with PRs 60542, 60237). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Audit use buffered backend

This is the next step after https://github.com/kubernetes/kubernetes/pull/60076

This PR fixes https://github.com/kubernetes/kubernetes/issues/53020, to address https://github.com/kubernetes/kubernetes/issues/53006 later

In this PR buffered backend, introduced in https://github.com/kubernetes/kubernetes/pull/60076, is used to replace ad-hoc solution for webhook and add an ability to enable buffering for the log audit backend.

```release-note
Log audit backend can now be configured to perform batching before writing events to disk.
```

/cc @sttts @tallclair @ericchiang @CaoShuFeng
2018-03-01 11:42:48 -08:00
Mik Vyatskov
881e6d4f6f Add buffering to the log audit backend
Signed-off-by: Mik Vyatskov <vmik@google.com>
2018-03-01 14:30:51 +01:00
Konrad Delong
86c5d0f603 Added dashboard banner passthrough to GCE kube-up. 2018-03-01 12:03:40 +01:00
Kubernetes Submit Queue
f82b38cfaa
Merge pull request #60470 from cjcullen/sched
Automatic merge from submit-queue (batch tested with PRs 60470, 59149, 56075, 60280, 60504). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add scheduling.k8s.io to the known groups for audit logging on GCE.

This lets PriorityClass objects get logged.

```release-note
NONE
```
2018-02-28 04:54:26 -08:00
Mike Danese
857690baf5 gce: add support for enabling TokenRequest feature 2018-02-27 18:54:03 -08:00
Kubernetes Submit Queue
7480face90
Merge pull request #59128 from bmoyles0117/master
Automatic merge from submit-queue (batch tested with PRs 60433, 59982, 59128, 60243, 60440). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[fluentd-gcp addon] Update to use Stackdriver Agent image.

Update the fluentd DaemonSet to use the Stackdriver Logging Agent container image. 
The Stackdriver Logging Agent container image uses fluentd v0.14.25.
We add a special label to each log record as a signal to logging backends to handle both new and legacy resource types.

**Release note:**
```release-note
[fluentd-gcp addon] Switch to the image, provided by Stackdriver.
```
2018-02-27 08:25:43 -08:00
Kubernetes Submit Queue
44c166cd73
Merge pull request #60020 from roberthbailey/kubelet-flags
Automatic merge from submit-queue (batch tested with PRs 59310, 60424, 60308, 60436, 60020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

 Move kubelet flag generation from the node to the client

Pass the kubelet flags through a new variable in kube-env (KUBELET_ARGS).

Remove vars from kube-env that were only used for kubelet flags.

This will make it simpler to gradually migrate to dynamic kubelet
config, because we can gradually replace flags with config file
options in a single place without worrying about the plumbing to
move variables from the client onto the node.

/cc @verult (re: https://github.com/kubernetes/kubernetes/pull/58171)

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
action required: [GCP kube-up.sh] Some variables that were part of kube-env are no longer being set (ones only used for kubelet flags) and are being replaced by a more portable mechanism (kubelet configuration file). The individual variables in the kube-env metadata entry were never meant to be a stable interface and this release note only applies if you are depending on them.
```
2018-02-27 07:11:55 -08:00
CJ Cullen
ad227abfdc Add scheduling.k8s.io to the known groups for audit logging on GCE. 2018-02-26 17:53:43 -08:00
Bryan Moyles
84a86cffce Update to use Stackdriver Agent image.
Prometheus is enabled by default.
2018-02-26 14:05:33 -05:00
wojtekt
10e423a69e Fix regional clusters startup 2018-02-26 15:20:15 +01:00
Robert Bailey
fe10c27ec0 Move kubelet flag generation from the node to the client, and
pass the kubelet flags through a new variable in kube-env
(KUBELET_ARGS).

Remove vars from kube-env that were only used for kubelet flags.

This will make it simpler to gradually migrate to dynamic kubelet
config, because we can gradually replace flags with config file
options in a single place without worrying about the plumbing to
move variables from the client onto the node.
2018-02-24 22:39:36 -08:00
Kubernetes Submit Queue
1f03d40170
Merge pull request #60320 from cjcullen/flags
Automatic merge from submit-queue (batch tested with PRs 59882, 59434, 57722, 60320, 51249). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Allow TTLs to be plumbed through to webhook authn/authz in gce scripts.

Just allows these flags to be configured from the startup scripts.
2018-02-24 18:43:36 -08:00
Kubernetes Submit Queue
a85f7d9fff
Merge pull request #58090 from serathius/pass-location-to-event-exporter
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pass location parameter to event exporter.

**What this PR does / why we need it**:
This PR makes event-exporter export cluster location together with events.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-23 23:15:43 -08:00
CJ Cullen
61a21e903f Allow TTLs to be plumbed through to webhook authn/authz in gce scripts. 2018-02-23 10:58:37 -08:00
Mike Danese
f82fa4dc9a gce: allow extra addons to be sourced form a url 2018-02-22 14:33:52 -08:00
Marek Siarkowicz
bbfcd681b5 Pass location parameter to event exporter.
Location passed based on ZONE from kube-env.
2018-02-21 12:54:29 +01:00
Zihong Zheng
15b13bfe69 Set pod priority on kube-proxy by default 2018-02-20 20:39:48 -08:00
Kubernetes Submit Queue
a6b8c06380
Merge pull request #59378 from ostromart/istio_addon_ostromart
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add code and yaml for Istio as an addon

This commit adds the following:
 - YAML for istio components to the addons directory (versions with
   and without auth)
 - logic to copy the appropriate YAML file into the addons manager
   dir, depending on ENABLE_ISTIO and ISTIO_AUTH_TYPE env vars
 - changes to addon manager script which allow plugins to exist in
   namespace other than kube-system. This is needed for Istio
   functionality
 - additional initialization for Istio in addon manager.

This has been manually tested - e2e tests are in progress but require additional code to enable istio in tests. See:
 - https://github.com/istio/istio/pull/2841
 - https://github.com/kubernetes/test-infra/pull/6612



**What this PR does / why we need it**:
Allow [Istio](https://istio.io/) to be installed as an addon. This simplifies the Istio setup process on k8s and GKE. Istio team is trying to make it easier for end users to get to a basic functional installation.  
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-20 10:35:29 -08:00
Kubernetes Submit Queue
3a60b0b4f2
Merge pull request #59686 from nicksardo/gce-roles
Automatic merge from submit-queue (batch tested with PRs 59683, 59964, 59841, 59936, 59686). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

GCE: Create cloud-provider roles and bindings via addons

**What this PR does / why we need it**:
This removes the `cloud-provider` role and role binding from the rbac boostrapper and replaces it with a policy applied via addon mgr. This also creates a new clusterrole allowing the service account to create events for any namespace.  

**Special notes for your reviewer**:
/assign @bowei @timstclair 
/cc timstclair

**Release note**:
```release-note
GCE: A role and clusterrole will now be provided with GCE/GKE for allowing the cloud-provider to post warning events on all services and watching configmaps in the kube-system namespace.
```
2018-02-16 16:31:40 -08:00
Martin Ostrowski
ec7f7a5c5f Add code and yaml for Istio as an addon
This commit adds the following:
 - YAML for istio components to the addons directory (versions with
   and without auth)
 - logic to copy the appropriate YAML file into the addons manager
   dir, depending on ENABLE_ISTIO and ISTIO_AUTH_TYPE env vars
 - changes to addon manager script which allow plugins to exist in
   namespace other than kube-system. This is needed for Istio
   functionality
 - additional initialization for Istio in addon manager.
2018-02-16 10:48:09 -08:00
Nick Sardo
911a082d65 Add cloud-provider policies to be applied via addon mgr 2018-02-15 14:49:33 -08:00
Kubernetes Submit Queue
d3bacb914c
Merge pull request #59657 from x13n/manual-fluentd-gcp-scaler
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Enable scaling fluentd-gcp resources using ScalingPolicy.

See https://github.com/justinsb/scaler for more details about ScalingPolicy resource.

**What this PR does / why we need it**:
This is adding a way to override fluentd-gcp resources in a running cluster. The resources syncing for fluentd-gcp is decoupled from addon manager.

**Special notes for your reviewer**:

**Release note**:
```release-note
fluentd-gcp resources can be modified via a ScalingPolicy
```

cc @kawych @justinsb
2018-02-15 03:42:14 -08:00
Kubernetes Submit Queue
bc9c6df31d
Merge pull request #59103 from Random-Liu/upload-container-runtime-log
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Upload container runtime log to sd/es.

I've verified this in my environment. My stackdriver has an extra `container-runtime` entry for node log, and it collects container runtime daemon log correctly.

@yujuhong @feiskyer @crassirostris @piosz 
@kubernetes/sig-node-pr-reviews @kubernetes/sig-instrumentation-pr-reviews 
Signed-off-by: Lantao Liu <lantaol@google.com>

**Release note**:

```release-note
Container runtime daemon (e.g. dockerd) logs in GCE cluster will be uploaded to stackdriver and elasticsearch with tag `container-runtime`
```
2018-02-14 03:33:21 -08:00
Kubernetes Submit Queue
cca7ccbff1
Merge pull request #59530 from rramkumar1/ipvs-plumbing
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add a new environment variable to the gce startup scripts called KUBE_PROXY_MODE

**What this PR does / why we need it**:
This PR adds a new environment variable called KUBE_PROXY_MODE to the startup scripts for gce. This variable will allow a user to specify the kube-proxy implementation they want to use, with the choices being ipvs or iptables (iptables is default). 

Next steps:

1. Need to remove use of feature gateway when IPVS goes GA
2. Need to add logic of loading required ipvs kernel modules in the scripts 

Question: If the proxier is IPVS, is it necessary to have the iptables sync period flags? 

**Release note**:
```release-note
None
```
2018-02-13 17:45:32 -08:00
Lantao Liu
8d920d095c Upload container runtime log to sd/es.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-02-13 18:25:02 +00:00
Daniel Kłobuszewski
6a48e343f0 Review #2 2018-02-13 10:57:29 +01:00