Commit Graph

196 Commits

Author SHA1 Message Date
Lantao Liu
2bc0532eb3 Add fluentd support. 2017-11-21 00:36:01 +00:00
Kubernetes Submit Queue
b2125f5aa8
Merge pull request #55509 from tallclair/psp-addons
Automatic merge from submit-queue (batch tested with PRs 54602, 54877, 55243, 55509, 55128). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

PodSecurityPolicies for addons

**What this PR does / why we need it**:

1. Colocate addon PodSecurityPolicy config with the addons (in a `podsecuritypolicies` subdirectory). 
2. Add policies for addons that are currently missing policies (not in the default GCE suite)
3. Remove HostPath SSL certs from several heapster deployments, so that heapster doesn't require a special PSP

**Which issue(s) this PR fixes**:
#43538

**Release note**:
```release-note
- Add PodSecurityPolicies for cluster addons
- Remove SSL cert HostPath volumes from heapster addons
```
2017-11-14 03:03:30 -08:00
Kubernetes Submit Queue
f5c29f51fa
Merge pull request #55506 from Random-Liu/fix-cri-fluentd
Automatic merge from submit-queue (batch tested with PRs 54460, 55258, 54858, 55506, 55510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix CRI fluentd config.

This should fix the cri-containerd stackdriver test failure:
```
Cluster level logging implemented by Stackdriver should ingest logs
```

I copied the pattern from a comment previously. However, it doesn't actually work properly. `\b` only matches word boundary, and seems to match the boundary of previous word in our case.

That's why we get the log with a leading space:
```
Nov 10 18:39:11.661: INFO: Unexpected error occurred: log entry ingested incorrectly, got --> <--I0101 00:00:00.000000       1 main.go:1] Text, want Text
```

@kubernetes/sig-node-bugs @kubernetes/sig-instrumentation-bugs 

Signed-off-by: Lantao Liu <lantaol@google.com>

```release-note
none
```
2017-11-11 10:45:27 -08:00
Kubernetes Submit Queue
dad41f8526
Merge pull request #54215 from mrahbar/elasticsearch_logging_discovery
Automatic merge from submit-queue (batch tested with PRs 54987, 55221, 54099, 55144, 54215). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

extracted elasticsearch-logging service name as environment variable

**What this PR does / why we need it**:
Deploying the cluster-addon fluentd-elasticsearch with customized resource definitions can cause elasticsearch discovery to fail because the service name `elasticsearch-logging` is hard-coded in  cluster/addons/fluentd-elasticsearch/es-image/elasticsearch_logging_discovery.go

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
-> none yet

**Special notes for your reviewer**:
The name of the  environment variable is ELASTICSEARCH_SERVICE_NAME. When non is given the fallback service-name  fallback is  `elasticsearch-logging`

```release-note
[fluentd-elasticsearch addon] Elasticsearch service name can be overridden via env variable ELASTICSEARCH_SERVICE_NAME
```
2017-11-10 14:51:33 -08:00
Tim Allclair
cd720c4759
Add optional addon PSPs 2017-11-10 13:57:33 -08:00
Lantao Liu
53d7494b9e Fix CRI fluentd config.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-11-10 20:55:56 +00:00
mrahbar
4ecd54f47f extracted elasticsearch-logging service name as environment variable ELASTICSEARCH_SERVICE_NAME with fallback on default 2017-11-10 14:14:22 +01:00
Dr. Stefan Schimanski
bec617f3cc Update generated files 2017-11-09 12:14:08 +01:00
Dr. Stefan Schimanski
012b085ac8 pkg/apis/core: mechanical import fixes in dependencies 2017-11-09 12:14:08 +01:00
Lantao Liu
70a0cdfa8e Add CRI log format support in fluentd. 2017-10-30 06:25:52 +00:00
Jeff Grafton
aee5f457db update BUILD files 2017-10-15 18:18:13 -07:00
Arve Knudsen
e04773a43d fluentd-elasticsearch add-on: Rename Elasticsearch Docker image tag 2017-10-03 17:54:31 +02:00
Arve Knudsen
64fcdf04c4 fluentd-elasticsearch add-on: Upgrade API versions 2017-10-01 12:28:16 +02:00
Arve Knudsen
1f6647d3cd fluentd-elasticsearch add-on: Upgrade to Elasticsearch/Kibana 5.6.2 2017-10-01 12:26:18 +02:00
Chen Rong
d23df051e1 update to rbac v1 in yaml file 2017-08-21 17:29:37 +08:00
Arve Knudsen
7e3b8a59dc Base Fluentd image off debian:stretch-slim for systemd with LZ4 2017-08-15 15:55:13 +02:00
Mik Vyatskov
1b0e32480c Increase kibana CPU limit to sped up the startup 2017-08-14 11:49:32 +02:00
Jeff Grafton
a7f49c906d Use buildozer to delete licenses() rules except under third_party/ 2017-08-11 09:32:39 -07:00
Jeff Grafton
33276f06be Use buildozer to remove deprecated automanaged tags 2017-08-11 09:31:50 -07:00
Kubernetes Submit Queue
70b4db2023 Merge pull request #50082 from crassirostris/cleanup-fluentd-es
Automatic merge from submit-queue (batch tested with PRs 48532, 50054, 50082)

Refactored the fluentd-es addon

Refactor fluentd-elasticsearch addon:

- Decrease the number of files by moving RBAC-related objects in the same files where they're used
- Move the fluentd configuration out of the image
- Don't use PVC to avoid leaking resources in e2e tests
- Fluentd now ingest docker and kubelet logs that are written to journald
- Disable X-Pack, because it's not free

Fixes https://github.com/kubernetes/kubernetes/issues/41462
Fixes https://github.com/kubernetes/kubernetes/issues/49816
Fixes https://github.com/kubernetes/kubernetes/issues/48973
Fixes https://github.com/kubernetes/kubernetes/issues/49450

@aknuds1 @coffeepac Could you please take a look?

```release-note
Fluentd DaemonSet in the fluentd-elasticsearch addon is configured via ConfigMap and includes journald plugin
Elasticsearch StatefulSet in the fluentd-elasticsearch addon uses local storage instead of PVC by default
```
2017-08-05 05:14:03 -07:00
Mik Vyatskov
46f53daef0 Refactored the fluentd-es addon files, moved the fluentd configuration to ConfigMap 2017-08-04 20:05:15 +02:00
Kubernetes Submit Queue
d15baf69e1 Merge pull request #48969 from ixdy/update-kazel
Automatic merge from submit-queue (batch tested with PRs 50103, 49677, 49449, 43586, 48969)

Run kazel on the entire tree

**What this PR does / why we need it**: part of #47558: auto-generate `BUILD` files on the entire tree, since this is what `gazelle` does, and it'll make subsequent reviews easier if less is changing.

**Release note**:

```release-note
NONE
```
/assign
/release-note-none
2017-08-03 16:43:41 -07:00
Kubernetes Submit Queue
ae0ca36695 Merge pull request #48722 from aknuds1/upgrade-fluentd-elasticsearch
Automatic merge from submit-queue (batch tested with PRs 48365, 49902, 49808, 48722, 47045)

Upgrade fluentd-elasticsearch addon to Elasticsearch/Kibana 5.5

This is a patch to upgrade the fluentd-elasticsearch addon to Elasticsearch/Kibana 5.5. Please provide feedback!

```release-notes
* Upgrade Elasticsearch/Kibana to 5.5.1 in fluentd-elasticsearch addon
* Switch to basing our image of Elasticsearch in fluentd-elasticsearch addon off the official one
* Switch to the official image of Kibana in fluentd-elasticsearch addon
* Use StatefulSet for Elasticsearch instead of ReplicationController, with persistent volume claims
* Require authenticating towards Elasticsearch, as Elasticsearch 5.5 by default requires basic authentication
```
2017-08-02 22:32:59 -07:00
Jeff Grafton
3579017b86 Run hack/update-bazel.sh to generate BUILD files 2017-08-02 18:33:25 -07:00
Arve Knudsen
0ed0f02c76 Upgrade Elasticsearch/Kibana to 5.5.1 and use official Kibana image 2017-08-02 19:40:19 +02:00
Matthieu Maquevice
93acc4c276 fix spelling 2017-07-31 10:59:29 +02:00
Victor Sartori
de50947f30 Bump image version on makefile and DS. 2017-07-07 14:55:54 -03:00
Victor Sartori
0e52082673 Add prometheus plugin on fluentd image. 2017-07-06 18:24:04 -03:00
Kubernetes Submit Queue
93ef2f2f59 Merge pull request #47860 from crassirostris/fluentd-log-to-stdio
Automatic merge from submit-queue (batch tested with PRs 47860, 47170)

Make fluentd log to stdio instead of a dedicated file

Lower verbosity also, to reduce volume of system logs exported to the backend.

Fix https://github.com/kubernetes/kubernetes/issues/43772

/cc @piosz
2017-06-26 14:03:04 -07:00
Piotr Szczesniak
77fd8a1898 Revert "Decrese fluentd cpu request" 2017-06-26 16:18:24 +02:00
Kubernetes Submit Queue
5f82f129df Merge pull request #46203 from simt2/fluentd-elasticsearch-rbac
Automatic merge from submit-queue (batch tested with PRs 46151, 47602, 47507, 46203, 47471)

Add RBAC support to fluentd-elasticsearch cluster addon

**What this PR does / why we need it**:
Adds rbac support to the fluentd-elasticsearch addon

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #46023 

**Special notes for your reviewer**:

**Release note**:

```release-note
Add RBAC support to fluentd-elasticsearch cluster addon
```
2017-06-23 05:08:28 -07:00
Kubernetes Submit Queue
509c4351df Merge pull request #47507 from yiqinguo/yiqinguo_es_addargs
Automatic merge from submit-queue (batch tested with PRs 46151, 47602, 47507, 46203, 47471)

es discovery support args apiserver-host and kubeconfig

Now discovery elasticsearch through kubernetes client,but now does not support specifying the apiserver-host or kubeconfig create client.
2017-06-23 05:08:26 -07:00
gmarek
8427d5a274 Decrease fluentd cpu request 2017-06-22 18:30:21 +02:00
Mik Vyatskov
1d1c126783 Make fluentd log to stdio instead of a dedicated file 2017-06-21 20:14:43 +02:00
yiqinguo
b0c57c081e es discovery support args apiserver-host and kubeconfig 2017-06-20 09:15:23 +08:00
simt2
4bc0da349d Add rbac support to fluentd-elasticsearch 2017-06-16 08:44:24 +02:00
Stanisław Szydło
9a02da7754 Update fluentd-es-ds.yaml
'fluentd-es-ds' container image ver. bump.
2017-05-29 19:09:57 +02:00
Stanislaw Szydlo
f9d46cf23c Update 'fluentd-elasticsearch' image.
Plugins updated:
 - fluent-plugin-elasticsearch 1.5.0 -> 1.9.5
 - fluent-plugin-kubernetes_metadata 0.24.0 -> 0.27.0
2017-05-29 16:14:52 +02:00
kb@itsvit.org
b23b919b05 Bump elasticsearch and kibana to 5.4.0 2017-05-24 14:07:15 +03:00
haoyuan
d3fd956dac remove the elasticsearch template 2017-05-17 19:20:14 +08:00
Kubernetes Submit Queue
1e6061b9ec Merge pull request #45763 from piosz/es-owners
Automatic merge from submit-queue

Added coffeepac to ElasticSearch owners

@coffeepac

@fgrzadkowski, could you please add @coffeepac to Kubernetes org?
2017-05-16 12:22:59 -07:00
Christian Koep
df80b76d1b
Refactor gcr.io/google_containers/elasticsearch to alpine
Signed-off-by: Christian Koep <christiankoep@gmail.com>
2017-05-15 17:52:39 +02:00
Piotr Szczesniak
da8f82cbd0 Added coffeepac to ElasticSearch owners 2017-05-13 07:48:09 +02:00
Mik Vyatskov
30f22ad683 Remove toleration from fluentd-elasticsearch 2017-04-05 11:27:14 +02:00
Kubernetes Submit Queue
13e2f5d9d7 Merge pull request #43379 from crassirostris/fluentd-gcp-docs
Automatic merge from submit-queue

Add readme for fluentd addons

/cc @steveperry-53
2017-03-23 02:08:56 -07:00
Mik Vyatskov
7ae8eee1cd Add readme for fluentd-elasticsearch addon 2017-03-22 11:27:48 +01:00
Piotr Szczesniak
8968ac5c36 Renamed fluentd-ds-ready annotation 2017-03-21 20:48:13 +01:00
Kubernetes Submit Queue
17793bccb9 Merge pull request #42630 from anguslees/elasticsearch
Automatic merge from submit-queue (batch tested with PRs 41830, 42630)

Arrange for elasticsearch to shutdown cleanly

Kubernetes initiates "graceful shutdown" by sending SIGTERM to pid 1, which
is exactly what elasticsearch is expecting (good!)

The way the existing startup scripts worked however, this signal arrived at
the shell wrapper, not elasticsearch, and the shell wrapper exited,
killing the container immediately (bad!)

Before this change:
```
    1 ?        Ss     0:00 /bin/sh -c /run.sh
    6 ?        S      0:00 /bin/bash /run.sh
   13 ?        S      0:00  \_ /bin/su -c /elasticsearch/bin/elasticsearch elasticsearch
   14 ?        Ss     0:00      \_ sh -c /elasticsearch/bin/elasticsearch
   15 ?        Sl    19:18          \_ /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java ... org.elasticsearch.bootstrap.Elasticsearch start
```
After this change:
```
    1 ?        Ssl    0:29 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java ... org.elasticsearch.bootstrap.Elasticsearch start
```
2017-03-10 16:21:20 -08:00
Angus Lees
3390deac31 Fix metav1 typo in previous commit
Fix typo in 469df12038 by using `metav1`
consistently rather than an error-producing mix of `metav1` and
`meta_v1`.
2017-03-10 11:24:22 +11:00
Mikhail Vyatskov
0d363f2892 Fix username case in OWNERS files 2017-03-07 16:45:40 -08:00