Commit Graph

109 Commits

Author SHA1 Message Date
Mik Vyatskov
94eeca8d2c Fixed startup script bug in kibana image 2016-11-09 16:35:34 +01:00
Kubernetes Submit Queue
48ef0faa0e Merge pull request #35216 from edsiper/fluentd-jemalloc
Automatic merge from submit-queue

cluster-addons: enable Jemalloc for Fluentd based images

**What this PR does / why we need it**:

This Pull Request includes two patches that enable the recommended use of Jemalloc memory allocator for container images that are based in Fluentd. The patches applies to the following cluster-addons:
- fluentd-es-image
- fluentd-gcp-image

**Which issue this PR fixes** 

This PR is part of the solution for issues:
-  kubernetes/kubernetes/issues/32762
-  GoogleCloudPlatform/fluent-plugin-google-cloud/issues/87

When Fluentd runs in high load environments, it's likely the default operating system memory allocator will generate a high fragmentation ending up in a high memory usage. In order to reduce fragmentation and decrease memory usage an alternative memory allocator as Jemalloc is used. 

![](https://cloud.githubusercontent.com/assets/369718/19498577/eaa9f324-954e-11e6-9a6b-6b30310a66a3.png)

For the record: fluentd-es-image uses [td-agent](https://docs.treasuredata.com/articles/td-agent) Fluentd package maintained by Treasure Data, which contains Jemalloc 4.2.1 (latest stable version). The google-fluentd package used in fluentd-gcp-image comes with Jemalloc 2.2.5, which have many known issues, I strongly suggest google-fluentd package gets updated.

**Special notes for your reviewer**:

In the research of this topic have been involved @piosz and @Crassirostris.
2016-11-06 05:26:58 -08:00
Mik Vyatskov
3b6353bdc8 Fixed kibana image and controller to work through proxy 2016-11-02 22:06:22 +01:00
Jan Chaloupka
4fde09d308 Replace client with clientset in code 2016-10-23 22:00:35 +02:00
Kubernetes Submit Queue
35943d6ad6 Merge pull request #34969 from Crassirostris/es-kibana-usage-update
Automatic merge from submit-queue

Update elasticsearch and kibana usage

```release-note
Updated default Elasticsearch and Kibana used for elasticsearch logging destination to versions 2.4.1 and 4.6.1 respectively.
```

Updated controllers for elasticsearch and kibana to use newer versions of images. Fixed e2e test because of elasticsearch backward incompatible API changes.

Fixed out of sync elasticsearch controller for coreos.

@piosz
2016-10-20 12:26:02 -07:00
Eduardo Silva
5e321af2ed cluser/addons: fluentd-elasticsearch: preload Jemalloc for td-agent
The current DockerFile build an image using td-agent package but it let
the service run with the default memory allocator provided by glibc.

In high load environments, is highly required to use a customized memory
allocator such as Jemalloc. Otherwise the service will face a high memory
fragmentation ending up in 'high memory' usage from a monitoring perspective.

td-agent package by default install Jemalloc and set the LD_PRELOAD
environment variable through it init script, but since the service is
launched through Docker the env variable needs to be set manually.

After this patch, when running td-agent container image now is possible
to see that Jemalloc is used:

  root@monotop:/proc/18810# cat maps |grep jemall
  7f251eddd000-7f251ee1b000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
  7f251ee1b000-7f251f01b000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
  7f251f01b000-7f251f01d000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
  7f251f01d000-7f251f01e000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2

For a reference about the memory usage difference between malloc v/s jemalloc
please refer to the following chart:

  https://goo.gl/dVYTmw

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>
2016-10-20 10:50:58 -06:00
Mik Vyatskov
9832ae1259 Update elasticsearch and kibana usage 2016-10-17 21:04:25 +02:00
Mik Vyatskov
7a582fcae5 Fixed kibana image 2016-10-17 17:45:59 +02:00
Kubernetes Submit Queue
fd1fd2a14c Merge pull request #34562 from Crassirostris/es-kibana-update
Automatic merge from submit-queue

Elasticsearch and Kibana update

```release-note
Updated Elasticsearch image from version 1.5.1 to version 2.4.1. Updated Kibana image from version 4.0.2 to version 4.6.1.
```

Updated es and kibana images. Made image versions match es/kibana versions they contain.

ref #19149
2016-10-17 03:04:33 -07:00
Mik Vyatskov
5b79b65bd3 Update elasticsearch and kibana images 2016-10-16 16:01:38 +02:00
Jess Frazelle
cde5559d09
Update gcloud docker commands to use gcloud docker -- ARGS
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```

Signed-off-by: Jess Frazelle <acidburn@google.com>
2016-10-10 13:42:34 -07:00
Mik Vyatskov
3fbde5ecfb Fixed elasticsearch cluster logging e2e test on GCI 2016-09-21 13:55:43 +02:00
Manuel de Brito Fontes
e2c50152dd Remove kubernetes metadata plugin 2016-08-23 08:21:46 -03:00
Manuel de Brito Fontes
5753a3196c Reduce the size of fluentd images 2016-08-23 08:21:45 -03:00
haoyuan
a1f20cbdf4 fix typo 2016-08-08 19:33:51 +08:00
Rick Buskens
77bda6fd16 Updated configuration to spawn multiple threads for processing 2016-08-01 15:08:11 -04:00
David McMahon
ef0c9f0c5b Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
pprokop
fa91b3e461 Adding option to deploy fluentd-elasticsearch in diffrent namespace then kube-system 2016-06-27 18:15:41 +02:00
Alex Robinson
bb51c94172 Add collection of the new glbc and cluster-autoscaler logs 2016-06-03 15:50:52 +00:00
Alex Robinson
8579509293 Rebuild elasticsearch image to include changes since 1.2 2016-06-01 20:48:20 +00:00
keontang
1520a01d97 Fixed #25127 2016-05-16 09:03:00 +08:00
Mike Zupan
dc643d2592 Fixing a simple typo 2016-03-06 23:49:35 -08:00
Alex Robinson
29eb6020fb Merge pull request #22074 from a-robinson/images
Update the logging images to parse sub-second docker timestamps.
2016-03-04 09:04:33 -08:00
Alex Robinson
feea376b5c Update the logging images to parse sub-second docker timestamps. 2016-03-03 10:21:30 -08:00
Alex Robinson
f8965a6f41 Fix parsing of kube logs to handle logs split across lines.
I didn't expect glog to split single log statements onto multiple lines,
but apparently it does if they're long enough. This groups them back
together appropriately.
2016-02-26 22:58:46 +00:00
Christopher Eck
c34ecf7100 Enable fluentd to parse the sub-second portion of timestamps coming from docker logs. 2016-02-18 11:07:27 -08:00
Alex Robinson
e5c5f76402 Update the default fluentd images to parse system logs. 2016-02-04 23:29:27 +00:00
Alex Robinson
592da85a81 Parse system logs into structured messages in fluentd.
This allows you to filter based on components of the log,
like their severity, in the developers console or elasticsearch.
2016-02-04 08:14:59 +00:00
Paul Morie
b672785d72 Add boilerplate checks for Dockerfiles 2016-02-03 18:35:26 -05:00
Paul Morie
05bd107301 Add boilerplate checks for Makefiles 2016-02-03 18:35:26 -05:00
Alex Robinson
6acf297231 Update the fluentd versions to include fixes for #19405. 2016-01-27 00:30:53 +00:00
Alex Robinson
ac13e851e2 Don't let fluentd pipe its own logs directly back into itself. 2016-01-27 00:30:53 +00:00
Alex Robinson
95a4ea3fc8 Increase the fluentd buffer chunk size to improve write throughput.
Also reduce the max wait between retries, 30 seconds should be more than
enough backoff.
2016-01-27 00:30:50 +00:00
gmarek
2d5c429f9d Add resource requests to addons 2015-12-17 10:13:15 +01:00
Minhan Xia
b4c72e56ce enforce node quorum to prevent es split brain problem 2015-12-14 18:16:04 -08:00
Matt Moore
b750d1dddc Drop the beta for GCR v2 images.
beta.gcr.io is no longer needed to pull through v2.
2015-11-14 12:19:02 -08:00
Matt Moore
eeb4eeb17c Move pause and fluentd-elasticsearch to GCR v2.
This scopes down the initially ambitious PR:
https://github.com/kubernetes/kubernetes/pull/14960 to replace just
`pause` and `fluentd-elasticsearch` to come through `beta.gcr.io`.

The v2 versions have been pushed under new tags, `pause:2.0` and
`fluentd-elastisearch:1.12`.

NOTE: `beta.gcr.io` will still serve images using v1 until they are repushed with v2.  Pulls through `gcr.io` will still work after pushing through `beta.gcr.io`, but will be served over v1 (via compat logic).
2015-10-06 16:39:07 -07:00
Tamer Tas
cb5f921df1 Add newline to the end of bash scripts 2015-09-18 19:35:05 +03:00
Eric Paris
04fc8ae3dd s|github.com/GoogleCloudPlatform/kubernetes|github.com/kubernetes/kubernetes| 2015-09-03 10:10:11 -04:00
Jimmi Dyson
e07da97447 Switch to fluentd kubernetes metadata plugin to enrich logs with more searchable kubernetes metadata 2015-09-02 10:33:15 +01:00
Alex Robinson
d02dc64036 Let fluentd-es log by not redirecting to a file within the container
and only using -q (warning logs only) rather than -qq (error logs only).
2015-08-27 21:41:44 +00:00
Alex Robinson
f6f8cb07ae Transfer maintainer responsibilities for fluentd configs/images. 2015-08-26 21:48:51 +00:00
Satnam Singh
278a3911fd Fetch fluentd td agent securely 2015-08-19 13:34:03 -07:00
Kris Rousey
ae6c64d9bb Moving everyone to unversioned client 2015-08-18 10:23:03 -07:00
Alex Robinson
c887ad75b8 Stop collecting monit logs via fluentd. 2015-08-07 00:12:21 +00:00
Mike Danese
17defc7383 run gofmt on everything we touched 2015-08-05 17:52:56 -07:00
Mike Danese
8e33cbfa28 rewrite go imports 2015-08-05 17:30:03 -07:00
Alex Robinson
cc02e50fc7 Collect more of the VM's log files (monit, salt) in fluentd-es. 2015-08-05 00:22:38 +00:00
Alex Robinson
69ab7026af Update fluentd-es to collect etcd logs and remove merge conflict cruft. 2015-07-30 22:21:01 +00:00
Satnam Singh
691c07763c Remove use of v1beta3 from ES logging 2015-07-06 18:47:45 -07:00