Automatic merge from submit-queue (batch tested with PRs 64142, 64426, 62910, 63942, 64548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update fluentd-elasticsearch addon
**What this PR does / why we need it**:
Elastic now provides a fully opensource version for their prebuild
docker images (elasticsearch, kibana and so on). To avoid running into
licensing conflicts for this addon example, we should rather use these
images instead of the premium ones (were we also have to disable premium
features manually right now).
This PR updates:
- fluentd from <=1.1.0 to ~>1.1.3
- elasticsearch and kibana from 5.6.2 to 6.2.4
- fluentd-elasticsearch-plugin from 2.4.1 to 2.9.1
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
https://github.com/fluent/fluentd/issues/1894
**Special notes for your reviewer**:
**Release note**:
```release-note
```
* elastic now provides a fully opensource version for their prebuild
docker images (elasticsearch, kibana and so on). To avoid running into
licensing conflicts for this addon example, we should rather use these
images instead of the premium ones (were we also have to disable premium
features manually right now)
* remove disable flags for xpack, since *-oss images do not include this anymore
* bump elasticsearch and kibana version from 5.6.4 to 6.2.4
* use oss version from elastic as baseimg for kibana and elasticsearch
* bump fluentd version to ~>1.1.3
* bump gem 'fluent-plugin-elasticsearch' to '~>2.9.1' to allow usage of elasticsearch 6.x
* bump fluentd-es-image to v2.1.0
* fix elasticserach run.sh to align with new elasticsearch upstream container structure
This is the 2nd attempt. The previous was reverted while we figured out
the regional mirrors (oops).
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest. To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today). For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it. Nice and
visible, easy to keep track of.
The current DockerFile build an image using td-agent package but it let
the service run with the default memory allocator provided by glibc.
In high load environments, is highly required to use a customized memory
allocator such as Jemalloc. Otherwise the service will face a high memory
fragmentation ending up in 'high memory' usage from a monitoring perspective.
td-agent package by default install Jemalloc and set the LD_PRELOAD
environment variable through it init script, but since the service is
launched through Docker the env variable needs to be set manually.
After this patch, when running td-agent container image now is possible
to see that Jemalloc is used:
root@monotop:/proc/18810# cat maps |grep jemall
7f251eddd000-7f251ee1b000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
7f251ee1b000-7f251f01b000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
7f251f01b000-7f251f01d000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
7f251f01d000-7f251f01e000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
For a reference about the memory usage difference between malloc v/s jemalloc
please refer to the following chart:
https://goo.gl/dVYTmw
Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>
We can then avoid the following warning:
```
WARNING: The '--' argument must be specified between gcloud specific args on the left and DOCKER_ARGS on the right. IMPORTANT: previously, commands allowed the omission of the --, and unparsed arguments were treated as implementation args. This usage is being deprecated and will be removed in March 2017.
This will be strictly enforced in March 2017. Use 'gcloud beta docker' to see new behavior.
```
Signed-off-by: Jess Frazelle <acidburn@google.com>
I didn't expect glog to split single log statements onto multiple lines,
but apparently it does if they're long enough. This groups them back
together appropriately.
