kubernetes/cluster/addons/fluentd-elasticsearch/fluentd-es-image/Dockerfile
Eduardo Silva 5e321af2ed cluser/addons: fluentd-elasticsearch: preload Jemalloc for td-agent
The current DockerFile build an image using td-agent package but it let
the service run with the default memory allocator provided by glibc.

In high load environments, is highly required to use a customized memory
allocator such as Jemalloc. Otherwise the service will face a high memory
fragmentation ending up in 'high memory' usage from a monitoring perspective.

td-agent package by default install Jemalloc and set the LD_PRELOAD
environment variable through it init script, but since the service is
launched through Docker the env variable needs to be set manually.

After this patch, when running td-agent container image now is possible
to see that Jemalloc is used:

  root@monotop:/proc/18810# cat maps |grep jemall
  7f251eddd000-7f251ee1b000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
  7f251ee1b000-7f251f01b000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
  7f251f01b000-7f251f01d000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2
  7f251f01d000-7f251f01e000 ... /opt/td-agent/embedded/lib/libjemalloc.so.2

For a reference about the memory usage difference between malloc v/s jemalloc
please refer to the following chart:

  https://goo.gl/dVYTmw

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>
2016-10-20 10:50:58 -06:00

45 lines
1.6 KiB
Docker

# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This Dockerfile will build an image that is configured
# to run Fluentd with an Elasticsearch plug-in and the
# provided configuration file.
# TODO(a-robinson): Use a lighter base image, e.g. some form of busybox.
# The image acts as an executable for the binary /usr/sbin/td-agent.
# Note that fluentd is run with root permssion to allow access to
# log files with root only access under /var/log/containers/*
# Please see http://docs.fluentd.org/articles/install-by-deb for more
# information about installing fluentd using deb package.
FROM gcr.io/google_containers/ubuntu-slim:0.4
MAINTAINER Alex Robinson "arob@google.com"
MAINTAINER Jimmi Dyson "jimmidyson@gmail.com"
# Ensure there are enough file descriptors for running Fluentd.
RUN ulimit -n 65536
# Disable prompts from apt.
ENV DEBIAN_FRONTEND noninteractive
# Copy the Fluentd configuration file.
COPY td-agent.conf /etc/td-agent/td-agent.conf
COPY build.sh /tmp/build.sh
RUN /tmp/build.sh
ENV LD_PRELOAD /opt/td-agent/embedded/lib/libjemalloc.so
# Run the Fluentd service.
ENTRYPOINT ["td-agent"]