ae0ca36695
Automatic merge from submit-queue (batch tested with PRs 48365, 49902, 49808, 48722, 47045) Upgrade fluentd-elasticsearch addon to Elasticsearch/Kibana 5.5 This is a patch to upgrade the fluentd-elasticsearch addon to Elasticsearch/Kibana 5.5. Please provide feedback! ```release-notes * Upgrade Elasticsearch/Kibana to 5.5.1 in fluentd-elasticsearch addon * Switch to basing our image of Elasticsearch in fluentd-elasticsearch addon off the official one * Switch to the official image of Kibana in fluentd-elasticsearch addon * Use StatefulSet for Elasticsearch instead of ReplicationController, with persistent volume claims * Require authenticating towards Elasticsearch, as Elasticsearch 5.5 by default requires basic authentication ``` |
||
|---|---|---|
| .. | ||
| es-image | ||
| fluentd-es-image | ||
| env-configmap.yaml | ||
| env-secret.yaml | ||
| es-clusterrole.yaml | ||
| es-clusterrolebinding.yaml | ||
| es-service.yaml | ||
| es-serviceaccount.yaml | ||
| es-statefulset.yaml | ||
| fluentd-es-clusterrole.yaml | ||
| fluentd-es-clusterrolebinding.yaml | ||
| fluentd-es-ds.yaml | ||
| fluentd-es-serviceaccount.yaml | ||
| kibana-deployment.yaml | ||
| kibana-service.yaml | ||
| OWNERS | ||
| README.md | ||
Elasticsearch Add-On
This add-on consists of a combination of Elasticsearch, Fluentd and Kibana. Elasticsearch is a search engine that is responsible for storing our logs and allowing for them to be queried. Fluentd sends log messages from Kubernetes to Elasticsearch, whereas Kibana is a graphical interface for viewing and querying the logs stored in Elasticsearch.
Elasticsearch
Elasticsearch is deployed as a StatefulSet, which is like a Deployment, but allows for maintaining state on storage volumes.
Authentication
Elasticsearch has basic authentication enabled by default, in our configuration the credentials are at their default values, i.e. username 'elastic' and password 'changeme'. In order to change them, please read up on the official documentation.
Initialization
The Elasticsearch Statefulset manifest specifies that there shall be an
init container executing
before Elasticsearch containers themselves, in order to ensure that the kernel state variable
vm.max_map_count is at least 262144, since this is a requirement of Elasticsearch.
You may remove the init container if you know that your host OS meets this requirement.
Storage
The Elasticsearch StatefulSet will claim a storage volume 'elasticsearch-logging', of the standard StorageClass, that by default will be 100 Gi per replica. Please adjust this to your needs (including possibly choosing a more suitable StorageClass).
Fluentd
Fluentd is deployed as a DaemonSet which spawns a pod on each node that reads logs, generated by kubelet, container runtime and containers and sends them to Elasticsearch.
Please note that for Fluentd to work, every Kubernetes node must be labeled
beta.kubernetes.io/fluentd-ds-ready=true, as otherwise Fluentd will ignore them.
Learn more at: https://kubernetes.io/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana