a7ed23be8e
* elastic now provides a fully opensource version for their prebuild docker images (elasticsearch, kibana and so on). To avoid running into licensing conflicts for this addon example, we should rather use these images instead of the premium ones (were we also have to disable premium features manually right now) * remove disable flags for xpack, since *-oss images do not include this anymore * bump elasticsearch and kibana version from 5.6.4 to 6.2.4 * use oss version from elastic as baseimg for kibana and elasticsearch * bump fluentd version to ~>1.1.3 * bump gem 'fluent-plugin-elasticsearch' to '~>2.9.1' to allow usage of elasticsearch 6.x * bump fluentd-es-image to v2.1.0 * fix elasticserach run.sh to align with new elasticsearch upstream container structure
84 lines
4.2 KiB
Markdown
84 lines
4.2 KiB
Markdown
# Elasticsearch Add-On
|
|
|
|
This add-on consists of a combination of [Elasticsearch][elasticsearch],
|
|
[Fluentd][fluentd] and [Kibana][kibana]. Elasticsearch is a search engine
|
|
that is responsible for storing our logs and allowing for them to be queried.
|
|
Fluentd sends log messages from Kubernetes to Elasticsearch, whereas Kibana
|
|
is a graphical interface for viewing and querying the logs stored in
|
|
Elasticsearch.
|
|
|
|
**Note:** this addon should **not** be used as-is in production. This is
|
|
an example and you should treat it as such. Please see at least the
|
|
[Security](#security) and the [Storage](#storage) sections for more
|
|
information.
|
|
|
|
## Elasticsearch
|
|
|
|
Elasticsearch is deployed as a [StatefulSet][statefulSet], which is like
|
|
a Deployment, but allows for maintaining state on storage volumes.
|
|
|
|
### Security
|
|
|
|
Elasticsearch has capabilities to enable authorization using the [X-Pack
|
|
plugin][xPack]. For the sake of simplicity this example uses the fully open
|
|
source prebuild images from elastic that do not contain the X-Pack plugin. If
|
|
you need these features, please consider building the images from either the
|
|
"basic" or "platinum" version. After enabling these features, follow [official
|
|
documentation][setupCreds] to set up credentials in Elasticsearch and Kibana.
|
|
Don't forget to propagate those credentials also to Fluentd in its
|
|
[configuration][fluentdCreds], using for example [environment
|
|
variables][fluentdEnvVar]. You can utilize [ConfigMaps][configMap] and
|
|
[Secrets][secret] to store credentials in the Kubernetes apiserver.
|
|
|
|
### Initialization
|
|
|
|
The Elasticsearch StatefulSet manifest specifies that there shall be an
|
|
[init container][initContainer] executing before Elasticsearch containers
|
|
themselves, in order to ensure that the kernel state variable
|
|
`vm.max_map_count` is at least 262144, since this is a requirement of
|
|
Elasticsearch. You may remove the init container if you know that your host
|
|
OS meets this requirement.
|
|
|
|
### Storage
|
|
|
|
The Elasticsearch StatefulSet will use the [EmptyDir][emptyDir] volume to
|
|
store data. EmptyDir is erased when the pod terminates, here it is used only
|
|
for testing purposes. **Important:** please change the storage to persistent
|
|
volume claim before actually using this StatefulSet in your setup!
|
|
|
|
## Fluentd
|
|
|
|
Fluentd is deployed as a [DaemonSet][daemonSet] which spawns a pod on each
|
|
node that reads logs, generated by kubelet, container runtime and containers
|
|
and sends them to Elasticsearch.
|
|
|
|
**Note:** in order for Fluentd to work, every Kubernetes node must be labeled
|
|
with `beta.kubernetes.io/fluentd-ds-ready=true`, as otherwise the Fluentd
|
|
DaemonSet will ignore them.
|
|
|
|
Learn more in the [official Kubernetes documentation][k8sElasticsearchDocs].
|
|
|
|
### Known problems
|
|
|
|
Since Fluentd talks to the Elasticsearch service inside the cluster, instances
|
|
on masters won't work, because masters have no kube-proxy. Don't mark masters
|
|
with the label mentioned in the previous paragraph or add a taint on them to
|
|
avoid Fluentd pods scheduling there.
|
|
|
|
[fluentd]: http://www.fluentd.org/
|
|
[elasticsearch]: https://www.elastic.co/products/elasticsearch
|
|
[kibana]: https://www.elastic.co/products/kibana
|
|
[xPack]: https://www.elastic.co/products/x-pack
|
|
[setupCreds]: https://www.elastic.co/guide/en/x-pack/current/setting-up-authentication.html#reset-built-in-user-passwords
|
|
[fluentdCreds]: https://github.com/uken/fluent-plugin-elasticsearch#user-password-path-scheme-ssl_verify
|
|
[fluentdEnvVar]: https://docs.fluentd.org/v0.12/articles/faq#how-can-i-use-environment-variables-to-configure-parameters-dynamically
|
|
[configMap]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/
|
|
[secret]: https://kubernetes.io/docs/concepts/configuration/secret/
|
|
[statefulSet]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset
|
|
[initContainer]: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
|
|
[emptyDir]: https://kubernetes.io/docs/concepts/storage/volumes#emptydir
|
|
[daemonSet]: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
|
|
[k8sElasticsearchDocs]: https://kubernetes.io/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana
|
|
|
|
[]()
|