97 lines
4.7 KiB
Markdown
97 lines
4.7 KiB
Markdown
# Elasticsearch Add-On
|
|
|
|
This add-on consists of a combination of [Elasticsearch][elasticsearch],
|
|
[Fluentd][fluentd] and [Kibana][kibana]. Elasticsearch is a search engine
|
|
that is responsible for storing our logs and allowing for them to be queried.
|
|
Fluentd sends log messages from Kubernetes to Elasticsearch, whereas Kibana
|
|
is a graphical interface for viewing and querying the logs stored in
|
|
Elasticsearch.
|
|
|
|
**Note:** this addon should **not** be used as-is in production. This is
|
|
an example and you should treat it as such. Please see at least the
|
|
[Security](#security) and the [Storage](#storage) sections for more
|
|
information.
|
|
|
|
## Elasticsearch
|
|
|
|
Elasticsearch is deployed as a [StatefulSet][statefulSet], which is like
|
|
a Deployment, but allows for maintaining state on storage volumes.
|
|
|
|
### Security
|
|
|
|
Elasticsearch has capabilities to enable authorization using the [X-Pack
|
|
plugin][xPack]. For the sake of simplicity this example uses the fully open
|
|
source prebuild images from elastic that do not contain the X-Pack plugin. If
|
|
you need these features, please consider building the images from either the
|
|
"basic" or "platinum" version. After enabling these features, follow [official
|
|
documentation][setupCreds] to set up credentials in Elasticsearch and Kibana.
|
|
Don't forget to propagate those credentials also to Fluentd in its
|
|
[configuration][fluentdCreds]. You can utilize [ConfigMaps][configMap] and
|
|
[Secrets][secret] to store credentials in the Kubernetes apiserver.
|
|
|
|
### Initialization
|
|
|
|
The Elasticsearch StatefulSet manifest specifies that there shall be an
|
|
[init container][initContainer] executing before Elasticsearch containers
|
|
themselves, in order to ensure that the kernel state variable
|
|
`vm.max_map_count` is at least 262144, since this is a requirement of
|
|
Elasticsearch. You may remove the init container if you know that your host
|
|
OS meets this requirement.
|
|
|
|
### Storage
|
|
|
|
The Elasticsearch StatefulSet will use the [EmptyDir][emptyDir] volume to
|
|
store data. EmptyDir is erased when the pod terminates, here it is used only
|
|
for testing purposes. **Important:** please change the storage to persistent
|
|
volume claim before actually using this StatefulSet in your setup!
|
|
|
|
## Fluentd
|
|
|
|
Fluentd is deployed as a [DaemonSet][daemonSet] which spawns a pod on each
|
|
node that reads logs, generated by kubelet, container runtime and containers
|
|
and sends them to Elasticsearch.
|
|
|
|
Learn more in the [official Kubernetes documentation][k8sElasticsearchDocs].
|
|
|
|
## Building
|
|
|
|
Both images are now being hosted in quay.io and are built locally via `make` and `docker`.
|
|
To build locally run `make build` and then `make push` to publish.
|
|
|
|
### Known problems
|
|
|
|
Since Fluentd talks to the Elasticsearch service inside the cluster, instances
|
|
on masters won't work, because masters have no kube-proxy. Don't mark masters
|
|
with the label mentioned in the previous paragraph or add a taint on them to
|
|
avoid Fluentd pods scheduling there.
|
|
|
|
[fluentd]: http://www.fluentd.org/
|
|
[elasticsearch]: https://www.elastic.co/products/elasticsearch
|
|
[kibana]: https://www.elastic.co/products/kibana
|
|
[xPack]: https://www.elastic.co/products/x-pack
|
|
[setupCreds]: https://www.elastic.co/guide/en/x-pack/current/setting-up-authentication.html#reset-built-in-user-passwords
|
|
[fluentdCreds]: https://github.com/uken/fluent-plugin-elasticsearch#user-password-path-scheme-ssl_verify
|
|
[configMap]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/
|
|
[secret]: https://kubernetes.io/docs/concepts/configuration/secret/
|
|
[statefulSet]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset
|
|
[initContainer]: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
|
|
[emptyDir]: https://kubernetes.io/docs/concepts/storage/volumes#emptydir
|
|
[daemonSet]: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
|
|
[k8sElasticsearchDocs]: https://kubernetes.io/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana
|
|
|
|
## Running EFK stack in production
|
|
|
|
The configurations provided here for Elasticsearch, Fluentd & Kibana (EFK),
|
|
are a starting point and should not be considered to be production ready.
|
|
|
|
If you like to run these tools in a production environment you could use the
|
|
[Helm](https://helm.sh) charts, provided by the Helm community, which are used
|
|
by a lot of people and therefore are widely tested. You can find them all via the
|
|
[Helm Hub](https://hub.helm.sh/).
|
|
|
|
The source of the mentioned charts can be found here:
|
|
|
|
* [Elasticsearch](https://github.com/helm/charts/tree/master/stable/elasticsearch)
|
|
* [Fluentd-elasticsearch](https://github.com/kiwigrid/helm-charts/tree/master/charts/fluentd-elasticsearch)
|
|
* [Kibana](https://github.com/helm/charts/tree/master/stable/kibana)
|