Elastic Stack, formerly known as ELK Stack, is a collection of open-source tools developed by Elastic. It is designed for log management, data visualization, and real-time analytics. The Elastic Stack consists of four core components
Elasticsearch is a distributed, RESTful search and analytics engine. It stores and indexes data, allowing for fast and efficient searching, filtering, and aggregations. Elasticsearch provides scalability and fault tolerance, making it suitable for handling large volumes of data.
Logstash is a data processing pipeline that collects, transforms, and enriches data from various sources. It supports a wide range of input sources such as log files, syslog, databases, and message queues. Logstash allows for data parsing, filtering, and normalization before sending it to Elasticsearch for indexing.
Kibana is a powerful data visualization and exploration tool. It provides a web-based interface for creating interactive dashboards, charts, and visualizations based on data stored in Elasticsearch. Kibana allows users to explore and analyze data, monitor system metrics, and gain insights from log data.
Beats are lightweight data shippers that collect and send data to the Elastic Stack. They are designed to gather specific types of data, such as logs, metrics, or network packets. Beats provide a simple and efficient way to ingest data into Elasticsearch.
The Elastic Stack is commonly used for log analysis, monitoring, and observability. It enables organizations to centralize and analyze logs, metrics, and other data sources in real-time. With Elasticsearch’s powerful search capabilities, Logstash’s data processing capabilities, and Kibana’s visualization capabilities, users can gain valuable insights from their data and detect issues or anomalies more efficiently.
Elastic Stack can be extended and integrated with other tools and plugins to meet specific requirements, making it a versatile and widely adopted solution for log management and data analytics.
here we will try to install elastic stack in docker container.
Prerequisite
- VM with docker & docker-compose installed
You can install docker engine following this article:
Install Elastic ELK Stack on Docker Container
Clone elastic stack manifest
git clone https://github.com/elkninja/elastic-stack-docker-part-one.git cd elastic-stack-docker-part-one/
set some additional
sysctl -w vm.max_map_count=262144
in the dotenv, modify memory limit
# Increase or decrease based on the available host memory (in bytes) ES_MEM_LIMIT=2147483648 KB_MEM_LIMIT=2147483648 LS_MEM_LIMIT=2147483648
modify dotenv to set password and docker compose up
nano .env docker compose up -d
and wait until all healthy
check running docker container
done!
