How to Install Elastic Stack on Ubuntu 22.04 LTS

Photo by Justin Morgan on Unsplash

Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.

Overview:

In the world of modern software development and IT operations, efficient logging and monitoring are essential for maintaining the health, security, and performance of applications and systems. The ELK Stack, comprised of Elasticsearch, Logstash, and Kibana, is a powerful toolset that enables organizations to achieve centralized logging, real-time analysis, and visualization of data. In this tutorial, we will guide you through the process of setting up and configuring the ELK Stack from scratch, helping you gain insights into your system’s behavior and troubleshoot issues effectively.

Objectives:

1. Install Elasticsearch:

• Add the Elasticsearch APT repository and configure access over HTTPS.
• Install OpenJDK 11 (Java) on Ubuntu.
• Set up the JAVA_HOME environment variable for Java-related configurations.

2. Configure and Verify Elasticsearch:

• Download and install the public signing key for Elasticsearch.
• Configure the APT repository for Elasticsearch 8.
• Update the package index and install Elasticsearch.
• Start and enable the Elasticsearch service.
• Configure Elasticsearch settings in the elasticsearch.yml file.
• Verify the status of the Elasticsearch service.

3. Install and Configure Logstash:

• Install Logstash to collect data from different sources.
• Start and enable the Logstash service.
• Configure Logstash settings in the logstash.yml file.

4. Visualize Data with Kibana:

• Install Kibana to provide a graphical interface for log file interpretation.
• Start and enable the Kibana service.
• Configure Kibana settings in the kibana.yml file.
• Access the Kibana dashboard via a web browser.

5. Install and Configure Filebeat:

• Install Filebeat to collect and ship log files.
• Configure Filebeat to send event data to Elasticsearch.
• Enable the Filebeat system module and load the index template.
• Start and enable the Filebeat service.

6. Verify Data Flow:

• Confirm Elasticsearch’s reception of data from Filebeat.
• Access Elasticsearch and Kibana via a web browser to visualize collected data.

Prerequisite:
- Ubuntu Server with 22.04 LTS
- Java 8 or higher version
- 2 CPU and 4 GB RAM

Install Elasticsearch:

1. Add the Elasticsearch APT repository:

Install the apt-transport-https package to access repository over HTTPS

sudo apt install apt-transport-https

Install Java on Ubuntu 22.04 LTS:

Let’s install OpenJDK 11 on Ubuntu

sudo apt install openjdk-11-jdk

Let’s verify java version

java -version

Setting the JAVA_HOME Environment Variables

To define the environment variable open a text file.

sudo nano /etc/environment

Paste the below variable into the file:

JAVA_HOME="/usr/lib/jvm/java-11-openjdk-amd64"

Load the environment variable using below command

source /etc/environment

Let’s verify JAVA_HOME variable

echo $JAVA_HOME

Install ElasticSearch 8 on Ubuntu 22.04 LTS

Download and install the public signing key:

Save the repository definition to /etc/apt/sources.list.d/elastic-8.x.list:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

Update the package index and install Elasticsearch:

sudo apt-get update
sudo apt-get install elasticsearch

Start and enable Elasticsearch:

sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch

Verify Elasticsearch status:

sudo systemctl status elasticsearch

Configure Elasticsearch on Ubuntu 22.04 LTS

sudo nano /etc/elasticsearch/elasticsearch.yml

Go to Network section and uncomment network.host and replace your system IP

network.host: 0.0.0.0

And you need to add this line in discovery section as shown in below:

discovery.seed_hosts:[]

Then second step is go to the BEGIN SECURITY AUTO CONFIGURATION and here you need to replace this true with false as shown in below:

xpack.security.enabled: false

After changing in configuration file you need to restart so run the below command:

sudo systemctl restart elasticsearch

Testing Elasticsearch on Ubuntu 22.04 LTS

Let’s test Elasticsearch using curl command by sending HTTP request

curl -X GET "localhost:9200"

You can access using browser

http://systemIP:9200

**Installing Logstash:**

Logstash is a tool that collects data from different sources. The data it collects is parsed by Kibana and stored in Elasticsearch.

Install Logstash using following command:

sudo apt-get install logstash

Start the Logstash service:

sudo systemctl start logstash

Enable the Logstash service:

sudo systemctl enable logstash

To check the status of the service, run the following command:

sudo systemctl status logstash

Configure Logstash on Ubuntu 22.04 LTS

sudo nano /etc/logstash/logstash.yml

**Visualizing Data with Kibana:**
Install Kibana on Ubuntu 22.04 LTS

It is recommended to install Kibana next. Kibana is a graphical user interface for parsing and interpreting collected log files

Run the following command to install Kibana:

sudo apt-get install kibana

Start the Kibana service:

sudo systemctl start kibana

Enable the Kibana service:

sudo systemctl enable kibana

Let’s check the status of Kibana:

sudo systemctl status kibana

Configure Kibana on Ubuntu 22.04 LTS

Open the kibana.yml configuration file for editing:

sudo nano /etc/kibana/kibana.yml

Uncomment this below lines and localhost replace with 0.0.0.0 (means any ip_address):

server.port: 5601

server.host: "localhost"

elasticsearch.hosts: ["http://localhost:9200"]

After changing configuration file you need to restart kibana

sudo systemctl restart kibana

Accessing Kibana on Ubuntu 22.04 LTS

To access Kibana, open a web browser and browse to the following address:

http://ip_address:5601

Kibana dashboard:

Install Filebeat on Ubuntu 22.04 LTS

Filebeat is a lightweight plugin used to collect and ship log files. It is the most commonly used Beats module. One of Filebeat’s major advantages is that it slows down its pace if the Logstash service is overwhelmed with data.

Install Filebeat using following command:

sudo apt-get install filebeat

Configure Filebeat on Ubuntu 22.04 LTS

Filebeat, by default, sends data to Elasticsearch. Filebeat can also be configured to send event data to Logstash.

Open configuration file using below command:

sudo nano /etc/filebeat/filebeat.yml

Under the Elasticsearch output section, comment out the following lines:

#output.elasticsearch:
#Array of hosts to connect to.
#hosts: ["localhost:9200"]

Under the Logstash output section, uncomment in the following two lines:

output.logstash
hosts: ["localhost:5044"]

Enable the Filebeat system module:

sudo filebeat modules enable system

Load the index template:

sudo filebeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts= ["0.0.0.0:9200"]'

Start and enable the Filebeat service:

sudo systemctl start filebeat
sudo systemctl enable filebeat

Verify Elasticsearch Reception of Data:

curl -XGET http://ip_address:9200/_cat/indices?v

You can access in browser also:

http://ip_address:9200/_cat/indices?v

Success!!!!

Conclusion!

In this tutorial we’ve gained a solid understanding of how to build, configure, and effectively use the ELK Stack for centralized logging and monitoring. This knowledge will empower you to make informed decisions, detect anomalies, and troubleshoot issues within your systems, ultimately leading to enhanced operational efficiency and better application performance. Hopefully this tutorial was fun and informative. Thank you for following along, happy coding!