How To Install ELK Stack In Linux

Hello Everyone

Welcome to CloudAffaire and this is Debjeet.

In this series, we will explore one of the most popular log management tools in DevOps better known as ELK (E=Elasticserach, L=Logstash, K=Kibana) stack.

Elastic Stack

What Is ELK stack in DevOps?

The ELK Stack is a collection of three open-source products — Elasticsearch, Logstash, and Kibana — all developed, managed and maintained by Elastic. Elasticsearch is an open source, full-text search and analysis engine, based on the Apache Lucene search engine. Logstash is a log aggregator that collects data from various input sources, executes different transformations and enhancements and then ships the data to various supported output destinations. Kibana is a visualization layer that works on top of Elasticsearch, providing users with the ability to analyze and visualize the data. Together, these different components are most commonly used for monitoring, troubleshooting and securing IT environments, business intelligence and web analytics.

How To Install ELK Stack In Linux?

Step 1: Configure yum repository for your ELK stack.

#######################################
## How To Install ELK Stack In Linux ##
#######################################
## Prerequisites: One Linux system with internet access
## Linux OS: CentOs 7
## IP: 192.168.0.10

## ------------------------
## Configure yum repository
## ------------------------
## Download and install the public signing key
sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

## Create the repository file
sudo vi /etc/yum.repos.d/elasticsearch.repo
---------------------
[elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
---------------------
:wq

#######################################

## How To Install ELK Stack In Linux ##

#######################################

## Prerequisites: One Linux system with internet access

## Linux OS: CentOs 7

## IP: 192.168.0.10

## ------------------------

## Configure yum repository

## ------------------------

## Download and install the public signing key

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

## Create the repository file

sudo vi /etc/yum.repos.d/elasticsearch.repo

---------------------

[elasticsearch]

name=Elasticsearch repository for 7.x packages

baseurl=https://artifacts.elastic.co/packages/7.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

---------------------

:wq

Step 2: Install elasticsearch.

## ---------------------
## Install elasticsearch
## ---------------------
## Install elasticsearch
sudo yum install elasticsearch

## Enable and start elasticsearch
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch 
sudo systemctl status elasticsearch

## Check if elasticsearch installed successfully
curl -X GET "localhost:9200/?pretty"

## View elasticsearch configuration
sudo cat /etc/elasticsearch/elasticsearch.yml

## ---------------------

## Install elasticsearch

## ---------------------

## Install elasticsearch

sudo yum install elasticsearch

## Enable and start elasticsearch

sudo systemctl enable elasticsearch

sudo systemctl start elasticsearch

sudo systemctl status elasticsearch

## Check if elasticsearch installed successfully

curl -X GET "localhost:9200/?pretty"

## View elasticsearch configuration

sudo cat /etc/elasticsearch/elasticsearch.yml

Step 3: Install Kibana.

## --------------
## Install kibana
## --------------
## Install kibana
sudo yum install kibana

## Enable and start kibana
sudo systemctl enable kibana
sudo systemctl start kibana
sudo systemctl status kibana

## View and eidt kibana configuration to connect remotely
sudo vi /etc/kibana/kibana.yml
---------------------
server.host: "192.168.0.10"
---------------------
:wq

## Stop firewall
## or open kibana default port using firewall-cmd --add-port=5601/tcp --permanent
sudo systemctl stop firewalld

## Restart kibana
sudo systemctl restart kibana

## View Kibana dashboard
## Open below url in your browser, it may take some time for kibana to get started
192.168.0.10:5601

## --------------

## Install kibana

## --------------

## Install kibana

sudo yum install kibana

## Enable and start kibana

sudo systemctl enable kibana

sudo systemctl start kibana

sudo systemctl status kibana

## View and eidt kibana configuration to connect remotely

sudo vi /etc/kibana/kibana.yml

---------------------

server.host: "192.168.0.10"

---------------------

:wq

## Stop firewall

## or open kibana default port using firewall-cmd --add-port=5601/tcp --permanent

sudo systemctl stop firewalld

## Restart kibana

sudo systemctl restart kibana

## View Kibana dashboard

## Open below url in your browser, it may take some time for kibana to get started

192.168.0.10:5601

How To Install ELK Stack In Linux

Step 4: Install Logstash.

## ----------------
## Install Logstash
## ----------------
## Install logstash
sudo yum install logstash

## Enable and start logstash
sudo systemctl start logstash
sudo systemctl enable logstash
sudo systemctl status logstash

## View logstash configuration files
sudo ls /etc/logstash/

## Stash your first log from commandline
## After starting Logstash, wait until you see "Pipeline main started" and then enter hello world at the command prompt
sudo /usr/share/logstash/bin/logstash -e 'input { stdin { } } output { stdout {} }'

hello world

## {
##      "@version" => "1",
##       "message" => "hello world",
##    "@timestamp" => 2020-02-16T06:49:51.384Z,
##          "host" => "system1"
## }

## To stop logstash press control + d

## ----------------

## Install Logstash

## ----------------

## Install logstash

sudo yum install logstash

## Enable and start logstash

sudo systemctl start logstash

sudo systemctl enable logstash

sudo systemctl status logstash

## View logstash configuration files

sudo ls /etc/logstash/

## Stash your first log from commandline

## After starting Logstash, wait until you see "Pipeline main started" and then enter hello world at the command prompt

sudo /usr/share/logstash/bin/logstash -e 'input { stdin { } } output { stdout {} }'

hello world

## {

## "@version" => "1",

## "message" => "hello world",

## "@timestamp" => 2020-02-16T06:49:51.384Z,

## "host" => "system1"

## }

## To stop logstash press control + d

To get more details on ELK, please refer below documentation.

https://www.elastic.co/guide/index.html