ELK-Tutorial

Elastic Search

Elastic search is a indexing and querying based over apache’s lucene engine.

It helps download link: elastic search

  cd elasticsearch
  bin/elasticsearch

You can check if elasticsearch is up and running at localhost:9200

Open another terminal and proceed for kibana.

Kibana helps visualizing data using after querying using elastic search.

kibana

download link: kibana

  cd kibana
  bin/kibana

You can access kibana GUI at localhost:5601

Logstash helps normazling data from various data soruces such as apache, log events, sql and other data sources.

Also, logstash enhances data us varius filters such as geoip, grok, matcher and many more.
Open another terminal and proceed for logstash

Logstash

download link: logstash

download sample-data from logs

unzip the logs into a folder

download apache.conf file from github repo apache.conf

Now open apache.conf in text editor and change the path field to location of log we downloaded above.

In apache.conf, change

file{
path => path_of_logs_downloaded/logs_file
.
.
}

  cd logstash
  bin/logstash -f "file_path"/apache.conf
  Change file_path to path of file where you have stored apache.conf

Indexing

Open kibana in your browser, default localhost:5601.
Go to mangament
Select index patterns under kibana
Create Index Patterns
Type logstash-* and click on next step
Select @timestamp

Kibana console

Go to Discover, you will see your index pattern
As it shows data fro last 15 minutes, we have to change the time range
select range of year from jan, 2014 to dec 2015. (as data has timestamps within that period)
select absolute
You can see your data

New visualization

Go to visualization
select add new filter and choose data table
select paramters as specified in figure. You are free to explore.
Save the visualization with a name
Add 2-3 other visualizations and save them

Dashboard

Go to dashboard
Select add from above. You will see all of your saved visualizations.
Select from them and create a dashboard

References

https://www.elastic.co