ELK is a famous and powerful log tool. ELK stands for Elasticsearch, Logstash and Kibana.
Preview:
Prerequisites:
[root@iZ259zncm66Z kibana-6.4.0-linux-x86_64]# java -version java version "1.8.0_25" Java(TM) SE Runtime Environment (build 1.8.0_25-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)
Install ELK:
download and unzip the following files.
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.0.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.4.0.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.4.0-linux-x86_64.tar.gz
Configuration:
- Run elasticsearch.
cd path/to/elasticsearch
bin/elasticsearch
curl http://localhost:9200/
- Run Kibana
cd path/to/kibana
open config/kibana.yml and set elasticsearch.url to point at your Elasticsearch instance, set server.host to external ip
bin/kibana
curl http://localhost:5601
3.Run Logstash on each cosmos node.
cd path/to/logstash
create a cosmos.conf.
bin/logstash -f cosmos.conf
cosmos.conf:
input {
file {
path => "/root/.gaiad/out.log"
codec => multiline {
pattern => "^[\s\}]"
what => "previous"
}
}
}
filter {
if [message] =~ ".*Block\{.*" {
dissect {
mapping => {
"message" => "%{level}[%{logtime}]"
}
}
kv { }
}else if [message] =~ "^E" {
dissect {
mapping => {
"message" => "%{level}[%{logtime}]"
}
}
kv { }
}else{
drop { }
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"] #edit this value to your elasticsearch host
}
#stdout { codec => rubydebug }
}
Then you can manage your logs in one log management server. (Perhaps you will need wait a couple of minutes to see logs on kibana)
Additional:
i droped some logs like precommited in cosmos.conf. if you want to see them, replace drop {} with:
dissect {
mapping => {
"message" => "%{level}[%{logtime}]"
}
}
kv { }
I hope you will enjoy this amazing tool.
