[Linux] ELK (Elasticsearch + Logstash + Kibana) 설치

환경

OS : CentOS7

apache : 2.4.6

JAVA : 

ElasticSearch 5.0.2

Logstash 5.0.2

Kibana 5.0.2

1. JAVA 설치                                                                               

# curl -LO -H "Cookie: oraclelicense=accept-securebackup-cookie" \ "http://download.oracle.com/otn-pub/java/jdk/8u151-b12/e758a0de34e24606bca991d704f6dcbf/jdk-8u151-linux-i586.rpm"

# rpm -Uvh jdk-8u151-linux-x64.rpm

# vi /etc/profile

export JAVA_HOME=/usr/java/default

export PATH=$PATH:$JAVA_HOME/bin

export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar

# source /etc/profile

# alternatives --config java    // 방금 설치한 JAVA version을 선택해줍니다.

2. ElasticSearch 설치                                                                    

1. 먼저 저장소 설정을 해줍니다.

# vi /etc/yum.repos.d/elasticsearch.repo

[elasticsearch-5.x]

name=Elasticsearch repository for 5.x packages

baseurl=https://artifacts.elastic.co/packages/5.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

2. Elasticsearch 설치

# yum -y install elasticsearch

3. Elasticsearch 시작

# systemctl start elasticsearch

# systemctl enable elasticsearch

4. 테스트

# curl http://127.0.0.1:9200

# curl http://127.0.0.1:9200/_aliases?pretty

# curl -X PUT "http://127.0.0.1:9200/test_index"

# curl http://127.0.0.1:9200/_aliases?pretty

# curl http://127.0.0.1:9200/test_index/_settings?pretty

# curl -X PUT "http://127.0.0.1:9200/test_index/doc01/1" -d '{

    "subject" : "Test Post No.1",

    "description" : "This is the initial post",

    "content" : "This is the test message for using Elasticsearch."

}'

# curl "http://127.0.0.1:9200/test_index/_mapping/doc01?pretty"

# curl "http://127.0.0.1:9200/test_index/doc01/1?pretty"

# curl "http://127.0.0.1:9200/test_index/doc01/_search?q=description:initial&pretty=true “

3. Kibana 설치                                                                             

1. Kibana 설치

# yum -y install kibana

# chown -R kibana. /etc/pki/tls/*

2. 설정파일 수정

# vi /etc/kibana/kibana.yml

server.host: "0.0.0.0"

server.name: "dlp.srv.world"

elasticsearch.url: "http://localhost:9200"

server.ssl.enabled: true

server.ssl.certificate: /etc/pki/tls/certs/server.crt

server.ssl.key: /etc/pki/tls/certs/server.keyi

3. Kibana 시작

# systemctl start kibana 

# systemctl enable kibana

4. 방화벽 설정

# firewall-cmd --add-port=5601/tcp --permanent 

# firewall-cmd --reload 

4. Logstash 설치                                                                        

1. Logstash 설치

# yum -y install logstash

2. 설정파일 수정

# vi /etc/logstash/conf.d/sshd.conf

input {

  file {

    type => "seucure_log"

    path => "/var/log/secure"

  }

}

filter {

  grok {

    add_tag => [ "sshd_fail" ]

    match => { "message" => "Failed %{WORD:sshd_auth_type} for %{USERNAME:sshd_invalid_user} from %{IP:sshd_client_ip} port %{NUMBER:sshd_port} %{GREEDYDATA:sshd_protocol}" }

  }

}

 

output {

  elasticsearch {

    index => "sshd_fail-%{+YYYY.MM}"

  }

}

3. 

# chgrp logstash /var/log/secure

# chmod 640 /var/log/secure

# systemctl start logstash

# systemctl enable logstash

4. 로그가 잘 수집되는지 확인

# curl localhost:9200/_cat/indices?v

# curl localhost:9200/sshd_fail-2017.05/_search?pretty

5. 웹서버의 로그 파일을 가져오기 위해서 filebeat를 설치해줍니다.

# yum -y install filebeat

# systemctl start filebeat 

# systemctl enable filebeat

# /usr/share/filebeat/scripts/import_dashboards -only-index

6. 그래서 kibana에 접속해서 확인해봅니다.

http://IP주소:5601 또는 http://도메인 주소:5601