This is an old revision of the document!
Elasticsearch is used index and make all logs searchable.
Kibana provides a web interface to the search and configure visualization.
Syslog-ng can send logs directly to elasticsearch, so the setup is like ELK without using logstash.
Install syslog-ng, this will replace the current rsyslog.
:~# aptitude install syslog-ng
https://bitbucket.org/snippets/iansamuel/EEkay
puppet apply /etc/puppet/manifests/syslog-ng-mod-elasticsearch.pp
https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
:~# wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - :~# echo "deb http://packages.elastic.co/elasticsearch/1.7/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.7.list :~# aptitude update && aptitude install elasticsearch default-jre-headless :~# update-rc.d elasticsearch defaults 95 10 :~# service elasticsearch start * Starting Elasticsearch Server
https://www.elastic.co/downloads/kibana
:~# cd /usr/src :/usr/src# wget -c https://download.elastic.co/kibana/kibana/kibana-4.1.2-linux-x64.tar.gz :/usr/src# cd /usr/local/ :/usr/local# tar xvzf /usr/src/kibana-*gz :/usr/local# ln -s kibana-4.1.2-linux-x64 kibana :/usr/local# echo "/usr/local/kibana/bin/kibana | logger -t kibana &" >> /etc/rc.local :/usr/local# /usr/local/kibana/bin/kibana | logger -t kibana &
/etc/syslog-ng/conf.d/elasticsearch.conf:
@include "scl/elasticsearch/plugin.conf" destination d_elastic { elasticsearch( index("syslog-ng") ); }; log { source(s_src); destination(d_elastic); flags(flow-control); };
:~# service syslog-ng reload * Reload system logging syslog-ng
Visit your kibana URL http://hostname:5601/
Set your index pattern to syslog-ng to match the index() of syslog-ng.
Start by clicking on the Discover tab to verify your syslog data is being captured and categorized correctly.