Elasticsearch is used index and make all logs searchable.

Kibana provides a web interface to the search and configure visualization.

Syslog-ng can send logs directly to elasticsearch, so the setup is like ELK without using logstash.

Install syslog-ng, this will replace the current rsyslog.

:~# aptitude install syslog-ng

Code: https://bitbucket.org/snippets/iansamuel/LLEag

puppet apply /etc/puppet/manifests/syslog-ng-mod-elasticsearch.pp

https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html

:~# wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

:~# echo "deb http://packages.elastic.co/elasticsearch/1.7/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.7.list

:~# aptitude update && aptitude install elasticsearch default-jre-headless

:~# update-rc.d elasticsearch defaults 95 10

:~# service elasticsearch start
 * Starting Elasticsearch Server

https://www.elastic.co/downloads/kibana

:~# cd /usr/src

:/usr/src# wget -c https://download.elastic.co/kibana/kibana/kibana-4.1.2-linux-x64.tar.gz

:/usr/src# cd /usr/local/

:/usr/local# tar xvzf /usr/src/kibana-*gz

:/usr/local# ln -s kibana-4.1.2-linux-x64 kibana

:/usr/local# echo "/usr/local/kibana/bin/kibana | logger -t kibana &" >> /etc/rc.local

:/usr/local# /usr/local/kibana/bin/kibana | logger -t kibana &

/etc/syslog-ng/conf.d/elasticsearch.conf:

@include "scl/elasticsearch/plugin.conf"

destination d_elastic {
       elasticsearch(
         index("syslog-ng")
       );
};

log {
    source(s_src);
    destination(d_elastic);
    flags(flow-control);
};

:~# service syslog-ng reload 
 * Reload system logging syslog-ng

Visit your kibana URL http://hostname:5601/

Set your index pattern to syslog-ng to match the index() of syslog-ng.

Start by clicking on the Discover tab to verify your syslog data is being captured and categorized correctly.