Elasticsearch is used index and make all logs searchable.
Kibana provides a web interface to the search and configure visualization.
Syslog-ng can send logs directly to elasticsearch, so the setup is like ELK without using logstash.
Install syslog-ng, this will replace the current rsyslog.
:~# aptitude install syslog-ng
Puppet Manifest: https://bitbucket.org/snippets/iansamuel/LLEag
puppet apply /etc/puppet/manifests/syslog-ng-mod-elasticsearch.pp
https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
https://www.elastic.co/downloads/kibana
Puppet Manifest: https://bitbucket.org/snippets/iansamuel/yqj5k
puppet apply /etc/puppet/manifests/elasticsearch-kibana.pp
/etc/syslog-ng/conf.d/elasticsearch.conf:
@include "scl/elasticsearch/plugin.conf" source s_net { udp(); }; # All interfaces destination d_elastic { elasticsearch( host("localhost") index("syslog-ng") ); }; log { source(s_net); destination(d_elastic); flags(flow-control); };
:~# service syslog-ng reload * Reload system logging syslog-ng
Visit your kibana URL http://hostname:5601/
Set your index pattern to syslog-ng to match the index() of syslog-ng.
Start by clicking on the Discover tab to verify your syslog data is being captured and categorized correctly.