====== Syslog-ng / Elasticsearch / Kibana ======

Elasticsearch is used index and make all logs searchable.

Kibana provides a web interface to the search and configure visualization.

Syslog-ng can send logs directly to elasticsearch, so the setup is like ELK without using logstash.

<note>
I am using Ubuntu 14.04 LTS Server.
</note>

====== Installation ======

===== Syslog-ng =====

Install syslog-ng, this will replace the current rsyslog.

<code>
:~# aptitude install syslog-ng
</code>

<note tip>
Syslog-ng 3.7 can run elasticsearch directly, because the version in Ubuntu 14.04 is 3.5 we use a glue module.
</note>

Puppet Manifest: https://bitbucket.org/snippets/iansamuel/LLEag

<code>
puppet apply /etc/puppet/manifests/syslog-ng-mod-elasticsearch.pp
</code>

===== ElasticSearch / Kibana =====

https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html \\
https://www.elastic.co/downloads/kibana

Puppet Manifest: https://bitbucket.org/snippets/iansamuel/yqj5k

<code>
puppet apply /etc/puppet/manifests/elasticsearch-kibana.pp
</code>
====== Configuration ======

===== Syslog-ng =====

**/etc/syslog-ng/conf.d/elasticsearch.conf**:
<code>
@include "scl/elasticsearch/plugin.conf"

source s_net { udp(); };  # All interfaces

destination d_elastic {
       elasticsearch(
         host("localhost")
         index("syslog-ng")
       );
};

log {
    source(s_net);
    destination(d_elastic);
    flags(flow-control);
};
</code>

<code>
:~# service syslog-ng reload 
 * Reload system logging syslog-ng
</code>


===== Kibana =====

Visit your kibana URL http://hostname:5601/

Set your index pattern to syslog-ng to match the index() of syslog-ng.

Start by clicking on the //Discover// tab to verify your syslog data is being captured and categorized correctly.