Logstash is a log processing supertool written in ruby and packaged using the java implementation of ruby, jruby. In other words, it's a ruby app that runs on java. Logstash accepts just about any type of data you can throw at it (apache logs file, syslog data, Windows logs, random data from the 80's), parses it and creates meaningful, structured, searchable information out of it.

Elasticsearch is a high performance database that utilizes the search engine library, Apache Lucene.

Kibana is a powerful browser-based dashboard built with HTML and Javascript that is the primary interface to the mounds of data stored in Elasticsearch.

Together these create the ELK stack. Logstash munges the data, elasticsearch sifts through the data and kibana presents the data.

Installation

Check here for the latest builds

1) Install java

apt-get install openjdk-7-jre-headless

2) Download and install logstash and elasticsearch

wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.4.deb
wget https://download.elasticsearch.org/logstash/logstash/packages/debian/logstash_1.4.2-1-2c0f5a1_all.deb
wget https://download.elasticsearch.org/logstash/logstash/packages/debian/logstash-contrib_1.4.2-1-efd53ef_all.deb
dpkg -i elasticsearch-1.3.4.deb
dpkg -i logstash_1.4.2-1-2c0f5a1_all.deb
dpkg -i logstash-contrib_1.4.2-1-efd53ef_all.deb

Optionally, you can follow these steps to add the repos to your system and install using your package manager. [HINT: This makes upgrading easier]

3) Configure

3) Install kibana

wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.1.tar.gz
tar -xzvf kibana-3.1.1.tar.gz

sudo vi ~/kibana-3.0.1/config.js
elasticsearch: "http://"+window.location.hostname+":80",


sudo mkdir -p /var/www/kibana3

sudo cp -R ~/kibana-3.0.1/* /var/www/kibana3/

4) Install nginx

Download the modifed nginx config from here