Logstash is a log processing supertool written in ruby and packaged using the java implementation of ruby, jruby. In other words, it's a ruby app that runs on java. Logstash accepts just about any type of data you can throw at it (apache logs file, syslog data, Windows logs, random data from the 80's), parses it and creates meaningful, structured, searchable information out of it.
Elasticsearch is a high performance database that utilizes the search engine library, Apache Lucene.
Kibana is a powerful browser-based dashboard built with HTML and Javascript that is the primary interface to the mounds of data stored in Elasticsearch.
Together these create the ELK stack. Logstash munges the data, elasticsearch sifts through the data and kibana presents the data.
Installation
Check here for the latest builds
1) Install java
apt-get install openjdk-7-jre-headless
2) Download and install logstash and elasticsearch
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.4.deb
wget https://download.elasticsearch.org/logstash/logstash/packages/debian/logstash_1.4.2-1-2c0f5a1_all.deb
wget https://download.elasticsearch.org/logstash/logstash/packages/debian/logstash-contrib_1.4.2-1-efd53ef_all.deb
dpkg -i elasticsearch-1.3.4.deb
dpkg -i logstash_1.4.2-1-2c0f5a1_all.deb
dpkg -i logstash-contrib_1.4.2-1-efd53ef_all.deb
Optionally, you can follow these steps to add the repos to your system and install using your package manager. [HINT: This makes upgrading easier]
3) Configure
3) Install kibana
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.1.tar.gz
tar -xzvf kibana-3.1.1.tar.gz
sudo vi ~/kibana-3.0.1/config.js
elasticsearch: "http://"+window.location.hostname+":80",
sudo mkdir -p /var/www/kibana3
sudo cp -R ~/kibana-3.0.1/* /var/www/kibana3/
4) Install nginx
Download the modifed nginx config from here