Logstash joins Elasticsearch
elasticsearch.com
Logstash joins Elasticsearch
1–10 of 60 posts
Re: Logstash joins Elasticsearch
#2I'm confused. Can someone explain to me why this is so obviously interesting, yet not worth discussing, that it stands - as of 2 hours after submission - at 75 points with zero comments?
Honestly, I've never heard of either company, although I obviously wish them the best of luck. Am I just out of touch?
Re: Logstash joins Elasticsearch
#3Logstash, Elasticsearch and Kibana are just fantastic. After being unsatisfied with a whole bunch of Logging As A Service providers (I tried loggly.com, logentries.com and splunkstorm.com) I spent an afternoon setting up Logstash and co and couldn't be happier.
There's a neat demo of Kibana here: http://demo.kibana.org/#/dashboard/elasticsearch/Logstash%20...
The only thing that isn't fully baked in with this stack is alerts (e.g. sending an email if a certain error log message comes in), but you can do that using Logstash filters and outputs, although there's no pretty UI.
There are some excellent Chef cookbooks for setting up Logstash and friends too:
- Logstash: https://github.com/lusis/chef-logstash
- Elasticsearch: https://github.com/elasticsearch/cookbook-elasticsearch
- Kibana: https://github.com/lusis/chef-kibana
Re: Logstash joins Elasticsearch
#4Logstash is really great and Jordan is approachable and very helpful. To all interested, I recommend joining their IRC channel (#logstash on Freenode) and talking to the people there a bit.
Congrats :)
Re: Logstash joins Elasticsearch
#5For anyone who can't immediately see the significance..this is Elasticsearch's entry into real-time log analytics. There is plenty of room for innovation and financial opportunity in this area, given the success of the $5 billion valued Splunk along with companies like SumoLogic and LogLogic.
What's most interesting is that Elasticsearch seems like a completely open source (and widely used) offering of a product that Splunk charges close to oracle pricing for.
Shameless plug: If you're looking for an opportunity at a well-funded true real-time analytics company in silicon valley...feel free to ping me. There's lots of exciting and fun work to do in this area.
Re: Logstash joins Elasticsearch
#6Logstash is awesome. We use it at Swiftype to index all our logs and it's super helpful nailing down support requests and bugs (using Kibana).
Since you can access the logs via the Elasticsearch API, we made users' recent logs available to them in our dashboard: https://swiftype.com/blog/api-logs.html
Re: Logstash joins Elasticsearch
#7I'm confused. Can someone explain to me why this is so obviously interesting, yet not worth discussing, that it stands - as of 2 hours after submission - at 75 points with zero comments? Honestly, I've never heard of either company, although I obviously wish them the best of luck. Am I just out of touch?
Logstash + Elasticsearch + Kibana is the biggest thing in opensource operational tools since Nagios.
Re: Logstash joins Elasticsearch
#8logstash + elasticsearch are pretty amazing. however, if you are generating a high rate of log entries you may want to consider using mozilla hekad instead (http://hekad.readthedocs.org/en/latest/). on our servers logstash was running around 20% CPU during quite periods while hekad was running around 1-2% CPU. while during busy periods i think logstash was going up to 100% CPU while hekad was sitting around 20-30% CPU.
hekad is written in go which compiles down to native code while logstash is written in jruby which is not the most performant runtime.
Re: Logstash joins Elasticsearch
#9I'm confused. Can someone explain to me why this is so obviously interesting, yet not worth discussing, that it stands - as of 2 hours after submission - at 75 points with zero comments? Honestly, I've never heard of either company, although I obviously wish them the best of luck. Am I just out of touch?
There's always something to learn and there's always somet technology or company we don't know (yet).
Of the three, you should at least could have heard of ElasticSearch: it is a general purpose search server based on Apache Lucene like its cousin Apache SOLR (maybe you know it), but with REST API, JSON support etc..
You can use it for searching stuff and one (but not the only one) scenario is logging. In this case you use some other software (like Logstash but I think there are others too) to collect logs from different sources (i.e. syslog for operating systems, ruby apps via gems, raw tcp etc..) and according to a friend of mine that uses it for this purpose it's very good for its speed, easy to use, to scale etc..
N.B. I invite anyone more expert to elaborate/correct what I've said, I've just used Apache SOLR in the past and never tried ES
Re: Logstash joins Elasticsearch
#10I'm confused. Can someone explain to me why this is so obviously interesting, yet not worth discussing, that it stands - as of 2 hours after submission - at 75 points with zero comments? Honestly, I've never heard of either company, although I obviously wish them the best of luck. Am I just out of touch?
Logstash + Elasticsearch + Kibana is the biggest thing in opensource operational tools since Nagios.
I'd put CMSs (puppet, chef, etc.) in between, but otherwise I agree. These were the tools making a huge difference.