Druid: fast column-oriented distributed data store

Druid is quickly becoming the leading open source solution for building highly scalable analytics. We evaluated it for getstream.io. Unfortunately the setup and maintenance is still very labour intensive. For startups that's a concern. Many larger companies we spoke to were extremely happy about running Druid in production though.

Re: Druid: fast column-oriented distributed data store

#5

Post by jnordwick » Thu, Mar 31, 2016, 10:21 PM UTC

Every open source column database I've seen is very poor: text, no decent array oriented ability (give me the prevoius row), slow, json output, etc. When will somebody get it right?

Re: Druid: fast column-oriented distributed data store

#6

Post by ktamura » Thu, Mar 31, 2016, 10:42 PM UTC

jnordwick wrote: post #5

Every open source column database I've seen is very poor: text, no decent array oriented ability (give me the prevoius row), slow, json output, etc. When will somebody get it right?

This is simply false. PostgreSQL + cstore_ftw, Presto, Impala, etc. all support "array oriented ability" via window functions.

Re: Druid: fast column-oriented distributed data store

#7

Post by ciconia » Thu, Mar 31, 2016, 10:52 PM UTC

sspring1 wrote: post #2

Great, just what we need, a Druish database.

Funny, it doesn't look druish...

Re: Druid: fast column-oriented distributed data store

#8

Post by jnordwick » Thu, Mar 31, 2016, 11:07 PM UTC

ktamura wrote: post #6

jnordwick wrote: post #5

Every open source column database I've seen is very poor: text, no decent array oriented ability (give me the prevoius row), slow, json output, etc. When will somebody get it right?

This is simply false. PostgreSQL + cstore_ftw, Presto, Impala, etc. all support "array oriented ability" via window functions.

I mean like give me the preceding row from the current. Very useful for calculating anything from a simple rate of change to a properly doing a temporal database. Arrays as a column type doesn't get you that.

Re: Druid: fast column-oriented distributed data store

#9

Post by rosalinekarr » Thu, Mar 31, 2016, 11:13 PM UTC

Yeah, more database solutions that's what we need.

https://xkcd.com/927/

Re: Druid: fast column-oriented distributed data store

#10

Post by feral » Thu, Mar 31, 2016, 11:22 PM UTC

tschellenbach wrote: post #4

Druid is quickly becoming the leading open source solution for building highly scalable analytics. We evaluated it for getstream.io. Unfortunately the setup and maintenance is still very labour intensive. For startups that's a concern. Many larger companies we spoke to were extremely happy about running Druid in production though.

Second that experience:

When I was at Optimizely, my team chose Druid for a large-scale analytics application, after a pretty extensive benchmarking. It was very impressive, though not trivial to set up.

Blog post with a bit more detail: https://medium.com/engineers-optimizely/slicing-and-dicing-d...