Ask HN: What DB to use for huge time series?

#1

Post by BWStearns » Thu, Sep 25, 2014, 5:53 PM UTC

Hi HN, I wanted to know if anyone had good recommendations for a database for massive timeseries. I took a look at InfluxDB and Druid, both of which look promising but they're young projects and I don't want to strand myself with a deprecated component at the core of the system I'm working on. Does anyone have any suggestions/advice/experience they can share to provide some guidance here?

thanks in advance!

Re: Ask HN: What DB to use for huge time series?

#2

Post by sdab » Thu, Sep 25, 2014, 6:03 PM UTC

It would be useful to know what "huge" means here. And how you want to look up the data.

That said, I've used Cassandra in the past for timeseries data as one of the useful queries that can be made is a range query (if the composite key is set up correctly)

Re: Ask HN: What DB to use for huge time series?

#3

Post by caw » Thu, Sep 25, 2014, 6:04 PM UTC

My company is currently using Mongo, and while it works, I wouldn't recommend it. We're looking at Cassandra and Elasticsearch, which seems to be a lot more promising.

Re: Ask HN: What DB to use for huge time series?

#4

Post by pmalynin » Thu, Sep 25, 2014, 6:04 PM UTC

If its only numbers, then Graphite with Grafana for visualization. Otherwise MongoDB can be good as well.

Re: Ask HN: What DB to use for huge time series?

#5

Post by ankushio » Thu, Sep 25, 2014, 6:05 PM UTC

Have you considered opentsdb or graphite? I love graphite because of the nice frontend interface and functionality it provides for visualizing and transforming your metrics.

Re: Ask HN: What DB to use for huge time series?

#6

Post by arenaninja » Thu, Sep 25, 2014, 6:05 PM UTC

caw wrote: post #3

My company is currently using Mongo, and while it works, I wouldn't recommend it. We're looking at Cassandra and Elasticsearch, which seems to be a lot more promising.

The number of horror stories I've seen about mongo is up to around 10 this month alone.

I'm now glad I never made the jump... in the meantime, pgsql is still on my list

Re: Ask HN: What DB to use for huge time series?

#7

Post by TheAceOfHearts » Thu, Sep 25, 2014, 6:07 PM UTC

If you're up for considering a cloud service, you might want to check out Treasure Data (http://treasuredata.com/).

The free plan allows 10M records per month with a maximum capacity of 150M.

Full disclosure: I work there.

Re: Ask HN: What DB to use for huge time series?

#8

Post by virtuabhi » Thu, Sep 25, 2014, 6:07 PM UTC

To get a relevant recommendation, you'll have to describe two things at least - data and queries. How is the data generated? What is stored in the data? What are the queries you plan to run?

Re: Ask HN: What DB to use for huge time series?

#9

Post by whalesalad » Thu, Sep 25, 2014, 6:10 PM UTC

I have not used it but will be experimenting with influxdb soon for storing timeseries data.

Re: Ask HN: What DB to use for huge time series?

#10

Post by herman5 » Thu, Sep 25, 2014, 6:10 PM UTC

Have a look at TempoDB - built specifically for timeseries data (https://tempo-db.com/about/)