Ask HN: What DB to use for huge time series?

The data is going to be sensor readings. Just numeric readouts over time. There will probably be about 8-16 physical sensor points per person per reading. I'll want to retrieve slices of time rather than individual records and likely produce some averages/basic algebra over those slices in order to produce more meaningful data for the rest of the system which is pretty vanilla in terms of data requirements.

Kx (http://kx.com) has been around forever and has a good rep for this sort of thing.

Depending on how 'huge' your timeseries are, you might be pleasantly surprised with Postgres. Postgres scales to multiple TB just fine, and of course the software can be easier to write since you have SQL and ORMs to rely on. It's also an incredibly mature and stable software package, if you're worried about future-proofing.

Some (constantly-growing) timeseries can be stored on a per-row basis, while other (static or older) timeseries can be stored in a packed form (e.g. an array column).

I find that most of the time, "Big Data" isn't really all that big for modern hardware, and so going through all of the extra software work for specialized data stores isn't really all that necessary. YMMV, of course, depending on the nature of your queries.

Development of graphite is effectively dead. The datastore component (carbon and whisper) has design issues, and the official replacement (ceres) hasn't seen any commits this year. There are a some alternatives, though.

For data storage Cyanite [0] speaks the graphite protocol and stores the data in Cassandra. Alternately, InfluxDB [1] speaks the graphite protocol and stores the data in itself

To get the data back out, there's graphite-api [2] which can be hooked up to cyanite [3] or influxdb [4]. You can then connect any graphite dashboard you like, such as grafana [5], to it.

[0] https://github.com/pyr/cyanite [1] http://influxdb.com [2] https://github.com/brutasse/graphite-api [3] https://github.com/brutasse/graphite-cyanite [4] https://github.com/vimeo/graphite-influxdb [5] http://grafana.org

I have used http://influxdb.com/, I have used it with a few million records. Getting the data out is a bit slow because it goes over HTTP. Also make your InfluxDB library of choice can deal with HTTP chunking. I found that if you request a lot of data from InfluxDB and the system does not have enough memory, the process will silently die.

If you have mega huge data http://opentsdb.net/ seems pretty decent, however I have not tried it out.