InfluxDB vs. Cassandra for timeseries data • Hacker News Board

Re: InfluxDB vs. Cassandra for timeseries data

#31

Post by soundoflight » Thu, Sep 08, 2016, 4:32 PM UTC

bsg75 wrote: post #27

pauldix wrote: post #24

Earlier quoted context omitted.

We're working on the cardinality problem. Will be resolved in an upcoming release. Moving the index over to a disk based format that will hopefully still be fast and not sacrifice lookup performance.

Can you explain the cardinality problem in a bit more detail? Its come up more than once in this thread.

https://docs.influxdata.com/influxdb/v1.0/concepts/glossary/...

You want to keep the amount of different data that you are indexing/tagging on low. As an example with my situation, I was tracking what could be amounted to connections between nodes in a very large tree. I had a lot of distinct pairs, which means that I had a high cardinality. When the cardinality increases a query that used to take a millisecond to load could move to a couple seconds.

Re: InfluxDB vs. Cassandra for timeseries data

#32

Post by bsg75 » Thu, Sep 08, 2016, 4:48 PM UTC

soundoflight wrote: post #31

bsg75 wrote: post #27

Earlier quoted context omitted.

Can you explain the cardinality problem in a bit more detail? Its come up more than once in this thread.

https://docs.influxdata.com/influxdb/v1.0/concepts/glossary/... You want to keep the amount of different data that you are indexing/tagging on low. As an example with my situation, I was tracking what could be amounted to connections between nodes in a very large tree. I had a lot of distinct pairs, which means that I had a high cardinality. When the cardinality increases a query that used to take a millisecond to lo…

So InfluxDB v1.0 has issues with the cardinality of the "primary key" (or candidate keys) gets high?

At what level of keys or tags did you start to see query performance become problematic?

Re: InfluxDB vs. Cassandra for timeseries data

#33

Post by brianwawok » Fri, Sep 09, 2016, 2:13 AM UTC

bsg75 wrote: post #28

brianwawok wrote: post #21

Earlier quoted context omitted.

I don't understand this benchmark at all. It says performance of a 1000 node cluster, but then shows 100k inserts per second in Cassandra. Then later follow up comments say that this test was on a single machine. Without seeing the schema, 100k inserts / sec is reasonable for a single machine. For 1000 machines it would mean there is a pretty massive configuration issue. If you are going to benchmark a distributed sy…

Does it ever make sense to use Cassandra on a single node for anything but dev/test? I am under the impression that Cassandra's performance comes from its distribution capabilities.

It does not make sense to only use 1 node. It's not designed to be a fast 1 node DB.

In fact for most dev I use 3 nodes on my laptop, and most of our "unit" tests are multi-node as well (closer to integration tests by most measures).

Re: InfluxDB vs. Cassandra for timeseries data

#34

Post by brianwawok » Fri, Sep 09, 2016, 2:16 AM UTC

pauldix wrote: post #20

coredog64 wrote: post #17

Earlier quoted context omitted.

Hasn't that work already been done? Cyanite and KairosDB both plug in to the broader Graphite ecosystem (more or less) and use Cassandra as a data store. Time series data has also been a particular focus in the Cassandra community. DTCS was too complicated, so they came up with the easier and faster TWCS. I don't think this is on you, but I'd love to see a comparison with the latest stable 3.x and a multiple node clu…

We'll be doing comparisons against Kairos and OpenTSDB in the coming months. We just get asked about Cassandra specifically quite a bit.

If you want to test Cassandra, please test at least 9 nodes and have someone with Cassandra setup experience configure your cluster.