For Time Series Data there is also TempoDB
Cassandra vs MongoDB For Time Series Data
71–80 of 82 posts
Re: Cassandra vs MongoDB For Time Series Data
#72Earlier quoted context omitted.
I see. From what I know about Cassandra, this is a much more expensive write than doing it as a new row. To do this he has to be using dynamic columns, and those are stored as one serialized blob per row. So the more data you have in the row, the more expensive the deserialization/reserialization is with each column you add. For very large series this could be an issue. But it sounds like this is tolerable for his ap…
1.) You do not have to use dynamic columns for this. Unfortunately I've found in my own experience, as Cassandra has matured over the last year, alot of terminology has fallen in and out of fashion and its hard to recognize what is actually current. Dynamic columns in CQL3 has nothing to do with the behavior OP is talking about and in dynamic columns are sort-of a deprecated feature in Cassandra 1.2. In CQL3, OP's us…
nemothekid is correct. We are not using CQL, we are using Thrift and are running with real columns.
Re: Cassandra vs MongoDB For Time Series Data
#73I'm assuming this website is being served from S3 based on the HTTP headers. As I'm typing this, I get an empty Content-type: header. Is that a configuration oversight or is this par for the course if serving web pages from S3?
An odd oversight. Thanks for pointing it out. I'll see what's up with that.
Re: Cassandra vs MongoDB For Time Series Data
#74Would be really useful to get more background before/after and a lot more detail on the structures, the clustering and the queries required - and any perf improvements that were attempted previously successful or not - and any lessons learned from making the change.
Good ideas, I'll see if I can generate some time to post more about what we did.
Re: Cassandra vs MongoDB For Time Series Data
#75For what its worth, we have been using Cassandra for storing time series for about 2 years now at ~2k writes a sec. I would say every issue was self-induced and Cassandra has been amazingly patient with us. It works amazing with this scenario. We experimented with MongoDB a lot initially (along with riak, hbase, etc) and found about the same thing. Turns out using a database in a way its designed to work turns out in…
I understand the nervousness, but I was able to convert a thrift/hector time series model to exactly equivalent CQL3 without too much trouble. The (perhaps non-obvious) options involved were "WITH COMPACT STORAGE" for wide rows and "WITH CLUSTERING ORDER BY (blah DESC)" for a reversed comparator. http://www.datastax.com/documentation/cassandra/1.2/webhelp/...
Thanks for the pointers. We are also looking at that as a future plan. We are currently using Thrift which has been rightly pointed out and which I should have mentioned.
Re: Cassandra vs MongoDB For Time Series Data
#76Makes one wonder what the performance of Mysql/Memcached would have been. From my work experience, time-series data is quite stable in definition. I would see this more as a business case for a relational database than a NoSQL database.
Disk usage patterns are the problem with most relational stores (and were for Mongo). When you do a query on time series data, you want to look up a start, read some data, and stop when you hit the end point. Getting it laid out largely sequentially on disk is a big win there.
Re: Cassandra vs MongoDB For Time Series Data
#77Makes one wonder what the performance of Mysql/Memcached would have been. From my work experience, time-series data is quite stable in definition. I would see this more as a business case for a relational database than a NoSQL database.
Disk usage patterns are the problem with most relational stores (and were for Mongo). When you do a query on time series data, you want to look up a start, read some data, and stop when you hit the end point. Getting it laid out largely sequentially on disk is a big win there.
What GB/TByte Volume are we talking about here? Since memcache would be able to store some parts in RAM.
Re: Cassandra vs MongoDB For Time Series Data
#78For Time Series Data there is also TempoDB
how's that, anybody try using it ?
Re: Cassandra vs MongoDB For Time Series Data
#79i work on scada systems which usually come with a time series database built in so that operators can do some basic plotting. Often these products have a 10 or 15 year legacy. The scada vendors seem to be careful to avoid making the time series database and plotting tools which come with the HMI packages too powerful, as this might cut in to their sales of Historian type products. If it wasn't a commodity already, th…
What about Graphite http://graphite.wikidot.com/faq ? I've never tried but it is described as "Scalable Realtime Graphing". It seems to use an internal database so maybe it's not ok for you.. EDIT: I read another comment from you, you said it's rrd-like so it gets rid of old data, not what you're looking for..
The standard 1s time resolution on Graphite/Whisper also seemed to be a limiting factor for use with some of these systems, where you want to observe things on the order of milliseconds (or beyond).
Re: Cassandra vs MongoDB For Time Series Data
#80Earlier quoted context omitted.
TokuDB doesn't currently have a drop-in replacement strategy, so you'd need to migrate your whole cluster. A daunting task with hundreds of shards!
Such a thing could never exist, because the point of tokumx is to change the storage system, so at some point you have to change the storage over and that's just going to be a rewrite of all your data. It sucks but that's the way it is.
Yes, but if I could replace the replicas and let them refill one at a time, then replace the masters, I could do a rolling conversion.
But replication from MongoDB to TokuDB does not work.