Cassandra vs MongoDB For Time Series Data
relistan.com
Cassandra vs MongoDB For Time Series Data
1–10 of 82 posts
Re: Cassandra vs MongoDB For Time Series Data
#2KariosDB (https://github.com/proofpoint/kairosdb) is a time-series database for Casssandra. It's an OpenTSDB (http://opentsdb.net) rewrite that supports sub-second precision.
Re: Cassandra vs MongoDB For Time Series Data
#3One thing I always find interesting about these kinds of problems is that most DBs don't describe how they're implemented. It's easy to use the wrong tool, and then once you learn how e.g. Mongo is implemented, it's obvious, "oh, that's why things are slow".
I'd love to see http://eagain.net/articles/git-for-computer-scientists/, but for every DB technology.
Re: Cassandra vs MongoDB For Time Series Data
#4Would be really useful to get more background before/after and a lot more detail on the structures, the clustering and the queries required - and any perf improvements that were attempted previously successful or not - and any lessons learned from making the change.
Re: Cassandra vs MongoDB For Time Series Data
#5We had some major issues using MongoDB for time series data due to the write volume (real time sensor data). The solution for us was Riak, mainly because we never need to update a vector clock, leaving us without the need for conflict resolution (and last write wins is fine for sensor data).
Re: Cassandra vs MongoDB For Time Series Data
#6One thing I always find interesting about these kinds of problems is that most DBs don't describe how they're implemented. It's easy to use the wrong tool, and then once you learn how e.g. Mongo is implemented, it's obvious, "oh, that's why things are slow". I'd love to see http://eagain.net/articles/git-for-computer-scientists/ , but for every DB technology.
You nailed it. I think the issue is that people don't know what questions to ask when gathering the requirements.
I'd like to know more about how this part of the article came to be:
"This choice was made early on and it was supposed to be a temporary one."
HOW was that choice made. What requirements were out there. I think too many people choose Mongo because they believe it's "schemaless"[1] and faster for development, but don't look at the requirements for their actual use case.
[1] - There's always a schema. Either it's informally defined by your code or represented formally somewhere else.
Re: Cassandra vs MongoDB For Time Series Data
#7One thing I always find interesting about these kinds of problems is that most DBs don't describe how they're implemented. It's easy to use the wrong tool, and then once you learn how e.g. Mongo is implemented, it's obvious, "oh, that's why things are slow". I'd love to see http://eagain.net/articles/git-for-computer-scientists/ , but for every DB technology.
A pattern I've seen on a lot of the negative Mongo articles has been people using it for things they probably shouldn't.
I've yet to use it for any load, and am struggling to triangulate from all the articles I read on whether it does/doesn't scale efficiently.
All the issues I have hit so far have been self-inflicted, it is still one of the best new technologies I've used in years - but is has taken a while to stop thinking in SQL equivalents and start thinking natively.
Re: Cassandra vs MongoDB For Time Series Data
#8i work on scada systems which usually come with a time series database built in so that operators can do some basic plotting. Often these products have a 10 or 15 year legacy.
The scada vendors seem to be careful to avoid making the time series database and plotting tools which come with the HMI packages too powerful, as this might cut in to their sales of Historian type products.
If it wasn't a commodity already, the rash of startups which all seem to write their own tools for storing and plotting metrics from the operations of their servers and software services has certainly made the guts of a capable historian readily available for free.
for storing data open tsdb, based on hbase+hadoop, http://opentsdb.net/ kairosdb, based on cassandra, https://code.google.com/p/kairosdb/ timeseriesframework http://timeseriesframework.codeplex.com/
for plotting data I am hoping to find a library that allows for real time plotting and zooming, scrolling with the mouse wheel. so far I have found openhistorian http://openhistorian.codeplex.com/ kst http://sourceforge.net/projects/kst/ veusz http://home.gna.org/veusz/ chaco http://docs.enthought.com/chaco/ guiqwt https://code.google.com/p/guiqwt/ pyqtgraph http://www.pyqtgraph.org/ lots of D3 based libraries: http://selection.datavisualization.ch/
so there are lots of tools out there if you've got the patience to figure out which one is right for your application and glue it together
Re: Cassandra vs MongoDB For Time Series Data
#9We had some major issues using MongoDB for time series data due to the write volume (real time sensor data). The solution for us was Riak, mainly because we never need to update a vector clock, leaving us without the need for conflict resolution (and last write wins is fine for sensor data).
Why wouldn't generating some hashed _id work? It would then scale for writes easily (or, as in current mongodb, you could use hash-based indexes).
I'm just asking since Riak seemed much slower for me when I tried it.
Re: Cassandra vs MongoDB For Time Series Data
#10It would be surprising if, for data with a consistent, predictable structure, MongoDB had the best, most consistent, and most predictable performance. MongoDB's raison d'etre is that data doesn't always have consistent, predictable structures.
That said, there's cool stuff out there in the Mongo ecosystem. E.g., TokuMX is a whole new Mongo storage engine.