We use KDB for time series data but it is expensive. He's anyone checked this project out? http://www.monetdb.org/Home
i think monetdb is not supposed to be used on high write (or update) scenarios
Cassandra vs MongoDB For Time Series Data
21–30 of 82 posts
Re: Cassandra vs MongoDB For Time Series Data
#22One thing I always find interesting about these kinds of problems is that most DBs don't describe how they're implemented. It's easy to use the wrong tool, and then once you learn how e.g. Mongo is implemented, it's obvious, "oh, that's why things are slow". I'd love to see http://eagain.net/articles/git-for-computer-scientists/ , but for every DB technology.
A pattern I've seen on a lot of the negative Mongo articles has been people using it for things they probably shouldn't. I've yet to use it for any load, and am struggling to triangulate from all the articles I read on whether it does/doesn't scale efficiently. All the issues I have hit so far have been self-inflicted, it is still one of the best new technologies I've used in years - but is has taken a while to stop…
> whether [Mongo] does/doesn't scale efficiently
It doesn't. Three words: "global write lock". Writes block reads, reads block writes. Implications: if you run a query in production that doesn't hit an index, all traffic stops. The notablescan setting is a very, very good idea. This also means all queries must have an index, so Mongo ends up with more indexes than say, postgres would.
It's impossible to configure a clustered mongo environment to not lose data: http://aphyr.com/posts/284-call-me-maybe-mongodb
Sharding configuration is baroque, and limited.
Re: Cassandra vs MongoDB For Time Series Data
#23One thing I always find interesting about these kinds of problems is that most DBs don't describe how they're implemented. It's easy to use the wrong tool, and then once you learn how e.g. Mongo is implemented, it's obvious, "oh, that's why things are slow". I'd love to see http://eagain.net/articles/git-for-computer-scientists/ , but for every DB technology.
You nailed it. I think the issue is that people don't know what questions to ask when gathering the requirements. I'd like to know more about how this part of the article came to be: "This choice was made early on and it was supposed to be a temporary one." HOW was that choice made. What requirements were out there. I think too many people choose Mongo because they believe it's "schemaless"[1] and faster for developm…
It was one of those, "we need to do something now and this will work" solutions. We had a really talented consultant working for us, writing some of the early code. He was familiar with Mongo and wanted to go that route. Early on I said we should use Cassandra for this, but it took us quite some time to get to the point of being able to migrate. A testament to his code and the "this will be temporary" foreknowledge is the fact that we could swap data stores in a pretty massive sense without much in the way of outward-facing changes to our processing engine.
I think the hard thing for us up front was that trying to explain Cassandra data models to someone (and this guy is really really good) and then hand the rest of the work over to them to implement now, on a contracting rate is not a trivial problem. And we needed to hurry for both deadlines and burn rate.
Re: Cassandra vs MongoDB For Time Series Data
#24KariosDB ( https://github.com/proofpoint/kairosdb ) is a time-series database for Casssandra. It's an OpenTSDB ( http://opentsdb.net ) rewrite that supports sub-second precision.
Have you any experience using it?
Re: Cassandra vs MongoDB For Time Series Data
#25i work on scada systems which usually come with a time series database built in so that operators can do some basic plotting. Often these products have a 10 or 15 year legacy. The scada vendors seem to be careful to avoid making the time series database and plotting tools which come with the HMI packages too powerful, as this might cut in to their sales of Historian type products. If it wasn't a commodity already, th…
I had to look up "scada".
http://en.wikipedia.org/wiki/SCADA for those of you in the same boat
Re: Cassandra vs MongoDB For Time Series Data
#26We use KDB for time series data but it is expensive. He's anyone checked this project out? http://www.monetdb.org/Home
My go-to for any time-series data is a column-oriented database. fyi, Cassandra is row-oriented.
Well, but in a column-oriented database you would store your time series data in columns. Cassandra is "row-oriented" and you store the time series in rows. In both cases the data ends up sequentially on the disk, which is what you want.
Re: Cassandra vs MongoDB For Time Series Data
#27But is Cassandra web scale?
Re: Cassandra vs MongoDB For Time Series Data
#28Anyone uses cassandra with OrderedPartitioner and is it good ?(i know random is best)
Re: Cassandra vs MongoDB For Time Series Data
#29But is Cassandra web scale?
some vague graph from this year's Cassandra Datastax conf: https://pbs.twimg.com/media/BMfdOagCcAAxMVU.jpg
Re: Cassandra vs MongoDB For Time Series Data
#30For what its worth, we have been using Cassandra for storing time series for about 2 years now at ~2k writes a sec. I would say every issue was self-induced and Cassandra has been amazingly patient with us. It works amazing with this scenario. We experimented with MongoDB a lot initially (along with riak, hbase, etc) and found about the same thing. Turns out using a database in a way its designed to work turns out in your favor. That said hbase did well too, but it scared our ops team.
All the new changes in 1.2 and 2.0 with cql really make it seem like datastax is focused on being mysql and ignoring the time series use case though which makes me nervous.