Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB vs. Clustrix Benchmark

sergeitsar.blogspot.com

Post reply on HN
Print view View original

41–50 of 67 posts

Re: MongoDB vs. Clustrix Benchmark

#41

Post by true_religion »

PaulHoule wrote: post #28
va_coder wrote: post #23

First off, how do you download Clustrix? With MongoDB, simple as pie: http://www.mongodb.org/downloads Now, where are the docs? Again, Mongo has great docs http://wiki.mongodb.org/display/DOCS/Home How can I verify your claims? Oh that's right, you call a salesperson first....

Well, there are quite a few commercial database vendors that offer parallel and clustered RDBMS products, and many of them appear to be quite good. Unfortunately, they've got a terrible marketing problem. Before 1998 or so, a relational database was an expensive product that you got from a vendor like Oracle. Since then, a generation of people have grown up that think about using a commercial RDBMS the same way most…

On the other hand, allowing potential customers to easily try your product can destroy your chances at gaining them.

Here's a quick example:

A few years back, I was writing Smalltalk web systems and evaluating which commercial implementation to use. The two big ones that bubbled to the top of the list then were Gemstone and Cincom.

Cincom offered a relatively familiar set up which would require using sticky sessions for the web server, and another backend database like PostgreSQL.

Gemstone offered pearls on a platter: shared session state across all the backends, and a build in distributed object database. I thought it would be brilliant, and reading their blogs and documentations gave no hint of any pitfalls.

Cimcom was a download away---as were any of the free smalltalks---but Gemstone then required you to contact their sales staff before playing with it. So I made a mistake and chose Gemstone, and the system was built to target their as of yet unseen architecture. When I finally got to use Gemstone, it was a mess!

If I had that experience to start with rather than just their glowing self-produced reports, I'd have never picked Gemstone and instead gone with the familiar "kludgy" system that was "good enough".

Re: MongoDB vs. Clustrix Benchmark

#42

Post by sqrt17 »

fedd wrote: post #26
cheald wrote: post #24

Earlier quoted context omitted.

Indeed. A lot of the reason I like document databases so much is that you can solve problems differently (and many times, much more easily) than you could in a relational database. Compare: db.posts.find({tags: {$in: ["foo", "bar"]}}) to: SELECT * from posts JOIN taggings ON taggings.post_id = posts.id JOIN tags ON tags.tagging_id = taggings.id WHERE tags.tag IN ('foo', 'bar'); (Single query tag lookup; naively joins…

relational folks can denormalize when they want to

They get strangled by their DBA before they're finished with it.

And: the point of document stores is that you can denormalize (i.e. put lots of stuff into one data item) and _still_ use indices into the lots of stuff. Most commercial RDBMSes have means and ways to do that (e.g. storage of XML content), but afaik it's not standardized and would tie you closely to a single ($$$$$$) database vendor who will not hesitate to make you bleed whenever they can.

Re: MongoDB vs. Clustrix Benchmark

#43

Post by fedd »

sqrt17 wrote: post #42
fedd wrote: post #26

Earlier quoted context omitted.

relational folks can denormalize when they want to

They get strangled by their DBA before they're finished with it. And: the point of document stores is that you can denormalize (i.e. put lots of stuff into one data item) and _still_ use indices into the lots of stuff. Most commercial RDBMSes have means and ways to do that (e.g. storage of XML content), but afaik it's not standardized and would tie you closely to a single ($$$$$$) database vendor who will not hesitat…

the things like olap and intentional denormalization is not that dba are against of.

by denormalization i didnt mean keeping xml in a clob, but keeping the values in a row as if several tables are already joined into one wide table, thus eliminating the need in joins.

Re: MongoDB vs. Clustrix Benchmark

#44

Post by fedd »

JoachimSchipper wrote: post #37
kchodorow wrote: post #21

Earlier quoted context omitted.

Excellent point. MongoDB doesn't claim to be any faster than other dbs at the simple stuff (in fact, it's often slower because we haven't had years to optimize everything). The speed gains that people usually see are because they can just fetch one document, instead of doing complex joins or aggregations.

Serious question, because I don't know: what's the advantage over a denormalized SQL table?

because document stores store documents ready to be consumed by an app. these documents are tree-like, not tabular: jsons or xmls

Re: MongoDB vs. Clustrix Benchmark

#45

Post by fedd »

sergei wrote: post #25

I updated the post with the benchmark source.

you updated your post and i am waiting people here at hacker news to criticize your benchmark setup.

maybe i missed smth but i still saw only the suggestion for you now send your benchmark to mongodb guys.

edit: please discard, now i see there they say you should include joins to your clustrix benchmark. waiting for the reply

Re: MongoDB vs. Clustrix Benchmark

#46

Post by strlen »

Pet peeve: eventual consistency isn't for scalability and performance, it's for availability. In a well designed system, the whole debate only matters during in a failure condition: a strongly consistent system gives up availability upon a certain kind of failure, an eventually consistent system gives up consistency upon a certain kind of failure.

There are strongly consistent scalable "NoSQL" systems e.g., BigTable. Megastore even provides complex distributed cross row transactions.

In a well tuned system, loss of availability (in a failure scenario) could be minimize to seconds. What this means for performance is that you have systems which encounter second-long latency spikes upon failures.

It also isn't a binary switch:

1) Quorums can be used to achieve read-your-write consistency in the case of simple failures (loss of 1 node out of 3).

2) There's multiple kinds of relaxed consistency models. One of them is serializable consistency: you may get a stale read, but the the order of reads is the same as the order of writes. This is used by PNUTs and can be achieved by serializing the writes through a single master. This means that there's (again, for a short time period) loss of write availability in a failure, but there's no loss of read availability.

3) Paxos/multi-Paxos can be used to achieve atomic writes (all available nodes receive the write) while withstanding simple failures (similar to quorum protocols... and I believe multi-Paxos uses quorum protocols under the cover to improve liveness over "raw" Paxos). This is at the cost of higher latency (and complexity). [Edit: In this case, you're dealing with full-blown strong consistency, but with -- at the cost of latency -- the ability to tolerate certain kinds of simple failures/trivial partitions]

Clustrix looks interesting, but it looks like it addresses the scalability and performance issues with RDBMS, but not the availability feature. If an RDBMS were to drop the "A" and "I" in ACID (C in "ACID" means serializable view of the execution, which is not the same as the C in "CAP": the latter means all nodes in a cluster agreeing upon what the data is which isn't required for the former), it would be possible to build a highly available, low latency RDBMS; but it would also not be as useful without atomic and isolated cross row transactions.

[Disclaimer: I work on a Dynamo-style database, but fond of PNUTs as an architecture (more difficult to implement, but IMO a better fit for plurality of web applications) and generally fascinated and interested in distributed systems, databases and systems programming in general]

Re: MongoDB vs. Clustrix Benchmark

#48

Post by sergei »

strlen wrote: post #46

Pet peeve: eventual consistency isn't for scalability and performance, it's for availability. In a well designed system, the whole debate only matters during in a failure condition: a strongly consistent system gives up availability upon a certain kind of failure, an eventually consistent system gives up consistency upon a certain kind of failure. There are strongly consistent scalable "NoSQL" systems e.g., BigTable.…

A large part of the message I'm trying to convey is:

1. A DBMS is much more than just the interface. I'm going to write more on the subject. Whether you're using SQL, datalog, BSON, etc. -- there's a broader set of desirable features that's more important than the query language.

2. I'm not saying I agree with what the NoSQL folks are saying, or their justifications. Let's be honest: there's a prevalent sentiment that relational SQL based databases do not scale, and even further, that they somehow cannot scale. That's just not true.

I saw a video the other day of some guy at Google giving a talk about the database behind the app engine. At one point someone in the audience asked about scale and SQL. His response was "Well, how well does SQL scale?" Everyone in the room laughed.

Re: MongoDB vs. Clustrix Benchmark

#49

Post by jdf »

strlen wrote: post #46

Pet peeve: eventual consistency isn't for scalability and performance, it's for availability. In a well designed system, the whole debate only matters during in a failure condition: a strongly consistent system gives up availability upon a certain kind of failure, an eventually consistent system gives up consistency upon a certain kind of failure. There are strongly consistent scalable "NoSQL" systems e.g., BigTable.…

Eventual consistency often gives up the ability to develop applications against the system, because more reasoning about the state of the system has to be embedded in the application logic. For a better description of this problem, see this post from an AWS employee:

http://perspectives.mvdirona.com/2010/02/24/ILoveEventualCon...

Note that his "right answer" for performance (not availability, so it's a bit of an aside from your comment) is to dynamically repartition, which is what Clustrix does. Note that we can actually do this type of repartitioning on just the "hot" data, using MVCC to avoid any downtime (unlike Mongo, which blocks all writes).

By the way, the Google Megastore project I believe you're referring to is not actually eventually consistent in the same way that, say, Cassandra is. Megastore is fully ACID within an entity group, meaning that lowering eventually consistency down to the per-node level was not the solution that Google went with. BigTable is also not eventually consistent.

*Clustrix employee

Re: MongoDB vs. Clustrix Benchmark

#50

Post by jdf »

luigi wrote: post #31

He talks about populating MongoDB with "rows". No one should be using (or benchmarking) a technology unless they understand what it actually is, and what problems it aims to solve.

The term "rows" could certainly be replaced with "tuples", "objects", or whatever else you prefer without changing any meaning of the post. If you look at the benchmarks posted on Mongo's site

http://www.mongodb.org/display/DOCS/Benchmarks

you can see that most of them compare Mongo against MySQL. Certainly "rows" are being inserted into the latter.

Post reply on HN