Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things I wish I knew about MongoDB a year ago

snmaynard.com

Post reply on HN
Print view View original

91–100 of 111 posts

Re: Things I wish I knew about MongoDB a year ago

#91

Post by arielweisberg »

lmm wrote: post #76
arielweisberg wrote: post #44

Earlier quoted context omitted.

Shameless plug (hey Tokutek is doing it), in VoltDB replication is synchronous so it doesn't have this problem. Latency in the current version is nothing to write home about, but in V3 latency with replication is 600-1000 microseconds. Group commit to disk is every 1-2 milliseconds. V3 also allows reads to be load balanced across replicas and masters so you gain some additional read capacity from replication. V3 also…

>Now go figure out what you don't get to keep ;-) Cross-datacenter replication becomes a Really Bad Idea?

You don't have to give up cross DC replication if you do it asynchronously, but you lose cross shard consistency when there is a dirty fail over. This effects distributed transactions and series of single part transactions that depend on each other across different shards.

What Volt supports right now is actually asynchronous replication that does preserve cross shard consistency, but that is not going to last.

You can do synchronous multi-DC replication, but then you have Spanner and the associated latency of multiple data-center quorums.

There is also Calvin http://bit.ly/RGW9RY

Re: Things I wish I knew about MongoDB a year ago

#92

Post by jeffdavis »

andyzweb wrote: post #87
jeffdavis wrote: post #64

Earlier quoted context omitted.

"And the database world is moving towards scaling horizontally rather than vertically." I think that's an oversimplification. Two counterpoints: * Database systems will always need to make heavy use of locality. The speed of light means that synchronizing access over long distances (even medium distances -- light only goes about a foot per nanosecond) will always be a challenge. * Multi-core means vertical scaling is…

postgres does horrible with multicore unless you have many concurrent pooled connections. fork-and-forget

I think you're referring to postgres's inability to use more than one core per query, which is true (or mostly true... there are quite a few helper processes that take on some of the work).

For many smaller queries, postgres does great on multi-core, and pgbouncer is a good connection pooler.

Re: Things I wish I knew about MongoDB a year ago

#93

Post by DanWaterworth »

leif wrote: post #84
DanWaterworth wrote: post #66

Earlier quoted context omitted.

In my experience, it's not just throughput that's important, but also 99th percentile latency. If I understand fractal trees correctly, you sometimes need to rewrite all of your elements on disk. How do you do this without causing lag?

That's happily not how fractal trees work, at all. We have a few talks online describing how they work. Zardosht has one here http://vimeo.com/m/26471692 . I thought Bradley had a more detailed one at MIT but I can't find it right now. (EDIT: found it! http://video.mit.edu/watch/lecture-19-how-tokudb-fractal-tre... ) Basically, I think you're thinking of the COLA. What we implement does have a literal tree structure,…

Thank you very much for the links. Is my understanding correct?

Essentially, a fractal tree is a B-Tree (or perhaps a B+Tree?) with buffers on each branch (per child). Operations get added to these buffers and when one becomes full, the operations get passed to the corresponding child node. Operations are applied when they reach the node that is responsible for the data concerned.

Re: Things I wish I knew about MongoDB a year ago

#94

Post by leif »

DanWaterworth wrote: post #93
leif wrote: post #84

Earlier quoted context omitted.

That's happily not how fractal trees work, at all. We have a few talks online describing how they work. Zardosht has one here http://vimeo.com/m/26471692 . I thought Bradley had a more detailed one at MIT but I can't find it right now. (EDIT: found it! http://video.mit.edu/watch/lecture-19-how-tokudb-fractal-tre... ) Basically, I think you're thinking of the COLA. What we implement does have a literal tree structure,…

Thank you very much for the links. Is my understanding correct? Essentially, a fractal tree is a B-Tree (or perhaps a B+Tree?) with buffers on each branch (per child). Operations get added to these buffers and when one becomes full, the operations get passed to the corresponding child node. Operations are applied when they reach the node that is responsible for the data concerned.

That's about it! It's like a B+Tree in that the data resides in the leaves (well, and in the buffers), except that the fanout is a lot lower because we save room in the internal nodes for the buffers.

Re: Things I wish I knew about MongoDB a year ago

#95

Post by bsg75 »

jeremyjh wrote: post #88
dia80 wrote: post #5

Genuine question: In what use cases does mongo kick mysql's ass? I've used it a couple of times in hobby projects and enjoyed not maintaining a schema. I read so many of these 'gotcha' style articles and for example one commenter here wants to have a manual "recently dirty" flag to combat the master / slave lag mentioned in the article. I know it's faster (tm) but once you have to take in to account all this low leve…

One issue with MySQL in large databases is that schema changes are extremely expensive, so much so that you'll be making design decisions around it (e.g. how do we implement this feature without executing our two-day alter table statement). Not all RDBMS have this problem to such a degree but none can escape it entirely. A lot of the gotchas that he notes are related to design trade-offs with different default behavi…

> e.g. how do we implement this feature without executing our two-day alter table statement?

With MySQL being so prevalent, IMO this is the principal reason why the concept that RDBMS' are hard to work with exists. None of the other major platforms have this issue, but if a developer's only exposure is to MySQL, then the idea of schema alters are always fearsome.

This is not to say that ALTER TABLE statements are always quick - if constraint checks or default values are included, there can be long runtimes in PostgreSQL, MSSQL, Oracle, etc. - but significant downtime to add an empty nullable column is just plain stupid for a platform that has been around as long as MySQL.

MySQL has forced a major population segment of developers to toss the advantages of DB side validation and rich query languages for the purpose of _avoiding_MySQL_. Sure the whole object-relational impedance mismatch exists, SQL is hard, yada-yada, but I have never seen these reasons cause as much angst as taking a service offline to add a column to a table.

10gen should have a picture of Monty in their CFO's office.

Re: Things I wish I knew about MongoDB a year ago

#96

Post by DanWaterworth »

leif wrote: post #94
DanWaterworth wrote: post #93

Earlier quoted context omitted.

Thank you very much for the links. Is my understanding correct? Essentially, a fractal tree is a B-Tree (or perhaps a B+Tree?) with buffers on each branch (per child). Operations get added to these buffers and when one becomes full, the operations get passed to the corresponding child node. Operations are applied when they reach the node that is responsible for the data concerned.

That's about it! It's like a B+Tree in that the data resides in the leaves (well, and in the buffers), except that the fanout is a lot lower because we save room in the internal nodes for the buffers.

I'm a little confused. I thought fractal trees worked cache obliviously or am I mistaken?

Re: Things I wish I knew about MongoDB a year ago

#97

Post by leif »

DanWaterworth wrote: post #96
leif wrote: post #94

Earlier quoted context omitted.

That's about it! It's like a B+Tree in that the data resides in the leaves (well, and in the buffers), except that the fanout is a lot lower because we save room in the internal nodes for the buffers.

I'm a little confused. I thought fractal trees worked cache obliviously or am I mistaken?

In theory, it is cache oblivious, and the CO-DAM model informs our decisions about the implementation, but no, the implementation itself isn't actually cache oblivious. Shh, don't tell on us.

A "fractal tree" is defined by our marketing team as "whatever it is we actually implement." If you want to talk cache-oblivious data structures, we can talk about things like the COLA and cache-oblivious streaming B-trees which have rigorous definitions in the literature. At some point, if you want to achieve a certain level of detail, you have to pick one or the other in order to continue the conversation.

Re: Things I wish I knew about MongoDB a year ago

#98

Post by ibotty »

thibaut_barrere wrote: post #72
gbog wrote: post #43

Earlier quoted context omitted.

So you'd rather install a completely new storage engine and learn to use it than check a doc and install an extension to postgres?

The thing is that 10gen did a really, really good job at polishing the install process and documenting it to get people started. No surprise to see the GIS part of MongoDB is built-in instead of an extension of some kind. I know a couple of people who used PG without even knowing there was a GIS extension.

is it a good thing to have everything built in?

Re: Things I wish I knew about MongoDB a year ago

#99

Post by alex137 »

leif wrote: post #94
DanWaterworth wrote: post #93

Earlier quoted context omitted.

Thank you very much for the links. Is my understanding correct? Essentially, a fractal tree is a B-Tree (or perhaps a B+Tree?) with buffers on each branch (per child). Operations get added to these buffers and when one becomes full, the operations get passed to the corresponding child node. Operations are applied when they reach the node that is responsible for the data concerned.

That's about it! It's like a B+Tree in that the data resides in the leaves (well, and in the buffers), except that the fanout is a lot lower because we save room in the internal nodes for the buffers.

Hi. I thought the block size is much bigger in fractal trees (like 4MiB instead of 4KiB) than in B-trees hence the fanout would be about the same?

I'm trying to experiment with these ideas on my side, but can't quite grok how large the buffers must be at each level of the tree.

Let's take a concrete example like: 2^40 (1T) records with an 8-byte key and an 8-byte value. In a traditional B-tree with a 8KiB block size and assuming links take 8 bytes too and assume for a while that blocks are completely full. So, that's 1G leaf blocks, a fanout of about 1K and hence & full 4-level trees: 1 root block, 1K level-2 blocks, 1M level-3 blocks, 1G leaf blocks.

In this setting, I understand that a fractal tree will attach a buffer to each internal node. How large will they be at level 1 (root), level 2, and level 3 ?

Re: Things I wish I knew about MongoDB a year ago

#100

Post by diminoten »

dleblanc wrote: post #28
diminoten wrote: post #11

Earlier quoted context omitted.

Mongo isn't so important to this question as ODB vs RDBMS. Here's some light reading: http://en.wikipedia.org/wiki/Object-relational_impedance_mis... MongoDB is just ODB, and MySQL is just RDB. Besides, postgres is the real future!

Not sure what Wu-Tang has to do with this, but.... Seriously though, is Mongo an ODB, or a document oriented database? ODBs/OODBs imply a much different use-case and functionality, and I think we ought not conflate the two.

You're right, I didn't know there was a difference.
Post reply on HN