Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling MongoDB at Mailbox

tech.dropbox.com

Post reply on HN
Print view View original

31–40 of 78 posts

Re: Scaling MongoDB at Mailbox

#31

Post by mjallday »

stonewhite wrote: post #18

I dont understand how MongoDB can stay as a shiny tool given its long list of shortcomings. It is so elastic that I think it really has no true form (object cache, mq, relational??, key-value bucket). Lately I haven't read any news about a company migrating to Mongo, but rather most were either departing from mongo because of some catastrophic outage or some mission impossible operation to handle the shortcomings, li…

Same reason MySQL sticks around when there are alternatives that make it harder to shoot yourself in the foot: Easy to get started, works well enough to make an MVP.

Re: Scaling MongoDB at Mailbox

#32

Post by jessaustin »

Interesting:

...a single Python process using gevent and pymongo can copy a large MongoDB collection in half the time that mongodump (written in C++) takes, even when the MongoDB client and server are on the same machine.

Re: Scaling MongoDB at Mailbox

#33

Post by MBCook »

jessaustin wrote: post #32

Interesting: ...a single Python process using gevent and pymongo can copy a large MongoDB collection in half the time that mongodump (written in C++) takes, even when the MongoDB client and server are on the same machine.

I would guess this is for the reason posted in the article: the Python code is multithreaded (sort of, gevent) but the C++ code is only single threaded.

Re: Scaling MongoDB at Mailbox

#34

Post by jessaustin »

MBCook wrote: post #33
jessaustin wrote: post #32

Interesting: ...a single Python process using gevent and pymongo can copy a large MongoDB collection in half the time that mongodump (written in C++) takes, even when the MongoDB client and server are on the same machine.

I would guess this is for the reason posted in the article: the Python code is multithreaded (sort of, gevent) but the C++ code is only single threaded.

Sure, but that seems to suggest a meta-reason: the standard utility included with the package isn't asynchronous [sort of orthogonal to threadedness] because it would be a PITA to write an asynchronous utility in C++.

Re: Scaling MongoDB at Mailbox

#35

Post by leif »

jessaustin wrote: post #34
MBCook wrote: post #33

Earlier quoted context omitted.

I would guess this is for the reason posted in the article: the Python code is multithreaded (sort of, gevent) but the C++ code is only single threaded.

Sure, but that seems to suggest a meta -reason: the standard utility included with the package isn't asynchronous [sort of orthogonal to threadedness] because it would be a PITA to write an asynchronous utility in C++.

That's not true, I've had no trouble writing very fast threaded C++ programs with the mongo client.

Re: Scaling MongoDB at Mailbox

#36

Post by jessaustin »

leif wrote: post #35
jessaustin wrote: post #34

Earlier quoted context omitted.

Sure, but that seems to suggest a meta -reason: the standard utility included with the package isn't asynchronous [sort of orthogonal to threadedness] because it would be a PITA to write an asynchronous utility in C++.

That's not true, I've had no trouble writing very fast threaded C++ programs with the mongo client.

1. That's not what I said.

2. I'm sure the Mongo devs would accept pull requests for mongodump.

Re: Scaling MongoDB at Mailbox

#37

Post by wheaties »

raj_bains wrote: post #13

You need consistency (distributed mvcc), row-level locking and scale-out architecture. Clustrix might be a better fit for the workload. Clustrix customer Twoo.com has 336 core deployment (168 master, 168 slave, 21 nodes each), they have millions of users and billions of transactions per day. Their application still thinks it's talking to a single MySQL database and they don't have a DBA and they have never thought ab…

Wish someone would do that for Postgres.

Re: Scaling MongoDB at Mailbox

#38

Post by leif »

jessaustin wrote: post #36
leif wrote: post #35

Earlier quoted context omitted.

That's not true, I've had no trouble writing very fast threaded C++ programs with the mongo client.

1. That's not what I said. 2. I'm sure the Mongo devs would accept pull requests for mongodump.

1. > it would be a PITA to write an asynchronous utility in C++

2. Doubt it, and there's no reason for me to write one for them.

Re: Scaling MongoDB at Mailbox

#39

Post by jessaustin »

leif wrote: post #38
jessaustin wrote: post #36

Earlier quoted context omitted.

1. That's not what I said. 2. I'm sure the Mongo devs would accept pull requests for mongodump.

1. > it would be a PITA to write an asynchronous utility in C++ 2. Doubt it, and there's no reason for me to write one for them.

Asynchronous ≠ multithreaded. TFA actually tried both ways, measured the difference, and found async to be much faster.

Re: Scaling MongoDB at Mailbox

#40

Post by tkellogg »

lucian1900 wrote: post #27
MichaelGG wrote: post #3

>one performance issue that impacted us was MongoDB’s database-level write lock. People give MSSQL shit for having row-level locks (if you don't use their MVCC option), yet how is it that Mongo runs with a database-wide option and people don't immediately laugh and walk away? Is the hype so powerful that people just shrug about a huge mutex?

Yes, MongoDB really is that bad. Never use it.

Use rethinkdb instead. Its not perfect but it mainly hits the same niche but does it right.
Post reply on HN