Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling MongoDB at Mailbox

tech.dropbox.com

Post reply on HN
Print view View original

51–60 of 78 posts

Re: Scaling MongoDB at Mailbox

#51

Post by staccatomeasure »

rdtsc wrote: post #49
misiti3780 wrote: post #45

Earlier quoted context omitted.

or - mongo is great for some problems and not so great for other problems. if there is one thing you can figure out hanging out on HN long enough - it's cool to hate on mongodb

> it's cool to hate on mongodb Tens of thousands of computer geeks all randomly chose to hate a product. Clearly a coincidence or unlucky alignment of stars, nothing to do with said product, of course.

It could just be the case that everyone complains about their database and the HN crowd uses a lot of MongoDB. There's certainly more total MySql/MSSQL griping, whether or not gripers/users is higher for MongoDB isn't really obvious.

Re: Scaling MongoDB at Mailbox

#52

Post by rdtsc »

staccatomeasure wrote: post #51
rdtsc wrote: post #49

Earlier quoted context omitted.

> it's cool to hate on mongodb Tens of thousands of computer geeks all randomly chose to hate a product. Clearly a coincidence or unlucky alignment of stars, nothing to do with said product, of course.

It could just be the case that everyone complains about their database and the HN crowd uses a lot of MongoDB. There's certainly more total MySql/MSSQL griping, whether or not gripers/users is higher for MongoDB isn't really obvious.

As I pointed out MongoDB inc (ex 10gen) made some decision in relation to how they marketed and set up their product that made many turn their heads.

Re: Scaling MongoDB at Mailbox

#53

Post by stonewhite »

Pxtl wrote: post #46
threeseed wrote: post #43

Earlier quoted context omitted.

Simple. There are some use cases for which MongoDB will be 5x faster than any other system. It is relatively unique among databases in that it is a document store as opposed to a relational system. It is also very easy to use and manage.

> It is also very easy to use and manage. This is something that SQL database fans don't get. A small start-up can't afford to have a team of guys fiddling with incredibly obscure performance tuning settings. A developer might not know how to optimize a slow-running Postgre query, but he knows how to cache data and denormalize in a manner that makes the query he wants to run fast. And a document-store plays nice with…

One thing you don't know or faced yet is that mongo fails, and sometimes it fails so hard that you would rather kick some rocks.

For example, if you start a background index generation on an eventually consistent replica set, indexing on secondary nodes are done foreground. Which means you only accept reads from slaves but slaves are unresponsive because of the index generation. In this state, if you try to do anything fancy your data will go corrupt. Only way out is to wait through the outage (which I find it pretty hard to do so). This is still not solved in 2.4, waiting for 2.6.

Replica sets with all secondaries which can't elect a primary because it lost a node, or the mostly random primary-secondary switches that drops all connections, seldomly primary reelecting itself meanwhile dropping connections for no apparent reasons. Mongo offers tin foil hats for integrity, consistency and reliability. So yeah, I'd rather examine and understand why an SQL query is slow. Because it is at least deterministic, which in mongo nothing really is.

Postgres supports free form json, XML or hstore document formats by the way, couchDb has its own specific features as a document db too. I still don't see why people want to go on with mongo this bad.

Re: Scaling MongoDB at Mailbox

#54

Post by jshen »

Pxtl wrote: post #46
threeseed wrote: post #43

Earlier quoted context omitted.

Simple. There are some use cases for which MongoDB will be 5x faster than any other system. It is relatively unique among databases in that it is a document store as opposed to a relational system. It is also very easy to use and manage.

> It is also very easy to use and manage. This is something that SQL database fans don't get. A small start-up can't afford to have a team of guys fiddling with incredibly obscure performance tuning settings. A developer might not know how to optimize a slow-running Postgre query, but he knows how to cache data and denormalize in a manner that makes the query he wants to run fast. And a document-store plays nice with…

"A developer might not know how to optimize a slow-running Postgre query, but he knows how to cache data and denormalize in a manner that makes the query he wants to run fast"

And you can cache and denormalize with postgres just as easily, and it will probably perform better and not corrupt your data.

What mongo makes easy is things like replication and failover. It can go horribly wrong on that front, but until you see that, it is much easier to get up and running with replication and failover.

Re: Scaling MongoDB at Mailbox

#55

Post by bonzoesc »

bernatfp wrote: post #30

I'm aware of MongoDB issues with scaling, and I'm trying to avoid it as much as I can, but I still can't find any other alternative that offers geolocated queries.

From firsthand experience, the PostGIS extension for PostgreSQL is fantastic.

A couple of my coworkers at Basho have done geospatial work with Riak, our scalable, distributed database: http://basho.com/indexing-the-zombie-apocalypse-with-riak/

Re: Scaling MongoDB at Mailbox

#56

Post by nasalgoat »

kareemk wrote: post #26
nasalgoat wrote: post #25

Earlier quoted context omitted.

My tests with it showed a drop in disk size of about 20-30%. Unfortunately the read/write improvements weren't high enough to justify the massive engineering feat it would take to replace MongoDB with TokuMX, mostly due to the lack of commercial support for Toku.

Our disk size shrank to 10% of stock mongodb size. The migration is easy now that they've released a tool to replicate from a stock mongo to TokuMX. They also do have commercial support that we've paid for and their team has been incredibly helpful and responsive.

Perhaps once my commercial agreement with 10gen expires in a couple of months I'll take another look.

Re: Scaling MongoDB at Mailbox

#57

Post by kb19 »

MichaelGG wrote: post #3

>one performance issue that impacted us was MongoDB’s database-level write lock. People give MSSQL shit for having row-level locks (if you don't use their MVCC option), yet how is it that Mongo runs with a database-wide option and people don't immediately laugh and walk away? Is the hype so powerful that people just shrug about a huge mutex?

I think that's how it was in previous versions of Mongo but it was fixed in later versions. I'm probably wrong though.

Re: Scaling MongoDB at Mailbox

#58

Post by Diederich »

stonewhite wrote: post #53
Pxtl wrote: post #46

Earlier quoted context omitted.

> It is also very easy to use and manage. This is something that SQL database fans don't get. A small start-up can't afford to have a team of guys fiddling with incredibly obscure performance tuning settings. A developer might not know how to optimize a slow-running Postgre query, but he knows how to cache data and denormalize in a manner that makes the query he wants to run fast. And a document-store plays nice with…

One thing you don't know or faced yet is that mongo fails, and sometimes it fails so hard that you would rather kick some rocks. For example, if you start a background index generation on an eventually consistent replica set, indexing on secondary nodes are done foreground. Which means you only accept reads from slaves but slaves are unresponsive because of the index generation. In this state, if you try to do anythi…

I've been running, on a five minute cron, kill -9 on the master mongo instance in our QA lab for a good long time. 24/7.

There's a program running that, 100 times per second, reads a document, MD5's the field, and writes it back. At the same time, it reads a file from the local filesystem, MD5's it, and writes it back. The document and the local filesystem file started with the same value.

After a few thousand kill -9's on the master instance, the local file and the mongo document are still identical.

I've been running MongoDB in production since 2010.

It's definitely possible to use Mongo in a way that isn't safe for your particular use case. But we're doing it correctly.

I haven't lost a bit of data in more than three years of MongoDB.

Mongo has a lot of limitations. We're currently researching various 'big data' solutions, because for us, Mongo doesn't fit that.

For ease of development (in dynamic languages, where your in-program data structures and in-database documents look almost identical), safety, and lack of headaches, MongoDB has been a consistent win for me and the teams I've been on.

Re: Scaling MongoDB at Mailbox

#59

Post by [deleted] »

justinsb wrote: post #11
lucisferre wrote: post #4

Earlier quoted context omitted.

The problems people encounter with SQL performance generally (or at least also) are on the read side where query optimization can become a pretty hard problem. Basically, the short answer is people are using SQL and noSQL very differently (as one would expect). cue: someone telling me SQL query optimization is easy and people are just idiots.

It's a valid point taken by itself. But I would respectfully suggest that it's not a particularly useful statement, because you're not comparing SQL to anything. "Complex SQL queries are hard" because "complex queries are hard". If there was a magical tool where you could do complex querying and always have the right answer in the minimal time, then these discussions would never arise. (To a debatable extent, that is…

[deleted]

Re: Scaling MongoDB at Mailbox

#60

Post by mcot2 »

It's funny to read these comments trashing mongodb. Perhaps people should do a little less HN ranting and a little more serious study about the problem they need to solve. The design, benefits and tradeoffs of mongodb are well understood. If they don't fit your application, don't use it.
Post reply on HN