Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why MongoDB Never Worked Out at Etsy

mcfunley.com

Post reply on HN
Print view View original

21–30 of 73 posts

Re: Why MongoDB Never Worked Out at Etsy

#21

Post by PommeDeTerre »

matthewrnyc wrote: post #17
untog wrote: post #2

Genuine question- who is using MongoDB successfully in production, and at scale? I'm not aware of anyone myself- I hear of it being used in hackathons etc because its so quick to set up, but I'd be curious to know what people are using it with.

I'm using it for the analytics suite at my company (large ecommerce multinational). Its weird at first, coming from a background of Access then SQLite then MySQL/PHPMyAdmin but you get used to it. I essentially treat it like a gigantic python dictionary object. The sharding is too much of a ball-ache to set up so I've created an optimal way of distributing/mapping files across our cluster to make use of all machines.…

Where exactly is the net gain in your situation?

So right off the bat, you've lost the querying power that SQL offers. When dealing with data that's intended to be analyzed, that sounds like a pretty big loss.

Clearly its built-in sharding support, which is often touted as one of its biggest benefits, wasn't suitable for you. So you had to invest some time and effort coming up with an alternate system. That sounds like a loss to me.

When it comes to the schema issue, it sounds like you haven't actually reduced the effort or work in any way, but merely pushed it somewhere else. Like you admit, you still do have to deal with the schema, it's just handled within the application logic, rather than the database. That sounds worse to me, especially if there is more than one application using the database.

I'm just not seeing the benefit.

Re: Why MongoDB Never Worked Out at Etsy

#22

Post by francesca »

PommeDeTerre wrote: post #8
dindresto wrote: post #3

Earlier quoted context omitted.

They have a list of who's using MongoDB in production. http://www.mongodb.org/display/DOCS/Production+Deployments

How up-to-date is that list? I mean, in terms of removing entries that no longer apply. Some date from 2009 to 2011. Are these systems still in place, and actively being used today? I've heard of or directly witnessed enough situations where somebody with influence, but maybe not much actual technical experience, pushes for the use of a NoSQL database of some sort. Yes, the project is implemented and often does end u…

That page is updated regularly. You can also check out more in-depth case studies of MongoDB deployments here: http://www.10gen.com/customers

And there's s growing list of stories at 10gen.com/presentations

Some good ones to point out: Analytica: http://www.10gen.com/presentations/mongosv-2012/exploring-pu...

Apollo Group (The University of Phoenix) http://www.10gen.com/presentations/mongosv-2012/how-we-evalu...

AOL : http://www.10gen.com/presentations/managing-large-scale-data...

Github: http://www.10gen.com/presentations/mongosv-2012/mongodb-anal...

Banjo: http://www.10gen.com/presentations/real-time-location-based-...

Telefonica: http://www.10gen.com/presentations/mongodb-uk-2012/MongoDB-o...

MapMyFitness: http://www.10gen.com/presentations/mongodb-seattle-2012-mong...

Sailthru: http://www.10gen.com/presentations/mongodb-seattle-2012-sail...

Stripe: http://www.10gen.com/presentations/high-availability-mongodb...

eBay: http://www.10gen.com/presentations/mongodb-ebay

These are all large deployments. It's a mix of small startups, startups that grew up and large engineering companies (like eBay and Apollo group).

Re: Why MongoDB Never Worked Out at Etsy

#23

Post by jasonmccay »

druiid wrote: post #12

Whenever I see articles come up like this one mentioning MongoDB, I wonder not why people decided to go with Mongo, but why they didn't go with some of the alternatives out there? For my part, we use Couchbase to great success and it fixes many of the complaints against MongoDB. Then there's Riak and countless others with well established quality installations. To me MongoDB seems the buzzword NoSQL engine that gets…

So, if someone asked you why they should use "Riak and countless others" and not MongoDB, what would you really say? Also, you seemed to imply that Riak was a go-to solution (my wording) while implying that MongoDB was more of a fringe "buzzword" technology ... when, counting features, I think most would acknowledge MongoDB as being more mainstream.

There are a large number of well-established and quality installations of MongoDB. It works really well at both small and large scale and with a bit of tweaking (like any technology), can perform nicely.

Re: Why MongoDB Never Worked Out at Etsy

#24

Post by jeremyjh »

druiid wrote: post #12

Whenever I see articles come up like this one mentioning MongoDB, I wonder not why people decided to go with Mongo, but why they didn't go with some of the alternatives out there? For my part, we use Couchbase to great success and it fixes many of the complaints against MongoDB. Then there's Riak and countless others with well established quality installations. To me MongoDB seems the buzzword NoSQL engine that gets…

I do not see any of those other NoSQL databases as really being equivalent. MongoDB intends to be a general-purpose application database. It has many of the features developers expect from MySQL/Postgres, such as arbitrary numbers of indexed fields, partial record updates, aggregation queries (simpler than Map/Reduce) and many others. Couchbase may be much closer in feature-set but its developers claim they do not really compete with Mongo.

I do not see Riak or Cassandra as competing at all. In fact I would expect most applications that use Riak or Cassandra are also using a general-purpose database as well (such as MySQL or Mongo). You could use some of those databases as a general purpose database but it would be more work for little benefit. It makes more sense to me to use Riak or Cassandra for use-cases that really need high-throughput and unlimited write-scalability and use an app database for things like user accounts and preference management and all the little things that can take up a lot of development time but will never have really demanding runtime requirements (for 99.99% of internet apps).

Re: Why MongoDB Never Worked Out at Etsy

#25

Post by francesca »

monstrado wrote: post #13

Most of the production deployments I find on the internet are around 3-5 nodes. Are there any production clusters that are running 500-600+ nodes?

Disney runs over 1400 instances according to this presentation: http://www.10gen.com/presentations/mongosv-2011/a-year-with-...

Also foursquare runs a very large MongoDB deployment. http://www.10gen.com/presentations/mongodb-foursquare-cloud-...

Craigslist: http://www.10gen.com/customers/craigslist

Shutterfly also has a very large deployment: http://www.10gen.com/customers/shutterfly

Re: Why MongoDB Never Worked Out at Etsy

#26

Post by jeremyjh »

This is totally reasonable. MongoDB, more than any other "NoSQL" database, directly competes with MySQL/Postgres as a general-purpose application database. I don't see a need to have more than one for most applications - at least as long as there is only one development/support team for that application.

Re: Why MongoDB Never Worked Out at Etsy

#27

Post by druiid »

jeremyjh wrote: post #24
druiid wrote: post #12

Whenever I see articles come up like this one mentioning MongoDB, I wonder not why people decided to go with Mongo, but why they didn't go with some of the alternatives out there? For my part, we use Couchbase to great success and it fixes many of the complaints against MongoDB. Then there's Riak and countless others with well established quality installations. To me MongoDB seems the buzzword NoSQL engine that gets…

I do not see any of those other NoSQL databases as really being equivalent. MongoDB intends to be a general-purpose application database. It has many of the features developers expect from MySQL/Postgres, such as arbitrary numbers of indexed fields, partial record updates, aggregation queries (simpler than Map/Reduce) and many others. Couchbase may be much closer in feature-set but its developers claim they do not re…

Good points for sure. I think though I'd personally look at the different solutions on both an architectural and feature basis. A good number of the reasons that the original article listed as issues they came across, were outside the realm of features available in the actual MongoDB system (more or less) such as problems with logging, monitoring, backups, etc and were more architectural issues. To be certain, these can (and probably will) be issues with other systems to investigate.

Re: Why MongoDB Never Worked Out at Etsy

#28

Post by MichaelSalib »

PommeDeTerre wrote: post #21
matthewrnyc wrote: post #17

Earlier quoted context omitted.

I'm using it for the analytics suite at my company (large ecommerce multinational). Its weird at first, coming from a background of Access then SQLite then MySQL/PHPMyAdmin but you get used to it. I essentially treat it like a gigantic python dictionary object. The sharding is too much of a ball-ache to set up so I've created an optimal way of distributing/mapping files across our cluster to make use of all machines.…

Where exactly is the net gain in your situation? So right off the bat, you've lost the querying power that SQL offers. When dealing with data that's intended to be analyzed, that sounds like a pretty big loss. Clearly its built-in sharding support, which is often touted as one of its biggest benefits, wasn't suitable for you. So you had to invest some time and effort coming up with an alternate system. That sounds li…

right off the bat, you've lost the querying power that SQL offers.

Can you point me to any reasonably-priced database systems that allow me to execute SQL queries against a cluster of shared-nothing machines running commodity hardware? I'm not interested in paying $20-100K/TB/year for Vertica and friends.

SQL can be great if you have vast amounts of money or if your data can fit onto a single machine. When neither of those things are true though, nosql DBs become important.

Re: Why MongoDB Never Worked Out at Etsy

#29

Post by druiid »

MichaelSalib wrote: post #28
PommeDeTerre wrote: post #21

Earlier quoted context omitted.

Where exactly is the net gain in your situation? So right off the bat, you've lost the querying power that SQL offers. When dealing with data that's intended to be analyzed, that sounds like a pretty big loss. Clearly its built-in sharding support, which is often touted as one of its biggest benefits, wasn't suitable for you. So you had to invest some time and effort coming up with an alternate system. That sounds li…

right off the bat, you've lost the querying power that SQL offers. Can you point me to any reasonably-priced database systems that allow me to execute SQL queries against a cluster of shared-nothing machines running commodity hardware? I'm not interested in paying $20-100K/TB/year for Vertica and friends. SQL can be great if you have vast amounts of money or if your data can fit onto a single machine. When neither of…

Well are we talking with shard capabilities built-in, or having to roll your own? If you can roll your own then I suggest looking at Galera (we use this for our production MySQL stuff) http://www.codership.com

If it has to have built-in sharding then I can't think of anything off-hand.

Re: Why MongoDB Never Worked Out at Etsy

#30

Post by aroman »

emperorcezar wrote: post #11

Oh look, another "We thought Mongo was a silver bullet and found out that was wrong" post.

Except, as others have said, that is not what this article is saying at all. That said, your comment seems to imply that you just read the headline, and (perhaps understandably) didn't actually read the article.

I've learned, especially on HN, that article titles can be extremely misleading.

Post reply on HN