Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why MongoDB Never Worked Out at Etsy

mcfunley.com

Post reply on HN
Print view View original

61–70 of 73 posts

Re: Why MongoDB Never Worked Out at Etsy

#61

Post by Groxx »

julian37 wrote: post #47
thefreeman wrote: post #7

Earlier quoted context omitted.

I think his point was more "Why you shouldn't try to run two different DBMS in production"

That would be a silly point to make. Many people are running multiple DBMS in production without regrets. Each DBMS has strengths and weaknesses in different areas, and for many businesses it doesn't make sense to shove everything into a single DBMS just because things are easier to manage that way. His point was "in our specific scenario the benefits of using MongoDB were outweighed by the difficulties in having to…

And it depends on how you define "DB". We've recently started using Redis for a number of things, and it has been incomprehensibly faster than our SQL database when it comes to rapidly finding and inserting records in (the equivalent of) a billion-row table. It's also far cheaper to run than significantly beefing up the database server just because of that one bottleneck. Easily worth the time and trouble of setting it up (neither of which was very much).

Re: Why MongoDB Never Worked Out at Etsy

#62

Post by tillk »

mattmanser wrote: post #59
julian37 wrote: post #47

Earlier quoted context omitted.

That would be a silly point to make. Many people are running multiple DBMS in production without regrets. Each DBMS has strengths and weaknesses in different areas, and for many businesses it doesn't make sense to shove everything into a single DBMS just because things are easier to manage that way. His point was "in our specific scenario the benefits of using MongoDB were outweighed by the difficulties in having to…

Are there? Who are these magical people? If you've got data sitting in one place and then data sitting in another, it's generally a massive fucking pain in the ass. Speaking from my experience anyway. Bonus points if one of them was written by a company in the early 2000s who didn't trust those newfangled RDBMSes to get it right. Extra bonus points if they thought UTF-8 was for sissies.

I don't consider myself to be magical, but anyway — since you were asking.

We primarily use MySQL and CouchDB (BigCouch). Our user records (accounts and payment) are stored the RDBMS, while the data the users created is in BigCouch. We enjoy the schemaless nature, durability and scalability of BigCouch a lot.

Depending on your definition of "database system", we also have a hefty Solr index (for search), some Redis (no persistence only to connect systems/services via pubsub) and Memcache (cache).

Re: Why MongoDB Never Worked Out at Etsy

#63

Post by mminer »

stesch wrote: post #60

Wondering why everybody is using MySQL if Postgresql is supposed to be better. Are there any (startup) success stories involving Postgresql?

Perhaps one reason is that more hosting providers support MySQL but not PostgreSQL. Amazon AWS, Google Cloud SQL, and numerous others offer hosted MySQL solutions but not PostgreSQL. I'm unsure how much overall usage such service providers account for though; it would be an interesting stat.

Re: Why MongoDB Never Worked Out at Etsy

#64

Post by jasonmccay »

tillk wrote: post #62
mattmanser wrote: post #59

Earlier quoted context omitted.

Are there? Who are these magical people? If you've got data sitting in one place and then data sitting in another, it's generally a massive fucking pain in the ass. Speaking from my experience anyway. Bonus points if one of them was written by a company in the early 2000s who didn't trust those newfangled RDBMSes to get it right. Extra bonus points if they thought UTF-8 was for sissies.

I don't consider myself to be magical, but anyway — since you were asking. We primarily use MySQL and CouchDB (BigCouch). Our user records (accounts and payment) are stored the RDBMS, while the data the users created is in BigCouch. We enjoy the schemaless nature, durability and scalability of BigCouch a lot. Depending on your definition of "database system", we also have a hefty Solr index (for search), some Redis (…

Yep ... in interacting with many developers, setups like this are almost commonplace.

Re: Why MongoDB Never Worked Out at Etsy

#65

Post by benatkin »

druiid wrote: post #15
darkxanthos wrote: post #14

Earlier quoted context omitted.

Did you read the article? His main reason had nothing to do with Mongo itself.

Yes, yes I did. My question is why it was selected in the first place, rather than a period of exploration of the benefits and drawbacks of the different NoSQL solutions (which perhaps might have stopped this particular test-case from being a failure). The article doesn't mention those details though. Also, we DO implement a two DB setup... Couchbase and MySQL. They both have their place.

Then you would know that they were doing this in 2010, before Couchbase.

Re: Why MongoDB Never Worked Out at Etsy

#66

Post by ashcairo »

dindresto wrote: post #3
untog wrote: post #2

Genuine question- who is using MongoDB successfully in production, and at scale? I'm not aware of anyone myself- I hear of it being used in hackathons etc because its so quick to set up, but I'd be curious to know what people are using it with.

They have a list of who's using MongoDB in production. http://www.mongodb.org/display/DOCS/Production+Deployments

I loved this comment 'I noticed that "mongodb.org" is not on the Production Deployments List above. It appears this site uses Confluence which I believe uses PostgreSQL(or MySQL). It's hard to have confidence in an organization that does not even use its own technology.'

Re: Why MongoDB Never Worked Out at Etsy

#67

Post by eLobato »

untog wrote: post #2

Genuine question- who is using MongoDB successfully in production, and at scale? I'm not aware of anyone myself- I hear of it being used in hackathons etc because its so quick to set up, but I'd be curious to know what people are using it with.

CERN. We got too excited and started using it for EVERYTHING (it started just being part of the LHC data analyzing project) and it didn't work in some cases, but for some projects it fitted in perfectly.

http://lanyrd.com/2012/mongonyc/stqzc/

Re: Why MongoDB Never Worked Out at Etsy

#68

Post by lmm »

I'm surprised to hear this coming from Etsy, a place I thought of as doing deployment right.

All these things should be simple. You already have (or should have) a unified system for dealing with logging/monitoring/graphing/init scripts/backup across multiple services that are far more different from each other than they are from mongodb (Sharding strategy and slow queries are probably an application-level concern). It shouldn't be hard - in fact it should be trivial - to add one more service. At last.fm (disclaimer: my experience was brief and getting on for two years ago) it felt like we were running every database under the sun, but we had a unified system for doing deployment/monitoring/everything, so it was no bother to add one more if an application wanted it.

Re: Why MongoDB Never Worked Out at Etsy

#69

Post by ry0ohki »

Devilboy wrote: post #58
ry0ohki wrote: post #34

Earlier quoted context omitted.

I run two sites, one is the perfect use-case for MongoDB - http://www.AUsedCar.com , it's a used car search engine. We've seen nothing but benefits by switching to it from MS SQL Server. Queries are way faster etc... It's a great use case because 99.9% of DB interactions are read-only searches. My other site, http://www.BudgetSimple.com on the other hand is using SQL Server (in the process of porting to MySQL). It wo…

Can you give me an example of a query that was made much faster?

The biggest difference was pulling a random record. This is an odd use case for most people, and not even a common one for me, but say I wanted to show you a random car near your location. The more common use-cases that were sped up were any geo-location searches, ie you're searching within X miles.

Re: Why MongoDB Never Worked Out at Etsy

#70

Post by lincolnbryant »

ryoung wrote: post #55
lincolnbryant wrote: post #50

Earlier quoted context omitted.

This also makes it sound like whoever intervened to rewrite said feature in sharded mysql had an easy time. Usually this would not be an obvious port. However we don't know the technical nature of the feature or specifically why it failed.

I'm the developer that helped migrate the data from mongo to mysql (under mcfunley's supervision). Even a straightforward data migration becomes complicated when you have to do it without affecting the production feature or consumers of your public api (parallel writes to both dbs, snapshot and move the historical data, switch reads to the new db, etc). In addition, we took the opportunity to move the feature to a sh…

Congrats then! I've seen mongo features start simple then end containing a bunch of embedded lists of primary keys to SQL or other data stores, rather than what would be a bridge table or two. Not as elegant if your not putting everything there (as some people here say here is mandatory to prevent the overhead running mongo on top of everything else like you mentioned
Post reply on HN