Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why MongoDB Never Worked Out at Etsy

mcfunley.com

Post reply on HN
Print view View original

51–60 of 73 posts

Re: Why MongoDB Never Worked Out at Etsy

#51

Post by mark_story »

ry0ohki wrote: post #34
untog wrote: post #2

Genuine question- who is using MongoDB successfully in production, and at scale? I'm not aware of anyone myself- I hear of it being used in hackathons etc because its so quick to set up, but I'd be curious to know what people are using it with.

I run two sites, one is the perfect use-case for MongoDB - http://www.AUsedCar.com , it's a used car search engine. We've seen nothing but benefits by switching to it from MS SQL Server. Queries are way faster etc... It's a great use case because 99.9% of DB interactions are read-only searches. My other site, http://www.BudgetSimple.com on the other hand is using SQL Server (in the process of porting to MySQL). It wo…

> It's a great use case because 99.9% of DB interactions are read-only searches.

Did you ever consider a datasource like elasticsearch? If yes what made you choose mongo?

Re: Why MongoDB Never Worked Out at Etsy

#52

Post by ry0ohki »

mark_story wrote: post #51
ry0ohki wrote: post #34

Earlier quoted context omitted.

I run two sites, one is the perfect use-case for MongoDB - http://www.AUsedCar.com , it's a used car search engine. We've seen nothing but benefits by switching to it from MS SQL Server. Queries are way faster etc... It's a great use case because 99.9% of DB interactions are read-only searches. My other site, http://www.BudgetSimple.com on the other hand is using SQL Server (in the process of porting to MySQL). It wo…

> It's a great use case because 99.9% of DB interactions are read-only searches. Did you ever consider a datasource like elasticsearch? If yes what made you choose mongo?

No, and I actually used to work for a company that made a similar type of search engine!

That probably would have worked as well (don't think I considered that specific solution). Mongo came up on top because of it's wide use (among other things), ie it's pretty easy to find support and lots of stories about how to scale it under different scenarios.

Re: Why MongoDB Never Worked Out at Etsy

#53

Post by jshen »

untog wrote: post #2

Genuine question- who is using MongoDB successfully in production, and at scale? I'm not aware of anyone myself- I hear of it being used in hackathons etc because its so quick to set up, but I'd be curious to know what people are using it with.

We use it for a bunch of stuff at Disney. Games, websites, etc.

Re: Why MongoDB Never Worked Out at Etsy

#54

Post by bonobo »

gfodor wrote: post #45
bonobo wrote: post #5

I was expecting a post describing why MongoDB specifically weren't fit to their use case, but the TL;DR version is basically: " Before you get too excited, the reason for the failure is probably not any of the ones you're imagining. Mainly it's this: adding another kind of production database was a huge waste of time. " The blog title is misleading IMO. It could as well be titled "Why [any other DBMS] Never Worked Ou…

Your proposed title is very confusing and meta. The current title is only misleading if you are going into it with biased expectations about what it's going to teach you. The article is interesting precisely because it is not just another "NoSQL sucks" screed.

What I meant was that the article was not about MongoDB at all. I think thefreeman summarized it better than me on his reply: the author is talking about his experiences on running more than one DBMS in production, not about MongoDB in particular.

I'm not even saying if the article was interesting or not. It surely has it's merits, it touches a point worth discussing (the downsides of trying to cover one DBMS's weakness throwing in another DBMS in the mix), but I was uninterested. I was expecting to hear why MongoDB wasn't fit for his use case, to better understand when to use NoSQL and when not to, so yes, I was indeed biased waiting for something different—but, in my defense, I say I was biased because the title made me think this way.

Re: Why MongoDB Never Worked Out at Etsy

#55

Post by ryoung »

lincolnbryant wrote: post #50
vph wrote: post #42

The bottom line is that MongoDB and MySQL are two different persistent data structures. MySQL is a more powerful data structure that can do more things. MongoDB is less powerful, but is more efficient at certain things. Due to pre-mature optimization or shortsightedness, some folks are romanticized with the efficiency of a less powerful data structure (MongoDB) and fail to realize that their application really need t…

This also makes it sound like whoever intervened to rewrite said feature in sharded mysql had an easy time. Usually this would not be an obvious port. However we don't know the technical nature of the feature or specifically why it failed.

I'm the developer that helped migrate the data from mongo to mysql (under mcfunley's supervision). Even a straightforward data migration becomes complicated when you have to do it without affecting the production feature or consumers of your public api (parallel writes to both dbs, snapshot and move the historical data, switch reads to the new db, etc). In addition, we took the opportunity to move the feature to a sharded architecture and to rethink the schema. Anyway, you're right in that it wasn't exactly an obvious or easy port.

Re: Why MongoDB Never Worked Out at Etsy

#56

Post by bobfunk »

ddorian43 wrote: post #40
bobfunk wrote: post #37

Earlier quoted context omitted.

We're using it for Webpop ( http://www.webpop.com ) and have generally been very happy with it. We were very well aware of its characteristics when choosing our DB, and didn't go in expecting any magic Web Scale or somehow getting a HA setup with plenty of durability with just one server. For a multitenant CMS where you want to store documents with custom schemas, need more than just a key/value store and want some c…

Are you using mongodb for the analytics and how do you store/index/query custom fields?

We use mongodb for the analytics as well yes. It's a less obvious choice there than for the CMS part, but it's a good enough fit, in-place updates can be really handy and we prefer not having 2 different databases.

We don't index custom fields and for queries where that would be required we do the actual querying with ElasticSearch, but for simple filters on a custom field or the like, Mongo does fine.

Re: Why MongoDB Never Worked Out at Etsy

#57

Post by tobyjsullivan »

I'm so glad to see this post. I remember having a conversation with someone from Etsy at one point and they made an offhand comment about MongoDB having been a terrible idea with a hint that there was a longer story to it than we had time for. I've been curious about the story ever since.

Finally, some closure!

Re: Why MongoDB Never Worked Out at Etsy

#58

Post by Devilboy »

ry0ohki wrote: post #34
untog wrote: post #2

Genuine question- who is using MongoDB successfully in production, and at scale? I'm not aware of anyone myself- I hear of it being used in hackathons etc because its so quick to set up, but I'd be curious to know what people are using it with.

I run two sites, one is the perfect use-case for MongoDB - http://www.AUsedCar.com , it's a used car search engine. We've seen nothing but benefits by switching to it from MS SQL Server. Queries are way faster etc... It's a great use case because 99.9% of DB interactions are read-only searches. My other site, http://www.BudgetSimple.com on the other hand is using SQL Server (in the process of porting to MySQL). It wo…

Can you give me an example of a query that was made much faster?

Re: Why MongoDB Never Worked Out at Etsy

#59

Post by mattmanser »

julian37 wrote: post #47
thefreeman wrote: post #7

Earlier quoted context omitted.

I think his point was more "Why you shouldn't try to run two different DBMS in production"

That would be a silly point to make. Many people are running multiple DBMS in production without regrets. Each DBMS has strengths and weaknesses in different areas, and for many businesses it doesn't make sense to shove everything into a single DBMS just because things are easier to manage that way. His point was "in our specific scenario the benefits of using MongoDB were outweighed by the difficulties in having to…

Are there? Who are these magical people?

If you've got data sitting in one place and then data sitting in another, it's generally a massive fucking pain in the ass.

Speaking from my experience anyway. Bonus points if one of them was written by a company in the early 2000s who didn't trust those newfangled RDBMSes to get it right. Extra bonus points if they thought UTF-8 was for sissies.

Post reply on HN