Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was MongoDB Ever the Right Choice?

simplethread.com

Post reply on HN
Print view View original

171–180 of 224 posts

Re: Was MongoDB Ever the Right Choice?

#171

Post by zjaffee »

matthewmacleod wrote: post #30

I would argue that MongoDB is not—and has never been—the best choice for solving any particular technical problem. But it had some other "advantages" over other, better solutions – in that it was easier to set up, didn't require schema definition, had a passable clustering story etc. I have worked with at least one company that had been built using MongoDB as a primary data store from day one. This caused untold pain…

I have to absolutely disagree with the point you're making about mongodb never being something an experienced developers should use.

An experience developer understands when the goal is to deliver something as fast as possible vs building something more resilient.

If you're building an internal tool, particularly one that won't be storing data that would later be used for analytics, there's absolutely no reason to not go with mongodb over a relational database considering you can get hosted versions of both for the same price. For a less than a week long project, you could be saving at least 20% of the time spent not having to worry about a datamodel.

Re: Was MongoDB Ever the Right Choice?

#173

Post by bshipp »

I feel like Bill Murray in Punxsutawney. Didn't we just see this article yesterday?

https://news.ycombinator.com/item?id=19492562

Here was my comment from yesterday morning:

>Question 1: What problems am I trying to solve?

I wish I had really thought about this when I first wrote my largest web scraper. At the time, I was still relatively new to database design and programming in general. This web scraper, out of the thousands I've written in the interim, is--of course--the one that is still going strong many years later.

I eshewed MongoDB for all the reasons given to me on the internet and, because I was slowly gaining competence with SQL, ended up building a large and complex pipeline to send the data right into Postgres. In retrospect, this was a serious design mistake, and one that I regret the most.

Although I still contend that the data did eventually need to be normalized, I now believe that I was doing it far too early. By ingesting the JSON stream into a parser, splitting it up, generating foreign keys, and then forcing the whole works into a single Postgres database I severely limited the capacity of my web scraper (and also guaranteed the need for a very powerful server to run it).

Had I initially dumped all results into MongoDB (or some other efficient document store) and then, separately, parsed the output into normalized SQL, I would have dramatically simplified the operation, maintenance, and debugging of my web scraper. Plus it would have been much simpler to spawn work jobs on to different machines instead of trying to break up huge monolithic processes with poorly defined endpoints. There have been many lessons learned.

In short, Mongo likely serves a very good purpose for high-speed data storage and manipulation (although it's hardly alone in this space). However, it's still likely not a great all-around solution and works best when supported by an ACID-based normalized RDBMS. Unless, of course, things have dramatically changed in recent updates.

Re: Was MongoDB Ever the Right Choice?

#174

Post by _Marak_ »

wmij wrote: post #112
_Marak_ wrote: post #70

It's been over 10 years and CouchDB has not let me down once. Still using it in production everyday.

I also really like CouchDB and can attest to it being a reliable part of the stack. I'm using it on a personal project (Node/Nano/Couch) that has the potential for needing to store a lot of data but hasn't gotten there yet, so I can't speak from experience yet on performance/scalability. It has been so far great in production and also great for the normal CRUD parts of this project. The main reasons I chose Couch ove…

I've had decent success so far with CouchDB 2.0 and Mango for performing ad hoc queries.

I believe Mango will automatically create the view index on the fly for any query you make and save the index for later. Before CouchDB 2.0 this would usually have to be done manually before the query was run.

Re: Was MongoDB Ever the Right Choice?

#175

Post by mbell »

Thaxll wrote: post #167
matthewmacleod wrote: post #30

I would argue that MongoDB is not—and has never been—the best choice for solving any particular technical problem. But it had some other "advantages" over other, better solutions – in that it was easier to set up, didn't require schema definition, had a passable clustering story etc. I have worked with at least one company that had been built using MongoDB as a primary data store from day one. This caused untold pain…

"passable clustering" you talk about PG which has nothing, so when you start to re-implement sharding yourself you're in trouble. Fortnite uses MongoDB for everything, they have a huge load ( 7.5M CCU is insane ) and don't have more problems than if they choose something else. MongoDB gets bad press on Reddit / HN probably because of past marketing campaigns, but nowdays it's a very solid solution.

> "passable clustering" you talk about PG which has nothing, so when you start to re-implement sharding yourself you're in trouble.

Or you can just use Citus.

Re: Was MongoDB Ever the Right Choice?

#176

Post by sebringj »

After using mongodb for many years, I haven't run into a use case where I should have used it over SQL. The only benefit is rapid prototyping for me personally and it initially felt awesome to have mongoose in node be the schema as it was super easy to modify. But you pay for it big time as you go and find that anything difficult means you'll have to do backflips and more requests to get things done and get used to very strange and verbose nested json queries. It's the 10-15% use cases that are really hard. NoSQL is especially annoying when needing any type of joins because despite your best planning, it still happens. There were many cases I needed to fallback on postgres to get stuff done and have a second source of data to sync and then I was wondering why I didn't just do it all in postgres first.

Re: Was MongoDB Ever the Right Choice?

#177

Post by scarface74 »

giulianob wrote: post #133
fooker wrote: post #108

Earlier quoted context omitted.

The point is that the query I mentioned requires joins. You can of course get the same information from key value pairs, it will just require a number of scans over all your data, which doesn't scale if you need the queries to be fast. On the RDBMS side, there has been more than three decades of research on optimizing patterns like this. You don't want to try and reinvent that. If you can know for sure from the start…

MongoDB has supported some types of joins for years now, you can do quite a bit with MongoDB aggregation pipelines. https://docs.mongodb.com/manual/reference/operator/aggregati... As requirements change, you do need to migrate your data into a schema that makes sense and that's regardless of whether you're using SQL or MongoDB.

Yes Mongo supports joins. But, I wouldn’t use them. Application servers scale much easier than database servers. You’re not getting any efficiency gains from doing server side joins over just reading documents from the left side and doing an “in” query with the ids from the right side. Assuming you are doing the equivalent of a left outer join.

In fact, if you are using C#. You could use the same LINQ syntax either way.

Re: Was MongoDB Ever the Right Choice?

#178

Post by giulianob »

scarface74 wrote: post #177
giulianob wrote: post #133

Earlier quoted context omitted.

MongoDB has supported some types of joins for years now, you can do quite a bit with MongoDB aggregation pipelines. https://docs.mongodb.com/manual/reference/operator/aggregati... As requirements change, you do need to migrate your data into a schema that makes sense and that's regardless of whether you're using SQL or MongoDB.

Yes Mongo supports joins. But, I wouldn’t use them. Application servers scale much easier than database servers. You’re not getting any efficiency gains from doing server side joins over just reading documents from the left side and doing an “in” query with the ids from the right side. Assuming you are doing the equivalent of a left outer join. In fact, if you are using C#. You could use the same LINQ syntax either w…

The OP just said it would be hard to do that query (regardless of performance) and I'm saying it's totally capable.

Re: Was MongoDB Ever the Right Choice?

#179

Post by ralusek »

netok wrote: post #131
scarface74 wrote: post #63

Earlier quoted context omitted.

I’m a big fan of Mongo for the use case you described - searching by ID and all information in one document. But people don’t seem to understand that there are plenty of scenarios where you really either don’t know the schemes in advance and/or the “schema” is defined by an external source. I worked for a company that sold software that allowed users to create forms that could be filled out either on the web or via a…

EAV.

JSONB > EAV

Re: Was MongoDB Ever the Right Choice?

#180

Post by dajohnson89 »

rooam-dev wrote: post #122
wefarrell wrote: post #118

Earlier quoted context omitted.

It’s been a while since I’ve worked with Mongo but I remember having significant performance issues with indexed arrays, particularly for compound indexes. They weren’t solvable by any tweaking, it was a fundamental problem with that feature, to the point where we wondered why they allowed it at all. It’s possible they’ve fixed it by now.

Thanks, we'll add to our "eye keeping" list.

If you "abuse" MongoDB and treat it like a relational store, certain queries will be difficult to write and very poorly performing. At least that was our experience, 2 years ago.
Post reply on HN