Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was MongoDB Ever the Right Choice?

simplethread.com

Post reply on HN
Print view View original

101–110 of 224 posts

Re: Was MongoDB Ever the Right Choice?

#101

Post by ajcodez »

scarface74 wrote: post #63
dijit wrote: post #17

Since everyone is sharing their opinion and experience with mongodb I think I’ll share mine. As an appeal to authority I would like to mention that I have relevant vocational qualifications on the subject (more geared towards scalability and operations). Although I don’t believe it really matters - it will to those who assume I don’t understand best practice. MongoDB itself is not /really/ a valid choice in many scen…

I’m a big fan of Mongo for the use case you described - searching by ID and all information in one document. But people don’t seem to understand that there are plenty of scenarios where you really either don’t know the schemes in advance and/or the “schema” is defined by an external source. I worked for a company that sold software that allowed users to create forms that could be filled out either on the web or via a…

There’s nothing stopping you from creating tables and indexes on the fly using SQL but it requires explicit commands.

> What would an RDBMS have brought us?

It would probably work using a forms table, fields table, submissions table, and values table.

Re: Was MongoDB Ever the Right Choice?

#102

Post by scarface74 »

ajcodez wrote: post #101
scarface74 wrote: post #63

Earlier quoted context omitted.

I’m a big fan of Mongo for the use case you described - searching by ID and all information in one document. But people don’t seem to understand that there are plenty of scenarios where you really either don’t know the schemes in advance and/or the “schema” is defined by an external source. I worked for a company that sold software that allowed users to create forms that could be filled out either on the web or via a…

There’s nothing stopping you from creating tables and indexes on the fly using SQL but it requires explicit commands. > What would an RDBMS have brought us? It would probably work using a forms table, fields table, submissions table, and values table.

And then what happens when they add a field to the form and the table already has a million rows? What happens when they decide that the numeric field should have strings?

It would probably work using a forms table, fields table, submissions table, and values table.

I didn’t ask “would it have worked”, I asked “what would have bought us”.

Re: Was MongoDB Ever the Right Choice?

#103

Post by sonnyblarney »

mruts wrote: post #15
sonnyblarney wrote: post #10

I actually think NoSQL might be better for MVP and prototyping. Why? Because they can handle churn more quickly, and you don't have to worry about scale. Once the data layer starts to resonate around a solution ... then pick the right horse for it.

Why not just use a SQL db that handles json and stick everything in a couple columns? Changing databases is a lot of work, and as far as I can tell, mongo isn’t really providing much value over SQL dbs that already support schemaless json columns.

The ability to query, index etc. I believe is much more limited than regular Mongo, but you can correct me if I'm wrong. Also, it feels like a cludge.

RDBMS to me is to expensive in the churn and I've run into some deadly query scenarios.

As a non-storage expert I would just love one of those 2000's era 'object databases' to start with. We used to use them a lot in networking because they suited well to topology. I'm not sure they're much of a thing anymore however as I suggest JSON DB's are pretty close to it so there's no need for it.

But they fit developer's mentality really well.

Re: Was MongoDB Ever the Right Choice?

#104

Post by scarface74 »

fooker wrote: post #100
scarface74 wrote: post #63

Earlier quoted context omitted.

I’m a big fan of Mongo for the use case you described - searching by ID and all information in one document. But people don’t seem to understand that there are plenty of scenarios where you really either don’t know the schemes in advance and/or the “schema” is defined by an external source. I worked for a company that sold software that allowed users to create forms that could be filled out either on the web or via a…

What happens when you need to do something like "Select browser user agent from all users who filled forms for a particular set of clients after a given date." ? This would fit in a single SQL query which is expected to perform reasonably well, with an unstructured database optimizing this query will take months of work.

How so? In our case, meta data like the userid, browser agent, date entered, etc was always added to the object before it was stored and those fields were indexed. They are just name value pairs.

Re: Was MongoDB Ever the Right Choice?

#105

Post by StavrosK »

matthewmacleod wrote: post #30

I would argue that MongoDB is not—and has never been—the best choice for solving any particular technical problem. But it had some other "advantages" over other, better solutions – in that it was easier to set up, didn't require schema definition, had a passable clustering story etc. I have worked with at least one company that had been built using MongoDB as a primary data store from day one. This caused untold pain…

In my experience, when there's something that offers additional guarantees for additional work, it's very rarely the right solution to completely forgo the guarantees. This goes for schemas, typing in languages, etc.

Maybe the additional work is too much for the benefit, but it's not that there's no benefit. Forgoing the guarantees usually leads to much more extra work. I think Python's "no types anywhere" was a reaction to Java's "types everywhere", but neither is optimal. Something with gradual typing or type inference has much more bang for its buck, so whenever I see "I use MongoDB because there's no schema to worry about", I always think that it's going to come back and bite someone in the ass.

Re: Was MongoDB Ever the Right Choice?

#106

Post by skywhopper »

As with anything, it depends on the project. I’m working on an internal service that uses Mongo as a single merged cache for a lot of mostly unchanging data from various data stores with different credentials for each, which are distributed around the world, that we otherwise have to fetch through multiple comparatively slow API calls. For this, Mongo is perfect: no messing with schemas as they change, unannounced, from upstream; I can index just the fields I want to search on; Mongo will expire things for me on a TTL; the query API is simpler than the API we’re caching from; and we get results 20-50x faster. We looked into FoundationDB and Postgres, but they require a lot more initial setup. ElasticSearch is the closest solution, but it needs a lot more info about the schema up front and its query language is a nightmare compared to Mongo’s, for no real gain in functionality that I can see.

Is Mongo the right tool to build your entire business on top of? Probably not, but it can be the best tool for the right job.

Re: Was MongoDB Ever the Right Choice?

#107

Post by ajcodez »

scarface74 wrote: post #102
ajcodez wrote: post #101

Earlier quoted context omitted.

There’s nothing stopping you from creating tables and indexes on the fly using SQL but it requires explicit commands. > What would an RDBMS have brought us? It would probably work using a forms table, fields table, submissions table, and values table.

And then what happens when they add a field to the form and the table already has a million rows? What happens when they decide that the numeric field should have strings? It would probably work using a forms table, fields table, submissions table, and values table. I didn’t ask “would it have worked”, I asked “what would have bought us”.

> And then what happens when they add a field to the form and the table already had a million rows?

Maybe I don’t follow. You would insert a row into the fields table. Millions of rows should be fine.

> I asked “what would have bought us”.

I don’t know. None of this matters really, as long as the service works.

Re: Was MongoDB Ever the Right Choice?

#108

Post by fooker »

scarface74 wrote: post #104
fooker wrote: post #100

Earlier quoted context omitted.

What happens when you need to do something like "Select browser user agent from all users who filled forms for a particular set of clients after a given date." ? This would fit in a single SQL query which is expected to perform reasonably well, with an unstructured database optimizing this query will take months of work.

How so? In our case, meta data like the userid, browser agent, date entered, etc was always added to the object before it was stored and those fields were indexed. They are just name value pairs.

The point is that the query I mentioned requires joins. You can of course get the same information from key value pairs, it will just require a number of scans over all your data, which doesn't scale if you need the queries to be fast. On the RDBMS side, there has been more than three decades of research on optimizing patterns like this. You don't want to try and reinvent that.

If you can know for sure from the start that you'll never need queries like this, then of course something like Mongo will be awesome. But requirements change, hence this article.

Re: Was MongoDB Ever the Right Choice?

#109

Post by aaron42net »

TimFogarty wrote: post #61
bonesss wrote: post #53

Earlier quoted context omitted.

I evaluated several distributed databases for a healthcare-related system. The ability to lose messages in sharding scenarios, and the specifics of how one would recover them, made me think I could never support MongoDB for anything more serious than Reddit.

The Jepsen tests [1] have been run against MongoDB - while older versions presented edge-case opportunities for data loss, that's no longer the case with recent versions. The Jepsen tests also specifically test sharded clusters. From Aphyr's report: > MongoDB 3.6.4’s sharded clusters offer comparable safety to non-sharded deployments. These tests are now integrated into MongoDB's regular test suite. Maybe MongoDB was…

Wasn't it only has of version 3.4 that Jepsen stopped finding single-node data loss bugs in MongoDB? So it's been 3 years that MongoDB has been suitable for single-node data storage, and apparently 5 months that it's been reasonable to use in a sharded deployment.

Perhaps in another decade, MongoDB can shed its well-earned reputation for eating data.

Re: Was MongoDB Ever the Right Choice?

#110

Post by joncrane »

TimFogarty wrote: post #61
bonesss wrote: post #53

Earlier quoted context omitted.

I evaluated several distributed databases for a healthcare-related system. The ability to lose messages in sharding scenarios, and the specifics of how one would recover them, made me think I could never support MongoDB for anything more serious than Reddit.

The Jepsen tests [1] have been run against MongoDB - while older versions presented edge-case opportunities for data loss, that's no longer the case with recent versions. The Jepsen tests also specifically test sharded clusters. From Aphyr's report: > MongoDB 3.6.4’s sharded clusters offer comparable safety to non-sharded deployments. These tests are now integrated into MongoDB's regular test suite. Maybe MongoDB was…

The thing is, it's too late. When MongoDB was in its heyday, it got known as something that loses records. Not much you can do now when for every post affirming its consistency, there are two about how someone tested it and it failed consistency checks.
Post reply on HN