Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I Migrated Away From MongoDB

svs.io

Post reply on HN
Print view View original

151–160 of 213 posts

Re: Why I Migrated Away From MongoDB

#151

Post by taligent »

reinhardt wrote: post #128
taligent wrote: post #123

Earlier quoted context omitted.

The plural of anecdote is not data. You know that right ? In my case every NoSQL developer I've met had a lot of experience in SQL as well. But do you have ask why so called inexperienced users are choosing NoSQL in the first place ? Hint: it's because most RDBMS are ridiculously complex and inflexible.

The last line sounds pretty familiar after replacing "NoSQL" with "PHP" and "RDBMS" with "decent languages".

I completely agree. This whole NoSQL/RDBMS situation is programming language debate being played out over again.

And just like not every programming language works for every situation. Neither does every database.

Re: Why I Migrated Away From MongoDB

#152

Post by taligent »

nerd_in_rage wrote: post #131
taligent wrote: post #114

Earlier quoted context omitted.

This is complete and utter nonsense. Facebook and Instagram itself I imagine would care VERY much if data went missing as it would mean not just (a) user data going missing but (b) potentially huge inconsistencies in the social graph. Unless you have some evidence that those companies do not care about data durability ?

What? It's the absolute truth. If someone's wall post or tweet goes missing, it's totally inconsequential. "Social graph" - LOL.

Sure. But if a person or a photo data object goes missing ?

Facebook has different data stores for each of their features so I imagine the loss of a person would be pretty nasty as would a photo data object that multiple people have tagged or commented on.

My point is that when you are the size of Facebook, Twitter etc a loss of data in 0.001% of cases equates to a LOT of data and relationships.

Re: Why I Migrated Away From MongoDB

#153

Post by einhverfr »

taligent wrote: post #114
gregjor wrote: post #90

Earlier quoted context omitted.

I'll add that email and Facebook postings and Instagram photos don't require the same kind of data consistency and durability that medical records, bank transactions, or rail car movements require. If Google loses some old emails or Facebook loses the last thousand "likes" on a Jersey Shore fan page that's not much of a catastrophe, and they don't guarantee their data anyway. For those use cases Mongo or Cassandra et…

This is complete and utter nonsense. Facebook and Instagram itself I imagine would care VERY much if data went missing as it would mean not just (a) user data going missing but (b) potentially huge inconsistencies in the social graph. Unless you have some evidence that those companies do not care about data durability ?

If it is that important, they'd be using an ACID-compliant solution, right?

You don't use Best-guess Attempt at Sorta-consistent Eventualities for important data ;-)

Re: Why I Migrated Away From MongoDB

#154

Post by taligent »

pessimizer wrote: post #130
taligent wrote: post #123

Earlier quoted context omitted.

The plural of anecdote is not data. You know that right ? In my case every NoSQL developer I've met had a lot of experience in SQL as well. But do you have ask why so called inexperienced users are choosing NoSQL in the first place ? Hint: it's because most RDBMS are ridiculously complex and inflexible.

The plural of insulting everyone's intelligence who disagrees with you is not authority. >But do you have ask why so called inexperienced users are choosing NoSQL in the first place? Hint: it's because most RDBMS are ridiculously complex and inflexible. If we're going to have a cliche fight, this one is called having your cake and eating it too. Either inexperienced users are gravitating to NoSQL or they aren't. Oper…

I never meant to imply that inexperienced users were definitely moving to NoSQL. I would imagine that orders of magnitude more of them are still using MySQL due to its pervasiveness.

Only that if they were the current complexity of RDBMS would be a big factor. Accessing a database as a REST service like say CouchDB or having fluid schemas like MongoDB is much easier to handle than ER models.

Re: Why I Migrated Away From MongoDB

#155

Post by lttlrck »

nerd_in_rage wrote: post #131
taligent wrote: post #114

Earlier quoted context omitted.

This is complete and utter nonsense. Facebook and Instagram itself I imagine would care VERY much if data went missing as it would mean not just (a) user data going missing but (b) potentially huge inconsistencies in the social graph. Unless you have some evidence that those companies do not care about data durability ?

What? It's the absolute truth. If someone's wall post or tweet goes missing, it's totally inconsequential. "Social graph" - LOL.

You think FB doesn't consider durability and consistency essential? With almost one billion users?!

Wow.

Whether it is consequential or not is completely beside the point. As is whether you understand the "social graph".

Re: Why I Migrated Away From MongoDB

#156

Post by lttlrck »

dccoolgai wrote: post #3

"To be honest, the decision to use MongoDb was an ill-thought out one. Lesson learned - thoroughly research any new technology you introduce into your stack, know well the strengths and weaknesses thereof and evaluate honestly whether it fits your needs or not - no matter how much hype there is surrounding said technology." I think you are not alone in learning this lesson with this particular technology. Fortunately…

"You have to put the whole dataset on RAM?"

This is hardly a hidden feature...

Re: Why I Migrated Away From MongoDB

#157

Post by lttlrck »

dccoolgai wrote: post #3

"To be honest, the decision to use MongoDb was an ill-thought out one. Lesson learned - thoroughly research any new technology you introduce into your stack, know well the strengths and weaknesses thereof and evaluate honestly whether it fits your needs or not - no matter how much hype there is surrounding said technology." I think you are not alone in learning this lesson with this particular technology. Fortunately…

"You have to put the whole dataset on RAM?"

This is hardly a hidden feature...

Re: Why I Migrated Away From MongoDB

#158

Post by dkhenry »

stevencorona wrote: post #10

The downside, or challenge, with NoSQL (generally speaking) is that you need to handle your aggregations ahead of time - you need to know what queries you'll want to run in the future when you store your data. If you have some new aggregation you want to keep, you'll need to re-process the data (with Hadoop or something else). It's the trade-off of being able to scale reads and writes horizontally. And unless you nee…

This is also true of SQL databases. It just depends on _what_ your aggregating. In this case if you need to aggregate the count of something then in a DB like MySQL you can lean on the index to get a quick count, but if you need to SUM or AVG you will be doing just as much CPU level work as a NoSQL solution. The difference is in MySQL its a simple query with the AVG operator, in MongoDB its a Map-Reduce query which is much harder to write.

In general you need to know what your doing under the hood and how either solution effects your problem domain. Where I work we need to aggregate billions of data points on demand. This can't be done in real time without pre-aggregating the results ( and even then it takes tons of I/O just to process the aggregated data set )

Re: Why I Migrated Away From MongoDB

#159

Post by dkhenry »

Competly aside from the Article. The level of vitriolic discourse in this topic is astounding. I am amazed that as a community discussions of Database engines can draw out such mean spirited anger. I have never down voted as many comments on HN in a single thread then I have on this topic. I don't care which side of the debate you come down on. There is no excuse for belittleing and insulting others in a technical forum. Thats right I am looking at you

    gregjor, gaius, and zemo
In this case it appears to mostly be those arguing for Postgre, but I wouldn't care if you were arguing for sunshine and unicorns there is a way to behave civilly and your not doing it.

Re: Why I Migrated Away From MongoDB

#160

Post by stickfigure »

einhverfr wrote: post #145
stickfigure wrote: post #98

Earlier quoted context omitted.

I'm not sure what this has to do with SQL vs NoSQL. There are myriad differences between storage systems which have little to do with referential integrity constraints or data validation. Polymorphic structures are a PITA to model in relational tables, but they fit very well in schemaless, document-oriented NoSQL systems. This has nothing to do with data validation. Rapidly changing schemas are a PITA in RDBMSes, esp…

The basic tradeoff is between flexible input and flexible output. I do think it is sound advice to push the relational database model (this overlaps but is not identical to the relational algebra model) to the breaking point before going elsewhere. If you don't do data validation you can't do ad hoc data transformation later reliably. Strict schemas are always an investment in the future for that reason. > Polymorphi…

JSON and XML types give you all the pain of a NoSQL system with none of the benefits of an RDBMS. No thanks. My tools elegantly map database fields to my objects; doing this with serialization formats is incredibly crude and brittle. I do it for some edge cases already; I am looking forward to removing that code.

A schemaless datastore does not mean a schemaless app. Furthermore, the application layer is a far better place to do validation simply because it has more knowledge of the real world. An RDBMS can constraint an age field to be integer; the application can constrain it to be positive and within the probable lifespan of a human.

How many old schemas does your application have to support? Only as many as I want to; reprocessing removes the old, which I tend to do fairly quickly. I have done a lot of schema migration over the last three years on GAE and the process is vastly easier than it is in RDBMS-land. There's no need to stop the world.

Hey, I love Postgres. I spent most of my (almost two-decade-long) career building apps on many different RDBMSes. But there are many applications that are better suited to alternatives, especially with large-scale consumer-facing web apps. These threads annoy me; we should all be looking at new tools saying "wow, I wonder what this is good for" rather than smugly burning all the heretics that turn away from to the One True Idol.

Post reply on HN