Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I Migrated Away From MongoDB

svs.io

Post reply on HN
Print view View original

201–210 of 213 posts

Re: Why I Migrated Away From MongoDB

#201

Post by Ingaz »

rbranson wrote: post #14

I'm no fan of MongoDB, but this same advice goes for any NoSQL data store. I am an Apache Cassandra contributor and community MVP, but my advice stays the same: it's best just to start with a SQL database and go from there. Read some books and learn it well: the "SQL Cookbook" from O'Reilly is great, and so is "The Art of SQL." Premature optimization continues to be the root of all evil.

Funny.

The same problem exists with OLAP databases.

People trying to do MOLAP without building ROLAP. They are thinking that MS SASS, ESSBASE, Cognos or Qlikview will do some "magic" that eliminate the need to carefully think about data

Re: Why I Migrated Away From MongoDB

#202

Post by pnathan »

einhverfr wrote: post #139
pnathan wrote: post #95

Earlier quoted context omitted.

What I like about Mongo is that it's very free-form; I can splat some data in there and not be too concerned about it until later on. I don't really like dealing with file systems for holding data. Of course Mongo ain't the best at relational data... :o

But this is the fundamental trade-off isn't it? If you aren't concerned about your data on input then you can't reliably transform it on output, right? In this regard fixed schemas slow down one side of development (handling input) but speed up another side (handling output, and related tasks like reporting).

This is ABSOLUTELY the tradeoff. I am not using Mongo for long-term data storage. I am using it for hacking systems together which need a place to store data. Possibly this data will be manually queried and examined later on, possibly this data lives only a day or two. The entire conversation changes when you are wanting structured data output that is stable and scalable for 10 years.

Re: Why I Migrated Away From MongoDB

#203

Post by karterk »

sray wrote: post #93
gaius wrote: post #81

Earlier quoted context omitted.

When you go to the supermarket, you may see products calling themselves "quality" or "luxury" or similar. Words like "powerful" and "scalable" are like that.

Thanks, but you didn't really explain anything. In what specific ways does it fail to scale well?

Scaling has a number of components to it - it's not just about the absolute number of requests. There are always trade-offs (read up about the CAP theorem). My experience with Mongo has been that it has very inconsistent performance. Such inconsistency makes capacity planning very difficult. Also - stuff like failover, sharding etc. are not elegant.

Re: Why I Migrated Away From MongoDB

#204

Post by einhverfr »

lmm wrote: post #197
einhverfr wrote: post #196

Earlier quoted context omitted.

>"If that level of indexing is sufficient I don't see why you can't just do the same thing in mongodb." What level of indexing is sufficient depends a great deal on the specifics of the database layout on disk. In InnoDB for example, sequential scans are very costly, and primary key lookups are very cheap. This is because the table is more or less contained in the primary key index and this must be traversed in key o…

That's all pretty interesting, but I still don't see what you get with postgres that you don't get with mongodb. Your database won't enforce your schema for you, but I don't see how that means "ad hoc reporting is impossible".

I have read through the Mongo db query docs and it does look like you can do some ad hoc retrieval queries, and some aggregation. But in the SQL world that's not really the same thing as ad hoc reporting.

I suppose "can't do" is too strong assuming your reporting matches your query. However these things look a lot simpler to do in SQL than in Mongo's approach, and I don't see how you can reliably transform data on output if you don't have a guaranteed reliable schema on input. Also I don't really understand how you would transform the data in this way with Mongo's API. I suppose you always could but it looks painful to my (admittedly uninitiated) eyes.

How many lines of code are required to express a 50 line SQL query doing 5 joins, complex filters and group-by transformations, etc?

Re: Why I Migrated Away From MongoDB

#205

Post by einhverfr »

lmm wrote: post #197
einhverfr wrote: post #196

Earlier quoted context omitted.

>"If that level of indexing is sufficient I don't see why you can't just do the same thing in mongodb." What level of indexing is sufficient depends a great deal on the specifics of the database layout on disk. In InnoDB for example, sequential scans are very costly, and primary key lookups are very cheap. This is because the table is more or less contained in the primary key index and this must be traversed in key o…

That's all pretty interesting, but I still don't see what you get with postgres that you don't get with mongodb. Your database won't enforce your schema for you, but I don't see how that means "ad hoc reporting is impossible".

I have read through the Mongo db query docs and it does look like you can do some ad hoc retrieval queries, and some aggregation. But in the SQL world that's not really the same thing as ad hoc reporting.

I suppose "can't do" is too strong assuming your reporting matches your query. However these things look a lot simpler to do in SQL than in Mongo's approach, and I don't see how you can reliably transform data on output if you don't have a guaranteed reliable schema on input.

Re: Why I Migrated Away From MongoDB

#206

Post by einhverfr »

lmm wrote: post #197
einhverfr wrote: post #196

Earlier quoted context omitted.

>"If that level of indexing is sufficient I don't see why you can't just do the same thing in mongodb." What level of indexing is sufficient depends a great deal on the specifics of the database layout on disk. In InnoDB for example, sequential scans are very costly, and primary key lookups are very cheap. This is because the table is more or less contained in the primary key index and this must be traversed in key o…

That's all pretty interesting, but I still don't see what you get with postgres that you don't get with mongodb. Your database won't enforce your schema for you, but I don't see how that means "ad hoc reporting is impossible".

I have read through the Mongo db query docs and it does look like you can do some ad hoc retrieval queries, and some aggregation. But in the SQL world that's not really the same thing as ad hoc reporting.

I suppose "can't do" is too strong assuming your reporting matches your query. However these things look a lot simpler to do in SQL than in Mongo's approach, and I don't see how you can reliably transform data on output if you don't have a guaranteed reliable schema on input.

Re: Why I Migrated Away From MongoDB

#207

Post by einhverfr »

lmm wrote: post #197
einhverfr wrote: post #196

Earlier quoted context omitted.

>"If that level of indexing is sufficient I don't see why you can't just do the same thing in mongodb." What level of indexing is sufficient depends a great deal on the specifics of the database layout on disk. In InnoDB for example, sequential scans are very costly, and primary key lookups are very cheap. This is because the table is more or less contained in the primary key index and this must be traversed in key o…

That's all pretty interesting, but I still don't see what you get with postgres that you don't get with mongodb. Your database won't enforce your schema for you, but I don't see how that means "ad hoc reporting is impossible".

I have read through the Mongo db query docs and it does look like you can do some ad hoc retrieval queries, and some aggregation. But in the SQL world that's not really the same thing as ad hoc reporting.

I suppose "can't do" is too strong assuming your reporting matches your query. However these things look a lot simpler to do in SQL than in Mongo's approach, and I don't see how you can reliably transform data on output if you don't have a guaranteed reliable schema on input.

Re: Why I Migrated Away From MongoDB

#208

Post by einhverfr »

lmm wrote: post #197
einhverfr wrote: post #196

Earlier quoted context omitted.

>"If that level of indexing is sufficient I don't see why you can't just do the same thing in mongodb." What level of indexing is sufficient depends a great deal on the specifics of the database layout on disk. In InnoDB for example, sequential scans are very costly, and primary key lookups are very cheap. This is because the table is more or less contained in the primary key index and this must be traversed in key o…

That's all pretty interesting, but I still don't see what you get with postgres that you don't get with mongodb. Your database won't enforce your schema for you, but I don't see how that means "ad hoc reporting is impossible".

I have read through the Mongo db query docs and it does look like you can do some ad hoc retrieval queries, and some aggregation. But in the SQL world that's not really the same thing as ad hoc reporting.

I suppose "can't do" is too strong assuming your reporting matches your query. However these things look a lot simpler to do in SQL than in Mongo's approach, and I don't see how you can reliably transform data on output if you don't have a guaranteed reliable schema on input.

Re: Why I Migrated Away From MongoDB

#209

Post by gregjor »

einhverfr wrote: post #190
gregjor wrote: post #75

Earlier quoted context omitted.

I don't know if there's a use case. I have never had one come up in my own projects. I do sometimes need to manage a large number of documents (unstructured data) but I have found the Unix file system sufficient for those, maybe augmented by metadata in the database. I've worked on big applications with big databases where DB operations and aggregations started to get too slow but so far I or someone smarter on the t…

I suspect that a lot of NoSQL folks come to NoSQL from MySQL. As for this part: >"When you have multiple applications accessing the same database, each with their own implementation of consistency rules and their own internal idea of the data schema, come back and explain how that is more fun than using an RDBMS and being able to sleep nights." The obvious answer from the MySQL/NoSQL folks is "What? That's crazy! Mul…

I wasn't able to figure out how to post a comment/reply on Blogger, sorry.

Thanks for pointing me to your article from the HN flame war about MongoDB. If only there were more useful articles like this and less flaming.

You wrote "...in MySQL, the application is king and the db a humble servant." That is exactly my experience, and it leads not only to terrible databases but to piles of awful code written around terrible databases. In 1974 Kernighan and Plauger wrote "Choose a data representation that makes your program simple" (Elements of Programming Style). Other esteemed programmers have said the same thing. A well-designed and normalized database schema with validations and access rules implemented in the database will reduce the amount of application code needed.

I've written about keeping data management code in the database where it belongs (http://typicalprogrammer.com/?p=101), but you make a much clearer argument with real examples. A few years ago the Ruby on Rails people championed the idea of "application databases" and writing HTTP+XML APIs, which probably seemed like a great idea given that few, if any, Rails applications ever had databases that other applications needed to access. Now the NoSQL folks have taken this even further, eliminating structure from the data and all ACID compliance, separating the data from integrity enforcement, all in the name of scalability problems their app might have if they get as big as Google.

I am stuck with MySQL most of the time because I work with broken web applications, and so far all but a couple have used MySQL. As a long-time Oracle and SQL Server programmer I almost cry trying to figure out some of the MySQL databases I come actoss. I am working on one now that uses GUUIDs as primary keys in every table, generated in PHP code, for reasons I can only attribute to ignorance.

Re: Why I Migrated Away From MongoDB

#210

Post by einhverfr »

gregjor wrote: post #209
einhverfr wrote: post #190

Earlier quoted context omitted.

I suspect that a lot of NoSQL folks come to NoSQL from MySQL. As for this part: >"When you have multiple applications accessing the same database, each with their own implementation of consistency rules and their own internal idea of the data schema, come back and explain how that is more fun than using an RDBMS and being able to sleep nights." The obvious answer from the MySQL/NoSQL folks is "What? That's crazy! Mul…

I wasn't able to figure out how to post a comment/reply on Blogger, sorry. Thanks for pointing me to your article from the HN flame war about MongoDB. If only there were more useful articles like this and less flaming. You wrote "...in MySQL, the application is king and the db a humble servant." That is exactly my experience, and it leads not only to terrible databases but to piles of awful code written around terrib…

>"That is exactly my experience, and it leads not only to terrible databases but to piles of awful code written around terrible databases. "

Not planning for the future tends to do that.

>"I am stuck with MySQL most of the time because I work with broken web applications, and so far all but a couple have used MySQL. As a long-time Oracle and SQL Server programmer I almost cry trying to figure out some of the MySQL databases I come actoss. I am working on one now that uses GUUIDs as primary keys in every table, generated in PHP code, for reasons I can only attribute to ignorance."

Ouch, that sounds painful. I have horror stories I could tell too. Ambiguous foreign keys? ouch.....

Post reply on HN