Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I Migrated Away From MongoDB

svs.io

Post reply on HN
Print view View original

61–70 of 213 posts

Re: Why I Migrated Away From MongoDB

#61

Post by trafficlight »

dccoolgai wrote: post #3

"To be honest, the decision to use MongoDb was an ill-thought out one. Lesson learned - thoroughly research any new technology you introduce into your stack, know well the strengths and weaknesses thereof and evaluate honestly whether it fits your needs or not - no matter how much hype there is surrounding said technology." I think you are not alone in learning this lesson with this particular technology. Fortunately…

>> "You have to put the whole dataset on RAM?"

I'm pretty new to the whole database thing, but how is MongoDB different from Postgres or Mysql in this respect? In a traditional database, the data is pulled directly from the hard drive. Why does Mongo suffer a performance hit and Mysql doesn't?

Re: Why I Migrated Away From MongoDB

#62

Post by DonnyV »

If he just did 10min of research he would of realized MongoDB isn't for him. Also by doing that research he would've realized that MongoDB has no data constraints. Thats all done in your model in your application.

Re: Why I Migrated Away From MongoDB

#63

Post by egiva »

rbranson wrote: post #14

I'm no fan of MongoDB, but this same advice goes for any NoSQL data store. I am an Apache Cassandra contributor and community MVP, but my advice stays the same: it's best just to start with a SQL database and go from there. Read some books and learn it well: the "SQL Cookbook" from O'Reilly is great, and so is "The Art of SQL." Premature optimization continues to be the root of all evil.

Thanks rbranson, this comment was very helpful. I also think that the logical fallacy of some of these arguments is that they´re absolutist - it´s better to follow your advice because it´s a very logical progression to start with a SQL database and then move into different things as you grow. Life is a progression and I think your quote about premature optimization is spot on - it IS the root of all evil because it´s absolutist and too rigid to fit the curve of everything that life (and tech, and work) throws at you!

Re: Why I Migrated Away From MongoDB

#64

Post by gregjor »

gaius wrote: post #48
gregjor wrote: post #42

You were fortunate to recognize that MongoDB was the wrong tool for your job, and lucky to be able to move to Postgres instead of continuing to throw your time and effort away. I see the ad hominem "you're an ignorant idiot" attacks already started, along with advice like using regexes to do case-insensitive searches. Watching the NoSQL "movement" encounter the problems RDBMSs fixed 20 years ago and then hand-wave an…

The thing with the NoSQL guys is that many of them seem not to be in a position to make an educated comparison. For example, an, uhh, enthusiastic MongoDB advocate recently informed me that MongoDB was superior to Oracle because in Oracle you had to poll a table to see if it changed. Except, no, that isn't actually true: http://docs.oracle.com/cd/B19306_01/appdev.102/b14251/adfns_... - and that document is from 2005.…

Programming is more like fashion than science in this regard. Every decade or so something truly new happens in the software world. All the rest is mostly sound and fury, signifying nothing. If you're young or new to programming it's easy to mistake the buzz around things like NoSQL for innovation when they are usually re-discoveries of old (and often discarded or obsolete) ideas dressed up in new clothes.

There's also the tendency to favor new shiny things and reject old crufty (but proven) things, to want to be part of what seems like the leading edge, to be that guy in the cube farm who is playing with the cool new stuff.

I have been programming longer than RDBMSs have been available, so I know from experience what it's like to manage large databases in application code, and how hard it can be to maintain consistency or do accurate queries and aggregation with half-baked tools. It's frustrating to see a new generation of programmers go through this, but it's human nature to ignore the past.

My fourteen year old son wears his pants pulled down below his waist, Vans shoes, hoodies, lots of hair. He looks pretty much like I did when I was fourteen back in the 1970s. The underlying technologies are the same: pants, shirt, shoes, sweater, hair. The only differences are superficial. To him that style is edgy and contemporary and something his parents don't get. NoSQL is the gangster fashion of programming right now.

Re: Why I Migrated Away From MongoDB

#65

Post by otterley »

trafficlight wrote: post #61
dccoolgai wrote: post #3

"To be honest, the decision to use MongoDb was an ill-thought out one. Lesson learned - thoroughly research any new technology you introduce into your stack, know well the strengths and weaknesses thereof and evaluate honestly whether it fits your needs or not - no matter how much hype there is surrounding said technology." I think you are not alone in learning this lesson with this particular technology. Fortunately…

>> "You have to put the whole dataset on RAM?" I'm pretty new to the whole database thing, but how is MongoDB different from Postgres or Mysql in this respect? In a traditional database, the data is pulled directly from the hard drive. Why does Mongo suffer a performance hit and Mysql doesn't?

Because MongoDB mmap(2)s its backing stores into its process memory space. It's a naive approach to persistence - it's very fast and simple, but if you overcommit (i.e. you store more in the database than you have memory available), page-thrashing results.

MySQL's InnoDB table engine, on the other hand, uses direct I/O (in the recommended scenario) and manages the buffer pool independently of the kernel. Its buffer pool manager is specifically designed for the typical workloads MySQL is used for (http://dev.mysql.com/doc/refman/5.5/en/innodb-buffer-pool.ht...) - as opposed to the naive LRU that most OSes employ for their filessytem buffers.

Re: Why I Migrated Away From MongoDB

#66

Post by gregjor »

gaius wrote: post #56
gregjor wrote: post #44

Earlier quoted context omitted.

Wrong. SQL databases (except for SQLite) are almost completely interchangeable because they are all based on the same relational model and they all implement the ANSI SQL standard with only minor deviations. If you have a lot of stored procedures and triggers -- executable code embedded in the database -- you will have to rewrite that. Oracle is in a world of its own in a lot of ways, but if you are moving to or from…

Not wrong. As a DBA with 15+ year experience, I can tell at a glance what an application was first developed against (Oracle, SQL Server, etc) or even where a dev started their career. You might as well say any OS is interchangeable as they're all just a kernel running processes.

I didn't claim RDBMSs are interchangeable or that there are no differences. I disputed the statement that "databases are only trivially interchangeable for trivial cases." That has not been my experience: changing out one RDBMS for another is not trivial, but it's easier and more controllable and predictable than changing out application code.

I also disputed the notion that reading a book first would enable someone to choose the right tool the first time. There's no substitute for experience and real expertise, and you only get that by trying things and making mistakes, or at least learning from other people's mistakes.

Re: Why I Migrated Away From MongoDB

#67

Post by zemo »

gaius wrote: post #48
gregjor wrote: post #42

You were fortunate to recognize that MongoDB was the wrong tool for your job, and lucky to be able to move to Postgres instead of continuing to throw your time and effort away. I see the ad hominem "you're an ignorant idiot" attacks already started, along with advice like using regexes to do case-insensitive searches. Watching the NoSQL "movement" encounter the problems RDBMSs fixed 20 years ago and then hand-wave an…

The thing with the NoSQL guys is that many of them seem not to be in a position to make an educated comparison. For example, an, uhh, enthusiastic MongoDB advocate recently informed me that MongoDB was superior to Oracle because in Oracle you had to poll a table to see if it changed. Except, no, that isn't actually true: http://docs.oracle.com/cd/B19306_01/appdev.102/b14251/adfns_... - and that document is from 2005.…

MongoDB is disproportionately liked by the inexperienced. There are things to like about it, and being able to have sparse secondary b-tree indexes on arbitrary data is, overall, pretty rad. For prototyping, being able to just toss some shit in there, especially when your data comes from an external service and you're not in control of your incoming data (very common these days), it works great for simple use cases and CRUD apps (which is a lion's share of new projects in the industry of the Internet). Being able to predict all the incoming data ahead of time isn't always doable, because a lot of services have underdocumented APIs, and every now and then you'll get data that's just ever-so-slightly different than what you expected. With Mongo, you can at least guarantee you're holding onto that, so that you can use this existing data in the future. That's way less contrived than it sounds. E.g., a third-party changes one of the data formats, adds a field or changes a datatype on one, for instance. If you have a strict schema, you might fail to write some of that data until you update everything. With Mongo, you can at least capture that data and get it working later.

I'm using it on one of my projects, and as I said, there are things to like about it, but graph traversals remain problematic. Overall, I'd say I'm pretty unhappy with MongoDB, and I wish I hadn't chosen it; it doesn't work well for my project. A lot of things I'm doing boil down to performing some kind of graph traversal, which is painful to do at runtime in Mongo (the potential solution space is too large to be precomputed). From what I've seen, MongoDB hasn't been working for me very well with a highly-connected data set.

Re: Why I Migrated Away From MongoDB

#68

Post by yxhuvud »

stickfigure wrote: post #43
rbranson wrote: post #14

I'm no fan of MongoDB, but this same advice goes for any NoSQL data store. I am an Apache Cassandra contributor and community MVP, but my advice stays the same: it's best just to start with a SQL database and go from there. Read some books and learn it well: the "SQL Cookbook" from O'Reilly is great, and so is "The Art of SQL." Premature optimization continues to be the root of all evil.

it's best just to start with a SQL database and go from there. This is bad advice. It's best to understand your problem domain and use the tools that are most appropriate. You see a lot of two types of posts on HN: * "I picked a NoSQL database for a problem domain with a better relational fit." Those posts look like this one. * "I picked an RDBMS for a problem domain with a better NoSQL fit." Those posts are usually…

The problem with your argument is that all problem will look like it will have polymorphic and ambigous data until you have mapped it out properly. What the post you replied to was basically an advice for new projects and developers to fully map out the problem and see where things are going.

It is a lot easier to start strict and organized and back down to less strictness with looser rules where you need it, than to do the opposite.

Re: Why I Migrated Away From MongoDB

#69

Post by gregjor »

bunderbunder wrote: post #55
gregjor wrote: post #44

Earlier quoted context omitted.

Wrong. SQL databases (except for SQLite) are almost completely interchangeable because they are all based on the same relational model and they all implement the ANSI SQL standard with only minor deviations. If you have a lot of stored procedures and triggers -- executable code embedded in the database -- you will have to rewrite that. Oracle is in a world of its own in a lot of ways, but if you are moving to or from…

SQL databases (except for SQLite) are almost completely interchangeable because they are all based on the same relational model and they all implement the ANSI SQL standard with only minor deviations. They only all implement the ANSI SQL standard for certain values of "ANSI SQL standard." Those values being SQL-89 or perhaps SQL-99. Beyond that it's a mess. SQL Server didn't get around to doing a really good job with…

I agree that there are numerous inconsistencies. I just have a different definition of "trivial cases" than you do. The kinds of things you mention -- real issues that come up in real applications -- fall outside of what most databases are doing in the kinds of web apps that I work on. I haven't had to deal with Oracle vs. SQL Server bulk load problems or Oracle's unique implementation of NULLs in years. Not waving them away, just saying that these kinds of issues are not all that common because most real-world databases are trivial cases.

Re: Why I Migrated Away From MongoDB

#70

Post by blaines »

gregjor wrote: post #42

You were fortunate to recognize that MongoDB was the wrong tool for your job, and lucky to be able to move to Postgres instead of continuing to throw your time and effort away. I see the ad hominem "you're an ignorant idiot" attacks already started, along with advice like using regexes to do case-insensitive searches. Watching the NoSQL "movement" encounter the problems RDBMSs fixed 20 years ago and then hand-wave an…

So do you believe there is a use case for a document-oriented database? I feel like your comment writes off a huge swath of useful technology.

I said this below, but I'll say it again. Data is malleable, and writing apps to fit around any datastore seems wrong. I write applications to fulfill their use case. When the needs of the application change, so could it's database (or other dependencies).

http://gigaom.com/cloud/mongodb-or-mysql-why-not-both/

Post reply on HN