Live data from Hacker News

Why I Migrated Away From MongoDB

svs.io

171–180 of 213 posts

Re: Why I Migrated Away From MongoDB

#171

As a relative idiot when it comes to this sort of thing, I'd like to insert the following supplementary question: what is the sort of application/dataset for which Mongo is particularly suited? I've used it on small projects, and have enjoyed it. Perhaps my data has just been simple/loosely-coupled enough to never run into these problems? I read a lot of posts like this on HN before every trying Mongo, so I've at lea…

"what is the sort of application/dataset for which Mongo is particularly suited" The majority of the NoSQL databases are based on Amazon's Dynamo: loosely coupled replication. MongoDB is one of the few (next to Hbase and a few others) that adopts Google BigTable's architecture: data is divided in Ranges, and each mongod node serves multiple Ranges. This means MongoDB is able to provide atomicity where it's harder wit…

You said that you don't like the availability issues then state "but at the moment MongoDB is about as good as it gets."?

Just. Wow.

Get your head out of the sand mate. MongoDB is nowhere NEAR as good as it gets.

Re: Why I Migrated Away From MongoDB

#172

Earlier quoted context omitted.

>> "You have to put the whole dataset on RAM?" I'm pretty new to the whole database thing, but how is MongoDB different from Postgres or Mysql in this respect? In a traditional database, the data is pulled directly from the hard drive. Why does Mongo suffer a performance hit and Mysql doesn't?

Because MongoDB mmap(2)s its backing stores into its process memory space. It's a naive approach to persistence - it's very fast and simple, but if you overcommit (i.e. you store more in the database than you have memory available), page-thrashing results. MySQL's InnoDB table engine, on the other hand, uses direct I/O (in the recommended scenario) and manages the buffer pool independently of the kernel. Its buffer p…

For what it's worth VoltDB uses main memory format as its "naive" approach to persistance too.

I would consider VoltDB naive in the approach to persistance because it gives you dramatic gains (2 orders of magnitude) in read/write performance for similar workloads only by redefining concurrency and durability out of the equation. Durability is no longer a property of each machine but is instead a property of the network. Concurrency is handled by executing queries fast and doing them in series. So it's like ACID without the C or D.

Re: Why I Migrated Away From MongoDB

#173

Competly aside from the Article. The level of vitriolic discourse in this topic is astounding. I am amazed that as a community discussions of Database engines can draw out such mean spirited anger. I have never down voted as many comments on HN in a single thread then I have on this topic. I don't care which side of the debate you come down on. There is no excuse for belittleing and insulting others in a technical fo…

In this case it appears to mostly be those arguing for Postgre

Curious. I have never found a database named "Postgre" to be used by anybody. Perhaps you can direct me to the download site.

I think the bigger issue is that there isn't a lot of discussion from the NoSQL crowd about what you give up when you go to a NoSQL solution. I think that sort of disclosure would help people weigh the options a lot better.

From the comments of some people here you'd almost think they would build an ERP app in Mongo....

Re: Why I Migrated Away From MongoDB

#174
post #113

Earlier quoted context omitted.

I think we did "sharding" with relational databases... Back in the 80s. Then we got fast hash joins and partitioning and it turns out that the disadvantages of sharding just aren't worth it. The NoSQL crew will figure this out too around 2030 :-p

Be a little careful with this level of disdain. Sharding may have been available in the 1980's, but what it lead to in some products is quite amazing. Consider Teradata's clustering ability which is sort of like sharding your database but without the disadvantages typically associated with it. Postgres-XC now offers something similar as a semi-fork of PostgreSQL. Basically what we are talking about here is a two-tier…

Oh absolutely, but what you're talking about there, people do with CICS today, and that's even older than the 80s. CICS is a technology I have a lot of respect for.

But my point is - when I need to use something like that, I know that's what I'm using. I don't imagine that it's some new invention. Hell, a lot of what the NoSQL guys think they're inventing, IBM did back then too - IMS.

Re: Why I Migrated Away From MongoDB

#175
post #117

Earlier quoted context omitted.

Why? The guys at FB or Google are just guys like you and me, don't believe the hype, they are not superheroes or gurus despite their much talked about interview process (the Google interviews I did were a walk in the park compared to GS, btw). Some will know more than me sure, but some'll know less.

Just guys like you or me except they build systems with billions of page views a month.

At my last job, we averaged 60k pages/sec, on what the NoSQL guys would call a "legacy" database, and there was plenty of headroom. It's not rocket science, just engineering. Companies like Google love to weave a mystique around what they do, it's in their interests after all, to convince their investors that what they do is magic. But I'm from back in the day when we were the same way about millions of pages per month, then per week, then per day...

Re: Why I Migrated Away From MongoDB

#176

Competly aside from the Article. The level of vitriolic discourse in this topic is astounding. I am amazed that as a community discussions of Database engines can draw out such mean spirited anger. I have never down voted as many comments on HN in a single thread then I have on this topic. I don't care which side of the debate you come down on. There is no excuse for belittleing and insulting others in a technical fo…

You can find this level of discourse in plenty of topics every day. Programmers draw blood over indenting with tabs or spaces. It's geek entertainment.

I have never been upvoted so much on HN. I admit to strong opinions and light sarcasm but you'll have to show me where I've been uncivil, belittling or insulting, except perhaps in response to people who insulted me.

Re: Why I Migrated Away From MongoDB

#177
I work for a company that operates in the energy industry. We utilize both RDBMS and "NoSQL", both have their purposes that they fit in well. We store customer account and configuration data in Postgres, and use Cassandra to store time-series statistics and high write volume data.

I have a background in data warehousing in both Oracle and SQL Server, and was part of the decision to use a polyglot persistence model. I've got at least a decade's worth of experience in the DW world, and more as a general developer before that, so I like to think I've got a relatively credible background in a variety of data stores.

I haven't looked at Mongo much - it's durability concerns and the write lock stuff pushed me away from it early on (I don't mean to disparage it, but that was where it was at when I evaluated it), but Cassandra's configurable consistency levels and operational story at a cluster level are what sold us for our time-series data (that, and the ability to construct a sparse timeline and multiplex reads/writes). For anything we need flexible querying with, we push it into specialized Postgres dbs.

The level of willful ignorance and vitrol in this thread is kind of amazing. Most of the really experienced DW guys I know are all looking at HBase, Cassandra and others because they fit a niche that we've all been looking for in certain data sets at really large scale. It doesn't mean we're ditching our relational data stores, it just means we're augmenting them with other tools because they fit the job at hand. To suggest that one tool is absolutely perfect for every scenarios seems a little short-sighted to me, possibly driven out of inexperience. I don't mean that as an insult - I know a lot of guys who've been working on the same data sets for 30 years who really do just need the one tool - however, you've got to realize there are other data sets and problems for which your hammer just won't fit.

Re: Why I Migrated Away From MongoDB

#178
post #174

Earlier quoted context omitted.

Be a little careful with this level of disdain. Sharding may have been available in the 1980's, but what it lead to in some products is quite amazing. Consider Teradata's clustering ability which is sort of like sharding your database but without the disadvantages typically associated with it. Postgres-XC now offers something similar as a semi-fork of PostgreSQL. Basically what we are talking about here is a two-tier…

Oh absolutely, but what you're talking about there, people do with CICS today, and that's even older than the 80s. CICS is a technology I have a lot of respect for. But my point is - when I need to use something like that, I know that's what I'm using. I don't imagine that it's some new invention. Hell, a lot of what the NoSQL guys think they're inventing, IBM did back then too - IMS.

"Those who don't understand Unix are condemned to reinvent it, poorly." – Henry Spencer

Re: Why I Migrated Away From MongoDB

#179

Mmm, not sure about some of the complains... - You can make case insensitive searches on the DB using regexes ( http://www.mongodb.org/display/DOCS/Advanced+Queries#Advance... ). A simple case-insensitive regex is not very bad performance-wise, but in general, case-insensitive searches should be avoided for search purposes (you can normalize to set everything to lower case or other equivalent trick) - The proper way…

- The proper way of doing an audit (and search later) is to make an independent collection with a reference to the other(s) document in a different collection. Then you can index by user, date, or any other field and leave the main collection alone. The described embedded access collection doesn't look very scalable. I think this point is very important even in the RDBMS side. There are cases, even with relational da…

> "I think this point is very important even in the RDBMS side."

This is true. After all you can't extract an audit trail from a deleted record. Simply in terms of information management it often makes sense to represent this as separate info.

> "There are cases, even with relational datastores that would preform better if the dataset was built to the query."

The big problem with doing it that way is that you are screwed as soon as requirements change. I recently blogged about not-1NF designs in PostgreSQL (nested data structures for subset constraints), and the lesson I took away was that you really don't want to have your select queries hitting the same tables you maintain for inserts. You want your queries hitting a normalized data structure even if the data comes in as something different.

Re: Why I Migrated Away From MongoDB

#180

Earlier quoted context omitted.

The last line sounds pretty familiar after replacing "NoSQL" with "PHP" and "RDBMS" with "decent languages".

I completely agree. This whole NoSQL/RDBMS situation is programming language debate being played out over again. And just like not every programming language works for every situation. Neither does every database.

I agree. BTW, the most lucid description of when to use relational vs non-relational models is not all that new. You can read it here:

http://db.cs.berkeley.edu/papers/Informix/www.informix.com/i...

In this model, there is virtually no difference between the complex data/no query db's of the paper then and NoSQL today in terms of use case. Interesting that a one-time executive of Informix was able to present such a long-lasting discussion of db technologies.....

Post reply on HN