Live data from Hacker News

Don't use MongoDB

pastebin.com

71–80 of 331 posts

Re: Don't use MongoDB

#71

No shit, nmongo. Anyone with half a brain can go look at the MongoDB codebase and deduce that it's amateur hour. It's start up quality code but it's supposed to keep your data safe . That's pretty much the issue here -- "cultural problems" is just another way of saying the same thing. Compare the code base of something like PostgreSQL to Mongo, and you'll see how a real database should be coded. Even MySQL looks like…

I think the discussion here also misses an important aspect of the conversation which is about application data modeling. Mongo will sooner or later reach a "stable" level as it matures just as mysql, postgres and all other datastores have done. I picked mongo due to the good fit it had to the problems I needed solved not only from the server perspective but from the modeling perspective. The ease of ad-hoc queries a…

These are not new approaches to data modelling.

Document databases, network databases and hierarchical databases (IMS, CODASYL etc) predate relational databases by decades.

Relational is the universal default for a simple reason. When first introduced it proved to be far better, in every conceivable way, than the technologies it replaced.

It's as simple as that. Relational is a slam-dunk, no-brainer for 99.99% of use cases.

Still, if you really want a fast, proven system for one of the older models, you can get IBM to host stuff for you on a z/OS or z/TPF instance, running IMS. It'll have more predictable performance than AWS to boot.

Re: Don't use MongoDB

#72
post #53

Earlier quoted context omitted.

"Schema-less" has the potential (if you use it properly) advantage of allowing gradual migration. As long as your code can handle all versions of objects in current use, you can deploy new code, then either migrate objects as they're updated/rewritten, and/or slowly migrate objects in the background. For certain types of schema changes in large enough data stores, this can be a killer feature. I remember one RDBMS se…

Even more common is when you have a mature application with a lot of users and you need to add new fields to f.ex the user table and you can't because alter table across a sharded db setup will take days or weeks so you end up creating a table that's a hashtable key, value and then proceed to pay the cost of joins against it. Most of my excitement around NoSql comes from hard earned pain not from "oh new shiny thing,…

@mechanical_fish yes and it was a mysql installation. Weird things happen with all systems once you push them up to the edge of performance both of the hardware and interconnections between servers.

Slow interconnect between servers caused me headaches in the past with mysql for replication. Shared switched did the same. Problems with locks under high contention did the same. Problems with the client libraries the same. In fact all storage systems have similar problems and pain. Some are just more battle tested than others.

Re: Don't use MongoDB

#73
post #16

There's a lot of anonymity going on here. A new HN account, an unknown company and product, and claims with no evidence. Why are't links to 10gen's Jira provided? Where's the test code that shows the problems they had with the write lock? This is an extremely shallow analysis.

And yet he makes some good points. Pretty much all of this is verifiable. I don't agree with a lot of his conclusions, but mostly his data is correct.

Look, I'm not the best person to do this..but...good points?

1 - Default writes are unsafe by default:

MongoDB supports a number of "write concerns":

* fire-and-forget or "unsafe"

* safe mode (only written to memory, but the data is checked for "correctness", like unique constraint violations)

* journal commit

* data-file commit

* replicate to N nodes

The last 4 can be mixed and matched. Most (all?) drivers allow this to be specified on a per-write basis. It's an incredible amount of flexibility. I don't know of any other store that lets you do that.

When a user registers, we do a journal commit ({j:true}), 'cuz you don't want to mess that up. When a user submits a score, we do a fire-and-forget, because, if we lose a few scores during the 100ms period between journal commit, it isn't the end of the world (for us, if it is for you, always use j:true)

The complaint is the default-behavior (which I think you can globally configure in most drivers) of the driver? Issue a pull request. Is the default table created in MySQL still MyISAM ?

2 and 6 - Lost Data

This is the most damning point. But what can I say? "No?" My word versus his? I haven't seen those issues in production, I hang out in their google groups and I don't recall seeing anyone bring that up - though I do tend to avoid anything complicated/serious and let the 10gens guys handle that. Maybe they did something wrong? Maybe they were running a development release? Maybe they did hit a really nasty MongoDB bug.

3 - Global Lock

MongoDB works best if your working set fits in memory. That should simply be an operation goal. Beyond that, three points. First, the global lock will yield, i believe (someone more informed can verify this). Second, the story gets better with every version and it's clearly high on 10gen's list.

Most importantly though, it's a constraint of the system. All systems have constraints. You need to test it out for your use-case. For a lot of people, the global lock isn't an issue, and MongoDB's performance tends to be higher than a lot of other systems. Yes it's a fact, but with respect to "don't use MongoDB", its FUD. It's an implementation detail, that you should be aware of, but it's the impact of that implementation details, if any, that we should be talking about.

3 and 4 - Sharding

Sharding is easy, rebalancing shards is hard. Sharding is something else which got better in 1.8 and 2.0, which the author thinks we ought to simply dismiss. I don't have enough experience with MongoDB shard management to comment more. I think the foursquare outage is somewhat relevant though (again, keeping in mind that things have improved a lot since then).

7 - "Things were shipped that should have never been shipped"

This is a good verifiable point? I remember using MySQL cluster when it first shipped. That was a disaster. I also remember using MySQL from a .NET project and opened up a good 3-4 separate bugs about concurrency issues where you could easily deadlock a thread trying to pull a connection from the connection pool.

I once had to use use clearcase. Talk about something that shouldn't have shipped.

This is essentially an attack on 10gen, that ISN'T verifiable. Again, it's his anonymous word versus no ones. Just talking about it is giving it unjust attention.

8 - Replication

It's unclear if this is replica sets or the older master-slave replication. Either way, again, I don't think this is verifiable. In fact, I can say that, relatively speaking, I see very few replica set questions in the groups. It works for me, but I have a very small data set, my data pieces themselves are small. Obviously some people are managing just fine (I'm not going to go through their who's who, I think we all know some of the big MongoDB installations).

9 - The "real" problem

We've all seen some pretty horrible things. I was using MySQL in 5.0 and there was some amazing bugs. There's a bug, which I think still exists, where SQL Server can return you the incorrect inserted id (no, not using @@identify, using scope_identity) when you use a multi-core system. MS spent years trying to fix it.

I guess I can say what 10gen never could...If you were using MongoDB prior to 1.8 on a single server, it's your own fault if you lost data. To me, replication as a means to provide durability never seemed crazy. It just means that you have to understand what's going on.

Look, I don't doubt that this guy really ran into problems. I just think they have a large data set with a heavy workload, they thought MongoDB was a silver bullet, and rather than being accountable for not doing proper testing, they want to try and burn 10gen.

They didn't act responsibly, and now they aren't being accountable.

Re: Don't use MongoDB

#74

Earlier quoted context omitted.

Attacking the messenger is shallow. How about you look at the points - whether valid or not - he or she raises instead and try to refute them? It matters little if that person is well known or someone entirely new. I don't see how the relative anonymity of a person is in any way related to his or her credibility. Besides, calling a position you don't agree with "trolling" with no further argumentation is 4chan level…

it's still a valid point as there are no references to back up any off the claims in the post. he should at least have included links to issues in their jira or some way of replicating the problem he is experiencing. as it stands now it's not fact based and could as much be opinion as there is no way to weight the merit of the claims against anything substantial :(

Now that's more of a valid argument.

I just dislike calling anyone who prefers to stay in relative anonymity (for whatever reason) or is simply new to a community "not credible", at least if it's only because of those attributes. It's a thinly veiled ad hominem.

Re: Don't use MongoDB

#75
post #19

Well, I worked in Vodafone (and Nokia) in very large (laaarge) projects, serving ~50 milions users. Years ago, no hope for NoSQL, we used MySQL. We hit at least 10/20 bugs, solved by 'hotpatch' from Sun. So? I think as developers we should get used to bugs and patches. Should I write a post "don't use MySQL?". We also hit several bugs in the generational garbage collector. Stop using Java? I don't feel the drama here…

> Should I write a post "don't use MySQL?" There have been plenty.

I assure you that, back when MySQL was the same age as Mongo is today, "don't use MySQL" was conventional wisdom... among those who could find and afford Oracle DBAs. ;)

(Though there weren't a lot of blog posts about it, because the word blog had not been invented yet; blogs developed along with... MySQL.)

It will be interesting to watch Mongo as it matures over the next ten years. Unlike MySQL, it is competing against ubiquitously-deployed, well-known, well-worn open-source RDBMS packages, so its history is unlikely to unfold in the same way that MySQL's did.

Re: Don't use MongoDB

#76

Earlier quoted context omitted.

I think the discussion here also misses an important aspect of the conversation which is about application data modeling. Mongo will sooner or later reach a "stable" level as it matures just as mysql, postgres and all other datastores have done. I picked mongo due to the good fit it had to the problems I needed solved not only from the server perspective but from the modeling perspective. The ease of ad-hoc queries a…

These are not new approaches to data modelling. Document databases, network databases and hierarchical databases (IMS, CODASYL etc) predate relational databases by decades. Relational is the universal default for a simple reason. When first introduced it proved to be far better, in every conceivable way, than the technologies it replaced. It's as simple as that. Relational is a slam-dunk, no-brainer for 99.99% of use…

very true but it's a resurgence of modeling alternatives which can only help to enrich our ability to write interesting applications. yes you can model a social network in a RDBMS but it's not as efficient or as flexible as using neo4j. or yes you can model a key value document in a RDBMS but again it's not a good fit. The right tool for the right problem. You don't build a house with only a hammer so why should we build applications only on one storage concept ?

Re: Don't use MongoDB

#77
post #30

I've used MongoDB in production since the 1.4 days. It should be noted that my apps are NOT write heavy. But, many of the author's points can be refuted by using version 2.0. Regarding the point of using getLastError(), the author is completely correct. But the problem is not so much that MongoDB isn't good, it's that developers start using it and expect it to behave like a relational DB. Start thinking in an asynchr…

Schema-less is imho a overrated feature. ORMs like DataMapper (Ruby) and NHibernate (.NET) can generate the schema on the fly for RMDBS, so no need for migrations pre-production. But when your application is in production you need migrations even with a "schema-less" db! See, rename a field and "all your data" is lost, unless you migrate the data from the old field to the new one..

My main problem with schema migrations was that once you reach 100 million records or so, those tend to lock down the DB server and take quite a while

Re: Don't use MongoDB

#78
post #53

Earlier quoted context omitted.

Schema-less is imho a overrated feature. ORMs like DataMapper (Ruby) and NHibernate (.NET) can generate the schema on the fly for RMDBS, so no need for migrations pre-production. But when your application is in production you need migrations even with a "schema-less" db! See, rename a field and "all your data" is lost, unless you migrate the data from the old field to the new one..

"Schema-less" has the potential (if you use it properly) advantage of allowing gradual migration. As long as your code can handle all versions of objects in current use, you can deploy new code, then either migrate objects as they're updated/rewritten, and/or slowly migrate objects in the background. For certain types of schema changes in large enough data stores, this can be a killer feature. I remember one RDBMS se…

Fair enough, but you can also have a schemaless store by using JSON fields in PostgreSQL or MySQL.

Re: Don't use MongoDB

#79
post #16

There's a lot of anonymity going on here. A new HN account, an unknown company and product, and claims with no evidence. Why are't links to 10gen's Jira provided? Where's the test code that shows the problems they had with the write lock? This is an extremely shallow analysis.

And yet he makes some good points. Pretty much all of this is verifiable. I don't agree with a lot of his conclusions, but mostly his data is correct.

The original author should provide the verifiable evidence.

Re: Don't use MongoDB

#80
Yes, if people are sing MongoDB for applications requiring ACID, it is probably not the best fit. However, there are many great use cases that MongoDB is a great fit for, sometimes characterized by needing a lot of read slaves for complex analytics, where data loss is not a lose-the-company proposition, rapid prototyping, etc. I just ported a Java GWT + Objectify appengine application to run on an EC2 with MongoDB and it was shockingly easy to do. Also, you an give up some write performance for increased data safety.

MongoDB (along with PostgreSQL, RDF data sotres, and sometimes Neo4J) is solidly in my preferred tool set.

Post reply on HN