Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't use MongoDB

pastebin.com

Post reply on HN
Print view View original

111–120 of 331 posts

Re: Don't use MongoDB

#111

Post by LeafStorm »

Just for comparison, CouchDB has had one major bug that could cause the loss of data, detailed here: http://couchdb.apache.org/notice/1.0.1.html

The bug was only triggered when the delayed_commits option was on (holds off on fsyncing when lots of write operations are coming in) and there was both a write conflict and a period of inactivity - when the database was shut down, any writes that happened afterwards would not be saved.

They immediately worked to develop a process that would prevent any data from being lost if you didn't shut down the server, then a week later had released an emergency bugfix version without the bug. Then later they released a tool that could recover any data lost from the bug if the database hadn't been compacted.

That's the kind of attitude database developers need to have towards data integrity.

Re: Don't use MongoDB

#112

Post by desas »

SemanticFog wrote: post #84
jacques_chester wrote: post #71

Earlier quoted context omitted.

These are not new approaches to data modelling. Document databases, network databases and hierarchical databases (IMS, CODASYL etc) predate relational databases by decades. Relational is the universal default for a simple reason. When first introduced it proved to be far better, in every conceivable way, than the technologies it replaced. It's as simple as that. Relational is a slam-dunk, no-brainer for 99.99% of use…

You're absolutely right -- RDBMSes were designed to solve problems with the nosql-type approaches that preceded them. The nosql bandwagon is blindly rolling into the past, where it will crash into the old problems of concurrency and consistency under load. BTW if you want nosql-style schema flexibility within an RDBMS, then a simple solution is to store XML or JSON in in a character blob. Keep the fields you need to…

Another solution is to use the hstore feature in postgres to store key value data.

Re: Don't use MongoDB

#113

Post by angelbob »

latch wrote: post #73
angelbob wrote: post #44

Earlier quoted context omitted.

And yet he makes some good points. Pretty much all of this is verifiable. I don't agree with a lot of his conclusions, but mostly his data is correct.

Look, I'm not the best person to do this..but...good points? 1 - Default writes are unsafe by default: MongoDB supports a number of "write concerns": * fire-and-forget or "unsafe" * safe mode (only written to memory, but the data is checked for "correctness", like unique constraint violations) * journal commit * data-file commit * replicate to N nodes The last 4 can be mixed and matched. Most (all?) drivers allow thi…

If you were using MongoDB prior to 1.8 on a single server, it's your own fault if you lost data. To me, replication as a means to provide durability never seemed crazy. It just means that you have to understand what's going on.

Well, except for that thing where the replication decided that the empty set was the most recent and blew everything else away. And those cases where keys went away.

Losing data, particularly when the server goes down, is fine. Even not writing data isn't terrible, though his points about not knowing whether it has been written in case of failure are really good ones. But corrupting data and then replicating that corrupted data is really, really bad. Often unfixably bad.

They didn't act responsibly, and now they aren't being accountable.

For the complaints about the default write stuff, sure. For everything else... Dunno. He brought up a lot of real, actual issues which were not documented MongoDB behavior. Yes, there's also a fair bit of complaining about the documented bits, and sure, boo-hoo, whatever. But the idea that 10gen is shipping stuff with serious data integrity bugs, and doing so knowing, doesn't seem out of line here.

And while MySQL also has some bad stuff, sure, it has nothing like as many data integrity bugs as MongoDB.

And I say all of this as a serious fan of MongoDB.

Re: Don't use MongoDB

#114

Post by angelbob »

goldmab wrote: post #78
vidarh wrote: post #53

Earlier quoted context omitted.

"Schema-less" has the potential (if you use it properly) advantage of allowing gradual migration. As long as your code can handle all versions of objects in current use, you can deploy new code, then either migrate objects as they're updated/rewritten, and/or slowly migrate objects in the background. For certain types of schema changes in large enough data stores, this can be a killer feature. I remember one RDBMS se…

Fair enough, but you can also have a schemaless store by using JSON fields in PostgreSQL or MySQL.

Not indexably. But you can do a hideous many-tables-per-real-table thing where each field gets a tall thin table in PostGRES or MySQL, do a lot of joins to get your data, and index the fields in that.

It's not as awful as it sounds, performance-wise. It is as awful as it sounds in terms of maintainability, of course.

Re: Don't use MongoDB

#115

Post by krig »

itaborai83 wrote: post #59
foobarbazetc wrote: post #47

No shit, nmongo. Anyone with half a brain can go look at the MongoDB codebase and deduce that it's amateur hour. It's start up quality code but it's supposed to keep your data safe . That's pretty much the issue here -- "cultural problems" is just another way of saying the same thing. Compare the code base of something like PostgreSQL to Mongo, and you'll see how a real database should be coded. Even MySQL looks like…

I'm curious and I might be missing more than half of my brain. Would you be willing to show some examples of bad coding on their source tree?

I looked at using BSON in a project a while back, and ended up scrapping it mainly due to perceived poor code quality. Plenty of potential errors ignored, unclear error messages, unsafe practices.

I was also turned off by the sloppy use of memory. Heap allocated objects returned from functions with poor checks to see if anyone manages that memory on the other side. Lots of instances of strcmp, strcpy and similar unsafe string/buffer manipulation functions.

It's been a while since I looked at it so I don't have any particular examples at hand, but that was my impression.

Re: Don't use MongoDB

#116

Post by MikeCampo »

Very interesting. I recently worked on a little side project using MongoDB and I noticed during testing that some records would disappear at random. Glad to see this has happened to others. I suppose it's time to check out Redis.

Re: Don't use MongoDB

#117

Post by Vivtek »

jacques_chester wrote: post #71
christkv wrote: post #55

Earlier quoted context omitted.

I think the discussion here also misses an important aspect of the conversation which is about application data modeling. Mongo will sooner or later reach a "stable" level as it matures just as mysql, postgres and all other datastores have done. I picked mongo due to the good fit it had to the problems I needed solved not only from the server perspective but from the modeling perspective. The ease of ad-hoc queries a…

These are not new approaches to data modelling. Document databases, network databases and hierarchical databases (IMS, CODASYL etc) predate relational databases by decades. Relational is the universal default for a simple reason. When first introduced it proved to be far better, in every conceivable way, than the technologies it replaced. It's as simple as that. Relational is a slam-dunk, no-brainer for 99.99% of use…

I agree entirely - I think when people rebel against "relational databases" they're actually just realizing that the normalization fetish can be harmful in many application cases.

You're better off with MySQL or PostgreSQL managing a key-value table where the value is a blob of JSON (or XML, which I've done in the past), then defining a custom index, which is pretty damn easy in PostgreSQL. Then you have hundreds of genius-years of effort keeping everything stable, and you still get NoSQL's benefits. Everybody wins.

Re: Don't use MongoDB

#118

Post by vannevar »

nikcub wrote: post #57

Links about Foursquare's problems with MongoDB. The site was down for a while when their 1.6 instance crashed: * http://blog.foursquare.com/2010/10/05/so-that-was-a-bummer/ * http://www.infoq.com/news/2010/10/4square_mongodb_outage * http://groups.google.com/group/mongodb-user/browse_thread/th... I like MongoDB, it is easy to setup, work with and to understand. I think it has an opportunity to become the mysql of nos…

It should be noted that this was not really a problem with MongoDB. Foursquare used a poorly-chosen shard key that caused a disproportionate load on one of its shards, and on top of that did not have proper system monitoring in place to alert them that a server was running out of RAM. It should also be noted that no data was lost in the process of resolving the problem.

Re: Don't use MongoDB

#119

Post by WayneDB »

moe wrote: post #106
antirez wrote: post #26

I appreciate the "public service" intend of this blog post, however: 1) It is wrong to evaluate a system for bugs now fixed (but you can evaluate a software development process this way, however it is not the same as MongoDB itself, since the latter got fixed). 2) A few of the problems claimed are hard to verify, like subsystems crashing, but users can verify or deny this just looking at the mailing list if MongoDB h…

1) It is wrong to evaluate a system for bugs now fixed I disagree. A project's errata is a very good indicator for the overall quality of the code and the team. If a database-systems history is littered with deadlock, data-corruption and data-loss bugs up to the present day then that's telling a story. 2) A few of the problems claimed are hard to verify The particular bugs mentioned in an anonymous pastie may be hard…

I agree with your responses to 1 and 2. I take issue with the example for 3 though because Redis is nowhere near the complexity or feature set of MongoDB.

Re: Don't use MongoDB

#120

Post by latch »

regularfry wrote: post #108
latch wrote: post #73

Earlier quoted context omitted.

Look, I'm not the best person to do this..but...good points? 1 - Default writes are unsafe by default: MongoDB supports a number of "write concerns": * fire-and-forget or "unsafe" * safe mode (only written to memory, but the data is checked for "correctness", like unique constraint violations) * journal commit * data-file commit * replicate to N nodes The last 4 can be mixed and matched. Most (all?) drivers allow thi…

I honestly have no dog in this race, but an argument which boils down to "MySQL is just as bad" is not one I'd choose to pursue.

I spent the time to write all that, and all you got from it is "MySQL is just as bad"...I obviously did a bad job.

edit:

I brought up MySQL because I think we all know that companies, you, me knowingly ship products with bug. In fact, you can look at public bug tracking for a bunch of major software and see bug fixes scheduled for future releases.

However, if you are going to accuse a database vendor of knowingly shipping data-corruption bugs, I think you absolutely have to back that up. It's slanderous. Obviously, if you think that, you also shouldn't use their product. But you either know something the rest of us don't, or you're a complete ass, if you make those kinds of statements without evidence.

Post reply on HN