Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB Performance & Durability

mikealrogers.com

Post reply on HN
Print view View original

21–30 of 52 posts

Re: MongoDB Performance & Durability

#21

Post by cbryan »

kristina wrote: post #14

I've tried to post this as a comment on the blog, but it's not showing up (moderated?): ----------- Full disclosure: I work for 10gen. You strategically posted this when my air conditioning was broken, so here are a few thoughts before I go find somewhere cooler. Since CouchDB is "not a competitor" to MongoDB, it's nice of you to put all this time into a public service. > MongoDB, by default , doesn’t actually have a…

"This is because you assume you'll run it on single server. MongoDB's documentation clearly, repeatedly, and earnestly tells people to run MongoDB on multiple servers."

Err, what?

I'm using MongoDB in production and I've looked at your documentation a bunch. After spending days with your docs open in a browser tab I can't say that it was especially clear on this point.

Perhaps I'm particularly ignorant, but I'd wager that not many other developers know that multiple MongoDB servers are currently required to achieve reasonably acceptable durability.

Re: MongoDB Performance & Durability

#22

Post by mikealrogers »

mathias_10gen wrote: post #15
janl wrote: post #13

Earlier quoted context omitted.

So you run naked during the time of the master rebuild? — The only sensible solution is to run two slaves at least, IMHO. — I haven't looked, but do you promote that?

Actually, that will be a recommended config for replica sets. Most of our slides already show 3 replicas per set. Also, most people have backups so its not really "running naked". See http://www.mongodb.org/display/DOCS/Backups for a few ways to backup mongodb. With LVM/EBS/ZFS or any other snapshotable filesystem, backups can be done almost instantly. With EBS you can even get an insta-slave from the snapshot.

wait, seriously?

the suggested default step 1 for MongoDB is to acquire 3 servers?

i mean, no other database suggests such a huge default configuration. even knowing that their datacenter can get hit by lightening and all a lot of large production sites don't even run with this kind of redundancy.

this seems like a pretty taxing workaround for not keeping an append-only transaction log.

Re: MongoDB Performance & Durability

#23

Post by mikealrogers »

angelbob wrote: post #12
forsaken wrote: post #2

Scary.

Eh. Yes and no. The Mongo guys are very, very upfront about this. Check their web site. There are good workarounds, they just require spending a bit more on hardware.

that's kind of like saying "we told you our parachutes won't open sometimes. well, not in our conference talks or in our marketing but it's on our wiki and we blogged about it".

if you want to see a message that is clearly delivered about (lack of) durability look at memcached. nobody misunderstands memcached's durability/consistency guarantees.

-Mikeal (after a few drinks)

Re: MongoDB Performance & Durability

#24

Post by mikealrogers »

kristina wrote: post #14

I've tried to post this as a comment on the blog, but it's not showing up (moderated?): ----------- Full disclosure: I work for 10gen. You strategically posted this when my air conditioning was broken, so here are a few thoughts before I go find somewhere cooler. Since CouchDB is "not a competitor" to MongoDB, it's nice of you to put all this time into a public service. > MongoDB, by default , doesn’t actually have a…

copied from my blog:

@kristina

for some reason wordpress wanted me to moderate your post so sorry for the delay in it showing up.

>> Whoopsy, got your emphasis wrong there ….. Seriously, though, this “unchecked” type >> of write is just supposed to be for stuff like analytics or sensor data, when you’re getting >> a zillion a second and don’t really care some get lost if the server crashes.

did the default change? the last time i attempted to a concurrent performance test this was one of the barriers i hit. my issue isn’t that you include this feature, it’s that it’s the default, i certainly believe there is a use case for it i just think it’s harmful as a default.

>> Since CouchDB is “not a competitor” to MongoDB, it’s nice of you to put all this time >> into a public service.

haha, that’s funny. i regularly use non-CouchDB databases and I get along great with all the people from other databases at conferences. even if i did feel like we were competing, i wouldn’t care. this post really is about reliability issues i don’t think your users are fully aware of and i honestly hope that you fix.

>> fsyncs are configurable. You can fsync once a second, never, or after every single insert, >> remove, and update if you wish.

that’s really good to hear. have you optimized for a “group commit” yet?

>> This is because you assume you’ll run it on single server. MongoDB’s documentation >> clearly, repeatedly, and earnestly tells people to run MongoDB on multiple servers.

I responded earlier to the complexity of actually keeping something available that depends on this. so i won’t cover it again.

>> That’s not to say that things never go wrong, MongoDB is definitely not perfect and has >> lots of room for improvement. I hope that users with questions and problems will >> contact us on the list, our wiki, the bug tracker, or IRC (or, heck, write a snarky blog >> post). Anything to contact the community and let us try to help. I wish every person >> who tried MongoDB had a great experience with it.

You make it sounds like this is all just a matter of bugs, it’s not, and i find blaming it on users who don’t use JIRA or get on IRC a little distasteful.

these issues are architectural and until you do something append-only they aren’t going to go away. someone mentioned earlier that you plan to do an append-only transaction log, if that’s accurate then it’s fantastic news.

Re: MongoDB Performance & Durability

#25

Post by jchrisa »

ww520 wrote: post #17
mjs wrote: post #11

There's a blog post where the MongoDB people explain why they haven't prioritised single server durability: http://blog.mongodb.org/post/381927266/what-about-durability Basically: (a) for real single server durability you need to turn off hardware buffering or have a battery-backed RAID controller to ensure your write really hit disk; (b) this won't help you if your disks fail, and this failure mode is as likely as a…

It's more interesting to see how they recover in case of failure, rather than how write is completely on disk. Machine failure will happen. Writes will not be fully flushed to disk. It's how one recovers from failure that defines how robust the system is.

Gotta say, as a database developer, this is what interests me as well. Let's assume you've had a datacenter-wide power failure, and you're running the repair process on your master and slaves -- what does that look like? How long does it take? What are the factors that influence whether or not it will complete successfully?

Re: MongoDB Performance & Durability

#26

Post by mathias_10gen »

mikealrogers wrote: post #22
mathias_10gen wrote: post #15

Earlier quoted context omitted.

Actually, that will be a recommended config for replica sets. Most of our slides already show 3 replicas per set. Also, most people have backups so its not really "running naked". See http://www.mongodb.org/display/DOCS/Backups for a few ways to backup mongodb. With LVM/EBS/ZFS or any other snapshotable filesystem, backups can be done almost instantly. With EBS you can even get an insta-slave from the snapshot.

wait, seriously? the suggested default step 1 for MongoDB is to acquire 3 servers? i mean, no other database suggests such a huge default configuration. even knowing that their datacenter can get hit by lightening and all a lot of large production sites don't even run with this kind of redundancy. this seems like a pretty taxing workaround for not keeping an append-only transaction log.

No, it is a suggested configuration, not the suggested one. And recommending three nodes is not that uncommon for distributed systems, because you cant have a quorum with only two nodes. Some users are more concerned with handling 10s or 100s of servers than having single-server durability.

That said, for most users two servers are fine. Other users don't need any replicas at all since they do a nightly dump from a stored source into mongodb, or they just take regular backups. There are many ways to achieve system-wide durability, not all of them require the database to be durable.

Re: MongoDB Performance & Durability

#27

Post by patrickaljord »

ericflo wrote: post #20
kristina wrote: post #14

I've tried to post this as a comment on the blog, but it's not showing up (moderated?): ----------- Full disclosure: I work for 10gen. You strategically posted this when my air conditioning was broken, so here are a few thoughts before I go find somewhere cooler. Since CouchDB is "not a competitor" to MongoDB, it's nice of you to put all this time into a public service. > MongoDB, by default , doesn’t actually have a…

> Isn't it irritating how people will write, "MongoDB lost my data" or "CouchDB is really slow" and provide no specifics? He did go into several specifics as to why MongoDB might lose your data. Nice jab at CouchDB though!

> He did go into several specifics as to why MongoDB might lose your data.

He was talking about that post http://www.korokithakis.net/node/119 that doesn't go into any specifics at all.

> Nice jab at CouchDB though!

But couchdb is slow though. I, like many, switched to mongodb because couchdb was just too slow and when I asked in IRC how to make it faster I was told to run a cluster of couchdb, so, not too different than mongodb ;)

Re: MongoDB Performance & Durability

#28

Post by wmf »

gfodor wrote: post #16

Ok, so the standard response seems to be "single server durability" isn't supported, but replication makes up for that. How can this be? If no single server guarantees durability of the writes, how can a cluster of those types of machines suddenly cause those writes to be durable? Maybe it's a pedantic argument, but it seems to me that semantically speaking you are simply relying upon luck that your replicated nodes…

They should say that replication makes up for it under the assumption of uncorrelated failures. As you point out, some people may agree with that assumption and some may not.

Re: MongoDB Performance & Durability

#29

Post by mikealrogers »

patrickaljord wrote: post #27
ericflo wrote: post #20

Earlier quoted context omitted.

> Isn't it irritating how people will write, "MongoDB lost my data" or "CouchDB is really slow" and provide no specifics? He did go into several specifics as to why MongoDB might lose your data. Nice jab at CouchDB though!

> He did go into several specifics as to why MongoDB might lose your data. He was talking about that post http://www.korokithakis.net/node/119 that doesn't go into any specifics at all. > Nice jab at CouchDB though! But couchdb is slow though. I, like many, switched to mongodb because couchdb was just too slow and when I asked in IRC how to make it faster I was told to run a cluster of couchdb, so, not too different…

it's easy to make something fast if you don't want durability, just look at memcached.

Re: MongoDB Performance & Durability

#30

Post by jchrisa »

patrickaljord wrote: post #27
ericflo wrote: post #20

Earlier quoted context omitted.

> Isn't it irritating how people will write, "MongoDB lost my data" or "CouchDB is really slow" and provide no specifics? He did go into several specifics as to why MongoDB might lose your data. Nice jab at CouchDB though!

> He did go into several specifics as to why MongoDB might lose your data. He was talking about that post http://www.korokithakis.net/node/119 that doesn't go into any specifics at all. > Nice jab at CouchDB though! But couchdb is slow though. I, like many, switched to mongodb because couchdb was just too slow and when I asked in IRC how to make it faster I was told to run a cluster of couchdb, so, not too different…

For what it's worth, we've managed to make CouchDB about 10x faster (throughput, not latency) in the last year, and we're just getting started on the optimizations.
Post reply on HN