Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB Performance & Durability

mikealrogers.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: MongoDB Performance & Durability

#41

Post by po »

ekidd wrote: post #35
mjs wrote: post #11

There's a blog post where the MongoDB people explain why they haven't prioritised single server durability: http://blog.mongodb.org/post/381927266/what-about-durability Basically: (a) for real single server durability you need to turn off hardware buffering or have a battery-backed RAID controller to ensure your write really hit disk; (b) this won't help you if your disks fail, and this failure mode is as likely as a…

We run MongoDB on development machines, and they're frequently shut down unexpectedly. But some our development databases take an hour or two to regenerate from scratch, so we prefer to run recovery. In our experience, MongoDB single-node recovery is very robust. It makes no guarantees of transactional integrity _between_ objects, but in our experience the individual objects have always been recovered intact. Accordi…

Just a thought here… there might be a difference between a development box that's mostly idle shutting down unexpectedly and a production server that's under heavy load, maybe starting to fall behind on some writes, freezing and crashing.

Just saying that you might want to test it in a ways that simulates an overwhelming workload, before you trust it.

Re: MongoDB Performance & Durability

#42

Post by cbryan »

kristina wrote: post #37
cbryan wrote: post #21

Earlier quoted context omitted.

"This is because you assume you'll run it on single server. MongoDB's documentation clearly, repeatedly, and earnestly tells people to run MongoDB on multiple servers." Err, what? I'm using MongoDB in production and I've looked at your documentation a bunch. After spending days with your docs open in a browser tab I can't say that it was especially clear on this point. Perhaps I'm particularly ignorant, but I'd wager…

We really, really want people to know they should run on multiple servers. Do you have any suggestions on making it clearer? Where did you look for information about running it in production (so I can add stuff about multiple servers to that page)?

Thanks for being open to suggestions. Maybe on the documentation homepage? http://www.mongodb.org/display/DOCS/Home

Re: MongoDB Performance & Durability

#43

Post by kristina »

po wrote: post #40
wmf wrote: post #28

Earlier quoted context omitted.

They should say that replication makes up for it under the assumption of uncorrelated failures . As you point out, some people may agree with that assumption and some may not.

You're right. I've had a whole datacenter lose power. I am definitely not comfortable with assuming that multiple servers somehow magically makes it safe. I think the word "durable" implies that the data is written onto a disk. I think of what they are talking about as "redundant". Redundant not-necessarily-durable data.

Writing to disk and transaction logs are nice, but they aren't magic bullets. What if a data center catches fire? More mundanely, I've heard ~6% of hard drive fail/year. Only replication can help you there.

I'd argue that durability is a sliding scale. You have to figure out how much risk you're willing to take and you cannot have a perfectly durable system.

Re: MongoDB Performance & Durability

#44

Post by angelbob »

tszming wrote: post #18
angelbob wrote: post #12

Earlier quoted context omitted.

Eh. Yes and no. The Mongo guys are very, very upfront about this. Check their web site. There are good workarounds, they just require spending a bit more on hardware.

Replication does not always work if you deploy MongoDB as VM since master and slave instances may be running in a single physical machine.

Sure. And again, they make that clear. It's quite easy to find out that MongoDB doesn't "have single-server durability". If, like me (and at least one commenter here) you say, "what does that mean?", you can quickly find an explanation on their web site, or in many other places.

Re: MongoDB Performance & Durability

#45

Post by angelbob »

mikealrogers wrote: post #23
angelbob wrote: post #12

Earlier quoted context omitted.

Eh. Yes and no. The Mongo guys are very, very upfront about this. Check their web site. There are good workarounds, they just require spending a bit more on hardware.

that's kind of like saying "we told you our parachutes won't open sometimes. well, not in our conference talks or in our marketing but it's on our wiki and we blogged about it". if you want to see a message that is clearly delivered about (lack of) durability look at memcached. nobody misunderstands memcached's durability/consistency guarantees. -Mikeal (after a few drinks)

I went to the MongoSF conference. I found they explained it extensively in their conference talks.

I don't know about their marketing, but it's easy to find on the web site.

Re: MongoDB Performance & Durability

#46

Post by angelbob »

cbryan wrote: post #21
kristina wrote: post #14

I've tried to post this as a comment on the blog, but it's not showing up (moderated?): ----------- Full disclosure: I work for 10gen. You strategically posted this when my air conditioning was broken, so here are a few thoughts before I go find somewhere cooler. Since CouchDB is "not a competitor" to MongoDB, it's nice of you to put all this time into a public service. > MongoDB, by default , doesn’t actually have a…

"This is because you assume you'll run it on single server. MongoDB's documentation clearly, repeatedly, and earnestly tells people to run MongoDB on multiple servers." Err, what? I'm using MongoDB in production and I've looked at your documentation a bunch. After spending days with your docs open in a browser tab I can't say that it was especially clear on this point. Perhaps I'm particularly ignorant, but I'd wager…

We deployed it to production recently, and we were well aware of this. I feel their documentation is quite clear on the point.

We're also using it single-server, but only for analytics where loss of a small amount of data isn't a big deal. I've made it very clear to all and sundry that if we're going to put any data that has more value into it, we first need at least one additional server instance.

Re: MongoDB Performance & Durability

#47

Post by mikealrogers »

kristina wrote: post #43
po wrote: post #40

Earlier quoted context omitted.

You're right. I've had a whole datacenter lose power. I am definitely not comfortable with assuming that multiple servers somehow magically makes it safe. I think the word "durable" implies that the data is written onto a disk. I think of what they are talking about as "redundant". Redundant not-necessarily-durable data.

Writing to disk and transaction logs are nice, but they aren't magic bullets. What if a data center catches fire? More mundanely, I've heard ~6% of hard drive fail/year. Only replication can help you there. I'd argue that durability is a sliding scale. You have to figure out how much risk you're willing to take and you cannot have a perfectly durable system.

traditionally durability isn't considered a sliding scale, it's a goal/priority which requires you to implement multiple features and fallbacks to handle everything from invalid writes, crash during write, to the data center catching on fire.

thinking about durability this way may work great for MongoDB but it isn't how durability is framed in the rest of the database world.

Re: MongoDB Performance & Durability

#48

Post by mathias_10gen »

mikealrogers wrote: post #47
kristina wrote: post #43

Earlier quoted context omitted.

Writing to disk and transaction logs are nice, but they aren't magic bullets. What if a data center catches fire? More mundanely, I've heard ~6% of hard drive fail/year. Only replication can help you there. I'd argue that durability is a sliding scale. You have to figure out how much risk you're willing to take and you cannot have a perfectly durable system.

traditionally durability isn't considered a sliding scale, it's a goal/priority which requires you to implement multiple features and fallbacks to handle everything from invalid writes, crash during write, to the data center catching on fire. thinking about durability this way may work great for MongoDB but it isn't how durability is framed in the rest of the database world.

A) Just because something is 'traditionally' done doesn't mean its mandatory. Databases 'traditionally' spoke SQL but I don't see you dinging anyone for breaking that tradition. You've used the Appeal to Tradition fallacy (look it up on wikipedia) many times, and it add nothing to your argument.

B) Durability is an important goal at a system-wide level, but that doesn't mean it needs to be handled at the database layer. In addition to the already mentioned replication and transaction log methods, it can also be handled at the block or fs layer using snapshots, or by admins using backup tools. It can even be handled by having a different Database of Record and using Mongo as a operational store. Mongo as software is agnostic; we provide the tools, but it is up to the user or admin to make the best decisions for their technical and business interests. If another layer of the stack provides sufficient protection against data loss, it is unnecessary to pay performance costs associated with doing it in the DB layer.

Re: MongoDB Performance & Durability

#49

Post by jchrisa »

keeto wrote: post #33
jchrisa wrote: post #30

Earlier quoted context omitted.

For what it's worth, we've managed to make CouchDB about 10x faster (throughput, not latency) in the last year, and we're just getting started on the optimizations.

Care to share how? I'm really interested in what optimizations you did for this.

Probably the biggest one was cutting down on the amount of Erlang message passing that it takes to append a batch of data to the end of the db file.

There was also an Erlang configuration change which allowed us to make use of a thread pool for disk-io so that fsyncs didn't block other activity.

Really it was a bunch of tiny things, that all added up. The catalyst was the creation and use of a few highly-concurrent benchmark suites that we could use to identify bottlenecks.

Re: MongoDB Performance & Durability

#50

Post by kristina »

mikealrogers wrote: post #23
angelbob wrote: post #12

Earlier quoted context omitted.

Eh. Yes and no. The Mongo guys are very, very upfront about this. Check their web site. There are good workarounds, they just require spending a bit more on hardware.

that's kind of like saying "we told you our parachutes won't open sometimes. well, not in our conference talks or in our marketing but it's on our wiki and we blogged about it". if you want to see a message that is clearly delivered about (lack of) durability look at memcached. nobody misunderstands memcached's durability/consistency guarantees. -Mikeal (after a few drinks)

Actually, it's more like saying, "we HIGHLY recommend having a backup parachute" (as, I'd imagine, most skydiving instructors would).

Also, I think I've always mention it in my talks.

I'm not sure how VoltDB escaped your wrath, it's an in-memory db that claims to be durable.

Post reply on HN