Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases at 14.4Mhz

blog.foundationdb.com

Post reply on HN
Print view View original

81–86 of 86 posts

Re: Databases at 14.4Mhz

#81

Post by ansible »

jedberg wrote: post #79
ansible wrote: post #73

Earlier quoted context omitted.

FDB has a substantial per-node license cost. It makes sense to beef up the hardware on individual nodes as much as possible first, then scale out to more nodes. https://foundationdb.com/pricing

It would be unfortunate if their pricing model drove poor architectural decisions, but in this case that doesn't apply. I suggested getting more storage by using the same number of nodes differently.

Ah, I mis-read your post, and was thinking you were suggesting more nodes rather than more disks. Sorry for that.

At any rate, I have quite a distaste for RAID-5, so I agree that it would be preferable to go with the higher RF instead.

Even more preferable to me would be to not use RAID-0, if FDB can just use multiple drive partitions as storage directly.

Re: Databases at 14.4Mhz

#82

Post by jedberg »

ansible wrote: post #81
jedberg wrote: post #79

Earlier quoted context omitted.

It would be unfortunate if their pricing model drove poor architectural decisions, but in this case that doesn't apply. I suggested getting more storage by using the same number of nodes differently.

Ah, I mis-read your post, and was thinking you were suggesting more nodes rather than more disks. Sorry for that. At any rate, I have quite a distaste for RAID-5, so I agree that it would be preferable to go with the higher RF instead. Even more preferable to me would be to not use RAID-0, if FDB can just use multiple drive partitions as storage directly.

Yeah, JBOD is preferable to RAID if your workload supports it. Less overhead and less things to break.

Re: Databases at 14.4Mhz

#83

Post by jrallison »

jedberg wrote: post #80
jrallison wrote: post #69

Earlier quoted context omitted.

A few reasons off the top of my head: 1) We're still interested in the nodes being as reliable as they can be. With RAID 5, we need two simultaneous disk failures to brick a node. With RAID 0 (to increase usable disk space), any of the 3 disks can brick the node. Even with 12 nodes and RF of 2, an order of magnitude more node failures would be more likely to disrupt our service. Perhaps this makes more sense in a lar…

I think you misinterpreted what I said. I explained it more clearly below. I suggested using the same 12 nodes but putting each one as a RAID 0, which would get you more reliability and more storage for the same cost. In your current config, two dead disks possibly bricks the system -- in the config I propose, you'd need four dead disks before anyone noticed. What I'm suggesting is that you think of the cluster more…

That was my interpretation of your comment, but I'm still not sure I follow. In my understanding, by using RAID 0, any single disk failure will brick a node. Each node would then have 3 disks that are ticking time bombs (multiplying the failure rate by 3). How is that more reliable?

In RAID 5, I can have 1 disk failure on a node with no problem. 2 disk failures on the same node, and I only lose 1 node of my 12 node cluster (I.E. cluster is fine). I can also theoretically lose 12 (1 on each node) + 2*(RF-1) disks, and gracefully repair the situation with 0 interruption.

What's the benefit of RAID 0 other than increased usable disk space and perhaps write performance? It seems you're decreasing reliability significantly for those gains.

Re: Databases at 14.4Mhz

#84

Post by maliki »

This sounds great compared to my anecdotal experience with DB write performance; but is there a collection of database performance benchmarks that this can be easily compared to?

The best source for DB benchmarking I know of is http://www.tpc.org/. The methodology is more complicated there, but the top results are around 8 million transactions per minute on $5 million systems. This FoundationDB result is more like 900 million transactions per minute on a system that costs $1.5 million a year to rent (so, approx $5 million to buy?).

The USD/transactions-per-minute metric is clear, but without a standard test suite (schema, queries, client count, etc.), comparing claims of database performance makes my head hurt.

Re: Databases at 14.4Mhz

#85

Post by Dave_Rosenthal »

maliki wrote: post #84

This sounds great compared to my anecdotal experience with DB write performance; but is there a collection of database performance benchmarks that this can be easily compared to? The best source for DB benchmarking I know of is http://www.tpc.org/ . The methodology is more complicated there, but the top results are around 8 million transactions per minute on $5 million systems. This FoundationDB result is more like 9…

I think you mean "900 million transactions per minute". Of course that overstates things since each TPC-C transaction entails a lot more than one write. TPC-C is about 2/3 read and 1/3 write, and each TPC transaction might do 20 low-level read+write operations (I'm actually not sure, but I think that's in the ballpark.)

In the NoSQL world many people have converged on a workload of 90% reads/10% writes to individual keys. We show 90/10 results on our performance page [1] but in this test we do 100% writes to stress the "transaction engine", which processes writes.

Since we have our SQL Layer [2] as well, we will run some more-comparable SQL tests in the future.

[1] https://foundationdb.com/key-value-store/performance

[2] https://foundationdb.com/layers/sql

Re: Databases at 14.4Mhz

#86

Post by maliki »

Dave_Rosenthal wrote: post #85
maliki wrote: post #84

This sounds great compared to my anecdotal experience with DB write performance; but is there a collection of database performance benchmarks that this can be easily compared to? The best source for DB benchmarking I know of is http://www.tpc.org/ . The methodology is more complicated there, but the top results are around 8 million transactions per minute on $5 million systems. This FoundationDB result is more like 9…

I think you mean "900 million transactions per minute". Of course that overstates things since each TPC-C transaction entails a lot more than one write. TPC-C is about 2/3 read and 1/3 write, and each TPC transaction might do 20 low-level read+write operations (I'm actually not sure, but I think that's in the ballpark.) In the NoSQL world many people have converged on a workload of 90% reads/10% writes to individual…

(Ah, yes, thank you, 900 million per minute. I typo'd.)

Nice performance page. It pre-answers my follow up question, which was how linear is scaling with more cores? Looks solid.

Post reply on HN