Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases at 14.4Mhz

blog.foundationdb.com

Post reply on HN
Print view View original

21–30 of 86 posts

Re: Databases at 14.4Mhz

#21

Post by misframer »

dchichkov wrote: post #18
jrallison wrote: post #16

Earlier quoted context omitted.

Don't know much about RAMCloud, but from the description: "Many more issues remain, such as whether we can provide higher-level features such as secondary indexes and multiple-object transactions without sacrificing the latency or scalability of the system. We are currently exploring several of these issues." Sounds it doesn't provide multi-key ACID transactions at the very least.

Yes, It looks like it doesn't support multi-key ACID transactions. As per: https://ramcloud.atlassian.net/wiki/display/RAM/Deciding+Whe... > If your application requires the higher-level data model features of a relational database, such as secondary indexes and transactions spanning multiple objects, then it may be difficult to get that application running on RAMCloud. The current RAMCloud data model is a key-value…

> Still. Aside from that, it is free, and open source and gives hundred times better latency?

It doesn't have the same guarantees. If you need those guarantees, then there's no comparison and it doesn't make sense to compare latencies.

Re: Databases at 14.4Mhz

#22

Post by wmf »

hendzen wrote: post #9

This is very impressive, however... See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809 (All credit for the idea in this comment is due to @aphyr) Basically because the transactions modified keys selected from a uniform distribution, the probability of contention was extremely low. AKA this workload is basically a data-parallel problem, somewhat lessening the impressiveness of the high throu…

Unfortunately this problem isn't specific to FoundationDB; the old "industry-standard" TPC-C benchmark has a similar low-contention design which has led to years of unrepresentative performance tuning and benchmarketing.

Re: Databases at 14.4Mhz

#23

Post by landryraccoon »

shortstuffsushi wrote: post #15

As someone who has no idea about the cost of high-scale computing like this, is $150/hr reasonable? It seems like an amount that's hard to sustain to me, but I have no idea if that's a steady, all the time rate, or a burst rate, or what. Or if it's a set up you'd actually ever even need -- seems like from the examples they mention (like the Tweets), they're above the need by a fair amount. Anyone else in this sort of…

Per the article, it's about 1/20th the cost that Google would charge you for the same number of writes.

Re: Databases at 14.4Mhz

#24

Post by dchichkov »

misframer wrote: post #21
dchichkov wrote: post #18

Earlier quoted context omitted.

Yes, It looks like it doesn't support multi-key ACID transactions. As per: https://ramcloud.atlassian.net/wiki/display/RAM/Deciding+Whe... > If your application requires the higher-level data model features of a relational database, such as secondary indexes and transactions spanning multiple objects, then it may be difficult to get that application running on RAMCloud. The current RAMCloud data model is a key-value…

> Still. Aside from that, it is free, and open source and gives hundred times better latency? It doesn't have the same guarantees. If you need those guarantees, then there's no comparison and it doesn't make sense to compare latencies.

I'm not so sure. Have the latencies claimed by FoundationDB been measured on multi-key transactions?

And if you need these guaranties, it seems like "Most of these operations can be implemented on top of the existing RAMCloud features". I would expect it to be possible to implement that and stay well withing the latency budget, provided that a single-key durable write is 100 times faster.

Re: Databases at 14.4Mhz

#25

Post by dmitrig01 »

Dave_Rosenthal wrote: post #19
hendzen wrote: post #9

This is very impressive, however... See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809 (All credit for the idea in this comment is due to @aphyr) Basically because the transactions modified keys selected from a uniform distribution, the probability of contention was extremely low. AKA this workload is basically a data-parallel problem, somewhat lessening the impressiveness of the high throu…

Actually, @aphyr's analysis is wrong. In fact it's actually worse that he says. There is an exactly 0% chance of conflicts between two transactions that each only write 20 random keys (as they do in this test). This is perhaps counterintuitive, but, because there are no reads, any serialization order is possible and therefore there is no chance of conflict. Equally wrong is his assumption that this has any bearing on…

I'm not an expert in this, but:

because there are no reads, any serialization order is possible and therefore there is no chance of conflict

I don't understand how this is the case. If clients A and B try to write to the same key, it can be serialized as {A,B} or {B,A}, but in either case, there is some kind of conflict... no?

Re: Databases at 14.4Mhz

#26

Post by nlavezzo »

dchichkov wrote: post #24
misframer wrote: post #21

Earlier quoted context omitted.

> Still. Aside from that, it is free, and open source and gives hundred times better latency? It doesn't have the same guarantees. If you need those guarantees, then there's no comparison and it doesn't make sense to compare latencies.

I'm not so sure. Have the latencies claimed by FoundationDB been measured on multi-key transactions? And if you need these guaranties, it seems like "Most of these operations can be implemented on top of the existing RAMCloud features". I would expect it to be possible to implement that and stay well withing the latency budget, provided that a single-key durable write is 100 times faster.

All of the transactions in the test noted in the blog are multi-key transactions. (20 key updates)

Re: Databases at 14.4Mhz

#27

Post by jchrisa »

bsaul wrote: post #10

Just watched the linked presentation about "flow" here : https://foundationdb.com/videos/testing-distributed-systems-... Is it really the first Distributed DB project to have built a simulator ? Because frankly, if that's the case, it seems revolutionary to me. Intuitively, it seems like bringing the same kind of quality improvement as unit testing did to regular software development. PS : i should add that this talk…

At Couchbase we have a plethora of simulators. Simulations of cluster topology changes, simulations of failure scenarios, simulations of workloads to estimate requried cluster sizes, etc. Here's one: https://github.com/couchbaselabs/cbfg

Re: Databases at 14.4Mhz

#28

Post by Dave_Rosenthal »

dmitrig01 wrote: post #25
Dave_Rosenthal wrote: post #19

Earlier quoted context omitted.

Actually, @aphyr's analysis is wrong. In fact it's actually worse that he says. There is an exactly 0% chance of conflicts between two transactions that each only write 20 random keys (as they do in this test). This is perhaps counterintuitive, but, because there are no reads, any serialization order is possible and therefore there is no chance of conflict. Equally wrong is his assumption that this has any bearing on…

I'm not an expert in this, but: because there are no reads, any serialization order is possible and therefore there is no chance of conflict I don't understand how this is the case. If clients A and B try to write to the same key, it can be serialized as {A,B} or {B,A}, but in either case, there is some kind of conflict... no?

Nope. Not unless someone actually read that same key in their transaction. Then, with optimistic concurrency at least, you might get a conflict at commit time that basically says "Hey, someone changed that key you read in the meantime so your write might not be correct anymore".

Re: Databases at 14.4Mhz

#29

Post by jrallison »

dchichkov wrote: post #24
misframer wrote: post #21

Earlier quoted context omitted.

> Still. Aside from that, it is free, and open source and gives hundred times better latency? It doesn't have the same guarantees. If you need those guarantees, then there's no comparison and it doesn't make sense to compare latencies.

I'm not so sure. Have the latencies claimed by FoundationDB been measured on multi-key transactions? And if you need these guaranties, it seems like "Most of these operations can be implemented on top of the existing RAMCloud features". I would expect it to be possible to implement that and stay well withing the latency budget, provided that a single-key durable write is 100 times faster.

Multi-key transactions means coordination. Coordination is not cheap. You'll be paying a hefty cost somewhere.

Re: Databases at 14.4Mhz

#30

Post by imanaccount247 »

wmf wrote: post #22
hendzen wrote: post #9

This is very impressive, however... See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809 (All credit for the idea in this comment is due to @aphyr) Basically because the transactions modified keys selected from a uniform distribution, the probability of contention was extremely low. AKA this workload is basically a data-parallel problem, somewhat lessening the impressiveness of the high throu…

Unfortunately this problem isn't specific to FoundationDB; the old "industry-standard" TPC-C benchmark has a similar low-contention design which has led to years of unrepresentative performance tuning and benchmarketing.

How is it unrepresentative? TPC-C involves several queries simulating new orders being entered, being updated, filled, paid, etc. It is very representative of the workload it is simulating. If you want to simulate a different workload, pick one of the other benchmarks instead of C.
Post reply on HN