Databases at 14.4Mhz

#1

Post by chton » Wed, Dec 10, 2014, 2:53 PM UTC

Databases at 14.4Mhz
blog.foundationdb.com

Re: Databases at 14.4Mhz

#2

Post by [deleted] » Wed, Dec 10, 2014, 3:01 PM UTC

[deleted]

Re: Databases at 14.4Mhz

#3

Post by lttlrck » Wed, Dec 10, 2014, 5:49 PM UTC

"Or, as I like to say, 14.4Mhz."

Sorry, I don't like that at all.

Re: Databases at 14.4Mhz

#4

Post by jrallison » Wed, Dec 10, 2014, 6:06 PM UTC

We've been using FoundationDB in production for about 10 months now. It's really been a game changer for us.

We continue to use it for more and more data access patterns which require strong consistency guarantees.

We currently store ~2 terabytes of data in a 12 node FDB cluster. It's rock solid and comes out of the box with great tooling.

Excited about this release! My only regret is I didn't find it sooner :)

Re: Databases at 14.4Mhz

#5

Post by mariusz79 » Wed, Dec 10, 2014, 6:33 PM UTC

MHz not Mhz.

Re: Databases at 14.4Mhz

#6

Post by felixgallo » Wed, Dec 10, 2014, 6:44 PM UTC

This looks very interesting and congratulations to the FoundationDB crew on some pretty amazing performance numbers.

One of the links leads to an interesting C++ actor preprocessor called 'Flow'. In that table, it lists the performance result of sending a message around a ring for a certain number of processes and a certain number of messages, in which Flow appears to be fastest with 0.075 sec in the case of N=1000 and M=1000, compared with, e.g. erlang @ 1.09 seconds.

My curiosity was piqued, so I threw together a quick microbenchmark in erlang. On a moderately loaded 2013 macbook air (2-core i7) and erlang 17.1, with 1000 iterations of M=1000 and N=1000, it averaged 34 microseconds per run, which compares pretty favorably with Flow's claimed 75000 microseconds. The Flow paper appears to maybe be from 2010, so it would be interesting to know how it's doing in 2014.

Re: Databases at 14.4Mhz

#7

Post by ctdonath » Wed, Dec 10, 2014, 6:48 PM UTC

lttlrck wrote: post #3

"Or, as I like to say, 14.4Mhz." Sorry, I don't like that at all.

Inclined to agree. Title made me think it was a retro-computing analysis of running dBASE II on an IBM PC/AT or some such.

Re: Databases at 14.4Mhz

#8

Post by [deleted] » Wed, Dec 10, 2014, 7:00 PM UTC

[deleted]

Re: Databases at 14.4Mhz

#9

Post by hendzen » Wed, Dec 10, 2014, 7:09 PM UTC

This is very impressive, however...

See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809

(All credit for the idea in this comment is due to @aphyr)

Basically because the transactions modified keys selected from a uniform distribution, the probability of contention was extremely low. AKA this workload is basically a data-parallel problem, somewhat lessening the impressiveness of the high throughput. Would be interesting to see it with a Zipfian distribution (or even better, a Biebermark [0])

[0] - http://smalldatum.blogspot.co.il/2014/04/biebermarks.html

Re: Databases at 14.4Mhz

#10

Post by bsaul » Wed, Dec 10, 2014, 7:34 PM UTC

Just watched the linked presentation about "flow" here : https://foundationdb.com/videos/testing-distributed-systems-...

Is it really the first Distributed DB project to have built a simulator ?

Because frankly, if that's the case, it seems revolutionary to me. Intuitively, it seems like bringing the same kind of quality improvement as unit testing did to regular software development.

PS : i should add that this talk is one of the best i've seen this year. The guy is extremely smart, passionate, and clear. (i just loved the The Hurst exponent part).