51–60 of 86 posts
Earlier quoted context omitted.
All of the transactions in the test noted in the blog are multi-key transactions. (20 key updates)
When claiming the 14.4MHz number, is that per 20-key update? Or a single key update in the 20-key update? And what's the latency on the 20-key update?
This is Foundation DB's announcement they are doing full ACID databases with a 14.4M writes per second capability. That is insanely fast in the data base world. Running in AWS with 32 c3.8xlarge configured machines. So basically NSA level data base capability for $150/hr. But perhaps more interesting is that those same machines on the open market are about $225,000. That's two rack, one switch and a transaction rate…
32 c3.2xlarge instances have 1920GB memory. Given 1 billion 16B+ 8..100B values the whole dataset fits just into memory.
The Cassandra test mentioned [1] sustained loss of 1/3 instances. That's very impressive! Would love to see how F-DB handles this type of real-life situation (hint hint for follow up blog post).
[1] http://googlecloudplatform.blogspot.cz/2014/03/cassandra-hit...
We've been using FoundationDB in production for about 10 months now. It's really been a game changer for us. We continue to use it for more and more data access patterns which require strong consistency guarantees. We currently store ~2 terabytes of data in a 12 node FDB cluster. It's rock solid and comes out of the box with great tooling. Excited about this release! My only regret is I didn't find it sooner :)
Earlier quoted context omitted.
At Couchbase we have a plethora of simulators. Simulations of cluster topology changes, simulations of failure scenarios, simulations of workloads to estimate requried cluster sizes, etc. Here's one: https://github.com/couchbaselabs/cbfg
I think the difference is that FoundationDB has only one, and it's not an external simulation. The actual code that runs in production can also deterministically simulate a cluster of itself. I do believe this is unique among publically-available distributed databases.
I'm only familiar with other key-value storage engines, not FoundationDB, but it seems like the goals are: "distributed key-value database, read latencies below 500 microseconds, ACID, scalability". I remember evaluating a few low latency key-value storage solutions, and one of these was Stanford's RAMCloud, which is supposed to give 4-5 microseconds reads, 15 microseconds writes, scale up to 10,000 boxes and provide…
But it doesn't store the data on disk. It should be compared to Memcached, not to a real storage engine.
* Automatic replication, crash recovery, and fail-over
(no loss of availability if a server fails)
* Durability guarantee: data is always replicated
and durable before operations return, without
significant performance hit (subject to the
requirement for persistent buffers on backups)."Or, as I like to say, 14.4Mhz." Sorry, I don't like that at all.
I would agree. While 14.4 million writes per second is impressive, it definitely doesn't fit the definition of the Hz unit (cycles per second).
We've been using FoundationDB in production for about 10 months now. It's really been a game changer for us. We continue to use it for more and more data access patterns which require strong consistency guarantees. We currently store ~2 terabytes of data in a 12 node FDB cluster. It's rock solid and comes out of the box with great tooling. Excited about this release! My only regret is I didn't find it sooner :)
Why do you need a 12 node cluster for 2 TB of data?
FoundationDB requires SSD drives, which the best we can get efficiently in our data center is ~670 GB of usable space (3x480GB raided).
12x670 = 8040 GB
We try to keep extra space available for node failures (FDB will immediately start replicating addition data if it notices data with less than the configured number of replicas).
Our dataset currently grows at a decent pace, so we over provision a bit as well.
Earlier quoted context omitted.
But it doesn't store the data on disk. It should be compared to Memcached, not to a real storage engine.
Unlike Memcached or Redis, RAMCloud provides durability: * Automatic replication, crash recovery, and fail-over (no loss of availability if a server fails) * Durability guarantee: data is always replicated and durable before operations return, without significant performance hit (subject to the requirement for persistent buffers on backups).
Nothing stops you from replicating memcached.
Earlier quoted context omitted.
Ah – I see what you mean now! If they are blind writes, not some kind of CAS operation (reading from and modifying the same key), then there is no conflict. I thought you meant that clients A and B do their writes, and only if someone at a later time were to observe the value at that key, a conflict would be caused (Heisenberg-style). Thanks for clarifying!
DBs that are consistent until you observe them, a funny thought. In some parallel universe, the DB is still consistent; one could argue.
Earlier quoted context omitted.
Right, because a database write is not generally a periodic event. The unit they want is the becquerel - 14.4MBq might even sound more impressive because not so many people are familiar with that unit!
I agree that Hz seems to imply a periodic event. But I disagree that the Bq is appropriate for this example; the becquerel quantifies radioactivity: > The becquerel (symbol Bq) (pronounced: 'be-kə-rel) is the SI derived unit of radioactivity. One Bq is defined as the activity of a quantity of radioactive material in which one nucleus decays per second. ( https://en.wikipedia.org/wiki/Becquerel )