FoundationDB: A distributed, unbundled, transactional key value store [pdf]
1–10 of 103 posts
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#2Two quotes from the paper that I think will motivate people to read it:
"Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date, no inconsistent data replicas have ever been found in our production clusters."
"For example, early versions of FDB depended on Apache Zookeeper for coordination, which was deleted after real-world fault injection found two independent bugs in Zookeeper (circa 2010) and was replaced by a de novo Paxos implementation written in Flow. No production bugs have ever been reported since."
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#3I’d love to see a good primer on data models and scenarios that are well suited to FDB.
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#4FDB is an awesome and unique piece of software (I attribute quite a bit of Snowflake's success to FDB). I've also had the pleasure of meeting some folks from the original team and they are true engineers. Does anyone know if/when Redwood (the new storage engine) has landed / will land?
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#5Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…
Ehhhh, doesn't align with my experience. I think FDB is actually really poorly tested. When I was evaluating it for replacement of the metadata key-value store at a major, public web services company we found that injecting faults into virtual NVMe devices on individual replicas would cause corrupt results returned to clients. We also found that it would just crash-loop on Linux systems with huge pages, because although someone from the project had written a huge-page-aware C++ allocator "for performance", evidently nobody had ever actually tried to use it, including the author.
It's also really, really weird that their non-scalable architecture hits a brick wall at 25 machines. Ignoring the correctness flaws, it only works if you can either design around that limit by sharding, and never off cross-shard transactions, or if you can assure yourself that your use case will never outgrow half a rack of equipment.
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#6Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…
What is the Flow referred to here?
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#7Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…
> de novo Paxos implementation written in Flow
That's... brave. Flow is a DSL built on top of C++?
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#8Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…
What is the Flow referred to here?
It's an async/await framework for C++. I'm not sure what the best source on this is, but here's a discussion: https://forums.foundationdb.org/t/why-was-flow-developed/171...
My understanding is that FDB relies heavily on deterministic simulations for testing, and that their async/await model is a big part of how they make sure they cover different possible interleavings in a deterministic way.
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#9I’d love to see a good primer on data models and scenarios that are well suited to FDB.
Their docs might be a good place to start.
https://apple.github.io/foundationdb/developer-guide.html#da...
Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]
#10Markus Pilman from Snowflake did an awesome talk on FoundationDB's testing at CMU's Quarantine Tech Talks (2020), How I Learned to Stop Worrying and Trust the Database: