Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FoundationDB: A distributed, unbundled, transactional key value store [pdf]

foundationdb.org

Post reply on HN
Print view View original

11–20 of 103 posts

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#11

Post by _vvhw »

ryanworl wrote: post #2

Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…

Thanks for the quotes, I've been wanting to read this paper for some time. Great to see they went through the consensus literature and made a decision to go with Active Disk Paxos, instead of stopping short and not fully understanding the consensus they're building on. The consensus and replication protocol is such a huge part of building a distributed database.

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#12

Post by bpicolo »

jeffbee wrote: post #5
ryanworl wrote: post #2

Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…

Ehhhh, doesn't align with my experience. I think FDB is actually really poorly tested. When I was evaluating it for replacement of the metadata key-value store at a major, public web services company we found that injecting faults into virtual NVMe devices on individual replicas would cause corrupt results returned to clients. We also found that it would just crash-loop on Linux systems with huge pages, because altho…

What were the strong contenders?

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#13

Post by rbranson »

jeffbee wrote: post #5
ryanworl wrote: post #2

Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…

Ehhhh, doesn't align with my experience. I think FDB is actually really poorly tested. When I was evaluating it for replacement of the metadata key-value store at a major, public web services company we found that injecting faults into virtual NVMe devices on individual replicas would cause corrupt results returned to clients. We also found that it would just crash-loop on Linux systems with huge pages, because altho…

Were there other distributed databases that did pass the fault injection testing?

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#14

Post by fnordpiglet »

jeffbee wrote: post #5
ryanworl wrote: post #2

Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…

Ehhhh, doesn't align with my experience. I think FDB is actually really poorly tested. When I was evaluating it for replacement of the metadata key-value store at a major, public web services company we found that injecting faults into virtual NVMe devices on individual replicas would cause corrupt results returned to clients. We also found that it would just crash-loop on Linux systems with huge pages, because altho…

Can you fix a point in time? Software evolves and I think a point I saw is that it wasn’t well tested then they changed once production workloads told them it needs to change.

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#15

Post by jeffbee »

rbranson wrote: post #13
jeffbee wrote: post #5

Earlier quoted context omitted.

Ehhhh, doesn't align with my experience. I think FDB is actually really poorly tested. When I was evaluating it for replacement of the metadata key-value store at a major, public web services company we found that injecting faults into virtual NVMe devices on individual replicas would cause corrupt results returned to clients. We also found that it would just crash-loop on Linux systems with huge pages, because altho…

Were there other distributed databases that did pass the fault injection testing?

There weren't any, which is why that particular shop elected to roll their own distributed system on top of rocks.

In general I think people who think they want to do FoundationDB owe themselves a serious contemplation of the cost/benefit of using Cloud Spanner instead. Obviously you cannot do your own fault injection testing of Spanner, but it does have end-to-end checksums.

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#16

Post by sgk284 »

_vvhw wrote: post #10

Markus Pilman from Snowflake did an awesome talk on FoundationDB's testing at CMU's Quarantine Tech Talks (2020), How I Learned to Stop Worrying and Trust the Database: https://www.youtube.com/watch?v=OJb8A6h9jQQ

Here's another excellent talk at Strangeloop on FoundationDB's simulation testing by Will Wilson in 2014: https://www.youtube.com/watch?v=4fFDFbi3toc

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#17

Post by e12e »

This seems like a good place to ask - are there any new and exiting FOSS "application" worth checking out? I recall from the initial publication of the source - there was references to a great sql layer? I don't know if a FOSS work-a-like ever materialized? Other things I'd hoped for was a network filesystem/blob layer, like maybe s3/nfs/webdavfs compatible? What are people building on top of foundationdb today?

Ed: i suppose various document/db applications - like IMAP might be a good fit too?

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#18

Post by sigstoat »

jtdev wrote: post #3

I’d love to see a good primer on data models and scenarios that are well suited to FDB.

this is limited by your creativity and willingness to make tradeoffs.

the only really general statement i can think of is that the "larger"/"longer" your transactions are, the harder a time you'll have getting it to cooperate with FDB. "small"/"fast" transactions will be easier to fit into its model.

(to likely replies: this isn't an absolute, see all the quotes. yes things like redwood will alleviate some of this, but not all.)

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#20

Post by sigstoat »

jeffbee wrote: post #15
rbranson wrote: post #13

Earlier quoted context omitted.

Were there other distributed databases that did pass the fault injection testing?

There weren't any, which is why that particular shop elected to roll their own distributed system on top of rocks. In general I think people who think they want to do FoundationDB owe themselves a serious contemplation of the cost/benefit of using Cloud Spanner instead. Obviously you cannot do your own fault injection testing of Spanner, but it does have end-to-end checksums.

> There weren't any, which is why that particular shop elected to roll their own distributed system on top of rocks.

that's nuts. rocks could've been added as a storage engine to fdb far more easily.

Post reply on HN