Live data from Hacker News

FoundationDB: A distributed, unbundled, transactional key value store [pdf]

foundationdb.org

31–40 of 103 posts

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#31

What is the backup / restore story in FoundationDB? How does it compare to postgresql?

Much much better. Single line backup/restore and Disaster Recovery mode that syncs second DC and able too switch on the fly with barely any configs (except one file).

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#33

Here's one of my favorite articles on FoundationDB, where it (FDB) passes Jepsen first try: https://web.archive.org/web/20150312112556/http://blog.found... > I ran FoundationDB Key-Value Store through every nemesis in Jepsen - including those that found failures in other databases - and FoundationDB passed all of them with flying colors. FoundationDB is one of the coolest pieces of technology I've used in the past de…

That's great that you are doing your own resiliency testing.

Having someone other than those officially on the Jepsen project run the Jepsen test is a good start. However, many databases have claimed to run the Jepsen tests themselves and pass, but when there is an actual paid engagement for a distributed database there are always issues that are found. That's generally true even for unpaid official runs as well although Zookeeper did pass existing tests. Every database is different and the paid engagement will design specific tests designed to break the database in question.

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#34
Have nothing but praise for FoundationDB. It has been by far the most rock solid distributed database I have ever had the pleasure of using. I used to manage HBase clusters, and the fact that I have never once had to worry about manually splitting "regions" is such a boon for administration...let alone JVM GC tuning.

We run several FDB clusters using 3-DC replication and have never once lost data. I remember when we wanted to replace all of the FDB hardware (one cluster) in AWS, and so we just doubled the cluster size, waited for data shuffling to calm down, and just started axing the original hardware. We did this all while performing over 100K production TPS.

One thing that makes the above seamless for all existing connections is that clients automatically update their "cluster file" in the event that new coordinators join or are reassigned. That alone is amazing...as you don't have to track down every single client and change / re-roll with new connection parameters.

Anyway, I talk this database up every chance I get. Keep up the awesome work.

- A very happy user.

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#37
post #7
post #2

Two quotes from the paper that I think will motivate people to read it: "Rigorous correctness testing via simulation makes FDB extremely reliable. In the past several years, CloudKit [59] has deployed FDB for more than 0.5M disk years without a single data corruption event. Additionally, we constantly perform data consistency checks by comparing replicas of data records and making sure they are the same. To this date…

> de novo Paxos implementation written in Flow That's... brave. Flow is a DSL built on top of C++?

Yeah it's their own language on top of c++ to help them with testing distributed systems with deterministic simulation.

Their talk from a while ago about it was something that really blew me away at the time [0]

[0] https://www.youtube.com/watch?v=4fFDFbi3toc

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#38

Here's one of my favorite articles on FoundationDB, where it (FDB) passes Jepsen first try: https://web.archive.org/web/20150312112556/http://blog.found... > I ran FoundationDB Key-Value Store through every nemesis in Jepsen - including those that found failures in other databases - and FoundationDB passed all of them with flying colors. FoundationDB is one of the coolest pieces of technology I've used in the past de…

This was the FDB team’s stock demo in the early days. It’s a killer move.

Re: FoundationDB: A distributed, unbundled, transactional key value store [pdf]

#39

FDB is an awesome and unique piece of software (I attribute quite a bit of Snowflake's success to FDB). I've also had the pleasure of meeting some folks from the original team and they are true engineers. Does anyone know if/when Redwood (the new storage engine) has landed / will land?

Founders are building a distributed systems simulation product now called Antithesis. My data fabric startup, Stardog, is a happy Antithesis early adopter customer. It’s helping us reproduce and fix non-deterministic bugs deterministically. Good stuff.
Post reply on HN