Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

fauna.com

Post reply on HN
Print view View original

11–20 of 105 posts

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#11

Post by wikibob »

jchrisa wrote: post #10
wikibob wrote: post #8

This looks like a fascinating approach. Unfortunately I’m not well versed enough to intelligently compare it to alternatives. Any plans to have Jepsen / Aphyr conduct a rigorous test and write a report on the results?

We have a whole suite of internal correctness verification tools, in addition to which we are also using Jepsen. We don't have an official report out yet, but you can get a preview of our progress here: https://fauna.com/blog/verifying-transactional-consistency-w...

Just my opinion, having Kyle write a formal report would give a huge bump to validating the stated properties :)

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#12

Post by jchrisa »

donpdonp wrote: post #9

thanks drift.com (and driftt.com) for driving me to learn how to block specific sites in uBlock. Audio 'pops' for a webpage chatroom is a new level of annoyance.

Thanks for the bump I needed to go fix that. Sound is off now.

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#14

Post by evanweaver »

wikibob wrote: post #11
jchrisa wrote: post #10

Earlier quoted context omitted.

We have a whole suite of internal correctness verification tools, in addition to which we are also using Jepsen. We don't have an official report out yet, but you can get a preview of our progress here: https://fauna.com/blog/verifying-transactional-consistency-w...

Just my opinion, having Kyle write a formal report would give a huge bump to validating the stated properties :)

Stay tuned...

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#15

Post by jchrisa »

agentultra wrote: post #13

I wish projects like this would publish their proofs with their protocols. It's interesting to be sure but I find all of the prose and diagrams to be too verbose. I'd much rather read the mathematical model of the transaction protocol.

You can read the paper that inspired the FaunaDB transaction protocol here: http://cs.yale.edu/homes/thomson/publications/calvin-sigmod1...

Or if you want a very concise description, see slide 13 from Daniel Abadi's presentation: https://www.slideshare.net/abadid/the-power-of-determinism-i...

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#16

Post by fernly »

The concept of the "global log" is mentioned many times in the article, but not much detail on this critical piece:

> ... the consensus protocol is only being used for inserting transactions into a global log. For every other part of the protocol, replicas can proceed completely independently from each other.

Clearly this central resource can't be distributed or replicated. Right? So it, and the hardware it runs on, and the communications links to it, are a single point of failure. True?

If the global log goes down, or can't be reached at all, or perhaps only from some region of the regionized application, what happens?

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#17

Post by jchrisa »

fernly wrote: post #16

The concept of the "global log" is mentioned many times in the article, but not much detail on this critical piece: > ... the consensus protocol is only being used for inserting transactions into a global log. For every other part of the protocol, replicas can proceed completely independently from each other. Clearly this central resource can't be distributed or replicated. Right? So it, and the hardware it runs on,…

The global log is replicated using a Raft-like consensus mechanism, so that all replicas have a copy of the log, and agree on its order. So even if one replica site is lost, the others have a full copy of the log, and can continue.

You can learn more about Raft from this excellent visualization: http://thesecretlivesofdata.com/raft/

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#19

Post by ummonk »

evanweaver wrote: post #2

We're here, ready for your questions to be consistently replicated across the WAN at the lowest latency information science allows ;-)

Not a database person, so I might be missing context, but why is FaunaDB the only implementation of this?

Intuitively, it seems to be exactly the way I would do a distributed database - distributed transactional storage with a global ordering of transactions that are based on ids rather than timestamps. The only thing that I wouldn't intuitively come up with is the actual consensus algorithm, but that already existed...

Both the tendency to use declarative rather than transactional queries and the tendency to use timestamps with clock skew rather than an id-based ordering always seemed surprising to me.

Re: Consistency Without Clocks: FaunaDB's Distributed Transaction Protocol

#20

Post by jamesblonde »

fernly wrote: post #16

The concept of the "global log" is mentioned many times in the article, but not much detail on this critical piece: > ... the consensus protocol is only being used for inserting transactions into a global log. For every other part of the protocol, replicas can proceed completely independently from each other. Clearly this central resource can't be distributed or replicated. Right? So it, and the hardware it runs on,…

You are correct that they serialize cross-partition transactions. So if your workload has a reasonable percentage of write/update/delete operations it is possible that you will bottleneck on the global coordinator. From this blog https://fauna.com/blog/distributed-acid-transaction-performa..., you can get 3300 transactions/second. Daniel Abadi claims you can get to 500,000 trans/sec - http://dbmsmusings.blogspot.com/2018/09/newsql-database-syst....

In any case, there is an open-source in-memory database called NDB (MySQL Cluster) that can get to 100 million write transactions per second. The echo chamber of SV has ignored it, but it's real: http://mikaelronstrom.blogspot.com/2015/03/200m-reads-per-se...

NDB has what i call lock-aware programming. When you read a row, you take either a read_lock or a write_lock, as it only supports READ_COMMITTED isolation level. But, it does not serialize cross-partition transactions. Transaction coordinators run independently, and it is up to the developer to ensure consistency by taking either read or write locks on rows.

Post reply on HN