Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Raft: Understandable Distributed Consensus (2014)

thesecretlivesofdata.com

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Raft: Understandable Distributed Consensus (2014)

#31

Post by ryanthemadone »

kfrzcode wrote: post #23

DLT technology discussions are entirely incomplete without consideration of Hedera Hashgraph [0], an aBFT, leaderless, fair and fast DLT using a gossip-about-gossip consensus mechanism. It's absolutely a more robust and scalable technology than Paxos or any other DLT for that matter. I'd love to know what the HN crowd thinks about Hedera as the trust layer of the internet but.... nobody around here seems to have any.…

Paxos isn't a DLT, it's a consensus algorithm — granted, DLTs tend to require a consensus algorithm, but they're not the same things.

As for Hedera Hashgraph being the trust layer of the internet, we tend to build the internet through the IETF and standards setting. Unfortunately HH is an endeavour from a private company so isn't especially likely to be taken on in that context.

I'd also wonder what you mean by the trust layer of the internet, what are the use cases that you'd like to see solved with such a trust layer?

Re: Raft: Understandable Distributed Consensus (2014)

#32

Post by huntaub »

benbjohnson wrote: post #6

Author here. I'm happy to answer any questions although this project was from 10+ years ago so I could be a little rusty. Over the years I've been trying to find better ways to do this kind of visualization but for other CS topics. Moving to video is the most realistic option but using something like After Effects takes A LOT of time and energy for long-form visualizations. It also doesn't produce a readable output f…

I just want you to know how much this visualization was appreciated. In my time working at AWS, I recommended this website to every one of our new hires to learn how distributed consensus works. Know that this has taught probably 50+ people. Thank you for what you’ve built.

Re: Raft: Understandable Distributed Consensus (2014)

#33

Post by weinzierl »

_russross wrote: post #19

I am in the minority who thinks Raft is overrated. I tried teaching Raft one year instead of Paxos but ended up switching back. While it was much easier to understand how to implement Raft, I think my students gained deeper insight when focusing on single-decision Paxos. There is a lightbulb moment when they first understand that consensus is a property of the system that happens first (and they can point at the mome…

This is an interesting insight into the educational side, but now I am curious about the implementation side. Raft is easier to implement but that's just one factor. Looking at real world usages there seems to be a draw. I could easily count as many Paxos implementations as Raft. Is this just historical or are there good reasons for a new project to still implement Oaxos?

Re: Raft: Understandable Distributed Consensus (2014)

#34

Post by eatonphil »

lifeinthevoid wrote: post #29
throwawaymaths wrote: post #25

Earlier quoted context omitted.

> A known limitation of the base Raft protocol is that partial/asymmetric network partitions can cause a loss of liveness [27, 32]. For instance, if a leader can no longer make progress because it cannot receive messages from the other nodes, it continues to send AE heartbeats to followers, preventing them from timing out and from electing a new leader who can make progress. (Howard, Abram et al) Me: note this can al…

> A known limitation of the base Raft protocol is that partial/asymmetric network partitions can cause a loss of liveness [27, 32]. For instance, if a leader can no longer make progress because it cannot receive messages from the other nodes, it continues to send AE heartbeats to followers, preventing them from timing out and from electing a new leader who can make progress. Real-world raft implementations make the l…

The thesis does describe doing this fwiw while the paper does not.

Re: Raft: Understandable Distributed Consensus (2014)

#35

Post by mbivert »

In case this is of interest, MIT's 6.5840[0], distributed systems, has a series of labs, implementing Raft in Go. Haven't made it through the whole thing yet, but it's quite entertaining so far.

The teachers provide you with some code templates, a bunch of tests, and a progressive way to implement it all.

[0]: https://pdos.csail.mit.edu/6.824/index.html

Re: Raft: Understandable Distributed Consensus (2014)

#36

Post by prydt »

I've run a reading group for distributed systems for the last 2 years now and I do think that Raft is a better introduction to Consensus than any Paxos paper I have seen (I mean the Paxos Made Simple paper literally has bugs in it). But when I learned consensus in school, we used Paxos and Multi-Paxos and I do believe that there was a lot to be gained by learning both approaches.

Heidi Howard has several amazing papers about how the differences between Raft and Multi-Paxos are very surface level and that Raft's key contribution is its presentation as well as being a more "complete" presentation since there are so many fragmented different presentations of Multi-Paxos.

As a bonus, one of my favorite papers I have read recently is Compartmentalized Paxos: https://vldb.org/pvldb/vol14/p2203-whittaker.pdf which is just a brilliant piece on how to scale Multi-Paxos

Re: Raft: Understandable Distributed Consensus (2014)

#37

Post by prydt »

convolvatron wrote: post #28
throwawaymaths wrote: post #12

Earlier quoted context omitted.

Weirdly it's also kinda worse is better: raft is non-deterministic and has an unboundedly long election cycle time. IIRC: - it assumes no hysteresis in network latencies and if there is a hysteresis it's possible that elections can be deterministically infinite. - this fact and the use of raft in production has caused real, large scale network outages. Paxos is of course a beast and hard to understand. There is an al…

don't forget about calm. its a shame that that isn't the default we reach for, and only struggle when we really need stronger latency. https://arxiv.org/abs/1901.01930

I think the CALM theorem and this whole line of research is so interesting and it is still carried on by the CRDT people. But I would love to see more of this.

I feel like it doesn't get as much attention as it deserves.

Re: Raft: Understandable Distributed Consensus (2014)

#38

Post by bfdes »

weinzierl wrote: post #33
_russross wrote: post #19

I am in the minority who thinks Raft is overrated. I tried teaching Raft one year instead of Paxos but ended up switching back. While it was much easier to understand how to implement Raft, I think my students gained deeper insight when focusing on single-decision Paxos. There is a lightbulb moment when they first understand that consensus is a property of the system that happens first (and they can point at the mome…

This is an interesting insight into the educational side, but now I am curious about the implementation side. Raft is easier to implement but that's just one factor. Looking at real world usages there seems to be a draw. I could easily count as many Paxos implementations as Raft. Is this just historical or are there good reasons for a new project to still implement Oaxos?

Paxos and Multi-Paxos have been around much longer than Raft. The paper that introduced Raft was published in 2014.

Re: Raft: Understandable Distributed Consensus (2014)

#39

Post by alexchamberlain »

_russross wrote: post #19

I am in the minority who thinks Raft is overrated. I tried teaching Raft one year instead of Paxos but ended up switching back. While it was much easier to understand how to implement Raft, I think my students gained deeper insight when focusing on single-decision Paxos. There is a lightbulb moment when they first understand that consensus is a property of the system that happens first (and they can point at the mome…

I've read both the Paxos and Raft papers a few times, and hacked on some implementations, but never quite got one over the line to working...

Raft strikes me as a particular set of decisions made within a Paxos framework, such as having 1 entity for Proposers, Acceptor and Followers. It's frustrating that there isn't a clearly written defacto paper on Paxos - the story style confused the monkeys out of me.

Re: Raft: Understandable Distributed Consensus (2014)

#40

Post by dang »

Related:

Raft Consensus Animated (2014) - https://news.ycombinator.com/item?id=32484584 - Aug 2022 (67 comments)

Raft Visualization - https://news.ycombinator.com/item?id=25326645 - Dec 2020 (35 comments)

Raft: Understandable Distributed Consensus - https://news.ycombinator.com/item?id=8271957 - Sept 2014 (79 comments)

Post reply on HN