Live data from Hacker News

In search of a simple consensus algorithm

rystsov.info

61–70 of 87 posts

Re: In search of a simple consensus algorithm

#61
post #53
post #24

Earlier quoted context omitted.

> consistent reads do not need a living master It's wrong. If you're fine with stale reads then you don't a living master, but if you want to have a guarantee that the read value is up-to-date then the living master is necessary. Etcd has a bug related to this - https://github.com/coreos/etcd/issues/741

When leader is not around, isn't a majority read sufficient?

It depends on the implementation. If you have a replication system that generates a monotonic sequence ID for each update, and you have all writes block on a majority of replicas acking an update, then you can just read from a majority and pick the result with the highest monotonic identifier. This will not be consistent unless the leadership mechanism demands that leadership is chosen based on the same criteria.

Re: In search of a simple consensus algorithm

#62

I am surprised: 1. I am technically pretty strong but I have no idea what this paper is about 2. So many people know this is about that it shot up to #1 on HN Can someone give a pointer (a link or two) to the lay, interested audience here about what the field IS. Just a sort of intro guide to someone who knows about programming and math, but has never heard the term paxos? I am curious, and I am sure many others are…

These consensus algorithms are awesome! They are the backbones of many highly available systems in modern DC's today. It's how people manage to get any sleep at all when taking care of systems that require very reliable databases.

Raft is a consensus algorithm that is touted as being easy to understand. It is well specified, compared to paxos, which leaves many implementation details up to the creator. Raft, however, is fairly explicitly specified on page 4 of this paper: https://raft.github.io/raft.pdf

Consensus algorithms allow us to build reliable castles out of constantly failing servers (sand). Systems like zookeeper, etcd, chubby, consul, and others use these algorithms to achieve high availability AND strong consistency (linearizability, the strongest possible) despite up to a majority of the cluster failing.

Re: In search of a simple consensus algorithm

#63
post #51
post #14

Earlier quoted context omitted.

Funnily enough, there is a paper on exactly what you're referring to from Microsoft Research called "Flexible Paxos": https://arxiv.org/pdf/1608.06696.pdf The gist of it is that there are realistically 2 quorums to be had: One for read/write to a cluster, and one for leader election. Assuming leader election is a relatively uncommon event (as it is in Paxos), you can require more quorum nodes for leader election whil…

> I literally spent part of an evening with a beer Just one? :) Consensus algorithms are _fascinating_! And +1 for https://blog.acolyer.org/ , I'll often read his blog over lunch. Another great paper is https://research.google.com/archive/paxos_made_live.html which talks about what it's like to realistically run Paxos. There's a ton of corner cases, engineering realities, and optimizations to be done to make Paxos wo…

Great comments and links! Pretty sure Google wrote chubby before raft existed.

Chubby paper: 2006 http://dl.acm.org/citation.cfm?id=1298487

Raft paper: 2014 http://dl.acm.org/citation.cfm?id=2643666

If you've got something battletested, there isn't a lot of value to rip it all up if it works within the given business requirements. Also they figured out how to implement Multi-Paxos, which is known for being difficult.

Re: In search of a simple consensus algorithm

#64

I am surprised: 1. I am technically pretty strong but I have no idea what this paper is about 2. So many people know this is about that it shot up to #1 on HN Can someone give a pointer (a link or two) to the lay, interested audience here about what the field IS. Just a sort of intro guide to someone who knows about programming and math, but has never heard the term paxos? I am curious, and I am sure many others are…

Lesley Lamport, Paxos inventor, describes it thus in the abstract to the original paper:

"[Paxos] provides a new way of implementing the state-machine approach to the design of distributed systems."

Original Paxos paper (written with somewhat whimsical style):

https://www.microsoft.com/en-us/research/publication/part-ti...

Follow up paper to explain Paxos more simply:

https://www.microsoft.com/en-us/research/publication/paxos-m...

See also: https://en.wikipedia.org/wiki/Paxos_(computer_science)

Re: In search of a simple consensus algorithm

#65
Good to see more leaderless consensus protocol implementations and that RAFT isn't be all and end all of all consensus problems.

One advantage not mentioned in the article, of leader based consensus algorithms is the ability to more easily implement read leases for faster reads.

Read leases can allow for fresh reads without having to run them through the quorum protocol, by trading off availability (due to leader failure) and also correctness in certain edge cases (since read leases will depend on ability for individual machines to measure time delta with reasonable accuracy, which may not be true on some weird VM scenarios).

Re: In search of a simple consensus algorithm

#66
I think this overstates the instability of strong leader-based consensus algorithms, or at least over-generalizes Raft's instability to apply to all consensus algorithms with a stronger leader.

In Raft, it's possible for multiple nodes to prevent leader election progress by being overly aggressive when requesting votes. It is also possible for a follower to knock out a perfectly healthy leader by being too quick time out the leader and start a new term.

Both of these limitations stem from the simplicity of Raft's leader election algorithm. To compensate, most Raft implementations I've seen have more conservative follower timeouts that extend the time to detect leader failure and elect a new one.

It's possible for a more optimized algorithm to get sub-second latencies for detecting and re-electing a leader, even in a latent (e.g. geo-distributed) environment. In other words, well within the commit window for the replica set based on network hop latencies.

Also, while the latency for individual writes in single-decree paxos can be closer to strong leader protocols, it is non-trivial to achieve the same level of throughput that is possible in Raft et al when writing to an ordered log, as you cannot start a paxos instance for a new log entry until all prior instances have been resolved. Raft can just add new values in the next append call (or just spam out appends for new messages w/o waiting for the replies for previous ones).

IME, I'd say both single-decree paxos and raft are probably equivalent in terms of understandability, but raft is a better base on which to build a fast high-throughput consensus protocol.

Re: In search of a simple consensus algorithm

#67
post #55
post #45

The article abuses Kolmogorov complexity... > When it is applied to the algorithms, it means that an algorithm with the shortest implementation is simpler. That is misleading. Kolmogorov complexity is the length of the shortest program (in a pre-defined language) that produces a given object. So, if the shortest program that produces Algorithm A is smaller than the shortest program that produces Algorithm B, then the…

Thank you! I'll rework this paragraph to be correct, I wanted to make an observation that the given data (two attempts to implement key-value storages with keeping the length of a program as short as possible) favour Gryadka but of course isn't wrong to make strong statements based just on one data point.

Yeah,

The problem isn't just the incomputable quality of Kolmogorov complexity but that fact that Kolmogorov complexity applies only to finite strings or things that can be meaningfully mapped to them. Especially, Kolmogorov doesn't apply directly to abstract algorithms or programs with multiple implementations.

Re: In search of a simple consensus algorithm

#68

I am surprised: 1. I am technically pretty strong but I have no idea what this paper is about 2. So many people know this is about that it shot up to #1 on HN Can someone give a pointer (a link or two) to the lay, interested audience here about what the field IS. Just a sort of intro guide to someone who knows about programming and math, but has never heard the term paxos? I am curious, and I am sure many others are…

These consensus algorithms are awesome! They are the backbones of many highly available systems in modern DC's today. It's how people manage to get any sleep at all when taking care of systems that require very reliable databases. Raft is a consensus algorithm that is touted as being easy to understand. It is well specified, compared to paxos, which leaves many implementation details up to the creator. Raft, however,…

Whoops, I meant to say "up to the largest minority failing", if a majority is lost then the gig is up! Too late to edit!

Re: In search of a simple consensus algorithm

#69
post #66

I think this overstates the instability of strong leader-based consensus algorithms, or at least over-generalizes Raft's instability to apply to all consensus algorithms with a stronger leader. In Raft, it's possible for multiple nodes to prevent leader election progress by being overly aggressive when requesting votes. It is also possible for a follower to knock out a perfectly healthy leader by being too quick time…

> cannot start a paxos instance for a new log entry until all prior instances have been resolved

It's wrong. In Grydka (Single-decree Paxos) all the keys are independent so it's possible to update them at the same time without blocking.

Grydka's throughput is comparable to Etcd on the same type of machines (4720 vs 5227 rps) and I never optimized for it (my goal was to fit 500 lines) so it's also wrong that it's "non-trivial to achieve the same level of throughput" - I did it by accident.

So I don't understand why Raft is a better base to build a fast high-throughput consensus protocol.

Re: In search of a simple consensus algorithm

#70
post #69
post #66

I think this overstates the instability of strong leader-based consensus algorithms, or at least over-generalizes Raft's instability to apply to all consensus algorithms with a stronger leader. In Raft, it's possible for multiple nodes to prevent leader election progress by being overly aggressive when requesting votes. It is also possible for a follower to knock out a perfectly healthy leader by being too quick time…

> cannot start a paxos instance for a new log entry until all prior instances have been resolved It's wrong. In Grydka (Single-decree Paxos) all the keys are independent so it's possible to update them at the same time without blocking. Grydka's throughput is comparable to Etcd on the same type of machines (4720 vs 5227 rps) and I never optimized for it (my goal was to fit 500 lines) so it's also wrong that it's "non…

This is true, but comes at the cost of not being able to preserve linearity across arbitrary keys, and you are still limited by the throughput on updates to a single key.
Post reply on HN