Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare Meerkat - Globally distributed consensus

blog.cloudflare.com

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Cloudflare Meerkat - Globally distributed consensus

#31

Post by mrkeen »

m11a wrote: post #3

This article is a bit hard for me to grasp the main ideas of because, given Cloudflare's requirements (e.g. no strong leaders), it immediately seems like they should be comparing to leaderless protocols like Paxos-class algorithms. Comparing to Raft and saying it's better because Meerkat is leaderless is confusing, because Raft is an adjustment to Paxos to specifically have strong leaders. So I'm 3/4 the way into the…

Agreed. When they describe the advantages over Raft, I can't help but read 'this is Paxos'.

Re: Cloudflare Meerkat - Globally distributed consensus

#32

Post by [deleted] »

pstoll wrote: post #4

TBH I’m doubtful of most people building their own crypto libs and distributed consensus implementations. But maybe cloudflare can pull it off. Good to see them pushing the state of distributed consensus. Take aways: * it’s not in prod yet. I suspect those many round trips are going to get expensive on median aka typical redistributed deployments. Curious to see how it goes once in the wild. * they say it isn’t likel…

[deleted]

Re: Cloudflare Meerkat - Globally distributed consensus

#33

Post by mrkeen »

s8kur wrote: post #9

Consensus is one of those areas where the interesting engineering and the number of people who actually need it are inversely correlated. Most "we need distributed coordination" turns out to be "we need one writer and a lock," which a single Postgres hands you for free: advisory locks, SELECT ... FOR UPDATE, SKIP LOCKED for work distribution. Linearizability without running Raft.. The real threshold is multi-region w…

Are you arguing that most teams that recognise a need for a distributed system chicken-out and use a single node instead, and if so, is that a good thing?

Re: Cloudflare Meerkat - Globally distributed consensus

#34

Post by inigyou »

nano_o wrote: post #27

What's interesting here is that this would be the first production implementation of an asynchronous consensus algorithm (QuePaxa). Paxos, Raft, etc. are all partially synchronous, meaning they rely on timeouts and only make progress if message delay is sufficiently small compared to timeout durations. QuePaxa doesn't rely on timeouts and makes progress even under wild fluctuations in message delay. The question is w…

Wasn't there an impossibility proof for consensus without timeouts? At the boundary between a consensus failure and success, there must be a certain message that, if you delay it enough, causes a consensus failure, and that implies either you wait forever for that message (deadlock) or you eventually give up waiting (timeout).

Re: Cloudflare Meerkat - Globally distributed consensus

#35

Post by wmf »

pyrolistical wrote: post #29
materielle wrote: post #13

Earlier quoted context omitted.

There’s a weird yet common attitude that nobody has to implement operating systems, databases, compilers, cryptography, consensus algorithms, etc because you should use off the shelf solutions. This is obviously true, and there are surely more web devs than, say, compiler engineers. But then the logic seems to go as far as to imply nobody does or should be doing these things. Where do they think the “off the shelf” s…

I predict we may be heading back to self implementations based off of papers since it is now cheaper to do so with AI coding agents. Yeah they kinda suck and make mistakes right now, but tech only gets better. It’s no different than 3D printing. Instead of cutting and joining a bunch of stock parts, you just print the desired part. In the same way printing with plastic kinda sucks, the tech will improve. Now you can…

Another 20 trillion to Antithesis.

Re: Cloudflare Meerkat - Globally distributed consensus

#38

Post by nano_o »

inigyou wrote: post #34
nano_o wrote: post #27

What's interesting here is that this would be the first production implementation of an asynchronous consensus algorithm (QuePaxa). Paxos, Raft, etc. are all partially synchronous, meaning they rely on timeouts and only make progress if message delay is sufficiently small compared to timeout durations. QuePaxa doesn't rely on timeouts and makes progress even under wild fluctuations in message delay. The question is w…

Wasn't there an impossibility proof for consensus without timeouts? At the boundary between a consensus failure and success, there must be a certain message that, if you delay it enough, causes a consensus failure, and that implies either you wait forever for that message (deadlock) or you eventually give up waiting (timeout).

It's impossible to do it deterministically (that's the famous FLP impossibility result), but if you accept to have liveness only with probability 1 then it's possible (for example, an early randomized asynchronous protocol is Ben-Or's protocol from 1983).

Re: Cloudflare Meerkat - Globally distributed consensus

#39

Post by aabhay »

ebeirne wrote: post #2

if youve ever fought a raft cluster on a bad network with leaders flapping, elections storming and latency spiking this genuinely doesnt seem that bad. i believe this will be very useful to those dealing with messy networks

Looking at the implementation sketches, this algorithm looks even trickier to implement than Paxos (already a notoriously tricky algorithm to implement) and on top of that, I think the failure case in this algorithm is subtle and different -- very long tail latencies. In Paxos/Raft the latencies are more likely bounded by the timeouts (not eliminated of course) so you can build other systems to expect certain delays, but in this case, you may write something, wait for an ack, then abandon and retry, then realize the old write succeeded, etc. ad infinitum.

Re: Cloudflare Meerkat - Globally distributed consensus

#40

Post by advisedwang »

This sounds like a very direct approach to linearizability - just put everything in a linear order.

But this includes READ operations too! You have to get global consensus for every read! Most distributed systems have additional complexity and latency on write paths so that reads can be completely local. If you can accept slow read operations this seems like a great trade off, but I think that is going to relegate this to niece usage.

Post reply on HN