Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we rolled our own consensus algorithm

quorumcontrol.com

Post reply on HN
Print view View original

21–30 of 44 posts

Re: Why we rolled our own consensus algorithm

#21

Post by kjeetgill »

eloff wrote: post #18
zzzcpan wrote: post #12

Earlier quoted context omitted.

If you have enough background in distributed systems to choose a consensus algorithm you are definitely capable of implementing one yourself and even invent your own. It's not a bad idea either, most existing consensus-based systems are absolute garbage. People just hugely overestimate how much those algorithms matter for production systems. EDIT: Note, I'm not talking about blockchains and byzantine fault tolerance,…

That's like saying if you have some crypto background you can implement your own crypto library. It's TERRIBLE advice. Read the Google paper on their chubby distributed locking system using paxos. They had a hard time getting it right - unless you think you're so much better than them. IIRC Leslie Lamport works (worked?) for them. Use an existing, well used, well scrutinized implementation of a known algorithm.

Are you sure? I thought Lamport has been at Microsoft Research since forever.

Re: Why we rolled our own consensus algorithm

#22

Post by dlubarov »

eloff wrote: post #18
zzzcpan wrote: post #12

Earlier quoted context omitted.

If you have enough background in distributed systems to choose a consensus algorithm you are definitely capable of implementing one yourself and even invent your own. It's not a bad idea either, most existing consensus-based systems are absolute garbage. People just hugely overestimate how much those algorithms matter for production systems. EDIT: Note, I'm not talking about blockchains and byzantine fault tolerance,…

That's like saying if you have some crypto background you can implement your own crypto library. It's TERRIBLE advice. Read the Google paper on their chubby distributed locking system using paxos. They had a hard time getting it right - unless you think you're so much better than them. IIRC Leslie Lamport works (worked?) for them. Use an existing, well used, well scrutinized implementation of a known algorithm.

> Use an existing, well used, well scrutinized implementation of a known algorithm.

Keep in mind that this is BFT consensus. Until recently, it was a niche area of research with very little real world usage. Even if an existing algorithm like PBFT met their needs, there are no well tested PBFT libraries.

Re: Why we rolled our own consensus algorithm

#25

Post by cdiddy2 »

davidgerard wrote: post #19
cdiddy2 wrote: post #10

Earlier quoted context omitted.

I believe bitcoins proof of work has been formally verified, however I can't find a source for that. There is also this https://iohk.io/research/papers/#9BKRHCSI that claims to be formally verified Proof of Stake.

Unless you're actually a computer scientist following the area ... all claims of solving the "blockchain trilemma" - decentralisation, security and efficiency - should be ignored unless and until they have an implementation to show you, and it survives the hostile Internet. Promises are cheap. e.g. IOHK is behind Cardano/ADA, which has a marvellous white paper making all sorts of mathematical promises ... and still r…

Luckily I am a computer scientist however it is extremely hard to keep up with everything going on in the blockchain space.

There must be some validity behind the idea of weighted random turn-taking though since it is similar to how the next version(serenity) of ethereum will work based

Re: Why we rolled our own consensus algorithm

#26

Post by bradhe »

zzzcpan wrote: post #15
cremp wrote: post #11

Earlier quoted context omitted.

Just to drive it home. Those who actually make their own encryption libraries know their stuff, and have been in the space for years. You should never, ever, roll your own encryption. If it's a requirement, then change your damn requirements, because just like consensus; it is not easy to get right, and the pros make mistakes.

Consensus is not crypto, it's ridiculously trivial in comparison and usually so useless, that nobody even bothers doing it correctly all the way to the end user.

bingo.

Re: Why we rolled our own consensus algorithm

#27

Post by drfuchs »

eloff wrote: post #18
zzzcpan wrote: post #12

Earlier quoted context omitted.

If you have enough background in distributed systems to choose a consensus algorithm you are definitely capable of implementing one yourself and even invent your own. It's not a bad idea either, most existing consensus-based systems are absolute garbage. People just hugely overestimate how much those algorithms matter for production systems. EDIT: Note, I'm not talking about blockchains and byzantine fault tolerance,…

That's like saying if you have some crypto background you can implement your own crypto library. It's TERRIBLE advice. Read the Google paper on their chubby distributed locking system using paxos. They had a hard time getting it right - unless you think you're so much better than them. IIRC Leslie Lamport works (worked?) for them. Use an existing, well used, well scrutinized implementation of a known algorithm.

No, Lamport does not now, and never has, worked at Google.

Re: Why we rolled our own consensus algorithm

#28

Post by polskibus »

zonotope wrote: post #13
polskibus wrote: post #9

Is there a TLA+ spec available?

We are still vetting and improving the consensus algorithm, but we're planning on publishing a full TLA+ spec once the algorithm stabilizes.

Wouldn't it be easier to start with TLA+ / PlusCal? It offers higher level of abstraction and helps catch specification bugs.

Re: Why we rolled our own consensus algorithm

#29

Post by therein »

kjeetgill wrote: post #21
eloff wrote: post #18

Earlier quoted context omitted.

That's like saying if you have some crypto background you can implement your own crypto library. It's TERRIBLE advice. Read the Google paper on their chubby distributed locking system using paxos. They had a hard time getting it right - unless you think you're so much better than them. IIRC Leslie Lamport works (worked?) for them. Use an existing, well used, well scrutinized implementation of a known algorithm.

Are you sure? I thought Lamport has been at Microsoft Research since forever.

You're correct. Lamport has joined MSR in 2001 right after publishing "Time, Clocks, and the Ordering of Events in a Distributed System" and has been there ever since.

I am not sure if he is still at the MV office but he was one of the reasons why MSR kept a presence in MV for a while.

Re: Why we rolled our own consensus algorithm

#30

Post by kenhwang »

sagichmal wrote: post #2

Not rolling your own consensus algorithm is approx. rules zero through ten of distributed systems programming. Nothing on this page provides any positive signal indicating unique (or even average) competence to the task. To wit, > We are borrowing all the best parts of existing technology and combining them in a unique and innovative way. To the best of my knowledge, none of the listed existing technologies have been…

Hell, people even have a hard time correctly implementing the handful of provably correct consensus algorithms that already exist. Just look through the issues section of every raft implementation.
Post reply on HN