Etcdircd: an ircd backed by etcd • Hacker News Board

Re: Etcdircd: an ircd backed by etcd

#11

Post by mbrock » Wed, Oct 18, 2017, 6:53 PM UTC

takeda wrote: post #10

mbrock wrote: post #6

Earlier quoted context omitted.

Netsplits on IRC are such a visceral experience of distributed consensus failure. I remember as a little teenager on IRC being like "omg what is happening, why are there parallel worlds where weird things happen and everything drifts in and out?!"

I'm not convinced this is a solution to the problem though. As you mentioned with traditional IRC during actual netsplit (not single node disconnecting, but two large networks separating) you will see two networks that will get out of sync with time. In this case (assuming it is implemented correctly) the etcd will just shut down and not allow anyone use it until netsplit ends. Another issue here is that consensus pr…

Yeah, that's a good point. Federated networks like IRC have their benefits. Most chatting doesn't need total consensus. Disordered messages are sometimes confusing but mostly harmless.

Re: Etcdircd: an ircd backed by etcd

#12

Post by geofft » Wed, Oct 18, 2017, 7:23 PM UTC

TheDong wrote: post #5

Kudos for making this AGPL. This is exactly the sort of software that should use such a license. User's of an IRC server deserve to know whether their operator has started recording messages, added or removed security features, or otherwise be free to review said code. All chat software should be AGPL.

> User's of an IRC server deserve to know whether their operator has started recording messages, added or removed security features, or otherwise be free to review said code.

I don't understand how this is related - it's probably easier to tap messages outside of the software via packet capture or something than to modify the software to record them.

Also, someone who's interested in recording messages without user consent is probably also totally fine with violating the license.

Re: Etcdircd: an ircd backed by etcd

#13

Post by wahern » Thu, Oct 19, 2017, 2:50 AM UTC

takeda wrote: post #10

mbrock wrote: post #6

Earlier quoted context omitted.

Netsplits on IRC are such a visceral experience of distributed consensus failure. I remember as a little teenager on IRC being like "omg what is happening, why are there parallel worlds where weird things happen and everything drifts in and out?!"

I'm not convinced this is a solution to the problem though. As you mentioned with traditional IRC during actual netsplit (not single node disconnecting, but two large networks separating) you will see two networks that will get out of sync with time. In this case (assuming it is implemented correctly) the etcd will just shut down and not allow anyone use it until netsplit ends. Another issue here is that consensus pr…

Egalitarian Paxos (EPaxos) has much better WAN performance than previous protocols.

  https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf

Most real-world Paxos implementations, like Raft (what etcd uses), simply use Paxos for electing a leader. Afterward, all updates are serialized through the leader. This was a performance optimization, but it works well only on LANs; it sucks for WANs because you've at least doubled the round-trip time for every commit and the leader becomes a bottleneck, so it's especially important to have low latency.

EPaxos was published before Raft and I couldn't find any benchmarks comparing them directly, but in the paper above EPaxos outperforms Mencius (a rotating single leader design) even within a single EC2 cluster. And EPaxos maintains much more consistent performance during latency spikes. So EPaxos may help close the performance gap between LAN and WAN clusters. In the context of IRC where you're geographically distributed _anyhow_, something like EPaxos may impose no appreciable penalty at all, at least if there's sufficient redundancy to ensure a quorum.

Re: Etcdircd: an ircd backed by etcd

#14

Post by lomnakkus » Thu, Oct 19, 2017, 8:15 AM UTC

wahern wrote: post #13

takeda wrote: post #10

Earlier quoted context omitted.

I'm not convinced this is a solution to the problem though. As you mentioned with traditional IRC during actual netsplit (not single node disconnecting, but two large networks separating) you will see two networks that will get out of sync with time. In this case (assuming it is implemented correctly) the etcd will just shut down and not allow anyone use it until netsplit ends. Another issue here is that consensus pr…

Egalitarian Paxos (EPaxos) has much better WAN performance than previous protocols. https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf Most real-world Paxos implementations, like Raft (what etcd uses), simply use Paxos for electing a leader. Afterward, all updates are serialized through the leader. This was a performance optimization, but it works well only on LANs; it sucks for WANs because you've at least doubl…

> Most real-world Paxos implementations, like Raft (what etcd uses), simply use Paxos for electing a leader.

Ah, the ambiguity of natural language. Just to make this absolutely clear: Raft is not an implementation of Paxos. The "like" here refers to the "electing a leader" bit.

(I assume you already know this, it's just that the sentence as written is ambiguous and I wanted to clarify. I initially read it the wrong way and felt an urge to 'correct' you.)