A Primer on Database Replication (2017) • Hacker News Board

Re: A Primer on Database Replication (2017)

#11

Post by zzzcpan » Sun, Jan 06, 2019, 3:32 PM UTC

hardwaresofton wrote: post #5

This is a pretty much complete introduction to database replication techniques. If you're more interested in the quorum-ing tech that's come about (algorithms like paxos and raft) and the underlying distributed systems research I've recently tried to compile all the paxos family of algorithms in a blog post[0]. There has been a lot of work recently (notably EPaxos, WPaxos, SDPaxos) on tuning and improving the algorit…

> https://vadosware.io/post/paxosmon-gotta-concensus-them-all

This is pretty good non-FUD post about distributed systems (unlike many posts related to consistency and consensus coming from distributed database startups and megacorps).

Although missing some important real world considerations in reasoning, namely latency in WAN setups (reducing number of RTTs isn't really about that).

Re: A Primer on Database Replication (2017)

#12

Post by zzzcpan » Sun, Jan 06, 2019, 3:40 PM UTC

ansible wrote: post #4

There are solutions to this now: https://cockroachlabs.com https://foundationdb.org Of course, there are still tradeoffs, but for many applications, a distributed database with strong consistency guarantees is likely to be a good choice for many projects.

The tradeoffs those solutions make are exactly the ones most projects cannot really make.

Re: A Primer on Database Replication (2017)

#13

Post by hardwaresofton » Mon, Jan 07, 2019, 12:53 AM UTC

zzzcpan wrote: post #11

hardwaresofton wrote: post #5

This is a pretty much complete introduction to database replication techniques. If you're more interested in the quorum-ing tech that's come about (algorithms like paxos and raft) and the underlying distributed systems research I've recently tried to compile all the paxos family of algorithms in a blog post[0]. There has been a lot of work recently (notably EPaxos, WPaxos, SDPaxos) on tuning and improving the algorit…

> https://vadosware.io/post/paxosmon-gotta-concensus-them-all This is pretty good non-FUD post about distributed systems (unlike many posts related to consistency and consensus coming from distributed database startups and megacorps). Although missing some important real world considerations in reasoning, namely latency in WAN setups (reducing number of RTTs isn't really about that).

I'm not sure I agree -- reducing the number of round trips is definitely important in reducing latency for the system in the WAN context, a quote from the SDPaxos paper:

> In the wide area, latency is dominated by network communication,which is decided by the number of round trips, and the distance tothe replica to contact. The test for Multi-Paxos is omitted becauseits disadvantage is obvious: client has to communicate with theremote leader, as long as it is not co-located with the leader.The replicas and clients for wide-area experiments are deployedin California (CA), Oregon (OR), Ohio (OH), Ireland (IRE) and Seoul(SEL). The sequencer of SDPaxos locates in CA. The round-triptimes (ping latencies) between these regions are shown in Table 1.

I think this case is even worse than most real world cases people are dealing with today. Google/Facebook might have data centers this spread out, but I feel like 90% of the people who read that post will be worried about multiple regions in America.