Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

cockroachlabs.com

Post reply on HN
Print view View original

21–29 of 29 posts

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#21

Post by cpursley »

legojoey17 wrote: post #20
ahahuaya wrote: post #8

Cockroachdb looks very exciting. I’m waiting for it’s Postgres compatibility to be far enough along that it works with Elixir/Phoenix. Is that likely to happen any time soon?

If you are willing to give it a shot, there is currently a fork of postgrex from someone in the Elixir community! It handles some of rough edges of the incompatibility. You can find it at https://hexdocs.pm/postgrex_cdb/readme.html and the source code at https://github.com/jumpn/postgrex . For any other issues you run into when using it, you may want to see the discussion about postgrex compatibility at https://githu…

I'd love to see a port of https://github.com/begriffs/postgrest to Elixir. Cockroachdb + Elixir-based Postgrest implementation would be the ultimate platform for realtime distributed web scale.

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#23

Post by zokier »

jinqueeny wrote: post #19
muxator wrote: post #5

I always longed for the existence of such an Ops-friendly DB, which - building on solid distributed systems concepts - could make easy and correct the hard things (consistency, geographical replication, effective possibility of zero downtime). Now that I have a good candidate, I find myself wondering whether the performances would be good enough to ditch traditional DBs (I know: I should define "workload" before thin…

Another good candidate is TiDB ( https://github.com/pingcap/tidb ). It has elastic scalability, ACID compliances, high availability, etc. At least, TiDB, CRDB, RethinkDB are open source :)

> At least, TiDB, CRDB, RethinkDB are open source :)

So is CockroachDB too?

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#24

Post by a-robinson »

zokier wrote: post #23
jinqueeny wrote: post #19

Earlier quoted context omitted.

Another good candidate is TiDB ( https://github.com/pingcap/tidb ). It has elastic scalability, ACID compliances, high availability, etc. At least, TiDB, CRDB, RethinkDB are open source :)

> At least, TiDB, CRDB, RethinkDB are open source :) So is CockroachDB too?

Yes: https://github.com/cockroachdb/cockroach

The parent poster was referring to CockroachDB as CRDB.

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#25

Post by jinqueeny »

a-robinson wrote: post #24
zokier wrote: post #23

Earlier quoted context omitted.

> At least, TiDB, CRDB, RethinkDB are open source :) So is CockroachDB too?

Yes: https://github.com/cockroachdb/cockroach The parent poster was referring to CockroachDB as CRDB.

Yep, thanks for your clarification!

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#26

Post by a-robinson »

toolslive wrote: post #22

Isn't "Follow-the-workload" something that is naturally achieved when Mencius is used iso Raft?

WAN-optimized consensus protocols like Mencius and Egalitarian Paxos indeed are better for writes because any node can commit a write in one round trip, but they don't allow for reads with zero round trips. Every read must go through the consensus machinery in order to ensure consistency, whereas in CockroachDB the leaseholder can serve reads without needing to consult other nodes.

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#27

Post by toolslive »

a-robinson wrote: post #26
toolslive wrote: post #22

Isn't "Follow-the-workload" something that is naturally achieved when Mencius is used iso Raft?

WAN-optimized consensus protocols like Mencius and Egalitarian Paxos indeed are better for writes because any node can commit a write in one round trip, but they don't allow for reads with zero round trips. Every read must go through the consensus machinery in order to ensure consistency, whereas in CockroachDB the leaseholder can serve reads without needing to consult other nodes.

why do reads have to go through the consensus machinery ? If they don't go through the machinery, read results will still be consistent, they just might not include the very last updates, but you cannot avoid that. Raft has the same problem (they all have, and it's unavoidable): suppose you read from the Leader (Master, whatever you call it); the leader sends you a perfect fresh consistent value; however, your network takes time to deliver it to you and meanwhile an update might have happened. It just shows that the key value store should provide atomic multi updates (or at least compare-and-swap )

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#28

Post by cube2222 »

toolslive wrote: post #27
a-robinson wrote: post #26

Earlier quoted context omitted.

WAN-optimized consensus protocols like Mencius and Egalitarian Paxos indeed are better for writes because any node can commit a write in one round trip, but they don't allow for reads with zero round trips. Every read must go through the consensus machinery in order to ensure consistency, whereas in CockroachDB the leaseholder can serve reads without needing to consult other nodes.

why do reads have to go through the consensus machinery ? If they don't go through the machinery, read results will still be consistent, they just might not include the very last updates, but you cannot avoid that. Raft has the same problem (they all have, and it's unavoidable): suppose you read from the Leader (Master, whatever you call it); the leader sends you a perfect fresh consistent value; however, your networ…

Without the consensus read you can get inconsistent values.

Suppose you write value a to leader A and then value b to leader A. Node B got both updates, node C got only the a value because of network timeouts.

Now you do a read from B of both values, you get the current a and b. Then you do a read from C, you receive the current a and the old b. Inconsistent.

Basically a violation of serializability and linearizability across the cluster in respect to the client.

Re: “Follow-The-Workload” Beats the Latency-Survivability Tradeoff in CockroachDB

#29

Post by toolslive »

cube2222 wrote: post #28
toolslive wrote: post #27

Earlier quoted context omitted.

why do reads have to go through the consensus machinery ? If they don't go through the machinery, read results will still be consistent, they just might not include the very last updates, but you cannot avoid that. Raft has the same problem (they all have, and it's unavoidable): suppose you read from the Leader (Master, whatever you call it); the leader sends you a perfect fresh consistent value; however, your networ…

Without the consensus read you can get inconsistent values. Suppose you write value a to leader A and then value b to leader A. Node B got both updates, node C got only the a value because of network timeouts. Now you do a read from B of both values, you get the current a and b. Then you do a read from C, you receive the current a and the old b. Inconsistent. Basically a violation of serializability and linearizabili…

No. In your example, value 'a' will not have been applied to the store in C yet, as there is a hole in the log: a set of updates that contains update 'b' (and possibly others). So the view in A will be more up to date than the view in C, but both will be consistent.
Post reply on HN