Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The end of a myth: Distributed transactions can scale

muratbuffalo.blogspot.com

Post reply on HN
Print view View original

71–80 of 95 posts

Re: The end of a myth: Distributed transactions can scale

#71

Post by deepGem »

amitport wrote: post #34
deepGem wrote: post #27

Earlier quoted context omitted.

The atomic clock is the key to enable distributed transactions and Google has a proprietary lock on their atomic clocks for Spanner.

Do you know what is exactly proprietary in spanner? AFAIK most (all?) of the ideas existed before in the theoretical clock synchronization literature.

Nothing proprietary in Spanner but I believe no other vendor has an atomic clock similar to what Google has. Hence they are not able to implement the paxos global transaction lock which requires that all servers participating in the transaction be perfectly synchronized. This is what one of the comments above refers to I think.

So while the spanner paper is open to be implemented by any vendor, they don't have the proprietary advantage that Google has - the atomic clock. So Yugabyte, CockroachDB don't rely on atomic clocks. I tried to get to the ground level basics of this, but I haven't understood this matter completely yet.

Re: The end of a myth: Distributed transactions can scale

#72

Post by deepGem »

withinboredom wrote: post #32
deepGem wrote: post #27

Earlier quoted context omitted.

The atomic clock is the key to enable distributed transactions and Google has a proprietary lock on their atomic clocks for Spanner.

You need this for global consistency, but for logical local (like a single entity) this is unnecessary.

Yeah, you have a much better understanding of this topic than me for sure :)

Re: The end of a myth: Distributed transactions can scale

#73

Post by amitport »

deepGem wrote: post #71
amitport wrote: post #34

Earlier quoted context omitted.

Do you know what is exactly proprietary in spanner? AFAIK most (all?) of the ideas existed before in the theoretical clock synchronization literature.

Nothing proprietary in Spanner but I believe no other vendor has an atomic clock similar to what Google has. Hence they are not able to implement the paxos global transaction lock which requires that all servers participating in the transaction be perfectly synchronized. This is what one of the comments above refers to I think. So while the spanner paper is open to be implemented by any vendor, they don't have the pr…

I guess using worse clocks would mean using a (slightly?) slower spanner, but I'm not sure what is the impact. In any case, if a big vendor (e.g., Amazon IBM Oracle Dell...) would want something on par with Google's clock they probably can achieve it (though I don't know much about these clocks).

Re: The end of a myth: Distributed transactions can scale

#74

Post by hestefisk »

Interesting paper. That said, I find the image at the top of the article and the play on words in NAM-DB disturbing, considering the many atrocities and war crimes the US committed in the Vietnam War. The authors should have been more thoughtful given the usefulness of the paper on pure technical merits.

Re: The end of a myth: Distributed transactions can scale

#76

Post by lightbendover »

JackSlateur wrote: post #62
jeffreygoesto wrote: post #21

Earlier quoted context omitted.

Do you have indications why CPU bound? I have never seen Systems other than for example linear algebra or special algorithms to max out a modern CPU. Almost all loads today are in love way or another memory bound.

Bah Systems are either limited by CPU (processing) or by CPU (waiting for IO, especially memory) Systems limited by memory as in "quantity of" are scarse

Only scarce since it's an easy distributed problem to solve compared to IO.

Re: The end of a myth: Distributed transactions can scale

#77

Post by mikaelronstrom »

giovannibonetti wrote: post #57
_448 wrote: post #55

Earlier quoted context omitted.

Why not many are talking about RonDB? This is like Erlang of the database world! People at Ericsson were really very smart and ahead of their time. FoundationDB, TiKV, CitusData's Postgres extension are very well known, but RonDB looks like a hidden gem. What are the practical issues with RonDB that it is not widely known(or used?)?

Because RonDB is not a SQL database, it is just a KV store.

RonDB is a KV store with SQL capabilities (the MySQL storage engine NDB is maintained by Oracle as part of MySQL NDB Cluster). Hopsworks is on top of this adding a new REST API (REST + gRPC) service that is already in the github tree and will ready for production usage in a few months.

Re: The end of a myth: Distributed transactions can scale

#78

Post by di4na »

amitport wrote: post #73
deepGem wrote: post #71

Earlier quoted context omitted.

Nothing proprietary in Spanner but I believe no other vendor has an atomic clock similar to what Google has. Hence they are not able to implement the paxos global transaction lock which requires that all servers participating in the transaction be perfectly synchronized. This is what one of the comments above refers to I think. So while the spanner paper is open to be implemented by any vendor, they don't have the pr…

I guess using worse clocks would mean using a (slightly?) slower spanner, but I'm not sure what is the impact. In any case, if a big vendor (e.g., Amazon IBM Oracle Dell...) would want something on par with Google's clock they probably can achieve it (though I don't know much about these clocks).

The problem is that the slower it is the more the window for error grows. And the harder it is to recover. It also impose a high boundary to latency. You cannot be faster than your clock error without risks.

Note that even Spanner had multiple downtime due to clock and/or network failures. In these case, any operation lal guarantees are lost. This makes it really dangerous.

Re: The end of a myth: Distributed transactions can scale

#80

Post by juliangamble »

I’m using Transactions on a Distributed Database with Cassandra 4.0: https://medium.com/building-the-open-data-stack/an-apache-ca...

I’m curious to see what advantages this paper offers to everyday Cloud Developers over using the latest version of Cassandra.

Post reply on HN