Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NewSQL databases fail to guarantee consistency and I blame Spanner

dbmsmusings.blogspot.com

Post reply on HN
Print view View original

31–40 of 319 posts

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#31

Post by vidarh »

felixgallo wrote: post #28
abadid wrote: post #2

I'm the author of that post. I'm happy to respond to comments on the post on this thread for the next several hours. You can also leave comments on the post itself, and I will respond there at any time.

why didn't you disclose your connections in your post? It appears you've written much the same article for FaunaDB's official blog in the past ( https://fauna.com/blog/distributed-consistency-at-scale-span... ).

The article specifically states that Calvin comes out of his research group and that FaunaDB was inspired by Calvin.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#32

Post by gregwebs »

In the post and another comment here it is stated that Calvin can handle any real-world workload. However, according to my reading of the Calvin paper, one must understand the update keys before starting the transaction. I also experienced limitations when trying to use FaunaDB: it doesn't support ad-hoc queries and it only allows for indexed queries.

I really like the Calvin protocol, and is does seem perfectly suited to many application workloads, but it is odd to me though, to see Calvin presented as purely superior to all alternatives. It seems like more research and work needs to be done to create systems that can address its shortcomings around transactions that query for keys (3.2.1 Dependent transactions in the paper), including ad-hoc queries, and even interactive queries (with a transaction).

Disclaimer: I work on TiDB/TiKV. TiKV (distributed K/V store) uses the percolator model for transactions to ensure linearizability, but TiDB SQL (built on top) allows write skew.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#34

Post by jrockway »

tytso wrote: post #25

The post talks about Spanner using "a specialized hardware solution that uses both GPS and atomic clocks to ensure a minimal clock skew across servers." But what it fails to mention is that is that this time is distributed using a software solution --- NTP, in fact. Google makes its "leap-smeared NTP network" available via Google's Public NTP service. And it's not expensive for someone to buy their own GPS clock and…

I agree. Attempting to remove the GPS and atomic clocks from the system seems like a premature optimization to me. They are cheap and the ability to order events according to an actual wall clock is a huge benefit.

Some basics for less-critical applications: https://github.com/jrockway/beaglebone-gps-clock/blob/master...

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#35

Post by sabujp »

romed wrote: post #29

I’ve seen comments on HN over the years in which someone Dunning-Kruegers their way into saying that TrueTime is easily replicated. I always wonder if they have sixteen senior SREs in their pocket, because that’s the level of production engineering Google applies to the problem. Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up. If yo…

the only solution to this is to launch your own satellites :)

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#36

Post by CydeWeys »

mehrdadn wrote: post #20
abadid wrote: post #2

I'm the author of that post. I'm happy to respond to comments on the post on this thread for the next several hours. You can also leave comments on the post itself, and I will respond there at any time.

The problem isn't a lack of very accurate clocks per se, right? What you need is an accurate bound on clock error, whatever it might be. It sounds to me that non-Googles are specifying bound parameters that, unlike Google, they cannot guarantee. Doesn't that mean the blame goes to whomever put inaccurate bound parameters into the software? Why blame the algorithm for garbage input?

The problem is that the bound on clock error directly affects your performance. So if you're willing to accept, say, a second in clock error, then all transactions will take a minimum of one second. That level of performance is going to be unacceptable in many situations.

The potential clock error on VMs without dedicated time-keeping hardware is so large that performance turns into absolute garbage.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#37

Post by Cshelton »

romed wrote: post #29

I’ve seen comments on HN over the years in which someone Dunning-Kruegers their way into saying that TrueTime is easily replicated. I always wonder if they have sixteen senior SREs in their pocket, because that’s the level of production engineering Google applies to the problem. Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up. If yo…

Also, I believe all the major cloud providers provide a "TrueTime" API service. I forgot the name that AWS uses, but you can call it on your EC2 instances and make sure your hosts are all in sync. It's pretty cool.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#38

Post by xapata »

dgreensp wrote: post #21

This is the clearest presentation of why CAP is misleading that I’ve ever seen! Wonderful. If the app programmer knows they don’t have global consistency, just consistency per partition, I wonder if in practice there are ways to achieve the necessary application-level guarantees such as in the photo-sharing case (not that it sounds that appealing to need to do so).

> application-level guarantees

If your database doesn't "guarantee" consistency, but instead provides eventual consistency, your application needs to be aware of how long "eventually" is. In effect, you'd be coding something similar to Spanner's timestamp uncertainty estimation. In the photo sharing example, it'd be reasonable to put a delay on sharing new photos with other users until after permissions changes would have propagated.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#40

Post by pdpi »

jessaustin wrote: post #19
haberman wrote: post #13

Earlier quoted context omitted.

Without some description of what "false claims" and "misinterpretation" you mean, this is a content-free post.

It has more "content" than the post to which it replies. "This is good." "No, it is bad, in these particular areas." [EDIT:] Granted, the post could still be wrong, but ISTM accusations of content-freedom tend to be projections.

"This is a good read" is a statement of opinion. You're free to assign whatever value you like to the op's opinion, but that opinion is "content" in and of itself.

"The article has false claim" is a statement of fact. A vague statement of fact about a non-obvious topic that puts no effort into providing concrete examples and reasoning is pretty much worthless.

Post reply on HN