Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NewSQL databases fail to guarantee consistency and I blame Spanner

dbmsmusings.blogspot.com

Post reply on HN
Print view View original

91–100 of 319 posts

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#91

Post by xenadu02 »

notatcomputer68 wrote: post #67
ianopolous wrote: post #64

Earlier quoted context omitted.

I've always had an issue with systems assuming a universal time, because physics (Special Relativity) tells us there is no such thing. Two events in different places will be viewed as having a different order depending on your frame of reference. All that really matters on a physical level is causal connections. I believe vector clocks capture this semantic. But they have other trade-offs.

Can't you solve that by just picking a reference frame though?

Yes, and we all pick the Earth’s gravity well as our reference frame. Differences due to elevation and latitude are too small to measure for purposes of transaction clocks.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#92

Post by jonas21 »

notatcomputer68 wrote: post #67
ianopolous wrote: post #64

Earlier quoted context omitted.

I've always had an issue with systems assuming a universal time, because physics (Special Relativity) tells us there is no such thing. Two events in different places will be viewed as having a different order depending on your frame of reference. All that really matters on a physical level is causal connections. I believe vector clocks capture this semantic. But they have other trade-offs.

Can't you solve that by just picking a reference frame though?

yeah, I’m pretty sure TrueTime is only defined on Earth, and GPS satellites do take relativity into account in their design.

http://www.astronomy.ohio-state.edu/~pogge/Ast162/Unit5/gps....

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#93

Post by antoncohen »

mr_pickles wrote: post #55

No mention of Clustrix? The Spanner paper was in 2012. Clustrix started working on a "NewSQL" database in 2006 and had a product out in 2010.

One big difference is that Clustrix (YC'06) was a single datacenter consistent cluster, while Spanner can do multi-region consistency. It will be interesting to see what MariaDB does with Clustrix, now that they have acquired them, fingers crossed that they open source the technology.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#94

Post by dsp1234 »

Twirrim wrote: post #68
abadid wrote: post #27

Earlier quoted context omitted.

Thanks. I added a parenthetical remark to the post to indicate that I was talking about DynamoDB's default settings.

It's not really the default settings, per se. You don't have to change any bit of configuration about your database to get consistency. The DynamoDB API gives you the GetItem API call and a boolean property to choose to make it a consistent read. It's left as a very simple task for developers leveraging DynamoDB to make the appropriate trade offs on consistent or inconsistent read. source: Used to work for AWS on a s…

From the GetItem docs[0]

"GetItem provides an eventually consistent read by default."

This seems to meet the definition of "DynamoDB's default settings"

[0] - https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen...

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#95

Post by wsy »

reinhardt1053 wrote: post #61
dekhn wrote: post #14

This reads like an advertisement for the researcher's system combined with some FUD.

would you mind to elaborate?

Just to mention a few points:

- The title implies that Spanner is at fault for something, then the article itself admits that Spanner is totally fine.

- Then it goes on to argue that this approach can't possibly work without physical devices for precise time. However, there is no proof whatsoever. Please be a researcher and design an experiment that demonstrates a violated consistency guarantee.

- Finally, not disclosing financial relationships with products which are praised in the article is really low, and has only been fixed after this has been pointed out here. And it is not marked as a later addendum, as usual for changed articles.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#96

Post by abadid »

reilly3000 wrote: post #70
abadid wrote: post #2

I'm the author of that post. I'm happy to respond to comments on the post on this thread for the next several hours. You can also leave comments on the post itself, and I will respond there at any time.

Is there a "best-of-both-worlds" approach that could work, or are these two approaches mutually exclusive? I have to imagine that time drift can eventually reconciled with some kind of time delta.

The two approaches seem mutually exclusive to me.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#97

Post by sneak »

Quekid5 wrote: post #78
nemothekid wrote: post #48

> Systems that guarantee consistency only experience a necessary reduction in availability in the event of a network partition. As networks become more redundant, partitions become an increasingly rare event. And even if there is a partition, it is still possible for the majority partition to be available In my experience, yes network partitions are incredibly rare. However 99% of my distributed ststem partitions hav…

I may be a bit of an old fart, but this is the exact reasoning behind my decision to never go with "distributed X" if there's a "single-machine X" where you can just vertically scale. If you can afford 3-5 machines/VMs for a cluster you can almost certainly afford a single machine/VM with 2-4x the resources/CPU and chances are that it'll perform just as well (or better) because it doesn't have network latency to cont…

This is good and sound advice. The logical outgrowth of this is that the vast majority of organizations don’t need to use distributed databases. (Those that do should probably opt for hosted ones first, and then maybe consider running their own if they have the explicit need, and the substantial SRE budget for it.)

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#98

Post by gregwebs »

freels wrote: post #85
gregwebs wrote: post #32

In the post and another comment here it is stated that Calvin can handle any real-world workload. However, according to my reading of the Calvin paper, one must understand the update keys before starting the transaction. I also experienced limitations when trying to use FaunaDB: it doesn't support ad-hoc queries and it only allows for indexed queries. I really like the Calvin protocol, and is does seem perfectly suit…

Support for dependent reads is a subject that the Calvin paper only touches on briefly, describing one strategy of using "reconnaissance reads" to determine the key set. In FaunaDB, this is formalized as an optimistic concurrency control mechanism that combines snapshot reads at the coordinator and read-write conflict detection within the transaction engine. By the time a transaction is ready to commit the entire key…

Thanks for the link to the explanation. It seems like FaunaDB is doing the hard work to show how Calvin can actually be used for all types of workloads.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#99

Post by jessaustin »

pdpi wrote: post #40
jessaustin wrote: post #19

Earlier quoted context omitted.

It has more "content" than the post to which it replies. "This is good." "No, it is bad, in these particular areas." [EDIT:] Granted, the post could still be wrong, but ISTM accusations of content-freedom tend to be projections.

"This is a good read" is a statement of opinion. You're free to assign whatever value you like to the op's opinion, but that opinion is "content" in and of itself. "The article has false claim" is a statement of fact. A vague statement of fact about a non-obvious topic that puts no effort into providing concrete examples and reasoning is pretty much worthless.

Adding qualifiers makes a statement have less content? Please try to imagine using this questionable argument in any other context...

It's a controversial topic and TFA was written by someone with a commercial interest. Everything is opinion. The level of comment-policing I see here makes me quite suspicious of everything in TFA.

[EDIT:] aargh I'll never learn every discussion that starts with "content-free" cliche goes nowhere... just downvote and move on...

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#100

Post by hinkley »

romed wrote: post #29

I’ve seen comments on HN over the years in which someone Dunning-Kruegers their way into saying that TrueTime is easily replicated. I always wonder if they have sixteen senior SREs in their pocket, because that’s the level of production engineering Google applies to the problem. Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up. If yo…

I’m sure someone has talked about this and proven or disproven it, but I’ve always been a little uneasy with the truetime protocol. I mean it’s faster than Raft, but for the use case maybe we are trying to fix the wrong problem?

If two events are independent, it matters very little what order we record them in the system of record.

My whole career we have been building cause and effect at transaction time but when we debug we stare at log files and time stamps like we are reading tea leaves, trying to figure out what situation A led to corrupted data in row B.

Maybe there’s a way to record this stuff instead of time stamps? Something DVCS style. Or maybe it’s provably intractable.

Post reply on HN