NewSQL databases fail to guarantee consistency and I blame Spanner • Hacker News Board

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#101

Post by toast0 » Fri, Sep 21, 2018, 4:11 PM UTC

Quekid5 wrote: post #78

nemothekid wrote: post #48

> Systems that guarantee consistency only experience a necessary reduction in availability in the event of a network partition. As networks become more redundant, partitions become an increasingly rare event. And even if there is a partition, it is still possible for the majority partition to be available In my experience, yes network partitions are incredibly rare. However 99% of my distributed ststem partitions hav…

I may be a bit of an old fart, but this is the exact reasoning behind my decision to never go with "distributed X" if there's a "single-machine X" where you can just vertically scale. If you can afford 3-5 machines/VMs for a cluster you can almost certainly afford a single machine/VM with 2-4x the resources/CPU and chances are that it'll perform just as well (or better) because it doesn't have network latency to cont…

I think part of this is that most of the common knowledge about scaling is hard fought from the 90s/2000 era. eBay got bigger and bigger Sun boxes to run Oracle, until they couldn't get anything bigger -- then they had a problem and had to shard their listings into categories, etc. In the last few Intel cpu generations, computation performance has had small gains, but addressable memory has doubled about every other release, you can now get machines with 2TB of ram, and it's not that expensive.

You can fit a lot of data in an in memory database of 2TB.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#102

Post by Quekid5 » Fri, Sep 21, 2018, 4:14 PM UTC

toast0 wrote: post #101

Quekid5 wrote: post #78

Earlier quoted context omitted.

I may be a bit of an old fart, but this is the exact reasoning behind my decision to never go with "distributed X" if there's a "single-machine X" where you can just vertically scale. If you can afford 3-5 machines/VMs for a cluster you can almost certainly afford a single machine/VM with 2-4x the resources/CPU and chances are that it'll perform just as well (or better) because it doesn't have network latency to cont…

I think part of this is that most of the common knowledge about scaling is hard fought from the 90s/2000 era. eBay got bigger and bigger Sun boxes to run Oracle, until they couldn't get anything bigger -- then they had a problem and had to shard their listings into categories, etc. In the last few Intel cpu generations, computation performance has had small gains, but addressable memory has doubled about every other…

That's a really good observation and I agree. I sometimes have to pinch myself to make sure I'm not dreaming when I see that there are actually pretty affordable machines with these insane amounts of RAM. (I think the first machine I personally[0] owned had 512K or something like that.)

[0] Well, my dad bought it, but y'know. He wasn't particularly interested, but I think he recognized an interest in computers in me :).

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#103

Post by freels » Fri, Sep 21, 2018, 4:17 PM UTC

gregwebs wrote: post #98

freels wrote: post #85

Earlier quoted context omitted.

Support for dependent reads is a subject that the Calvin paper only touches on briefly, describing one strategy of using "reconnaissance reads" to determine the key set. In FaunaDB, this is formalized as an optimistic concurrency control mechanism that combines snapshot reads at the coordinator and read-write conflict detection within the transaction engine. By the time a transaction is ready to commit the entire key…

Thanks for the link to the explanation. It seems like FaunaDB is doing the hard work to show how Calvin can actually be used for all types of workloads.

You're welcome! I've enjoyed following TiKV/TiDB's development as well. It is an interesting time for databases. :-)

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#104

Post by abadid » Fri, Sep 21, 2018, 4:18 PM UTC

haberman wrote: post #26

abadid wrote: post #16

Earlier quoted context omitted.

By controversial, I mean that in the database research community (where I spend most of my time), there is significant disagreement about global consensus vs. Spanner's choice of partitioned consensus.

Disagreement about what though? Does Spanner's solution have an objective problem? Do you or others in your community have specific reasons to believe that it cannot deliver on its promises?

Spanner's approach requires help from hardware and several full time employees maintaining and ensuring the uncertainty guarantees. This increases the cost of the maintaining the system, which for Cloud Spanner is partially passed on to the end users. If you can build a system that doesn't require time synchronization, yet doesn't have any significant drawbacks relative to what Spanner provides, you'd be better off using this alternative system.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#105

Post by hinkley » Fri, Sep 21, 2018, 4:18 PM UTC

notatcomputer68 wrote: post #67

ianopolous wrote: post #64

Earlier quoted context omitted.

I've always had an issue with systems assuming a universal time, because physics (Special Relativity) tells us there is no such thing. Two events in different places will be viewed as having a different order depending on your frame of reference. All that really matters on a physical level is causal connections. I believe vector clocks capture this semantic. But they have other trade-offs.

Can't you solve that by just picking a reference frame though?

HTTP has had a reference frame since 1.0. It’s still one of my favorite features of just about any protocol ever.

Cache headers can set expiration times or offsets, but the client and server both send their own time stamp in the message. So when the server says “expire this at noon” but your laptop’s clock is five minutes slow or three time zones away, you can still figure out what the server meant by noon because the server tells you what time it thinks it is now. The accuracy is limited by network delays but for the problem space getting the right answer to the nearest second is still pretty damned good.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#106

Post by amluto » Fri, Sep 21, 2018, 4:21 PM UTC

ianopolous wrote: post #64

romed wrote: post #29

I’ve seen comments on HN over the years in which someone Dunning-Kruegers their way into saying that TrueTime is easily replicated. I always wonder if they have sixteen senior SREs in their pocket, because that’s the level of production engineering Google applies to the problem. Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up. If yo…

I've always had an issue with systems assuming a universal time, because physics (Special Relativity) tells us there is no such thing. Two events in different places will be viewed as having a different order depending on your frame of reference. All that really matters on a physical level is causal connections. I believe vector clocks capture this semantic. But they have other trade-offs.

Special relativity has no such problem. In SR you can easily define a “time” coordinate everywhere such that all events can be timestamped with that coordinate and will respect causality.

GR at least usually has this property as well. (It doesn’t in the presence of closed timelike curves. It does in weak gravity and in the FLRW metric in cosmology. I’m not sure about the general strong gravity case.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#107

Post by apoverton » Fri, Sep 21, 2018, 4:21 PM UTC

hinkley wrote: post #100

romed wrote: post #29

I’ve seen comments on HN over the years in which someone Dunning-Kruegers their way into saying that TrueTime is easily replicated. I always wonder if they have sixteen senior SREs in their pocket, because that’s the level of production engineering Google applies to the problem. Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up. If yo…

I’m sure someone has talked about this and proven or disproven it, but I’ve always been a little uneasy with the truetime protocol. I mean it’s faster than Raft, but for the use case maybe we are trying to fix the wrong problem? If two events are independent, it matters very little what order we record them in the system of record. My whole career we have been building cause and effect at transaction time but when we…

I think if 2 events are independent then you can use an eventually consistent DB like Cassandra and move on with your day. The importance of something like TrueTime arises when you have an example like in the article. Step 1: remove your parents from seeing your photo albums. Step 2: post your spring break photo album

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#108

Post by mannykannot » Fri, Sep 21, 2018, 4:21 PM UTC

Rapzid wrote: post #87

mannykannot wrote: post #82

Earlier quoted context omitted.

GPS is not typically used to confirm a position that is known accurately by other means, and that is not its purpose. Only in those cases where there is a manifest conflict with independent spatial information will the problem be evident.

People are going to notice a 300m deviation due to landmarks and their eyes.

Rarely, if you are navigating at sea or in the air or in the woods... and even on the road, it is not uncommon for my GPS device to be clearly off without justifying the conclusion that there is a fault in a satellite.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#109

Post by sseth » Fri, Sep 21, 2018, 4:23 PM UTC

hinkley wrote: post #100

romed wrote: post #29

I’ve seen comments on HN over the years in which someone Dunning-Kruegers their way into saying that TrueTime is easily replicated. I always wonder if they have sixteen senior SREs in their pocket, because that’s the level of production engineering Google applies to the problem. Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up. If yo…

I’m sure someone has talked about this and proven or disproven it, but I’ve always been a little uneasy with the truetime protocol. I mean it’s faster than Raft, but for the use case maybe we are trying to fix the wrong problem? If two events are independent, it matters very little what order we record them in the system of record. My whole career we have been building cause and effect at transaction time but when we…

TrueTime is not a timestamp, but a time with a spread which reflects the maximum difference in time between two data centers.

If i understand correctly, if the size of this increases too much, this brings down the number of transactions per second which can be done, so keeping the spread small is key. Hence the atomic clocks etc.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#110

Post by jules » Fri, Sep 21, 2018, 4:23 PM UTC

The CAP theorem has been truly disastrous for databases. The CAP theorem simply says that if you have a database on 2 servers and the connection between those serves goes down, then queries against one server don't see new updates from the other server, so you either have to give up consistency (serve stale data) or give up availability (one of the servers refuses to process further requests). That's all that CAP is, but somehow half the industry has been convinced that slapping a fancy name on this concept is a justification for giving up consistency (even when the network and all servers are fully functional) to retain availability. The A for availability in CAP means that ALL database servers are fully available, which is unnecessary in practice, because clients can switch to the other servers. Giving up consistency introduces big engineering challenges. You're getting something that most people don't need in return for a large cost.