[deleted]
NewSQL databases fail to guarantee consistency and I blame Spanner
81–90 of 319 posts
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#82Earlier quoted context omitted.
So yes, Google can be very impressed with Google. But I'm not sure that's the issue here. Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't? I think it's a very common pattern that a product has some set of trailblazer users who find issues before the people who make the product. Also, I think you're over-i…
> Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't If GPS timing is bad, a lot of people will notice that their position on the map is incorrect, because that's the whole purpose of the GPS network. A 1 microsecond error is 300 meters.
GPS is not typically used to confirm a position that is known accurately by other means, and that is not its purpose. Only in those cases where there is a manifest conflict with independent spatial information will the problem be evident.
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#83The post talks about Spanner using "a specialized hardware solution that uses both GPS and atomic clocks to ensure a minimal clock skew across servers." But what it fails to mention is that is that this time is distributed using a software solution --- NTP, in fact. Google makes its "leap-smeared NTP network" available via Google's Public NTP service. And it's not expensive for someone to buy their own GPS clock and…
I feel like the author is making a huge omission by not talking about the properties of multi-raft sharded DBs coupled with the TrueTime-like APIs from cloud providers in 2018. He links to a post from the CEO of CockroachDB written in early 2016, and Amazon launched their TimeSync service in 2017. It's likely that Spencer would have something different to say about the matter today.
> I feel like the author is making a huge omission by not talking about the properties of multi-raft sharded DBs coupled with the TrueTime-like APIs from cloud providers in 2018.
I mean this is all fine and dandy if it works (plus, how do know that it actually works in all the edge cases?), but that's a HUGE amount of complexity. IMO you really need to have a very good reason to even consider anything this complex.
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#84> Systems that guarantee consistency only experience a necessary reduction in availability in the event of a network partition. As networks become more redundant, partitions become an increasingly rare event. And even if there is a partition, it is still possible for the majority partition to be available In my experience, yes network partitions are incredibly rare. However 99% of my distributed ststem partitions hav…
Losing a majority due to various faults is not equivalent to a network partition. In practice for the systems you are working with it may have a similar effect (or a much worse one!), but in theory it is possible to recover from these faults in a short amount of time as long as there are no network disruptions.
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#85In the post and another comment here it is stated that Calvin can handle any real-world workload. However, according to my reading of the Calvin paper, one must understand the update keys before starting the transaction. I also experienced limitations when trying to use FaunaDB: it doesn't support ad-hoc queries and it only allows for indexed queries. I really like the Calvin protocol, and is does seem perfectly suit…
Support for dependent reads is a subject that the Calvin paper only touches on briefly, describing one strategy of using "reconnaissance reads" to determine the key set.
In FaunaDB, this is formalized as an optimistic concurrency control mechanism that combines snapshot reads at the coordinator and read-write conflict detection within the transaction engine. By the time a transaction is ready to commit the entire key set is known. I wrote a blog post that goes into more detail here: https://fauna.com/blog/acid-transactions-in-a-globally-distr...
Better analytics support, including ad-hoc queries is on our roadmap. That being said, requiring indexes was a design choice: First and foremost, we want FaunaDB to be the best operational database. It is a lot easier to write fast, predictable queries if they cannot fall back to a table scan!
While FaunaDB requires you to create indexes to support the views your workload requires, the reward is a clearer understanding of how queries perform. We've built indexes to be pretty low commitment, however. You can add or drop them on the fly based on how your workload evolves over time.
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#86I'm the author of that post. I'm happy to respond to comments on the post on this thread for the next several hours. You can also leave comments on the post itself, and I will respond there at any time.
"Systems that guarantee consistency only experience a necessary reduction in availability in the event of a network partition." Many of the distributed clusters I've maintained had crap infrastructure and no change control, and parts of the clusters were constantly going down from lack of storage, CPU and RAM, or bad changes. The only reason the applications kept working were either (1) the not-broken vnodes continue…
It seems you have other problems (crap infrastructure and no change control) to deal with before the issues in this article become your biggest concern, but are not the cases you list themselves partition problems?
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#87Earlier quoted context omitted.
> Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't If GPS timing is bad, a lot of people will notice that their position on the map is incorrect, because that's the whole purpose of the GPS network. A 1 microsecond error is 300 meters.
GPS is not typically used to confirm a position that is known accurately by other means, and that is not its purpose. Only in those cases where there is a manifest conflict with independent spatial information will the problem be evident.
People are going to notice a 300m deviation due to landmarks and their eyes.
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#88Earlier quoted context omitted.
The problem isn't a lack of very accurate clocks per se, right? What you need is an accurate bound on clock error, whatever it might be. It sounds to me that non-Googles are specifying bound parameters that, unlike Google, they cannot guarantee. Doesn't that mean the blame goes to whomever put inaccurate bound parameters into the software? Why blame the algorithm for garbage input?
The problem is that the bound on clock error directly affects your performance. So if you're willing to accept, say, a second in clock error, then all transactions will take a minimum of one second. That level of performance is going to be unacceptable in many situations. The potential clock error on VMs without dedicated time-keeping hardware is so large that performance turns into absolute garbage.
I would understand if the complaint was that Spanner is too slow without expensively accurate clocks and synchronization. But the complaint is that Spanner fails to guarantee consistency, which doesn't make sense to me. The requirements clearly include giving a valid clock bound, so if you give an invalid clock bound, it's clearly your fault for getting incorrect results, not Spanner's!
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#89Any comments on how FoundationDB might be the same / different?
Re: NewSQL databases fail to guarantee consistency and I blame Spanner
#90Earlier quoted context omitted.
I've always had an issue with systems assuming a universal time, because physics (Special Relativity) tells us there is no such thing. Two events in different places will be viewed as having a different order depending on your frame of reference. All that really matters on a physical level is causal connections. I believe vector clocks capture this semantic. But they have other trade-offs.
Can't you solve that by just picking a reference frame though?
You need a quite elaborate system of a grid of synchronized, stationary clocks and rods to correctly define a reference frame, which is worse than what's required to come up with a "good enough on earth to some time resolution" true time.