Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NewSQL databases fail to guarantee consistency and I blame Spanner

dbmsmusings.blogspot.com

Post reply on HN
Print view View original

171–180 of 319 posts

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#171

Post by zzzcpan »

jshen wrote: post #152
jules wrote: post #110

The CAP theorem has been truly disastrous for databases. The CAP theorem simply says that if you have a database on 2 servers and the connection between those serves goes down, then queries against one server don't see new updates from the other server, so you either have to give up consistency (serve stale data) or give up availability (one of the servers refuses to process further requests). That's all that CAP is,…

"The A for availability in CAP means that ALL database servers are fully available" Is this true? I always thought it meant that clients could continue to read and write to "the database" which could include the client switching to another node. There is nothing in CAP theorem about latency, so switching, even if it adds high latency, is fine by CAP theorem. This lack of accounting for latency is what makes CAP theor…

From implementation point of view networks are asynchronous and therefore are always partitioned. We don't actually have a luxury of arbitrary CAP interpretations, we can't know whether other nodes are available or not at any given moment. So instead we have to make requests to other nodes and always choose either to wait for responses (or more complex communications to achieve consensus) and get C or not wait for anything and get A, although each node can be a bit behind on updates from other nodes. Thus the CAP choices are pretty much about latency: waiting for globally visible updates vs not waiting and getting low latency. Both can be mixed in various proportions to get consistency with good latency, but still reasonable tolerance of byzantine failures.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#172

Post by [deleted] »

hosay123 wrote: post #60
wpietri wrote: post #52

Earlier quoted context omitted.

So yes, Google can be very impressed with Google. But I'm not sure that's the issue here. Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't? I think it's a very common pattern that a product has some set of trailblazer users who find issues before the people who make the product. Also, I think you're over-i…

If some condition existed that exceeded GPS intended design, you most certainly wouldn't learn of it first from some random anecdote on HN.. more likely the front page of the BBC as the transportation system instantly collapses So the anecdote itself is noise, it's intended to show how seriously intractable a problem accurate time is, but it doesn't do that, instead it only demonstrates OP's lack of familiarity with…

[deleted]

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#173

Post by 21 »

zwegner wrote: post #167
21 wrote: post #71

Earlier quoted context omitted.

> Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't If GPS timing is bad, a lot of people will notice that their position on the map is incorrect, because that's the whole purpose of the GPS network. A 1 microsecond error is 300 meters.

> A 1 microsecond error is 300 meters. While the speed-of-light propagation is about 300 meters in a microsecond, isn't the final position error possibly much greater? For calculating position on Earth, you can think about a sphere expanding at the speed of light from each satellite. The 1 microsecond error here corresponds to a radius 300m bigger or smaller, which only corresponds to 300m horizontal distance on the…

Intuitively this seems wrong to me. If the satellite is overhead, the error would put you 300m into the ground so to speak. I'm not sure why you project that horizontally, and especially why you take the distance to the satellite into account.

As another sanity check, if the error for 1 us is 110 km, the error for 1 ns would be 110 m, and I suspect 1 ns error is not unusual for consumer electronics:

> To reduce this error level to the order of meters would require an atomic clock. However, not only is this impracticable for consumer GPS devices, the GPS satellites are only accurate to about 10 nano seconds (in which time a signal would travel 3m)

https://wiki.openstreetmap.org/wiki/Accuracy_of_GPS_data

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#174

Post by zzzcpan »

chubot wrote: post #157
idubrov wrote: post #89

Any comments on how FoundationDB might be the same / different?

I'm pretty sure FoundationDB still relies on the more traditional model of hardware: "many fast, identically-spec'd machines on a rack, with fast reliable networking". In other words, I don't think you can run Foundation DB across geographically dispersed data centers, as you can with Spanner, CockroachDB, and others. Running on cloud VMs is a very hostile environment for a database, as others have pointed out in thi…

FoundationDB could probably do better than CockroachDB in multi datacenter configuration, because of the different trade offs they made, but I still wouldn't expect any of them do that well enough for most applications.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#175

Post by apaprocki »

> a specialized hardware solution that uses both GPS and atomic clocks to ensure a minimal clock skew across servers

Maybe I'm wrong, but pretty much anyone with a DC is going to use Microsemi (previously Symmetricom) grandmaster clocks with all the bells-and-whistles including the internal Rubidium atomic oscillator along with PTP to keep everything in sync with multiple layers of redundancy. A specialized hardware solution that uses "both GPS and atomic clocks" to guarantee ~15ns RMS to UTC are just a shopping cart away.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#176

Post by zzzcpan »

erikb wrote: post #155
CydeWeys wrote: post #42

It's definitely true that putting the burden of consistency on developers (instead of on the DB) results in a lot more tricky work for developers. On my project, which started six years ago, we use Cloud Datastore, because Cloud Spanner hadn't come out yet. It results in complicated, painful code that would be completely unnecessary with stronger transactional guarantees. Some examples: https://github.com/google/nomu…

Why does anybody need to provide consistency? Often you don't have complete consistency anyways. There are bugs, there are time delays, there is parallel processing. Why even have the requirement?

Well, it's pretty hard to even find applications that require strong global consistency and are willing to sacrifice latency for that. Typically apps don't need much consistency at all and can sacrifice some data instead, like with most RDBMS setups in the wild. Beyond that SEC (strong eventual consistency) covers pretty much all consistency needs there are.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#177

Post by zzzcpan »

jules wrote: post #110

The CAP theorem has been truly disastrous for databases. The CAP theorem simply says that if you have a database on 2 servers and the connection between those serves goes down, then queries against one server don't see new updates from the other server, so you either have to give up consistency (serve stale data) or give up availability (one of the servers refuses to process further requests). That's all that CAP is,…

> slapping a fancy name on this concept is a justification for giving up consistency (even when the network and all servers are fully functional) to retain availability

This is a misconception. AP databases are not supposed to give up consistency, just not wait for all nodes to see updates. That's it. Consistency is still there, nodes resync, users always see their own updates and all that.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#178

Post by cloudwizard »

hosay123 wrote: post #60
wpietri wrote: post #52

Earlier quoted context omitted.

So yes, Google can be very impressed with Google. But I'm not sure that's the issue here. Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't? I think it's a very common pattern that a product has some set of trailblazer users who find issues before the people who make the product. Also, I think you're over-i…

If some condition existed that exceeded GPS intended design, you most certainly wouldn't learn of it first from some random anecdote on HN.. more likely the front page of the BBC as the transportation system instantly collapses So the anecdote itself is noise, it's intended to show how seriously intractable a problem accurate time is, but it doesn't do that, instead it only demonstrates OP's lack of familiarity with…

https://www.bbc.com/news/technology-35491962

Could this be it?

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#179

Post by paulddraper »

qaq wrote: post #170
paulddraper wrote: post #165

Earlier quoted context omitted.

Link? It depends what is meant by "commodity" I guess. The largest server AWS offers is only 64 physical cores (128 logical) and less than 4 TB RAM.

https://lenovopress.com/lp0647-thinksystem-sr950-server

Interesting, though I don't see the price or any other purchasing details (at least on that page).

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#180

Post by oconnor663 »

Quekid5 wrote: post #78
nemothekid wrote: post #48

> Systems that guarantee consistency only experience a necessary reduction in availability in the event of a network partition. As networks become more redundant, partitions become an increasingly rare event. And even if there is a partition, it is still possible for the majority partition to be available In my experience, yes network partitions are incredibly rare. However 99% of my distributed ststem partitions hav…

I may be a bit of an old fart, but this is the exact reasoning behind my decision to never go with "distributed X" if there's a "single-machine X" where you can just vertically scale. If you can afford 3-5 machines/VMs for a cluster you can almost certainly afford a single machine/VM with 2-4x the resources/CPU and chances are that it'll perform just as well (or better) because it doesn't have network latency to cont…

Does running on 3-5 machines give you the advantage of being able to apply kernel patches and restart machines without bringing down the whole system? That might matter to people with paying customers, even if in theory they could've replaced their 3 machines with one bigger one.
Post reply on HN