Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NewSQL databases fail to guarantee consistency and I blame Spanner

dbmsmusings.blogspot.com

Post reply on HN
Print view View original

191–200 of 319 posts

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#191

Post by adrianmonk »

tytso wrote: post #25

The post talks about Spanner using "a specialized hardware solution that uses both GPS and atomic clocks to ensure a minimal clock skew across servers." But what it fails to mention is that is that this time is distributed using a software solution --- NTP, in fact. Google makes its "leap-smeared NTP network" available via Google's Public NTP service. And it's not expensive for someone to buy their own GPS clock and…

It makes me wonder if, as an industry, we shouldn't just include hardware clocks in our basic expectations for a data center.

That is, in order to be considered a satisfactory, non bare bones data center, you'd need to supply on-site, hardware, redundant, maintained and monitored clocks, verifiably meeting certain standards of accuracy and precision. Just like right now people expect backup power, cooling, network connectivity, physical security, access, etc.

It doesn't seem like the burden would be that large for data center operators. The hardware costs don't sound large, and they already have people being paid to monitor and maintain things.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#192

Post by jandrese »

paulddraper wrote: post #179
qaq wrote: post #170

Earlier quoted context omitted.

https://lenovopress.com/lp0647-thinksystem-sr950-server

Interesting, though I don't see the price or any other purchasing details (at least on that page).

Pretty sure once you're talking about 12TB of memory it goes to "call for pricing".

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#193

Post by CydeWeys »

mehrdadn wrote: post #88
CydeWeys wrote: post #36

Earlier quoted context omitted.

The problem is that the bound on clock error directly affects your performance. So if you're willing to accept, say, a second in clock error, then all transactions will take a minimum of one second. That level of performance is going to be unacceptable in many situations. The potential clock error on VMs without dedicated time-keeping hardware is so large that performance turns into absolute garbage.

I would understand if the complaint was that Spanner is too slow without expensively accurate clocks and synchronization. But the complaint is that Spanner fails to guarantee consistency, which doesn't make sense to me. The requirements clearly include giving a valid clock bound, so if you give an invalid clock bound, it's clearly your fault for getting incorrect results, not Spanner's!

Spanner does guarantee consistency, thanks to its use of hardware atomic clocks and GPS. It's alternatives like CockroachDB that don't have this dedicated hardware that can fail to guarantee consistency if clocks get of sync (a problem that can't happen in Spanner).

Spanner is really fast and massively parallelizable.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#194

Post by CydeWeys »

wikibob wrote: post #151
CydeWeys wrote: post #42

It's definitely true that putting the burden of consistency on developers (instead of on the DB) results in a lot more tricky work for developers. On my project, which started six years ago, we use Cloud Datastore, because Cloud Spanner hadn't come out yet. It results in complicated, painful code that would be completely unnecessary with stronger transactional guarantees. Some examples: https://github.com/google/nomu…

I’m guessing that Nomulus is your project? I ran across it before and just wanted to say it’s really cool that this is open sourced.

I'm the tech lead of it. Glad to hear that you've heard of it before, and yeah, I think it's pretty cool it's open sourced too, which is why I made it happen! https://opensource.googleblog.com/2016/10/introducing-nomulu...

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#195

Post by dang »

zzzcpan wrote: post #49
haberman wrote: post #13

Earlier quoted context omitted.

Without some description of what "false claims" and "misinterpretation" you mean, this is a content-free post.

Too much is false to explain it, like half of the article. So I just called it out instead. I know it's not very convincing, but I'm not getting paid to write giant FUD/PR posts like they do. I work on distributed stuff completely independently.

That's fine, but it's against the site guidelines to post shallow dimissals here, so please don't.

https://news.ycombinator.com/newsguidelines.html

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#196

Post by kwillets »

evanweaver wrote: post #6
abadid wrote: post #2

I'm the author of that post. I'm happy to respond to comments on the post on this thread for the next several hours. You can also leave comments on the post itself, and I will respond there at any time.

Do these concerns also apply in an HTAP or OLAP context e.g. systems like Cloudera's Kudu, which uses Hybrid Time? Or maybe Volt which you also worked on?

Vertica seems to include some of these observations, such as global consistency and group commit. I believe these are easier to achieve (lower overhead) in OLAP due to fewer, larger transactions.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#197

Post by kwillets »

xtacy wrote: post #18
evanweaver wrote: post #6

Earlier quoted context omitted.

Do these concerns also apply in an HTAP or OLAP context e.g. systems like Cloudera's Kudu, which uses Hybrid Time? Or maybe Volt which you also worked on?

OLAP systems tend to be read only (for analytics) so the question of transactions and consistency isn't really applicable.

They don't load themselves.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#198

Post by Serow225 »

eaguyhn wrote: post #147
Serow225 wrote: post #117

Earlier quoted context omitted.

My experience is that it's unfortunately really hard to convince people who don't deeply understand distributed systems (but think they know) that just because a system is called 'HA' or can have an 'HA' mode turned on, that doing so has downsides. They freak out if you try to propose _not_ running the HA mode, because they don't (or aren't willing to) understand the potential downsides of dealing with split-brain, b…

It's a struggle for me, too. People sometimes rest on their assumptions even though new technologies and paradigms can invalidate old rules (I'm sure everyone has lots of examples). I found that constant communication and scenarios help bridge the gap between how they think things work and how you do. It doesn't always work - a lot of time people don't care to discuss things in detail. But sometimes it worked and the…

Thanks for your thoughts. And definitely yes re your last point! Like rubber duck debugging :)

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#199

Post by rkarthik007 »

CTO of YugaByte here. We firmly stand by our claims, and I wanted to explain more.

From the post by Daniel: YugaByte, however, continues to claim a guarantee of consistency. I would advise people not to trust this claim. YugaByte, by virtue of its Spanner roots, will run into consistency violations when the local clock on a server suddenly jumps beyond the skew uncertainty window. >>

The statement about YugaByte DB is incorrect.

1. With respect to CAP, both Cockroach DB (https://www.cockroachlabs.com/blog/limits-of-the-cap-theorem...) and YugaByte DB (https://docs.yugabyte.com/latest/faq/architecture/#how-can-y...) are CP databases with HA and there is really no difference in the claims.

2. With respect to Isolation level in ACID, YugaByte DB does not make the linearizability (called external consistency by Google Spanner) claim. YugaByte DB offers Snapshot Isolation (detects write-write conflicts) today and Serializable isolation (detect read-write and write-write conflicts) is in the roadmap (https://docs.yugabyte.com/latest/architecture/transactions/i...).

3. We have publicly claimed that we do rely on NTP and max clock skew bounds to guarantee consistency. For example, slide 43 of our NorCal DB Day talk (https://www.slideshare.net/YugaByte/yugabyte-db-architecture...) we mention we are “relying on bounded clock sync (NTP, AWS Time Sync, etc).”

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#200

Post by Serow225 »

how_gauche wrote: post #182
Serow225 wrote: post #117

Earlier quoted context omitted.

My experience is that it's unfortunately really hard to convince people who don't deeply understand distributed systems (but think they know) that just because a system is called 'HA' or can have an 'HA' mode turned on, that doing so has downsides. They freak out if you try to propose _not_ running the HA mode, because they don't (or aren't willing to) understand the potential downsides of dealing with split-brain, b…

A rational way of explaining it is -- what is my service level objective? If my SLO is for two nines of uptime, then I can be down for 3.65 days a year, and I can get away with single-homing something, or going with a simple hot-failover replicated setup. I just need to be pretty confident that I can fix it if it breaks within a few hours. If I need more nines of uptime than a single system could provide (and think a…

Thank you very much, especially that first paragraph. I think you've given me some good discussion tools, and I really appreciate that.
Post reply on HN