Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NewSQL databases fail to guarantee consistency and I blame Spanner

dbmsmusings.blogspot.com

Post reply on HN
Print view View original

151–160 of 319 posts

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#151

Post by wikibob »

CydeWeys wrote: post #42

It's definitely true that putting the burden of consistency on developers (instead of on the DB) results in a lot more tricky work for developers. On my project, which started six years ago, we use Cloud Datastore, because Cloud Spanner hadn't come out yet. It results in complicated, painful code that would be completely unnecessary with stronger transactional guarantees. Some examples: https://github.com/google/nomu…

I’m guessing that Nomulus is your project?

I ran across it before and just wanted to say it’s really cool that this is open sourced.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#152

Post by jshen »

jules wrote: post #110

The CAP theorem has been truly disastrous for databases. The CAP theorem simply says that if you have a database on 2 servers and the connection between those serves goes down, then queries against one server don't see new updates from the other server, so you either have to give up consistency (serve stale data) or give up availability (one of the servers refuses to process further requests). That's all that CAP is,…

"The A for availability in CAP means that ALL database servers are fully available"

Is this true? I always thought it meant that clients could continue to read and write to "the database" which could include the client switching to another node. There is nothing in CAP theorem about latency, so switching, even if it adds high latency, is fine by CAP theorem.

This lack of accounting for latency is what makes CAP theorem less useful than a lot of people realize IMO.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#154

Post by qaq »

toast0 wrote: post #101
Quekid5 wrote: post #78

Earlier quoted context omitted.

I may be a bit of an old fart, but this is the exact reasoning behind my decision to never go with "distributed X" if there's a "single-machine X" where you can just vertically scale. If you can afford 3-5 machines/VMs for a cluster you can almost certainly afford a single machine/VM with 2-4x the resources/CPU and chances are that it'll perform just as well (or better) because it doesn't have network latency to cont…

I think part of this is that most of the common knowledge about scaling is hard fought from the 90s/2000 era. eBay got bigger and bigger Sun boxes to run Oracle, until they couldn't get anything bigger -- then they had a problem and had to shard their listings into categories, etc. In the last few Intel cpu generations, computation performance has had small gains, but addressable memory has doubled about every other…

You can get commodity x86 server with 12TB RAM and 224 cores.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#155

Post by erikb »

CydeWeys wrote: post #42

It's definitely true that putting the burden of consistency on developers (instead of on the DB) results in a lot more tricky work for developers. On my project, which started six years ago, we use Cloud Datastore, because Cloud Spanner hadn't come out yet. It results in complicated, painful code that would be completely unnecessary with stronger transactional guarantees. Some examples: https://github.com/google/nomu…

Why does anybody need to provide consistency? Often you don't have complete consistency anyways. There are bugs, there are time delays, there is parallel processing. Why even have the requirement?

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#156

Post by erikb »

peterwwillis wrote: post #46

Distributed databases are like systems of government. Some are worse than others, but all of them suck. I'm not aware of any study that shows that X type of database reduces bugs, increases availability, and makes customers happier. Pick one that fits your application and deal with the suckiness.

Why not take this pain and think about other forms of architecture? Maybe a consistent source of all truth is a bullshit requirement?

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#157

Post by chubot »

idubrov wrote: post #89

Any comments on how FoundationDB might be the same / different?

I'm pretty sure FoundationDB still relies on the more traditional model of hardware: "many fast, identically-spec'd machines on a rack, with fast reliable networking".

In other words, I don't think you can run Foundation DB across geographically dispersed data centers, as you can with Spanner, CockroachDB, and others.

Running on cloud VMs is a very hostile environment for a database, as others have pointed out in this thread.

Though I'm happy to hear from someone who knows more about FoundationDB.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#158

Post by davidgay »

CydeWeys wrote: post #42

It's definitely true that putting the burden of consistency on developers (instead of on the DB) results in a lot more tricky work for developers. On my project, which started six years ago, we use Cloud Datastore, because Cloud Spanner hadn't come out yet. It results in complicated, painful code that would be completely unnecessary with stronger transactional guarantees. Some examples: https://github.com/google/nomu…

It's worth noting that Cloud Datastore's follow-on, Cloud Firestore, does provide strong consistency, and includes a "Datastore mode" that supports the Datastore API.

Firestore is currently in beta, but once it's GA we will be migrating all Datastore users to Firestore: https://cloud.google.com/datastore/docs/upgrade-to-firestore

Disclaimer: I work on Cloud Datastore/Firestore.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#159

Post by makomk »

21 wrote: post #71
wpietri wrote: post #52

Earlier quoted context omitted.

So yes, Google can be very impressed with Google. But I'm not sure that's the issue here. Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't? I think it's a very common pattern that a product has some set of trailblazer users who find issues before the people who make the product. Also, I think you're over-i…

> Is it really surprising that people who have extremely precise time needs and a whole team devoted to solving them would notice issues that other people wouldn't If GPS timing is bad, a lot of people will notice that their position on the map is incorrect, because that's the whole purpose of the GPS network. A 1 microsecond error is 300 meters.

If I remember correctly, there was a bug a couple of years back which caused an incorrect time offset between GPS and UTC time to be uploaded to some of the satellites - off by a handful of microseconds. Didn't affect navigation but it did trip a bunch of alerts on systems that relied on precise time. I don't think Google was the one that alerted the USAF to that though, in fact they may not have had sufficiently accurate timekeeping back then.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#160

Post by riku_iki »

Dave_Rosenthal wrote: post #119
idubrov wrote: post #89

Any comments on how FoundationDB might be the same / different?

FoundationDB adheres to the highest level of consistency (linearizability) and, unlike Spanner or its derivatives, does not rely on clocks to achieve this. I'm surprised FoundationDB is not mentioned in the article. Maybe it didn't help the author make his point, but it is both the most mature of the new breed of ACID+noSQL databases and completely free/open source.

One reason is may be because it is not 'SQL' DB, which is probably prerequisite for NewSQL category.
Post reply on HN