Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NewSQL databases fail to guarantee consistency and I blame Spanner

dbmsmusings.blogspot.com

Post reply on HN
Print view View original

251–260 of 319 posts

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#251

Post by ryanworl »

grogers wrote: post #245
Dave_Rosenthal wrote: post #119

Earlier quoted context omitted.

FoundationDB adheres to the highest level of consistency (linearizability) and, unlike Spanner or its derivatives, does not rely on clocks to achieve this. I'm surprised FoundationDB is not mentioned in the article. Maybe it didn't help the author make his point, but it is both the most mature of the new breed of ACID+noSQL databases and completely free/open source.

According to their docs, FoundationDB only provides serializable isolation, so it isn't the same as the others in the post which offer strict serializable isolation (multi-key version of linearizability). Without strong clocks you can't have strict serializablity and scale beyond a single log (but one log can get you pretty damn far). TBH most people probably only need serializable transactions anyway. https://apple.…

While that documentation isn’t wrong, FoundationDB also provides external consistency.

https://forums.foundationdb.org/t/keyspace-partitions-perfor...

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#252

Post by [deleted] »

toast0 wrote: post #163
qaq wrote: post #154

Earlier quoted context omitted.

You can get commodity x86 server with 12TB RAM and 224 cores.

It looks like 12TB ram is available in eight socket Intel servers? Those are sort of commodity, but availability is limited, and NUMA becomes a much larger issue than in the more easily available dual socket configurations. Looks like Epyc can do 4TB in a dual socket configuration.

[deleted]

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#253

Post by chx »

paulddraper wrote: post #165
qaq wrote: post #154

Earlier quoted context omitted.

You can get commodity x86 server with 12TB RAM and 224 cores.

Link? It depends what is meant by "commodity" I guess. The largest server AWS offers is only 64 physical cores (128 logical) and less than 4 TB RAM.

http://www.supermicro.com/products/system/4U/8048/SYS-8048B-...

Quad socket R1 (LGA 2011) supports Intel® Xeon® processor E7-8800 v4/v3, E7-4800 v4/v3 family (up to 24-Core)

Up to 12TB DDR4 (128GB 3DS LRDIMM); 96x DIMM slots (8x memory module boards: X10QBi-MEM2)

AFAIK the latest Skylake Xeons ("Scalable" - Platinum/Gold/Silver/Bronze) have regressed to 1.5TB support, see https://ark.intel.com/products/93794/Intel-Xeon-Processor-E7... http://www.colfax-intl.com/nd/downloads/Intel-Xeon-Scalable-... corroborated by https://ark.intel.com/products/120502/Intel-Xeon-Platinum-81...

Since a single 128GB stick costs about 2900 USD, 96 of them will run to ~280 000 USD plus the server so it's likely to be above 300 000K.

If you want to go with 6TB "only" then it's a lot, lot cheaper as 64GB RDIMM sticks can be had below 700 USD. The end result might cost closer to a third of the 12TB server than half of it.

RAM prices comparison: https://memory.net/memory-prices/

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#254

Post by jacques_chester »

wting wrote: post #246
bostik wrote: post #217

Earlier quoted context omitted.

> what is my service level objective? There are environments where flat time distribution for SLO calculation is not acceptable. ( cough betting exchange) If your traffic patterns are extremely spiky, such as weekly peaks hitting 15-20x of your base load, and where a big chunk of your business can come from those peaks, then most normal calculations don't apply. Let's say your main system that accepts writes is 10 mi…

Time based SLOs definitely have their limitations, but in this instance isn't it fairly easy to redefine the SLO in terms of requests rather than time?

This is one of the recommendations given in the Google SRE book: use request-level metrics for SLOs/SLIs where possible. As your systems grow larger the probability of total outage, which would be measured in time, becomes a smaller fraction of the probability of partial outage.

Since total outages are a special case of partial outages, use metrics that cleanly measure partial outages. That's request error metrics.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#255

Post by dvirsky »

300bps wrote: post #206
Quekid5 wrote: post #102

Earlier quoted context omitted.

That's a really good observation and I agree. I sometimes have to pinch myself to make sure I'm not dreaming when I see that there are actually pretty affordable machines with these insane amounts of RAM. (I think the first machine I personally[0] owned had 512K or something like that.) [0] Well, my dad bought it, but y'know. He wasn't particularly interested, but I think he recognized an interest in computers in me…

My first computer had 38911 BASIC BYTES FREE.

Luxury! My first computer (Sinclair ZX Spectrum) had 48KB RAM total, and IIRC 8KB of BASIC RAM available.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#256

Post by espeed »

hosay123 wrote: post #44
romed wrote: post #29

I’ve seen comments on HN over the years in which someone Dunning-Kruegers their way into saying that TrueTime is easily replicated. I always wonder if they have sixteen senior SREs in their pocket, because that’s the level of production engineering Google applies to the problem. Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up. If yo…

> Time SRE has at various points had take measures up to and including calling the USAF and telling them their satellites are fucked up It's another cute anecdote, but Google culture is full of these, always scant on details and always intended to show how big/smart/important/complex/indispensable their engineering is. "Had to" is a strong term here, it's made to sound like USAF could not possibly have noticed some d…

Google also reports software and hardware security vulnerabilities and infrastructure security issues to external companies, organizations, and stakeholders responsible for the the design, operation, and maintenance of external systems. Google isn't the only company that does this - other organizations do too - but at this level expertise is a scarce resource, and we're all in the same boat so it behooves everyone when those capable can and do cooperate and participate in keeping a vigilant watch. This ethos is one if the reason the West dominates.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#257

Post by nickerten »

evanweaver wrote: post #6
abadid wrote: post #2

I'm the author of that post. I'm happy to respond to comments on the post on this thread for the next several hours. You can also leave comments on the post itself, and I will respond there at any time.

Do these concerns also apply in an HTAP or OLAP context e.g. systems like Cloudera's Kudu, which uses Hybrid Time? Or maybe Volt which you also worked on?

I've worked on a system that loaded data into Kudu in near real time and simultaneously ran queries on the data. Kudu has no transactions, consistency is eventual which was sufficient due to our near-real time constraint, however you do need a stable NTP source. We have lost data when the cluster could not get a reliable NTP connection, decided to shut down and tablet servers' data files became corrupted.

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#258

Post by mmt »

qaq wrote: post #154
toast0 wrote: post #101

Earlier quoted context omitted.

I think part of this is that most of the common knowledge about scaling is hard fought from the 90s/2000 era. eBay got bigger and bigger Sun boxes to run Oracle, until they couldn't get anything bigger -- then they had a problem and had to shard their listings into categories, etc. In the last few Intel cpu generations, computation performance has had small gains, but addressable memory has doubled about every other…

You can get commodity x86 server with 12TB RAM and 224 cores.

If you're willing to go back a generation on CPU (and merely 192 cores), you can get 24TB.

https://www.supermicro.com/products/system/7U/7088/SYS-7088B...

Re: NewSQL databases fail to guarantee consistency and I blame Spanner

#260

Post by rtpg »

lamontcg wrote: post #144
Quekid5 wrote: post #78

Earlier quoted context omitted.

I may be a bit of an old fart, but this is the exact reasoning behind my decision to never go with "distributed X" if there's a "single-machine X" where you can just vertically scale. If you can afford 3-5 machines/VMs for a cluster you can almost certainly afford a single machine/VM with 2-4x the resources/CPU and chances are that it'll perform just as well (or better) because it doesn't have network latency to cont…

Also an "old fart", I would recommend postgresql first over any of these other databases. Solve the big data scaling problems when you actually have them. One database server with replication and failover is going to still solve 95-98% or more of the use cases on the web.

It unfortunately doesn’t adequately solve “tweak and update the database software in the middle of the day without requiring downtime” situation very well

We do rolling releases of software all the time but it’s pretty hard for us to do much optimisation of our DB setup without doing it in the middle of the night because of how all this stuff works.

Post reply on HN