Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSDs have become fast, except in the cloud

databasearchitects.blogspot.com

Post reply on HN
Print view View original

411–420 of 427 posts

Re: SSDs have become fast, except in the cloud

#411

Post by the-rc »

rowanG077 wrote: post #391
the-rc wrote: post #368

Earlier quoted context omitted.

Although it used them for years, the first mention by Google of PCIe switches was probably in the 2022 Aquila paper, which doesn't really talk about storage anyway...

I don't understand why you would expect Google to state that. They have been standard technology for almost 2 decades. You don't see google claiming they use jtag or using SPI flash or whatever. It's just not special.

Google didn't invent the Clos network, either, but it took years before they started talking about its adoption and with what kind of proprietary twists. Same with power supplies. You're right, a PCIe switch is not special, unless maybe it's integrated in some unconventional way. It's in Google's DNA to be cagey by default on a lot of details, to avoid giving ideas to the competition. Or misleading others down rabbit holes, like with shipping container datacenters.

Re: SSDs have become fast, except in the cloud

#412

Post by wmf »

asalahli wrote: post #407
wmf wrote: post #329

Earlier quoted context omitted.

It's so obviously wrong that I can't really explain it. Maybe someone else can. To believe that requires a complete misunderstanding of IaaS.

GP is probably refferring to blurbs like these > Amazon SimpleDB measures the machine utilization of each request and charges based on the amount of machine capacity used to complete the particular request (SELECT, GET, PUT, etc.), normalized to the hourly capacity of a circa 2007 1.7 GHz Xeon processor. See below for a more detailed description of how machine utilization charges are calculated. https://aws.amazon.co…

SimpleDB is over 15 years old. I guess it's the only service still using "normalized" pricing. Newer services like RDS tell you exactly which processor you're getting and how many cores.

Re: SSDs have become fast, except in the cloud

#414

Post by sitkack »

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

You are wrong, even on GCP. PD, persistent disk are all remote, but actual Local SSD are, local. There is still a bit of code that runs between the guest and the device. All of this is public information.

Re: SSDs have become fast, except in the cloud

#415

Post by sitkack »

the-rc wrote: post #411
rowanG077 wrote: post #391

Earlier quoted context omitted.

I don't understand why you would expect Google to state that. They have been standard technology for almost 2 decades. You don't see google claiming they use jtag or using SPI flash or whatever. It's just not special.

Google didn't invent the Clos network, either, but it took years before they started talking about its adoption and with what kind of proprietary twists. Same with power supplies. You're right, a PCIe switch is not special, unless maybe it's integrated in some unconventional way. It's in Google's DNA to be cagey by default on a lot of details, to avoid giving ideas to the competition. Or misleading others down rabbit…

No, it dismisses technology until it does a 180 and then pretends it innovated in ways everyone is too stupid to understand. Google exceptionalism 101.

Re: SSDs have become fast, except in the cloud

#416

Post by silent_cal »

pdimitar wrote: post #400
silent_cal wrote: post #374

Earlier quoted context omitted.

Isn't that expected? I would assume cloud stuff is slower because it's essentially an emulation of the real thing.

Why should it be expected? I'm being sold compute with quoted GHz CPU speeds, RAM and types of SSDs. I would vaguely expect it to not match my workstation, sure, but all throughout this thread (and others) people have cited outrageous disparities i.e. 5x less performance that you'd expect even if you managed your expectations to e.g. 2x less due to the cloud compute not being a bare metal machine. In other words, and…

Makes sense.

Re: SSDs have become fast, except in the cloud

#417

Post by wistlo »

motles wrote: post #405
wistlo wrote: post #364

At my job at a telco, I had a 13 billion record file to scan and index for duplicates and bad addresses. Consultants brought in to move our apps (some of which were Excel macros, others SAS scripts running on old desktop) to Azure. The Azure architects identified Postgres as the best tool. Consultants attempted to create a Postgres index in a small Azure instance but their tests would fail without completion (they we…

Was this on the L-series or managed disk?

I tried a variety of configurations. The E-series was one of them, as it's advertised as "Great for relational database servers, medium to large caches, and in-memory analytics." Premium and Premium V2, tried those at larger capacities I didn't need just to get higher IOPS.

None came within an order of magnitude of a Ryzen 7600/nVME mobo sitting in my son's old gaming case.

An option I did not try was Ultra disk, which I recall being significantly more expensive and was not part of the standard corporate offering. I wasn't itching to get dragged in front of the architecture review board again, anyway.

Re: SSDs have become fast, except in the cloud

#418

Post by jauntywundrkind »

namibj wrote: post #378
bcaxis wrote: post #281

Earlier quoted context omitted.

The main problem with consumer drives is the missing power loss protection (plp). M.2 drives just don't have space for the caps like an enterprise 2.5 u.2/u.3 drive will have. This matters when the DB calls a sync and it's expecting the data to be written safely to disk before it returns. A consumer drive basically stops everything until it can report success and your IOPS falls to like 1/100th of what the drive is c…

Sadly the solution, a firmware variant with ZNS instead of the normal random write block device, just isn't on offer (please tell if I'm wrong; I'd love one!). Because with ZNS you can get away with tiny caps, large enough to complete the in-flight blocks (not the buffered ones, just those that are already at the flash chip itself), plus one metadata journal/ring buffer page to store write pointers and zone status fo…

It's fucking wild to me the trajectory here:

OpenChannelIO: stop making such absurd totalizing systems that hide the actual flash! We don't need your fancy controllers! Just give us a bunch of flash that we can individually talk to & control ourselves from the host! You leave so much performance on the table! There's so many access patterns that wouldn't suck & would involve random rewrites of blocks!

Then after years of squabbling, eventually: ok we have negotiated for years. We will build new standards to allow these better access patterns that require nearly no drive controller intermediation, that have exponentially higher degrees of mechanistic sympathy of what flash actually does. Then we will never release drives mainstream & only have some hard to get unbelievably expensive enterprise drives that support it.

You are so f-ing spot on. This ZNS would be perfect for lower reliability consumer drives. But: market segmentation.

The situation is so fucked. We are so impeded from excelling, as a civilization, case number nine billion three hundred forty two.

Re: SSDs have become fast, except in the cloud

#419

Post by bmicraft »

seabrookmx wrote: post #333
zten wrote: post #264

Earlier quoted context omitted.

> On the contrary, young people often show up having learned on their super fast Apple SSD or a top of the line gaming machine with NVMe SSD. Yes, this is often a big surprise. You can test out some disk-heavy app locally on your laptop and observe decent performance, and then have your day completely ruined when you provision a slice of an NVMe SSD instance type (like, i4i.2xlarge) and discover you're only paying fo…

This doesn't stop at SSD's. Spin up an E2 VM in Google Cloud and there's a good chance you'll get a nearly 9 year Broadwell architecture chip running your workload!

What this tells me is that the price of running inefficient cpus seemingly isn't nearly as high as I thought it would or should be (in terms of usd/kWh)

Re: SSDs have become fast, except in the cloud

#420

Post by seabrookmx »

bmicraft wrote: post #419
seabrookmx wrote: post #333

Earlier quoted context omitted.

This doesn't stop at SSD's. Spin up an E2 VM in Google Cloud and there's a good chance you'll get a nearly 9 year Broadwell architecture chip running your workload!

What this tells me is that the price of running inefficient cpus seemingly isn't nearly as high as I thought it would or should be (in terms of usd/kWh)

Well they bill you for the instance not for some unit of computation. I'd imagine many users of E2 instances don't realize that they could be getting much much worse performance per vcore than if they picked a different instance type.

From Google's perspective, if the hardware is paid for, still reliable, and they can still make money on it, they can put new hardware in new racks rather than replacing the old hardware. This suggests Google's DC's aren't space constrained but I'm not surprised after looking at a few via satellite images!

Post reply on HN