SSDs have become fast, except in the cloud • Hacker News Board

Re: SSDs have become fast, except in the cloud

#21

Post by Retric » Tue, Feb 20, 2024, 5:27 PM UTC

vlovich123 wrote: post #14

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

Why do they fundamentally need to be network attached storage instead of local to the VM?

Redundancy, local storage is a single point of failure.

You can use local SSD’s as slow RAM, but anything on it can go away at any moment.

Re: SSDs have become fast, except in the cloud

#22

Post by zokier » Tue, Feb 20, 2024, 5:27 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

> SSDs in the cloud are attached over a network, and fundamentally have to be

Not on AWS. Instance stores (what the article is about) are physical local disks.

Re: SSDs have become fast, except in the cloud

#23

Post by pclmulqdq » Tue, Feb 20, 2024, 5:28 PM UTC

vlovich123 wrote: post #14

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

Why do they fundamentally need to be network attached storage instead of local to the VM?

Reliability. SSDs break and screw up a lot more frequently and more quickly than CPUs. Amazon has published a lot on the architecture of EBS, and they go through a good analysis of this. If you have a broken disk and you locally attach, you have a broken machine.

RAID helps you locally, but fundamentally relies on locality and low latency (and maybe custom hardware) to minimize the time window where you get true data corruption on a bad disk. That is insufficient for cloud storage.

Re: SSDs have become fast, except in the cloud

#24

Post by dimgl » Tue, Feb 20, 2024, 5:28 PM UTC

teaearlgraycold wrote: post #2

What’s a good small cloud competitor to AWS? For teams that just need two AZs to get HA and your standard stuff like VMs, k8s, etc.

I am _super_ impressed with Supabase. I can see how a lot of people may not like it given how reliant it is on Postgres, but I find it to be absolutely genius.

It basically allows me to forego having to make a server for the CRUD operations so I can focus on the actual business implications. My REST API is automatically managed for me (mostly with lightweight views and functions) and all of my other core logic is either spread out through edge functions or in a separate data store (Redis) where I perform the more CPU intensive operations related to my business.

There's some rough edges around their documentation and DX but I'm really loving it so far.

Re: SSDs have become fast, except in the cloud

#25

Post by mkoubaa » Tue, Feb 20, 2024, 5:30 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

Dumb question. Why does the network have to be slow? If the SSDs are two feet away from the motherboard and there's an optical connection to it, shouldn't it be fast? Are data centers putting SSDs super far away from motherboards?

Re: SSDs have become fast, except in the cloud

#26

Post by cogman10 » Tue, Feb 20, 2024, 5:30 PM UTC

There's a 4th option. Cost.

The fastest SSDs tend to also be MLC which tend to have much lower write life vs other technologies. This isn't unusual, increasing data density generally also makes it easier to increase performance. However, it's at the cost that the writes are typically done for a block/cell in memory rather than for single bits. So if one cell goes bad, they all fail.

But even if that's not the problem, there is a problem of upgrading the fleet in a cost effective mechanism. When you start introducing new tech into the stack, replacing that tech now requires your datacenters to have 2 different types of hardware on hand AND for the techs swapping drives to have a way to identify and replace that stuff when it goes bad.

Re: SSDs have become fast, except in the cloud

#27

Post by bugbuddy » Tue, Feb 20, 2024, 5:30 PM UTC

_Rabs_ wrote: post #15

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

So much of this. The amount of times I've seen someone complain about slow DB performance when they're trying to connect to it from a different VPC, and bottlenecking themselves to 100Mbits is stupidly high. Literally depending on where things are in a data center... If you're looking for closely coupled and on a 10G line on the same switch, going to the same server rack. I bet you performance will be so much more co…

Aren’t 10G and 100G connections standard nowadays in data centers? Heck, I thought they were standard 10 years ago.

Re: SSDs have become fast, except in the cloud

#28

Post by jsnell » Tue, Feb 20, 2024, 5:31 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

According to the submitted article, the numbers are from AWS instance types where the SSD is "physically attached" to the host, not about SSD-backed NAS solutions.

Also, the article isn't just about SSDs being no faster than a network. It's about SSDs being two orders of magnitude slower than datacenter networks.

Re: SSDs have become fast, except in the cloud

#29

Post by wmf » Tue, Feb 20, 2024, 5:31 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

No, the i3/i4 VMs discussed in the blog have local SSDs. The network isn't the reason local SSDs are slow.

Re: SSDs have become fast, except in the cloud

#30

Post by ssl-3 » Tue, Feb 20, 2024, 5:31 PM UTC

Filligree wrote: post #18

vlovich123 wrote: post #14

Earlier quoted context omitted.

Why do they fundamentally need to be network attached storage instead of local to the VM?

They don't. Some cloud providers (i.e. Hetzner) let you rent VMs with locally attached NVMe, which is dramatically faster than network-attached even factoring in the VM tax. Of course then you have a single point of failure, in the PCIe fabric of the machine you're running on if not the NVMe itself. But if you have good backups, which you should, then the juice really isn't worth the squeeze for NAS storage.

A network adds more points of failure. It does not reduce them.