SSDs have become fast, except in the cloud • Hacker News Board

Re: SSDs have become fast, except in the cloud

#11

Post by pclmulqdq » Tue, Feb 20, 2024, 5:22 PM UTC

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions.

SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, which was the backing technology when a lot of these network attached storage systems were invented, because they are fundamentally slow compared to networks, but it is a problem for SSD.

Re: SSDs have become fast, except in the cloud

#12

Post by brucethemoose2 » Tue, Feb 20, 2024, 5:23 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

Yeah this was my impression.

I am but an end user, but I noticed that disk IO for a certain app was glacial compared to a local test deployment, and I chalked it up to networking/VM overhead

Re: SSDs have become fast, except in the cloud

#13

Post by siliconc0w » Tue, Feb 20, 2024, 5:23 PM UTC

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast that they're close enough to memory so it's "your data surely fits on disk". This reduces the complexity of a lot of workloads so you can just buy a beefy server and do pretty insane caching/calculation/serving with just a single box or two for redundancy.

Re: SSDs have become fast, except in the cloud

#14

Post by vlovich123 » Tue, Feb 20, 2024, 5:24 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

Why do they fundamentally need to be network attached storage instead of local to the VM?

Re: SSDs have become fast, except in the cloud

#15

Post by _Rabs_ » Tue, Feb 20, 2024, 5:24 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

So much of this. The amount of times I've seen someone complain about slow DB performance when they're trying to connect to it from a different VPC, and bottlenecking themselves to 100Mbits is stupidly high.

Literally depending on where things are in a data center... If you're looking for closely coupled and on a 10G line on the same switch, going to the same server rack. I bet you performance will be so much more consistent.

Re: SSDs have become fast, except in the cloud

#16

Post by zokier » Tue, Feb 20, 2024, 5:25 PM UTC

> Since then, several NVMe instance types, including i4i and im4gn, have been launched. Surprisingly, however, the performance has not increased; seven years after the i3 launch, we are still stuck with 2 GB/s per SSD.

AWS marketing claims otherwise:

    Up to 800K random write IOPS
    Up to 1 million random read IOPS
    Up to 5600 MB/second of sequential writes
    Up to 8000 MB/second of sequential reads

https://aws.amazon.com/blogs/aws/new-storage-optimized-amazo...

Re: SSDs have become fast, except in the cloud

#17

Post by echelon » Tue, Feb 20, 2024, 5:25 PM UTC

siliconc0w wrote: post #13

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast that they're close enough to memory so it's "your data surely fits on disk". This reduces the complexity of a lot of workloads so you can just buy a beefy server and do pretty insane caching/calculation/serving with just a single box or two for redundancy.

The reasons to switch away from cloud keep piling up.

We're doing some amount of on-prem, and I'm eager to do more.

Re: SSDs have become fast, except in the cloud

#18

Post by Filligree » Tue, Feb 20, 2024, 5:26 PM UTC

vlovich123 wrote: post #14

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

Why do they fundamentally need to be network attached storage instead of local to the VM?

They don't. Some cloud providers (i.e. Hetzner) let you rent VMs with locally attached NVMe, which is dramatically faster than network-attached even factoring in the VM tax.

Of course then you have a single point of failure, in the PCIe fabric of the machine you're running on if not the NVMe itself. But if you have good backups, which you should, then the juice really isn't worth the squeeze for NAS storage.

Re: SSDs have become fast, except in the cloud

#19

Post by ejb999 » Tue, Feb 20, 2024, 5:26 PM UTC

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

How much faster would the network need to get, in order to meet (or at least approach) the speed of a local SSD? are we talking about needing to 2x or 3x the speed, or by factors of hundreds or thousands?

Re: SSDs have become fast, except in the cloud

#20

Post by zokier » Tue, Feb 20, 2024, 5:26 PM UTC

bombcar wrote: post #3

I think the obvious answer is there's not much demand, and keeping it "low" allows trickery and funny business with the virtualization layer (think: SAN, etc) that you can't do with "raw hardware speed".

There is no trickery with AWS instance stores, they are honest to god local disks.