SSDs have become fast, except in the cloud • Hacker News Board

Re: SSDs have become fast, except in the cloud

#51

Post by Nextgrid » Tue, Feb 20, 2024, 5:39 PM UTC

ejb999 wrote: post #19

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

How much faster would the network need to get, in order to meet (or at least approach) the speed of a local SSD? are we talking about needing to 2x or 3x the speed, or by factors of hundreds or thousands?

The problem isn't necessarily speed, it's random access latency. What makes SSDs fast and "magical" is their low random-access latency compared to a spinning disk. The sequential-access read speed is merely a bonus.

Networked storage negates that significantly, absolutely killing performance for certain applications. You could have a 100Gbps network and it still won't match a direct-attached SSD in terms of latency (it can only match it in terms of sequential access throughput).

For many applications such as databases, random access is crucial, thus why nowadays' mid-range consumer hardware often outperforms hosted databases such as RDS unless they're so overprovisioned on RAM that the dataset is effectively always in there.

Re: SSDs have become fast, except in the cloud

#52

Post by zokier » Tue, Feb 20, 2024, 5:39 PM UTC

pclmulqdq wrote: post #42

jsnell wrote: post #28

Earlier quoted context omitted.

According to the submitted article, the numbers are from AWS instance types where the SSD is "physically attached" to the host, not about SSD-backed NAS solutions. Also, the article isn't just about SSDs being no faster than a network. It's about SSDs being two orders of magnitude slower than datacenter networks.

It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.

What makes you think that?

Re: SSDs have become fast, except in the cloud

#53

Post by jeffbee » Tue, Feb 20, 2024, 5:39 PM UTC

siliconc0w wrote: post #13

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast that they're close enough to memory so it's "your data surely fits on disk". This reduces the complexity of a lot of workloads so you can just buy a beefy server and do pretty insane caching/calculation/serving with just a single box or two for redundancy.

"I will simply have another box for redundancy" is already a system so complex that having it in or out of the cloud won't make a difference.

Re: SSDs have become fast, except in the cloud

#54

Post by ddorian43 » Tue, Feb 20, 2024, 5:41 PM UTC

pclmulqdq wrote: post #42

jsnell wrote: post #28

Earlier quoted context omitted.

According to the submitted article, the numbers are from AWS instance types where the SSD is "physically attached" to the host, not about SSD-backed NAS solutions. Also, the article isn't just about SSDs being no faster than a network. It's about SSDs being two orders of magnitude slower than datacenter networks.

It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.

Do you have a link to explain this? I dont think its true.

Re: SSDs have become fast, except in the cloud

#55

Post by candiddevmike » Tue, Feb 20, 2024, 5:41 PM UTC

pclmulqdq wrote: post #42

jsnell wrote: post #28

Earlier quoted context omitted.

According to the submitted article, the numbers are from AWS instance types where the SSD is "physically attached" to the host, not about SSD-backed NAS solutions. Also, the article isn't just about SSDs being no faster than a network. It's about SSDs being two orders of magnitude slower than datacenter networks.

It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.

Depends on the cloud provider. Local SSDs are physically attached to the host on GCP, but that makes them only useful for temporary storage.

Re: SSDs have become fast, except in the cloud

#56

Post by jalk » Tue, Feb 20, 2024, 5:41 PM UTC

SteveNuts wrote: post #35

vlovich123 wrote: post #14

Earlier quoted context omitted.

Why do they fundamentally need to be network attached storage instead of local to the VM?

Because even if you can squeeze 100TB or more of SSD/NVMe in a server, and there are 10 tenants using the machine, you're limited to 10TB as a hard ceiling. What happens when one tenant needs 200TB attached to a server? Cloud providers are starting to offer local SSD/NVMe, but you're renting the entire machine, and you're still limited to exactly what's installed in that server.

How is that different from how cores, mem and network bandwidth is allotted to tenants?

Re: SSDs have become fast, except in the cloud

#57

Post by Twirrim » Tue, Feb 20, 2024, 5:42 PM UTC

Disclaimer: I work for OCI, opinion my own etc.

We offer faster NVMe drives in instances. Our E4 Dense shapes ship with SAMSUNG MZWLJ7T6HALA-00AU3, which supports Sequential Reads of 7000 MB/s, and Sequential Write 3800 MB/s.

From a general perspective, I would say the _likely_ answer to why AWS doesn't have faster NVMes at the moment is likely to be lack of specific demand. That's a guess, but that's generally how things go. If there's not enough specific demand being fed in through TAMs and the like for faster disks, upgrades are likely to be more of an after-thought, or reflecting supply chain.

I know there's a tendency when you engineer things, to just work around, or work with the constraints, and grumble amongst your team, but it's incredibly invaluable if you can make sure your account manager knows what shortcomings you've had to work around.

Re: SSDs have become fast, except in the cloud

#58

Post by mike_hearn » Tue, Feb 20, 2024, 5:42 PM UTC

pclmulqdq wrote: post #42

jsnell wrote: post #28

Earlier quoted context omitted.

According to the submitted article, the numbers are from AWS instance types where the SSD is "physically attached" to the host, not about SSD-backed NAS solutions. Also, the article isn't just about SSDs being no faster than a network. It's about SSDs being two orders of magnitude slower than datacenter networks.

It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.

They do this because they want SSDs to be in a physically separate part of the building for operational reasons, or what's the point in giving you a "local" SSD that isn't actually plugged into the real machine?

Re: SSDs have become fast, except in the cloud

#59

Post by avg_dev » Tue, Feb 20, 2024, 5:44 PM UTC

malfist wrote: post #45

siliconc0w wrote: post #13

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast that they're close enough to memory so it's "your data surely fits on disk". This reduces the complexity of a lot of workloads so you can just buy a beefy server and do pretty insane caching/calculation/serving with just a single box or two for redundancy.

I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower than CPU Cache. Samsung 990 Pro 2TB has a latency of 40 μs DDR4-2133 with a CAS 15 has a latency of 14 nano seconds. DDR4 latency is 0.035% of one of the fastest SSDs, or to put it another way, DDR4 is 2,857x faster than an SSD. L1 cache is typically accessible i…

pretty cool comparisons. quite some differences there.

tangent, I remember reading some post called something like "Latency numbers every programmer should know" and being slightly ashamed when I could not internalize it.

Re: SSDs have become fast, except in the cloud

#60

Post by geerlingguy » Tue, Feb 20, 2024, 5:46 PM UTC

bugbuddy wrote: post #27

_Rabs_ wrote: post #15

Earlier quoted context omitted.

So much of this. The amount of times I've seen someone complain about slow DB performance when they're trying to connect to it from a different VPC, and bottlenecking themselves to 100Mbits is stupidly high. Literally depending on where things are in a data center... If you're looking for closely coupled and on a 10G line on the same switch, going to the same server rack. I bet you performance will be so much more co…

Aren’t 10G and 100G connections standard nowadays in data centers? Heck, I thought they were standard 10 years ago.

Datacenters are up to 400 Gbps and beyond (many places are adopting 1+ Tbps on core switching).

However, individual servers may still operate at 10, 25, or 40 Gbps to save cost on the thousands of NICs in a row of racks. Alternatively, servers with multiple 100G connections split that bandwidth allocation up among dozens of VMs so each one gets 1 or 10G.