Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSDs have become fast, except in the cloud

databasearchitects.blogspot.com

Post reply on HN
Print view View original

211–220 of 427 posts

Re: SSDs have become fast, except in the cloud

#211

Post by BackBlast »

malfist wrote: post #45
siliconc0w wrote: post #13

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast that they're close enough to memory so it's "your data surely fits on disk". This reduces the complexity of a lot of workloads so you can just buy a beefy server and do pretty insane caching/calculation/serving with just a single box or two for redundancy.

I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower than CPU Cache. Samsung 990 Pro 2TB has a latency of 40 μs DDR4-2133 with a CAS 15 has a latency of 14 nano seconds. DDR4 latency is 0.035% of one of the fastest SSDs, or to put it another way, DDR4 is 2,857x faster than an SSD. L1 cache is typically accessible i…

You're missing the purpose of the cache. At least for this argument it's mostly for network responses.

HDD was 10ms, which was noticeable for cached network request that needs to go back out on the wire. This was also bottle necked by IOPS, after 100-150 IOPS you were done. You could do a bit better with raid, but not the 2-3 orders of magnitude you really needed to be an effective cache. So it just couldn't work as a serious cache, the next step up was RAM. This is the operational environment which redis and such memory caches evolved.

40 us latency is fine for caching. Even the high load 500-600us latency is fine for the network request cache purpose. You can buy individual drives with > 1 million read IOPS. Plenty for a good cache. HDD couldn't fit the bill for the above reasons. RAM is faster, no question, but the lower latency of the RAM over the SSD isn't really helping performance here as the network latency is dominating.

Rails conference 2023 has a talk that mentions this. They moved from a memory based cache system to an SSD based cache system. The Redis RAM based system latency was 0.8ms and the SSD based system was 1.2ms for some known system. Which is fine. It saves you a couple of orders of magnitude on cost and you can do much much larger and more aggressive caching with the extra space.

Often times these RAM caching servers are a network hop away anyway, or at least a loopback TCP request. Making the question of comparing SSD latency to RAM totally irrelevant.

Re: SSDs have become fast, except in the cloud

#212

Post by ianburrell »

youngtaff wrote: post #185
ianburrell wrote: post #118

Earlier quoted context omitted.

The reason for having most instances use network storage is that it makes possible migrating instances to other hosts. If the host fails, the network storage can be pointed at the new host with a reboot. AWS sends out notices regularly when they are going to reboot or migrate instances. Their probably should be more local instance storage types for using with instances that can be recreated without loss. But it is si…

> At work, someone used fast NVMe instance storage for Clickhouse which is a database. It was a huge hassle to copy data when instances were going to be restarted because the data would be lost. This post on how Discord RAIDed local NVMe volumes with slower remote volumes might be on interest https://discord.com/blog/how-discord-supercharges-network-di...

We moved to running Clickhouse on EKS with EBS volumes for storage. It can better survive instances going down. I didn't work on it so don't how much slower it is. Lowering the management burden was big priority.

Re: SSDs have become fast, except in the cloud

#213

Post by madars »

catherinecodes wrote: post #36
teaearlgraycold wrote: post #2

What’s a good small cloud competitor to AWS? For teams that just need two AZs to get HA and your standard stuff like VMs, k8s, etc.

Hetzner and Entrywan are pure-play cloud companies with good prices and support. Hetzner is based in Germany and Entrywan in the US.

Hetzner has a reputation for locking accounts for "identity verification" (Google "hetzner kyc" or "hetzner identity verification"). Might be worthwhile to go through a reseller just to avoid downtime like that.

Re: SSDs have become fast, except in the cloud

#214

Post by paulddraper »

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

> and fundamentally have to be

Can you expound?

Re: SSDs have become fast, except in the cloud

#215

Post by tptacek »

yolovoe wrote: post #195
pclmulqdq wrote: post #42

Earlier quoted context omitted.

It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.

You’re wrong. Instance local means SSD is physically attached to the droplet and is inside the server chassis, connected via PCIe. Sourece: I work on nitro cards.

"Attached to the droplet"?

Re: SSDs have become fast, except in the cloud

#216

Post by scottlamb »

kccqzy wrote: post #194
scottlamb wrote: post #145

Earlier quoted context omitted.

Interesting. I worked at Google until January 2021. I see 2019 dates on that PDF, but I wasn't aware of snap when I left. There was some alternate RPC approach (Pony Express, maybe? I get the names mixed up) that claimed 10 µs or so but was advertised as experimental (iirc had some bad failure modes at the time in practice) and was simply unavailable in many of the datacenters I needed to deploy in. Maybe they're two…

Snap was the external name for the internal project known as User Space Packet Service (abbreviated USPS) so naturally they renamed it prior to publication. I deployed an app using Pony Express in 2023 and it was available in the majority of cells worldwide. Pony Express supported more than just RPC though. The alternate RPC approach that you spoke of was called Void. It had been experimental for a long time and inde…

Interesting, thanks!

> If you and I still worked at Google I'd just give you an automon dashboard link showing latency an order of magnitude better than that to prove myself…

I believe you, and I think in principle we should all be getting the 50 µs latency you're describing within a datacenter with no special effort.

...but it doesn't match what I observed, and I'm not sure why. Maybe difference of a couple years. Maybe I was checking somewhere with older equipment, or some important config difference in our tests. And obviously my memory's a bit fuzzy by now but I know I didn't like the result I got.

Re: SSDs have become fast, except in the cloud

#217

Post by bombcar »

wmf wrote: post #153
doubled112 wrote: post #93

Earlier quoted context omitted.

Amusing is a good description. "Hardware degradation detected, please turn it off and back on again" I could do a migration with zero downtime in VMware for a decade but they can't seamlessly move my VM to a machine that works in 2024? Great, thanks. Amusing.

Cloud providers have live migration now but I guess they don't want to guarantee anything.

It's better (and better still with other providers) but I naively thought that "add more RAM" or "add more disk" was something they would be able to do with a reboot at most.

Nope, some require a full backup and restore.

Re: SSDs have become fast, except in the cloud

#218

Post by dekhn »

pclmulqdq wrote: post #143
jsnell wrote: post #71

Earlier quoted context omitted.

I think you're wrong about that. AWS calls this class of storage "instance storage" [0], and defines it as: > Many Amazon EC2 instances can also include storage from devices that are located inside the host computer, referred to as instance storage. There might be some wiggle room in "physically attached", but there's none in "storage devices located inside the host computer". It's not some kind of AWS-only thing eit…

That's the abstraction they want you to work with, yes. That doesn't mean it's what is actually happening - at least not in the same way that you're thinking. As a hint for you, I said " a network", not " the network." You can also look at public presentations about how Nitro works.

it sounds like you're trying to say "PCI switch" without saying "PCI switch" (I worked at Google for over a decade, including hardware division).

Re: SSDs have become fast, except in the cloud

#219

Post by jiggawatts »

computerdork wrote: post #202
jiggawatts wrote: post #187

There's a lot of talk about cloud network and disk performance in this thread. I recently benchmarked both Azure and AWS and found that: - Azure network latency is about 85 microseconds. - AWS network latency is about 55 microseconds. - Both can do better, but only in special circumstances such as RDMA NICs in HPC clusters. - Cross-VPC or cross-VNET is basically identical. Some people were saying it's terribly slow,…

Interesting. And would you happen to have the numbers on the performance of the local SSD? Is it's read and write throughput up to the level of modern SSD's?

It's pretty much like how the article said. The cloud local SSDs are notably slower than what you'd get in an ordinary laptop, let alone a high-end server.

I'm not an insider and don't have any exclusive knowledge, but from reading a lot about the topic my impression is that the issue in both clouds is the virtualization overheads.

That is, having the networking or storage go through any hypervisor software layer is what kills the performance. I've seen similar numbers with on-prem VMware, Xen, and Nutanix setups as well.

Both clouds appear to be working on next-generation VM SKUs where the hypervisor network and storage functions are offloaded into 100% hardware, either into FPGAs or custom ASICs.

"Azure Boost" is Microsoft's marketing name for this, and it basically amounts to both local and remote disks going through an NVMe controller directly mapped into the memory space of the VM. That is, the VM OS kernel talks directly to the hardware, bypassing the hypervisor completely. This is shown in their documentation diagrams: https://learn.microsoft.com/en-us/azure/azure-boost/overview

They're claiming up to 3.8M IOPS for a single VM, which is 3-10x what you'd get out of a single NVMe SSD stick, so... not too shabby at all!

Similarly, Microsoft Azure Network Adapter (MANA) is the equivalent for the NIC, which will similarly connect the VM OS directly into the network, bypassing the hypervisor software.

I'm not an AWS expert, but from what I've seen they've been working on similar tech (Nitro) for years.

Re: SSDs have become fast, except in the cloud

#220

Post by someguydave »

kwillets wrote: post #155

AWS docs and blogs describe the Nitro SSD architecture, which is locally attached with custom firmware. > The Nitro Cards are physically connected to the system main board and its processors via PCIe, but are otherwise logically isolated from the system main board that runs customer workloads. https://docs.aws.amazon.com/whitepapers/latest/security-desi... > In order to make the [SSD] devices last as long as possible…

Yeah I wonder how Nitro balances the latency & bandwidth demands of multiple VMs while also minimizing memory cache misses on the CPU (I am assuming it uses DMA to talk to the main CPU cores)
Post reply on HN