SSDs have become fast, except in the cloud • Hacker News Board

Re: SSDs have become fast, except in the cloud

#241

Post by jeffbee » Tue, Feb 20, 2024, 10:47 PM UTC

zokier wrote: post #230

jeffbee wrote: post #146

Earlier quoted context omitted.

I would start with the LBA format, which is likely to be suboptimal for compatibility.

somehow I4g drives don't like to get formatted # nvme format /dev/nvme1 -n1 -f NVMe status: INVALID_OPCODE: The associated command opcode field is not valid(0x2001) # nvme id-ctrl /dev/nvme1 | grep oacs oacs : 0 but the LBA format indeed is sus: LBA Format 0 : Metadata Size: 0 bytes - Data Size: 512 bytes - Relative Performance: 0 Best (in use)

It's a shame. The recent "datacenter nvme" standards involving fb, goog, et al mandate 4K LBA support.

Re: SSDs have become fast, except in the cloud

#242

Post by Ericson2314 » Tue, Feb 20, 2024, 10:47 PM UTC

The cloud really is a scam for those afraid of hardware

Re: SSDs have become fast, except in the cloud

#243

Post by kristopolous » Tue, Feb 20, 2024, 10:48 PM UTC

LeifCarrotson wrote: post #73

malfist wrote: post #45

Earlier quoted context omitted.

I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower than CPU Cache. Samsung 990 Pro 2TB has a latency of 40 μs DDR4-2133 with a CAS 15 has a latency of 14 nano seconds. DDR4 latency is 0.035% of one of the fastest SSDs, or to put it another way, DDR4 is 2,857x faster than an SSD. L1 cache is typically accessible i…

I wonder how many people have built failed businesses that never had enough customer data to exceed the DDR4 in the average developer laptop, and never had so many simultaneous queries it couldn't be handled by a single core running SQLite, but built the software architecture on a distributed cloud system just in case it eventually scaled to hundreds of terabytes and billions of simultaneous queries.

You're not considered serious if you don't. Kinda stupid.

Re: SSDs have become fast, except in the cloud

#244

Post by bravetraveler » Tue, Feb 20, 2024, 10:54 PM UTC

felixg3 wrote: post #174

queuebert wrote: post #165

Earlier quoted context omitted.

How do these machines manage the sharing of one local SSD across multiple VMs? Is there some wrapper around the I/O stack? Does it appear as a network share? Geniuinely curious...

Probably NVME namespaces [0]? [0]: https://nvmexpress.org/resource/nvme-namespaces/

Less fancy, quite often... at least on VPS providers [1]. They like to use reflinked files off the base images. This way they only store what differs.

1: Which is really a cloud without a certain degree of software defined networking/compute/storage/whatever.

Re: SSDs have become fast, except in the cloud

#245

Post by bravetraveler » Tue, Feb 20, 2024, 10:56 PM UTC

magicalhippo wrote: post #203

queuebert wrote: post #165

Earlier quoted context omitted.

How do these machines manage the sharing of one local SSD across multiple VMs? Is there some wrapper around the I/O stack? Does it appear as a network share? Geniuinely curious...

In say VirtualBox you can create a file backed on the physical disk, and attach it to the VM so the VM sees it as a NVMe drive. In my experience this is also orders of magnitude slower that true direct access, ie PCIe pass-through, as all access has to pass through the VM storage driver and so could explain what is happening.

The storage driver may have more impact on VBox. You can get very impressive results with 'virtio' on KVM

Re: SSDs have become fast, except in the cloud

#246

Post by bravetraveler » Tue, Feb 20, 2024, 10:58 PM UTC

icedchai wrote: post #225

queuebert wrote: post #165

Earlier quoted context omitted.

How do these machines manage the sharing of one local SSD across multiple VMs? Is there some wrapper around the I/O stack? Does it appear as a network share? Geniuinely curious...

With Linux and KVM/QEMU, you can map an entire physical disk, disk partition, or file to a block device in the VM. For my own VM hosts, I use LVM and map a logical volume to the VM. I assumed cloud providers did something conceptually similar, only much more sophisticated.

Files with reflinks are a common choice, the main benefit being: only storing deltas. The base OS costs basically nothing

LVM/block like you suggest is a good idea. You'd be surprised how much access time is trimmed by skipping another filesystem like you'd have with a raw image file

Re: SSDs have become fast, except in the cloud

#247

Post by zokier » Tue, Feb 20, 2024, 10:59 PM UTC

eisa01 wrote: post #122

Would this be a consequence of the cloud providers not being on the latest technology CPU-wise? At least I have the impression they are lagging, eg., still offering things like: z1d: Skylake (2017) https://aws.amazon.com/ec2/instance-types/z1d/ x2i: Cascade Lake (2019) and Ice lake (2021) https://aws.amazon.com/ec2/instance-types/x2i/ I have not been able to find instances powered by the 4th (Q1 2023) or 5th generati…

AWS was offering Sapphire Rapids instances before those CPUs became even publicly available

https://aws.amazon.com/about-aws/whats-new/2022/11/introduci...

Re: SSDs have become fast, except in the cloud

#248

Post by zokier » Tue, Feb 20, 2024, 11:02 PM UTC

StillBored wrote: post #228

Its worse than the article mentions. Because bandwidth isn't the problem its IOPS that are the problem. Last time (about a year ago) I ran a couple random IO benchmarks against a storage optimized instances and the random IOPs behavior is closer to a large spinning RAID array than SSDs if the disk size is over some threshold. IIRC, What it looks like is that there is a fast local SSD cache with a couple hundred GB of…

  # fio --name=read_iops_test   --filename=/dev/nvme1n1 --filesize=1500G   --time_based --ramp_time=1s --runtime=15s   --ioengine=io_uring --fixedbufs --direct=1 --verify=0 --randrep
  eat=0   --bs=4K --iodepth=256 --rw=randread   --iodepth_batch_submit=256  --iodepth_batch_complete_max=256
  read_iops_test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=256
  fio-3.32
  Starting 1 process
  Jobs: 1 (f=1): [r(1)][100.0%][r=2082MiB/s][r=533k IOPS][eta 00m:00s]
  read_iops_test: (groupid=0, jobs=1): err= 0: pid=34235: Tue Feb 20 22:57:00 2024
    read: IOPS=534k, BW=2086MiB/s (2187MB/s)(30.6GiB/15001msec)
      slat (nsec): min=713, max=255840, avg=31174.74, stdev=16248.45
      clat (nsec): min=1419, max=1175.6k, avg=443782.26, stdev=277389.66
      lat (usec): min=133, max=1240, avg=474.96, stdev=274.50
      clat percentiles (usec):
      |  1.00th=[  169],  5.00th=[  198], 10.00th=[  217], 20.00th=[  243],
      | 30.00th=[  265], 40.00th=[  285], 50.00th=[  306], 60.00th=[  334],
      | 70.00th=[  396], 80.00th=[  865], 90.00th=[  922], 95.00th=[  947],
      | 99.00th=[  996], 99.50th=[ 1012], 99.90th=[ 1045], 99.95th=[ 1057],
      | 99.99th=[ 1074]
    bw (  MiB/s): min= 2080, max= 2092, per=100.00%, avg=2086.72, stdev= 2.35, samples=30
    iops        : min=532548, max=535738, avg=534199.13, stdev=601.82, samples=30
    lat (usec)   : 2=0.01%, 100=0.01%, 250=23.06%, 500=50.90%, 750=0.28%
    lat (usec)   : 1000=24.90%
    lat (msec)   : 2=0.87%
    cpu          : usr=14.17%, sys=67.83%, ctx=156851, majf=0, minf=37
    IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
      submit    : 0=0.0%, 4=7.8%, 8=11.3%, 16=39.7%, 32=30.6%, 64=10.5%, >=64=0.1%
      complete  : 0=0.0%, 4=5.3%, 8=9.5%, 16=40.3%, 32=32.4%, 64=12.4%, >=64=0.1%
      issued rwts: total=8010661,0,0,0 short=0,0,0,0 dropped=0,0,0,0
      latency   : target=0, window=0, percentile=100.00%, depth=256

  Run status group 0 (all jobs):
    READ: bw=2086MiB/s (2187MB/s), 2086MiB/s-2086MiB/s (2187MB/s-2187MB/s), io=30.6GiB (32.8GB), run=15001-15001msec

  Disk stats (read/write):
    nvme1n1: ios=8542481/0, merge=0/0, ticks=3822266/0, in_queue=3822266, util=99.37%

tldr: random 4k reads pretty much saturate the available 2GB/s bandwidth (this is on m6id)

Re: SSDs have become fast, except in the cloud

#249

Post by Aurornis » Tue, Feb 20, 2024, 11:12 PM UTC

c0l0 wrote: post #48

Seeing the really just puny "provisioned IOPS" numbers on hugely expensive cloud instances made me chuckle (first in disbelief, then in horror) when I joined a "cloud-first" enterprise shop in 2020 (having come from a company that hosted their own hardware at a colo). It's no wonder that many people nowadays, esp. those who are so young that they've never experienced anything but cloud instances, seem to have little…

> It's no wonder that many people nowadays, esp. those who are so young that they've never experienced anything but cloud instances, seem to have little idea of how much performance you can actually pack in just one or two RUs today.

On the contrary, young people often show up having learned on their super fast Apple SSD or a top of the line gaming machine with NVMe SSD.

Many know what hardware can do. There’s no need to dunk on young people.

Anyway, the cloud performance realities are well know to anyone who works in cloud performance. It’s part of the game and it’s learned by anyone scaling a system. It doesn’t really matter what you could do if you build a couple RUs yourself and hauled them down to the data center, because beyond simple single-purpose applications with flexible uptime requirements, that’s not a realistic option.

Re: SSDs have become fast, except in the cloud

#250

Post by jltsiren » Tue, Feb 20, 2024, 11:15 PM UTC

malfist wrote: post #45

siliconc0w wrote: post #13

Core count plus modern nvme actually make a great case for moving away from the cloud- before it was, "your data probably fits into memory". These are so fast that they're close enough to memory so it's "your data surely fits on disk". This reduces the complexity of a lot of workloads so you can just buy a beefy server and do pretty insane caching/calculation/serving with just a single box or two for redundancy.

I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower than CPU Cache. Samsung 990 Pro 2TB has a latency of 40 μs DDR4-2133 with a CAS 15 has a latency of 14 nano seconds. DDR4 latency is 0.035% of one of the fastest SSDs, or to put it another way, DDR4 is 2,857x faster than an SSD. L1 cache is typically accessible i…

RAM is not as fast in practice as the specs claim, because there is a lot of overhead in accessing it. I did some latency benchmarking on my M2 Max MBP when I got it last year. As long as the working set fits in L1 cache, read latency is ~2 ns. Then it starts increasing slowly, reaching ~10 ns at 10 MiB. Then there is a rapid rise to ~100 ns at 100 MiB, followed by slow growth until ~10 GiB. Then the latency starts increasing rapidly again, reaching ~330 ns at 64 GiB.