Live data from Hacker News

Modern storage is plenty fast, but the APIs are bad

itnext.io

101–110 of 168 posts

Re: Modern storage is plenty fast, but the APIs are bad

#101
post #81

Earlier quoted context omitted.

Optane has been commercially available for five years already and it's not used in any device I'm aware of. Assuming it will find broad adoption at this point seems like a bad bet.

Consumer NVMe devices can deliver GB/s I/O and hundreds of thousands of iops. The article's point doesn't hinge on Optane at all.

please ask my several NVMe devices to take notice ! actual performance under Linux OS is far less than that, here

Re: Modern storage is plenty fast, but the APIs are bad

#102
This is such a big deal! The assumptions made when IO APIs where designed are so out-of-step with today's hardware that it really is a time to have a big rethink. In graphics, the last 20 years of API development have very much been focused on harnessing a GPU that have again and again outgrown the CPUs ability to feed it. So much have been learned, and we really need to apply this to both storage and networking.

Re: Modern storage is plenty fast, but the APIs are bad

#103

From the author's previous piece: https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-wi... > Our CTO, Avi Kivity, made the case for async at the Core C++ 2019 event. The bottom line is this; in modern multicore, multi-CPU devices, the CPU itself is now basically a network, the intercommunication between all the CPUs is another network, and calls to disk I/O are effectively another. There are good reasons why n…

The problem is forcing everyone to program in asynchronous way.

WinRT tried to go down that route (only asynchronous APIs), to drive developers into that path, but eventually they had to support synchronous as well due to the received resistence.

Re: Modern storage is plenty fast, but the APIs are bad

#104
post #74

Earlier quoted context omitted.

Here's what jeffbee is talking about: https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-di...

Yep. If you are sweating a microsecond, 100 nanoseconds is significant chunk of your budget. For this and possibly other reasons, a many-core CPU isn't always a great choice for hosted storage. If your goal is to export NVMe blocks over a network interface, you might be better off with an easier-to-program 4- or 8-core CPU. I don't like seeing 128 cores and a bunch of NVMe devices in the same box because it just caus…

With p2pdma[1], it seems you don't even need a fancy CPU to push GB/s.

[1]: https://www.youtube.com/watch?v=LDOlqgUZtHE (9:40)

Re: Modern storage is plenty fast, but the APIs are bad

#105

Earlier quoted context omitted.

Consumer NVMe devices can deliver GB/s I/O and hundreds of thousands of iops. The article's point doesn't hinge on Optane at all.

please ask my several NVMe devices to take notice ! actual performance under Linux OS is far less than that, here

I just tested my laptop with the Ubuntu benchmark tool on the partition editor. 3.5GB/s read on 100MB chunks.

Re: Modern storage is plenty fast, but the APIs are bad

#106
post #92

Earlier quoted context omitted.

Why not consider nVME in this case then as cheaper than RAM, slower than RAM, but faster than network storage? I don't know how you handle concurrency btwn VMs or virtualize that storage, but there must be some standard for that?

I think a lot of it depends what the machines are used for. I'm not actually the IT department, but I believe in my org, we started out with a SAN-backed high availability cluster, because the immediate initial need was getting basic infrastructure (wiki, source control, etc) off of a dedicated machine that was a single point of failure. But then down the road a different set of hosts were brought online that had fas…

I’m laughing a little bit because an old place I used to work had a similar setup. The SAN/NAS/whatever it was was pretty slow, provisioning VMs was slow, and as much as we argued that we didn’t need redundancy for a lot of our VMs (they were semi-disposable), the IT department refused to give us a fast non-redundant machine.

And then one day the SAN blew up. Some kind of highly unlikely situation where more disks failed in a 24h period than it could handle, and we lost the entire array. Most of the stuff was available on tapes, but rebuilding the whole thing resulted in a significant period of downtime for everyone.

It ended up being a huge win for my team, since we had been in the process of setting up Ansible scripts to provision our whole system. We grabbed an old machine and had our stuff back up and running in about 20 minutes, while everyone else was manually reinstalling and reconfiguring their stuff for days.

Re: Modern storage is plenty fast, but the APIs are bad

#107
post #64
post #36

Earlier quoted context omitted.

> I expect the price of NVMe drives to drop over the next few years until they're cheap enough that the majority of computers are running NVMe drives. Price no longer has anything to do with it. PC OEMs are simply not shipping SATA SSDs any more, and major drive vendors have started to discontinue their client (OEM) SATA SSD product lines. We're just waiting for the SATA-based PC install base to be retired.

I have 8 SATA SSDs in my workstation; are there motherboards that could run a similar NVMe setup?

You can get NVME PCIe cards which has on-board PCIe switch. Random example, here's[1] one with 4 M.2 slots sharing an x8 PCIe slot.

Obviously sustained bandwidth is limited to that of effectively two NVME devices, but if you're doing lots of random I/O I guess it's a win.

[1]: https://www.aliexpress.com/item/4000034598072.html

Re: Modern storage is plenty fast, but the APIs are bad

#108
post #103

From the author's previous piece: https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-wi... > Our CTO, Avi Kivity, made the case for async at the Core C++ 2019 event. The bottom line is this; in modern multicore, multi-CPU devices, the CPU itself is now basically a network, the intercommunication between all the CPUs is another network, and calls to disk I/O are effectively another. There are good reasons why n…

The problem is forcing everyone to program in asynchronous way. WinRT tried to go down that route (only asynchronous APIs), to drive developers into that path, but eventually they had to support synchronous as well due to the received resistence.

Before Herb Sutter's free lunch is over, not many writes in multi-threading code. Then today, everyone is writing multi-threading code, one way or another; and majority indirectly in newer languages like Go and Rust, or better frameworks like Actors, message passing, coroutines, and some noble souls who are capable enough, in classic pthread and other threading APIs.

Of course everyone is going to program in an async way; that's how nature works.

But it certainly will not be in a fashinon that is repulsive to you, just give it some time. Maybe 10 years.

Re: Modern storage is plenty fast, but the APIs are bad

#109
post #4

Also: Modern storage is plenty fast, but also not reliable for long term use. That is why I buy a new SSD every year and clone my current (worn out) SSD to the new one. I have several old SSDs that started to get unhealthy, well, according to my S.M.A.R.T utility that I used to check them. I could probably get away with using an SSD for another year, but will not risk the data loss. Anyone else do this?

When ssds fail they don’t lose your data, they just become unwritable. What you’re doing is unnecessary and wasteful.

I've only had two SSDs fail on me, and in both cases they died without any warning. Didn't get discovered during boot or anything. Two different brands, very different uses.

So while they _can_ fail in a graceful way, that's not been my experience.

Re: Modern storage is plenty fast, but the APIs are bad

#110
post #12

Earlier quoted context omitted.

Every year seems like a very short lifespan, but I guess every usecase is different. I definitely replace drive when SMART is starting to look bleak, but that is far more infrequent in my usecase I guess.

> Every year seems like a very short lifespan Yes but I forgot to mention I do a lot of heavy writes to it. It is common to see me creating a huge 20GB virtual machine disk image, using it for a few hours, then deleting it, before creating a new one in its place. I'm a huge virtualization freak.

> It is common to see me creating a huge 20GB virtual machine disk image, using it for a few hours, then deleting it

The SSD in my current desktop, a Samsung 960 Pro 1TB, has a warranty for 800 TBW or 5 years. So that's 800/5/365.25*1000 ~= 438 GB per day, every single day.

And it's been documented the Samsung drives can do a lot more than the warranty is good for.

Either you're doing something else weird, or you're not really wearing them out.

[1]: https://www.samsung.com/semiconductor/minisite/ssd/product/c...

Post reply on HN