Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern storage is plenty fast, but the APIs are bad

itnext.io

Post reply on HN
Print view View original

81–90 of 168 posts

Re: Modern storage is plenty fast, but the APIs are bad

#81

Post by Miraste »

arka2147483647 wrote: post #55
mehrdadn wrote: post #32

I liked most of the piece, but some bits rubbed me the wrong way: > I was taken by surprise by the fact that although every one of my peers is certainly extremely bright, most of them carried misconceptions about how to best exploit the performance of modern storage technology leading to suboptimal designs, even if they were aware of the increasing improvements in storage technology. > In the process of writing this…

I read that as that it was what he had at hand when running the tests. Also, if the speed and features are available as professional grade devices today, it will be available everywhere in a few years.

Optane has been commercially available for five years already and it's not used in any device I'm aware of. Assuming it will find broad adoption at this point seems like a bad bet.

Re: Modern storage is plenty fast, but the APIs are bad

#82

Post by sleepydog »

rtkwe wrote: post #57
mehrdadn wrote: post #32

I liked most of the piece, but some bits rubbed me the wrong way: > I was taken by surprise by the fact that although every one of my peers is certainly extremely bright, most of them carried misconceptions about how to best exploit the performance of modern storage technology leading to suboptimal designs, even if they were aware of the increasing improvements in storage technology. > In the process of writing this…

Yeah how many people are running apps on servers served at all or even partially by NVMe SSDs? Where I work for our on prem stuff it's basically all network storage.

For network storage some of his points are even stronger. Sure, the page cache becomes more useful as latency goes up but it also becomes more important to send more I/O at once, something that is hard to do with blocking APIs like read(2) and write(2). The page cache is pretty good at optimizing sequential I/O to do this, but not random I/O or workloads where you need to sync().

Re: Modern storage is plenty fast, but the APIs are bad

#83

Post by wtallis »

bob1029 wrote: post #54

One thing I have started to realize is that best case latency of an NVMe storage device is starting to overlap with areas where SpinWait could be more ideal than an async/await API. I am mostly advocating for this from a mass parallel throughput perspective, especially if batching is possible. I have started to play around with using LMAX Disruptor for aggregating a program's disk I/O requests and executing them in b…

On Linux, it's already a NVMe driver option to enable polling for (high priority) IO completion rather than sleeping until an interrupt. The latency of handling an interrupt and doing a couple of context switches is higher than the best-case latency for fast SSDs. The io_uring userspace API also has a polling mode.

Re: Modern storage is plenty fast, but the APIs are bad

#84

Post by foolmeonce »

blindm wrote: post #4

Also: Modern storage is plenty fast, but also not reliable for long term use. That is why I buy a new SSD every year and clone my current (worn out) SSD to the new one. I have several old SSDs that started to get unhealthy, well, according to my S.M.A.R.T utility that I used to check them. I could probably get away with using an SSD for another year, but will not risk the data loss. Anyone else do this?

I would add a new drive with zfs mirroring and enable simple compression. For most use cases it gets better read performance, ok write performance, and can tolerate both of the drives being a bit flaky so you can run it for a lot longer than the new drive alone.

Re: Modern storage is plenty fast, but the APIs are bad

#85

Post by mehrdadn »

Miraste wrote: post #81
arka2147483647 wrote: post #55

Earlier quoted context omitted.

I read that as that it was what he had at hand when running the tests. Also, if the speed and features are available as professional grade devices today, it will be available everywhere in a few years.

Optane has been commercially available for five years already and it's not used in any device I'm aware of. Assuming it will find broad adoption at this point seems like a bad bet.

3 years I think: https://en.wikipedia.org/wiki/3D_XPoint

> It was announced in July 2015 and is available on the open market under brand names Optane (Intel) and subsequently QuantX (Micron) since April 2017.

For comparison, look at how many decades it took SSDs to become commonplace: https://en.wikipedia.org/wiki/Solid-state_drive#Flash-based_...

Re: Modern storage is plenty fast, but the APIs are bad

#86

Post by bhewes »

aidenn0 wrote: post #64
wtallis wrote: post #36

Earlier quoted context omitted.

> I expect the price of NVMe drives to drop over the next few years until they're cheap enough that the majority of computers are running NVMe drives. Price no longer has anything to do with it. PC OEMs are simply not shipping SATA SSDs any more, and major drive vendors have started to discontinue their client (OEM) SATA SSD product lines. We're just waiting for the SATA-based PC install base to be retired.

I have 8 SATA SSDs in my workstation; are there motherboards that could run a similar NVMe setup?

Sure you could run 24 NVMes with highpoint pcie4 raids on a trx40 board. But then most still have like 10 sata ports so you can run those as well. It will be great when sata is replaced by U.2 but who knows when that happens.

Re: Modern storage is plenty fast, but the APIs are bad

#87

Post by aidenn0 »

wtallis wrote: post #75
aidenn0 wrote: post #64

Earlier quoted context omitted.

I have 8 SATA SSDs in my workstation; are there motherboards that could run a similar NVMe setup?

I wasn't including the workstation market when I referred to what PC OEMs are doing. Are you using 8 consumer SATA SSDs in your workstation? Is it for the sake of increased capacity, or for the sake of increased performance? Because it's pretty easy now to match the performance of an 8-drive SATA RAID-0 with a single NVMe drive, but 8TB consumer NVMe SSDs are still 50% more expensive than 8TB consumer SATA SSDs. (Als…

Increased capacity. I started with 4 spinning disks, replaced them with SSDs a while ago and then grew it to 8.

Re: Modern storage is plenty fast, but the APIs are bad

#88

Post by fibers »

eightysixfour wrote: post #79
aidenn0 wrote: post #64

Earlier quoted context omitted.

I have 8 SATA SSDs in my workstation; are there motherboards that could run a similar NVMe setup?

Yes, using PCIe expansion cards. I know of an AMD board that ships with 5 (3 on the board, 2 with a PCIe card). Could easily add more.

is this with threadripper boards?

Re: Modern storage is plenty fast, but the APIs are bad

#89

Post by ggm »

blindm wrote: post #4

Also: Modern storage is plenty fast, but also not reliable for long term use. That is why I buy a new SSD every year and clone my current (worn out) SSD to the new one. I have several old SSDs that started to get unhealthy, well, according to my S.M.A.R.T utility that I used to check them. I could probably get away with using an SSD for another year, but will not risk the data loss. Anyone else do this?

Can somebody please write up modern SSD and state of the world regarding data retention, modes, applicability for SSD replacing spinning rust "on the shelf" offline...

Re: Modern storage is plenty fast, but the APIs are bad

#90

Post by cahooon »

kazinator wrote: post #76

This is a really poor article. Only in very rare circumstances can developers change the API's. API's are not "bad"; they are built to various important requirements. Only some of those requirements have to do with performance. > “Well, it is fine to copy memory here and perform this expensive computation because it saves us one I/O operation, which is even more expensive”. "I/O operation" in fact refers to the API c…

I didn't get the impression that the author was suggesting to throw out the old APIs. It seems to me like the article is a proof of concept of new approaches that could be added as new APIs, only expected to be used by people who need them, using an approach that takes advantage of modern storage technology.

> "Random access may have to wastefully read larger blocks of the data than are actually requested by the application. The unused data gets cached, but if it's not going to be accessed any time soon, it means that something else got wastefully bumped out of the cache. Sequential access is likely to make use of an entire block."

I may have misread it, but I thought he addressed this in the article.

> "Random access files take a position as an argument, meaning there is no need to maintain a seek cursor. But more importantly: they don’t take a buffer as a parameter. Instead, they use io_uring’s pre-registered buffer area to allocate a buffer and return to the user. That means no memory mapping, no copying to the user buffer — there is only a copy from the device to the glommio buffer and the user get a reference counted pointer to that. And because we know this is random I/O, there is no need to read more data than what was requested."

Post reply on HN