Live data from Hacker News

io_uring is faster than mmap

bitflux.ai

51–60 of 143 posts

Re: io_uring is faster than mmap

#51

Someone who’s read it in more detail, it looks like the uring code is optimized for async, while the mmap code doesn’t do any prefetching so just chokes when the OS has to do work?

My first thought is that what's different here isn't async, per se, but parallelism. io_uring uses a kernel thread pool to service I/O requests, so you actually end up with multiple threads running in parallel handling bookkeeping work. AFAIU, SSD controllers also can service requests in parallel, even if the request stream is serialized. These two sources of parallelism is why the I/O results come back out-of-order.

Generic readahead, which is what the mmap case is relying on, benefits from at least one async thread running in parallel, but I suspect for any particular file you effectively get at most one thread running in parallel to fill the page cache.

What may also be important is the VM management. The splice and vmsplice syscalls came about because someone requested that Linux adopt a FreeBSD optimization--for sufficiently sized write calls (i.e. page size or larger), the OS would mark the page(s) CoW and zero-copy the data to disk or the network. But Linus measured that the cost of fiddling with VM page attributes on each call was too costly and erased most of the zero-copy benefit. So another thing to take note of is that the io_uring case doesn't induce any page faults at all or require any costly VM fiddling (the shared io_uring buffers are installed upfront), whereas in the mmap case there are many page faults and fixups, possibly as many as one for every 4K page. The io_uring case may even result in additional data copies, but with less cost than the VM fiddling, which is even greater now than 20 years ago.

Re: io_uring is faster than mmap

#52
post #42

Earlier quoted context omitted.

on server chips it's kind of ridiculous. 5th gen Epyc has 128 lanes of PCIEx5 for over 1TB/s of pcie bandwith (compared to 600GB/s RAM bandwidth from 12 channel ddr5 at 6400)

Your math is a bit off. 128 lanes gen5 is 8 times x16, which has a combined theoretical bandwidth of 512GB/s, and more like 440GB/s in practice after protocol overhead. Unless we are considering both read and write bandwidth, but that seems strange to compare to memory read bandwidth.

PCIe is full duplex while DDR5 is half duplex so in theory PCIe is higher. It's rare to max out PCIe in both directions though.

Re: io_uring is faster than mmap

#53

Very interesting article, thanks for publishing these tests! Is the manual loop unrolling really necessary to get vectorized machine code? I would have guessed that the highest optimization levels in LLVM would be able to figure it out from the basic code. That's a very uneducated guess, though. Also, curious if you tried using the MAP_POPULATE option with mmap. Could that improve the bandwidth of the naive in-memory…

YES! gcc and clang don't like to optimize this. But they do if you hardcode the size_bytes to an aligned value. It kind of makes sense, what if a user passes size_bytes as 3? With enough effort the compilers could handle this, but it's a lot to ask. I just ran MAP_POPULATE the results are interesting. It speeds up the counting loop. Same speed or higher as the my read() to a malloced buffer tests. HOWEVER... It takes…

Hmm, I expected some slowdown from POPULATE, but I thought it would still come out ahead. Interesting!

Re: io_uring is faster than mmap

#54

Earlier quoted context omitted.

Kernel doesn't really care about "on disk", it cares about "on filesystem". The "on disk" distinction is a simplification.

The kernel absolutely does care about the "on disk" distinction because it determines what driver to use.

The interface is handled by the kernel.

Re: io_uring is faster than mmap

#55
post #52
post #42

Earlier quoted context omitted.

Your math is a bit off. 128 lanes gen5 is 8 times x16, which has a combined theoretical bandwidth of 512GB/s, and more like 440GB/s in practice after protocol overhead. Unless we are considering both read and write bandwidth, but that seems strange to compare to memory read bandwidth.

PCIe is full duplex while DDR5 is half duplex so in theory PCIe is higher. It's rare to max out PCIe in both directions though.

happens frequently in fact when training neural nets on modern hw

Re: io_uring is faster than mmap

#56

Wait, PCIe bandwidth is higher than memory bandwidth now? That's bonkers, when did that happen? I haven't been keeping up. Just looked at the i9-14900k and I guess it's true, but only if you add all the PCIe lanes together. I'm sure there are other chips where it's even more true. Crazy!

"No." DDR5-8000 is 64GB/s per channel. Desktop CPUs have two channels. PCI-E 5.0 in x16 is 64GB/s. Desktops have one x16.

One x16 slot. They'll use PCIe lanes in other slots (x4, x1, M2 SSDs) and also for devices off the chipset (network, USB, etc). The current top AMD/Intel CPUs can do ~100GB/sec over 28 lanes of mostly PCIe 5.

Re: io_uring is faster than mmap

#57
> A few notes for the "um actually" haters commenting on Hacker News

Stay classy; any criticism is of course "hating", right?

The fact that your title is clickbaity and your results suspect should encourage you to get the most accurate picture, not shoot the messenger.

Re: io_uring is faster than mmap

#58

Cool. Original author here. AMA.

Like people mention, hugetlb,etc could be an improvement, but the core issue holding it it down probably has to do with mmap, 4k pages and paging behaviours, mmap will cause faults for each "small" 4k page not in memory, causing a kernel jump and then whatever machinery to fill in the page-cache (and bring up data from disk with the associated latency).

This in contrast with the io_uring worker method where you keep the thread busy by submitting requests and letting the kernel do the work without expensive crossings.

The 2g fully in-mem shows the CPU's real perf, the dip to 50gb is interesting, perhaps when going over 50% memory the Linux kernel evicts pages or something similar that is hurting perf, maybe plot a graph of perf vs test-size to see if there is an obvious cliff.

Re: io_uring is faster than mmap

#59
post #8

Shouldn't this be "io_uring is faster than mmap"? I guess that would not get much engagement though! That said, cool write up and experiment.

Lol. Thanks.

Its not even about clickbait for me, but I really dont want to go parse an article to figure out what is meant by "Memory is slow, Disk is fast". You want "clickbait" to make people click and think, we want descriptive tittles to know what the article is about before we read it. That used to be original purpose of tittles, we like it that way.

Its like as if youd label your food product "you wont believe this", and forced customers to figure what it is from ingredients list.

Re: io_uring is faster than mmap

#60
This is wrong, because your mmap code is being stalled for page faults (including soft page faults that you have when the data is in memory, but not mapped to your process).

The io_uring code looks like it is doing all the fetch work in the background (with 6 threads), then just handing the completed buffers to the counter.

Do the same with 6 threads that would first read the first byte on each page and then hand that page section to the counter, you'll find similar performance.

And you can use both madvice / huge pages to control the mmap behavior

Post reply on HN