Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

io_uring is faster than mmap

bitflux.ai

Post reply on HN
Print view View original

71–80 of 143 posts

Re: io_uring is faster than mmap

#71

Post by arghwhat »

lucketone wrote: post #65
ayende wrote: post #60

This is wrong, because your mmap code is being stalled for page faults (including soft page faults that you have when the data is in memory, but not mapped to your process). The io_uring code looks like it is doing all the fetch work in the background (with 6 threads), then just handing the completed buffers to the counter. Do the same with 6 threads that would first read the first byte on each page and then hand tha…

It would seem you summarised whole post. That’s the point: “mmap” is slow because it is serial.

mmap isn't "serial", the code that was using the mapping was "serial". The kernel will happily fill different portions of the mapping in parallel if you have multiple threads fault on different pages.

(That doesn't undermine that io_uring and disk access can be fast, but it's comparing a lazy implementation using approach A with a quite optimized one using approach B, which does not make sense.)

Re: io_uring is faster than mmap

#72

Post by nine_k »

hsn915 wrote: post #8

Shouldn't this be "io_uring is faster than mmap"? I guess that would not get much engagement though! That said, cool write up and experiment.

No. "io_uring faster than mmap" is sort of a truism: sequential page faults are slower than carefully orchestrated async I/O. The point of the article is that reading directly from a PCIe device, such as an NVMe flash, can actually be faster than caching things in RAM first.

Re: io_uring is faster than mmap

#73

Post by nextaccountic »

The real difference is that with io_uring and O_DIRECT you manage the cache yourself (and can't share with other processes, and the OS can't reclaim the cache automatically if under memory pressure), and with mmap this is managed by the OS.

If Linux had an API to say "manage this buffer you handled me from io_uring as if it were a VFS page cache (and as such it can be shared with other processes, like mmap), if you want it back just call this callback (so I can cleanup my references to it) and you are good to go", then io_uring could really replace mmap.

What Linux has currently is PSI, which lets the OS reclaim memory when needed but doesn't help with the buffer sharing thing

Re: io_uring is faster than mmap

#74

Post by rwmj »

modeless wrote: post #40

Wait, PCIe bandwidth is higher than memory bandwidth now? That's bonkers, when did that happen? I haven't been keeping up. Just looked at the i9-14900k and I guess it's true, but only if you add all the PCIe lanes together. I'm sure there are other chips where it's even more true. Crazy!

That's the promise (or requirement?) of CXL - have your memory managed centrally, servers access it over PCIe. https://en.wikipedia.org/wiki/Compute_Express_Link I wonder how many are actually using CXL. I haven't heard of any customers deploying it so far.

Re: io_uring is faster than mmap

#75

Post by Sesse__ »

inetknght wrote: post #33
loloquwowndueo wrote: post #32

Earlier quoted context omitted.

Share your code?

I don't work there any more (it was a decade ago) and I'm pretty busy right now with a new job coming up (offered today). Do you have kernel documentation that says that hugetlb doesn't work for files? I don't see that stated anywhere.

It's filesystem-dependent. In particular, tmpfs will work. To the best of my knowledge, no “normal” filesystems (e.g., ext4, xfs) will.

Re: io_uring is faster than mmap

#77

Post by amelius »

arghwhat wrote: post #71
lucketone wrote: post #65

Earlier quoted context omitted.

It would seem you summarised whole post. That’s the point: “mmap” is slow because it is serial.

mmap isn't "serial", the code that was using the mapping was "serial". The kernel will happily fill different portions of the mapping in parallel if you have multiple threads fault on different pages. (That doesn't undermine that io_uring and disk access can be fast, but it's comparing a lazy implementation using approach A with a quite optimized one using approach B, which does not make sense.)

OK, so we need a comparison between a multi threaded mmap approach and io_uring. Which would be faster?

Re: io_uring is faster than mmap

#78

Post by menaerus »

pixelpoet wrote: post #57

> A few notes for the "um actually" haters commenting on Hacker News Stay classy; any criticism is of course "hating", right? The fact that your title is clickbaity and your results suspect should encourage you to get the most accurate picture, not shoot the messenger.

People should just ignore such low-quality material and stop feeding the troll. Information found in both the first and second part of "Memory is slow, disk is fast" series is wrong on so many levels that it isn't worth correcting or commenting. It is obviously written with the help of the AI without actually fact-checking at all and all under the impression that the author is worthwhile which he isn't.

Just look at this bs:

> Early x86 processors took a few clocks to execute most instructions, modern processors have been able parallelize to where they can actually execute 2 instructions every clock.

Re: io_uring is faster than mmap

#79

Post by gpderetta »

menaerus wrote: post #78
pixelpoet wrote: post #57

> A few notes for the "um actually" haters commenting on Hacker News Stay classy; any criticism is of course "hating", right? The fact that your title is clickbaity and your results suspect should encourage you to get the most accurate picture, not shoot the messenger.

People should just ignore such low-quality material and stop feeding the troll. Information found in both the first and second part of "Memory is slow, disk is fast" series is wrong on so many levels that it isn't worth correcting or commenting. It is obviously written with the help of the AI without actually fact-checking at all and all under the impression that the author is worthwhile which he isn't. Just look at…

wait, P5 is not modern any more ? :D

Re: io_uring is faster than mmap

#80

Post by immibis »

arghwhat wrote: post #71
lucketone wrote: post #65

Earlier quoted context omitted.

It would seem you summarised whole post. That’s the point: “mmap” is slow because it is serial.

mmap isn't "serial", the code that was using the mapping was "serial". The kernel will happily fill different portions of the mapping in parallel if you have multiple threads fault on different pages. (That doesn't undermine that io_uring and disk access can be fast, but it's comparing a lazy implementation using approach A with a quite optimized one using approach B, which does not make sense.)

How do you do embarrassingly async memory access with mmap?
Post reply on HN