Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

io_uring is faster than mmap

bitflux.ai

Post reply on HN
Print view View original

91–100 of 143 posts

Re: io_uring is faster than mmap

#91

Post by kentonv »

nine_k wrote: post #69
avallach wrote: post #68

Maybe I'm misunderstanding, but after reading it sounds to me not like "io_uring is faster than mmap" but "raid0 with 8 SSDs has more throughput than 3 channel DRAM".

The title has been edited incorrectly. The original page title is "Memory is slow, Disk is fast", and it states exactly what you say: an NVMe RAID can offer more bandwidth than RAM.

No, the title edit is fair, where the original title is misleading.

Obviously, no matter how you read from disk, it has to go through RAM. Disk bandwidth cannot exceed memory bandwidth.*

But what the article actually tests is a program that uses mmap() to read from page cache, vs. a program that uses io_uring to read directly from disk (with O_DIRECT). You'd think the mmap() program would win, because the data in page cache is already in memory, whereas the io_uring program is explicitly skipping cache and pulling from disk.

However, the io_uring program uses 6 threads to pull from disk, which then feed into one thread that sequentially processes the data. Whereas the program using mmap() uses a single thread for everything. And even though the mmap() is pulling from page cache, that single thread still has to get interrupted by page faults as it reads, because the kernel does not proactively map the pages from cache even if they are available (unless, you know, you tell it to, with madvise() etc., but the test did not). So the mmap() test has one thread that has to keep switching between kernel and userspace and, surprise, that is not as fast as a thread which just stays in userspace while 6 other threads feed it data.

To be fair, the article says all this, if you read it. Other than the title being cheeky it's not hiding anything.

* OK, the article does mention that there exists CPUs which can do I/O directly into L3 cache which could theoretically beat memory bandwidth, but this is not actually something that is tested in the article.

Re: io_uring is faster than mmap

#93

Post by icedchai »

lordgilman wrote: post #90
immibis wrote: post #80

Earlier quoted context omitted.

How do you do embarrassingly async memory access with mmap?

You dereference a pointer.

From the application perspective, it's not truly async. On a deference, your app may be blocked indefinitely as data is paged into memory. In the early 2000's I worked on systems that made heavy use of mmap. In constrained ("dev") environments with slow disks, you could be blocked for several seconds...

Re: io_uring is faster than mmap

#94

Post by mrlongroots »

ayende wrote: post #60

This is wrong, because your mmap code is being stalled for page faults (including soft page faults that you have when the data is in memory, but not mapped to your process). The io_uring code looks like it is doing all the fetch work in the background (with 6 threads), then just handing the completed buffers to the counter. Do the same with 6 threads that would first read the first byte on each page and then hand tha…

Yes, it doesn't take a benchmark to find out that storage can not be faster than memory.

Even if you had a million SSDs and somehow were able to connect them to a single machine somehow, you would not outperform memory, because the data needs to be read into memory first, and can only then be processed by the CPU.

Basic `perf stat` and minor/major faults should be a first-line diagnostic.

Re: io_uring is faster than mmap

#95

Post by p_l »

worldsavior wrote: post #89

Probably off-topic, but could someone tell me what is the current status of io_uring from a security standpoint? It's still disabled on Android and some Linux distros, but it's being said it's safe now.

It's more that io_uring has not enough integration with LSMs and the like, so for example on Android it's disabled because Android heavily uses SElinux which would be bypassed by io_uring

Re: io_uring is faster than mmap

#96

Post by mrlongroots »

bawolff wrote: post #17

Shouldn't you also compare to mmap with huge page option? My understanding is its presicely meant for this circumstance. I don't think its a fair comparison without it. Respectfully, the title feels a little clickbaity to me. Both methods are still ultimately reading out of memory, they are just using different i/o methods.

You don't need hugepages for basic 5GB/s sequential scans. I don't know the exact circumstances that would cause TLB pressure, but this is not it.

You can maybe reduce the number of page faults, but you can do that by walking the mapped address space once before the actual benchmark too.

Re: io_uring is faster than mmap

#97

Post by jlokier »

icedchai wrote: post #93
lordgilman wrote: post #90

Earlier quoted context omitted.

You dereference a pointer.

From the application perspective, it's not truly async. On a deference, your app may be blocked indefinitely as data is paged into memory. In the early 2000's I worked on systems that made heavy use of mmap. In constrained ("dev") environments with slow disks, you could be blocked for several seconds...

This branch of the discussion is is about dereferencing on multiple threads concurrently. That doesn't block the application, each mmap'd dereference only blocks its own thread (same as doing read()).

In my own measurements with NVMe RAID, doing this works very well on Linux for storage I/O.

I was getting similar performance to io_uring with O_DIRECT, and faster performance when the data is likely to be in the page cache on multiple runs, because the multi-threaded mmap method shares the kernel page cache without copying data.

To measure this, replace the read() calls in the libuv thread pool function with single-byte dereferences, mmap a file, and call a lot of libuv async reads. That will make libuv do the dereferences in its thread pool and return to the main application thread having fetched the relevant pages. Make sure libuv is configured to use enough threads, as it doesn't use enough by default.

Re: io_uring is faster than mmap

#98

Post by alphazard »

mrlongroots wrote: post #94
ayende wrote: post #60

This is wrong, because your mmap code is being stalled for page faults (including soft page faults that you have when the data is in memory, but not mapped to your process). The io_uring code looks like it is doing all the fetch work in the background (with 6 threads), then just handing the completed buffers to the counter. Do the same with 6 threads that would first read the first byte on each page and then hand tha…

Yes, it doesn't take a benchmark to find out that storage can not be faster than memory. Even if you had a million SSDs and somehow were able to connect them to a single machine somehow, you would not outperform memory, because the data needs to be read into memory first, and can only then be processed by the CPU. Basic `perf stat` and minor/major faults should be a first-line diagnostic.

> storage can not be faster than memory

This is an oversimplification. It depends what you mean by memory. It may be true when using NVMe on modern architectures in a consumer use case, but it's not true about computer architecture in general.

External devices can have their memory mapped to virtual memory addresses. There are some network cards that do this for example. The CPU can load from these virtual addresses directly into registers, without needing to make a copy to the general purpose fast-but-volatile memory. In theory a storage device could also be implemented in this way.

Re: io_uring is faster than mmap

#99

Post by vlovich123 »

inetknght wrote: post #27
jared_hulbert wrote: post #21

Earlier quoted context omitted.

int fd = open(filename, O_RDONLY); void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); This doesn't work with a file on my ext4 volume. What am I missing?

What issue are you having? Are you receiving an error? This is the kind of question that StackOverflow or perhaps an LLM might be able to help you with. I highly suggest reading the documentation for mmap to understand what issues could happen and/or what a given specific error code might indicate; see the NOTES section: > Huge page (Huge TLB) mappings > For mappings that employ huge pages, the requirements for the a…

The most common problem is the system not having any support for hugetable allocated. Don’t have the specific things that need to be configured handy.

Re: io_uring is faster than mmap

#100

Post by benlwalker »

jared_hulbert wrote: post #30
josephg wrote: post #25

Earlier quoted context omitted.

I’d be fascinated to see a comparison with SPDK. That bypasses the kernel’s NVMe / SSD driver and controls the whole device from user space - which is supposed to avoid a lot of copies and overhead. You might be able to set up SPDK to send data directly into the cpu cache? It’s one of those things I’ve wanted to play with for years but honestly I don’t know enough about it. https://spdk.io/

spdk and I go way back. I'm confident it'd be about the same, possibly ~200-300MB/s more, I was pretty close to the rated throughput of the drives. Io_uring has really closed the gap that used to exist between the in kernel and userspace solutions. With the Intel connection they might have explicit support for DDIO. Good idea.

SPDK will be able to fully saturate the PCIe bandwidth from a single CPU core here (no secret 6 threads inside the kernel). The drives are your bottleneck so it won't go faster, but it can use a lot less CPU.

But with SPDK you'll be talking to the disk, not to files. If you changed io_uring to read from the disk directly with O_DIRECT, you wouldn't have those extra 6 threads either. SPDK would still be considerably more CPU efficient but not 6x.

DDIO is a pure hardware feature. Software doesn't need to do anything to support it.

Source: SPDK co-creator

Post reply on HN