Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

io_uring is faster than mmap

bitflux.ai

Post reply on HN
Print view View original

21–30 of 143 posts

Re: io_uring is faster than mmap

#21

Post by jared_hulbert »

inetknght wrote: post #5

Nice write-up with good information, but not the best. Comments below. Are you using linux? I assume so since stating use of mmap() and mention using EPYC hardware (which counts out macOS). I suppose you could use any other *nix though. > We'll use a 50GB dataset for most benchmarking here, because when I started this I thought the test system only had 64GB and it stuck.* So the OS will (or could) prefetch the file i…

int fd = open(filename, O_RDONLY); void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0);

This doesn't work with a file on my ext4 volume. What am I missing?

Re: io_uring is faster than mmap

#23

Post by jared_hulbert »

titanomachy wrote: post #12

Very interesting article, thanks for publishing these tests! Is the manual loop unrolling really necessary to get vectorized machine code? I would have guessed that the highest optimization levels in LLVM would be able to figure it out from the basic code. That's a very uneducated guess, though. Also, curious if you tried using the MAP_POPULATE option with mmap. Could that improve the bandwidth of the naive in-memory…

YES! gcc and clang don't like to optimize this. But they do if you hardcode the size_bytes to an aligned value. It kind of makes sense, what if a user passes size_bytes as 3? With enough effort the compilers could handle this, but it's a lot to ask.

I just ran MAP_POPULATE the results are interesting.

It speeds up the counting loop. Same speed or higher as the my read() to a malloced buffer tests.

HOWEVER... It takes a longer time overall to do the population of the buffer. The end result is it's 2.5 seconds slower to run the full test when compared to the original. I did not guess that one correctly.

time ./count_10_unrolled ./mnt/datafile.bin 53687091200 unrolled loop found 167802249 10s processed at 5.39 GB/s ./count_10_unrolled ./mnt/datafile.bin 53687091200 5.58s user 6.39s system 99% cpu 11.972 total time ./count_10_populate ./mnt/datafile.bin 53687091200 unrolled loop found 167802249 10s processed at 8.99 GB/s ./count_10_populate ./mnt/datafile.bin 53687091200 5.56s user 8.99s system 99% cpu 14.551 total

Re: io_uring is faster than mmap

#24

Post by mastax »

jared_hulbert wrote: post #20
bawolff wrote: post #17

Shouldn't you also compare to mmap with huge page option? My understanding is its presicely meant for this circumstance. I don't think its a fair comparison without it. Respectfully, the title feels a little clickbaity to me. Both methods are still ultimately reading out of memory, they are just using different i/o methods.

The original blog post title is intentionally clickbaity. You know, to bait people into clicking. Also I do want to challenge people to really think here. Seeing if the cached file data can be accessed quickly is the point of the experiment. I can't get mmap() to open a file with huge pages. void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); doesn't work. You can can see my code he…

MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).

Re: io_uring is faster than mmap

#25

Post by josephg »

jared_hulbert wrote: post #18
john-h-k wrote: post #13

Earlier quoted context omitted.

You mention modern server CPUs have capability to “read direct to L3, skipping memory”. Can you elaborate on this?

https://www.intel.com/content/www/us/en/io/data-direct-i-o-t... AMD has something similar. The PCIe bus and memory bus both originate from the processor or IO die of the "CPU" when you use an NVMe drive you are really just sending it a bunch of structured DMA requests. Normally you are telling the drive to DMA to an address that maps to the memory, so you can direct it cache and bypass sending it out on the DRAM bus.…

I’d be fascinated to see a comparison with SPDK. That bypasses the kernel’s NVMe / SSD driver and controls the whole device from user space - which is supposed to avoid a lot of copies and overhead.

You might be able to set up SPDK to send data directly into the cpu cache? It’s one of those things I’ve wanted to play with for years but honestly I don’t know enough about it.

https://spdk.io/

Re: io_uring is faster than mmap

#26

Post by jandrewrogers »

jared_hulbert wrote: post #20
bawolff wrote: post #17

Shouldn't you also compare to mmap with huge page option? My understanding is its presicely meant for this circumstance. I don't think its a fair comparison without it. Respectfully, the title feels a little clickbaity to me. Both methods are still ultimately reading out of memory, they are just using different i/o methods.

The original blog post title is intentionally clickbaity. You know, to bait people into clicking. Also I do want to challenge people to really think here. Seeing if the cached file data can be accessed quickly is the point of the experiment. I can't get mmap() to open a file with huge pages. void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); doesn't work. You can can see my code he…

Read the man pages, there are restrictions on using the huge page option with mmap() that mean it won’t do what you might intuit it will in many cases. Getting reliable huge page mappings is a bit fussy on Linux. It is easier to control in a direct I/O context.

Re: io_uring is faster than mmap

#27

Post by inetknght »

jared_hulbert wrote: post #21
inetknght wrote: post #5

Nice write-up with good information, but not the best. Comments below. Are you using linux? I assume so since stating use of mmap() and mention using EPYC hardware (which counts out macOS). I suppose you could use any other *nix though. > We'll use a 50GB dataset for most benchmarking here, because when I started this I thought the test system only had 64GB and it stuck.* So the OS will (or could) prefetch the file i…

int fd = open(filename, O_RDONLY); void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); This doesn't work with a file on my ext4 volume. What am I missing?

What issue are you having? Are you receiving an error? This is the kind of question that StackOverflow or perhaps an LLM might be able to help you with. I highly suggest reading the documentation for mmap to understand what issues could happen and/or what a given specific error code might indicate; see the NOTES section:

> Huge page (Huge TLB) mappings

> For mappings that employ huge pages, the requirements for the arguments of mmap() and munmap() differ somewhat from the requirements for mappings that use the native system page size.

> For mmap(), offset must be a multiple of the underlying huge page size. The system automatically aligns length to be a multiple of the underlying huge page size.

Ensure that the file is at least the page size, and preferably sized to align with a page boundary. Then also ensure that the length parameter (size_bytes in your example) is also aligned to a boundary.

There are also other important things to understand for these flags, which are described in the documentation, such as information available from /sys/kernel/mm/hugepages

https://www.man7.org/linux/man-pages/man2/mmap.2.html

Re: io_uring is faster than mmap

#28

Post by inetknght »

mastax wrote: post #24
jared_hulbert wrote: post #20

Earlier quoted context omitted.

The original blog post title is intentionally clickbaity. You know, to bait people into clicking. Also I do want to challenge people to really think here. Seeing if the cached file data can be accessed quickly is the point of the experiment. I can't get mmap() to open a file with huge pages. void* buffer = mmap(NULL, size_bytes, PROT_READ, (MAP_HUGETLB | MAP_HUGE_1GB), fd, 0); doesn't work. You can can see my code he…

MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).

> MAP_HUGETLB can't be used for mmaping files on disk

False. I've successfully used it to memory-map networked files.

Re: io_uring is faster than mmap

#29

Post by mischief6 »

jared_hulbert wrote: post #23
titanomachy wrote: post #12

Very interesting article, thanks for publishing these tests! Is the manual loop unrolling really necessary to get vectorized machine code? I would have guessed that the highest optimization levels in LLVM would be able to figure it out from the basic code. That's a very uneducated guess, though. Also, curious if you tried using the MAP_POPULATE option with mmap. Could that improve the bandwidth of the naive in-memory…

YES! gcc and clang don't like to optimize this. But they do if you hardcode the size_bytes to an aligned value. It kind of makes sense, what if a user passes size_bytes as 3? With enough effort the compilers could handle this, but it's a lot to ask. I just ran MAP_POPULATE the results are interesting. It speeds up the counting loop. Same speed or higher as the my read() to a malloced buffer tests. HOWEVER... It takes…

it could be interesting to see what ispc does with similar code.

Re: io_uring is faster than mmap

#30

Post by jared_hulbert »

josephg wrote: post #25
jared_hulbert wrote: post #18

Earlier quoted context omitted.

https://www.intel.com/content/www/us/en/io/data-direct-i-o-t... AMD has something similar. The PCIe bus and memory bus both originate from the processor or IO die of the "CPU" when you use an NVMe drive you are really just sending it a bunch of structured DMA requests. Normally you are telling the drive to DMA to an address that maps to the memory, so you can direct it cache and bypass sending it out on the DRAM bus.…

I’d be fascinated to see a comparison with SPDK. That bypasses the kernel’s NVMe / SSD driver and controls the whole device from user space - which is supposed to avoid a lot of copies and overhead. You might be able to set up SPDK to send data directly into the cpu cache? It’s one of those things I’ve wanted to play with for years but honestly I don’t know enough about it. https://spdk.io/

spdk and I go way back. I'm confident it'd be about the same, possibly ~200-300MB/s more, I was pretty close to the rated throughput of the drives. Io_uring has really closed the gap that used to exist between the in kernel and userspace solutions.

With the Intel connection they might have explicit support for DDIO. Good idea.

Post reply on HN