Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async hazard: MMAP is blocking IO

huonw.github.io

Post reply on HN
Print view View original

41–50 of 119 posts

Re: Async hazard: MMAP is blocking IO

#41

Post by rbanffy »

a-dub wrote: post #26
mjb wrote: post #9

I like this point - it's no secret that mmap can make memory access cost the same as an IO (swap can too) - but the interaction with async schedulers isn't immediately obvious. The cost can, sometimes, be even higher than this post says, because of write back behavior in Linux. Mmap is an interesting tool for system builders. It's super powerful, and super useful. But it's also kind of dangerous because the gap betwe…

it outsources buffer management and user thread i/o scheduling to the kernel. for some use cases it's a great way to simplify implementation or boost performance. for others it may not perform as well. the blog post points (in my mind) at some more general advice when programming which is not to mix and match paradigms unless you really know what you're doing. if you want to do user space async io, cool. if using ker…

Making it work asynchronously would require the compiler to split the memory access into two parts, a non-blocking IO dispatch and a blocking access to the mapped address. The OS would need to support that, however, and the language would need to keep track of what is a materialised array and what’s not.

Re: Async hazard: MMAP is blocking IO

#42

Post by dsp_person »

> One possible implementation might be to literally have the operating system allocate a chunk of physical memory and load the file into it, byte by byte, right when mmap is called… but this is slow, and defeats half the magic of memory mapped IO: manipulating files without having to pull them into memory

This doesn't defeat the purpose necessarily. How about for example, implementing a text editor: I want the best performance by loading the existing file initially (say it is <1MB), and the convenience and robustness of any writes to this memory being efficiently written to disk.

Re: Async hazard: MMAP is blocking IO

#43

Post by cbsmith »

mjb wrote: post #9

I like this point - it's no secret that mmap can make memory access cost the same as an IO (swap can too) - but the interaction with async schedulers isn't immediately obvious. The cost can, sometimes, be even higher than this post says, because of write back behavior in Linux. Mmap is an interesting tool for system builders. It's super powerful, and super useful. But it's also kind of dangerous because the gap betwe…

I'm surprised this is seen as a liability of mmap rather than a cooperative scheduler that isn't using native kernel threads. This is the deal you make with the devil when you use cooperative scheduling without involving the kernel, so I'm surprised it is news to people working with cooperative schedulers. These faults can happen even if you never explicitly memory map files (particularly since executables and shared libraries are often memory mapped into processes), so page faults are a blocking hazard for cooperative schedulers even without mmap.

The MMU in the hardware is aggressively parallel, and the only thread being blocked on the page fault is the one touching the page that needs to be swapped in. In reality, you can get heavily parallelized IO using mmap (indeed, it works quite well when you have a ton of IO you'd like to execute in parallel).

Re: Async hazard: MMAP is blocking IO

#44

Post by cmrdporcupine »

dividuum wrote: post #13
xoranth wrote: post #3

Earlier quoted context omitted.

On Linux, you might be able to use userfaultfd to make it async...

I don't see how that would work. The memory access causing the page fault still blocks, but now another thread handles paging in the requested data. So without coordination between those two, nothing really changes. Sounds easier to just use nonblocking reads directly. Thanks for the pointer to userfaultfd. Didn't know that existed.

Yeah. Part of the problem is that userfaultfd isn't itself quite flexible enough. What you might want to do is release the faulted thread to do some other work, letting it "know" what it can come back later when the data is available, but there's no mechanism to make that happen. Instead it's going to be entirely blocked until the fault can be resolved.

Re: Async hazard: MMAP is blocking IO

#45

Post by danarmak »

nemetroid wrote: post #37
danarmak wrote: post #23

Earlier quoted context omitted.

No, if the memory-mapped page you're accessing is in RAM, then you're just reading the RAM; there is no page fault and no syscall and nothing blocks. You could say that any non-register memory access "blocks" but I feel that's needlessly confusing. Normal async code doesn't "block" in any relevant sense when it accesses the heap.

So what is the definition of "blocking" here? That it takes more than 1 µs?

That the process/thread enters kernel mode and then is suspended waiting for IO or for some other event. As long as the thread is running your code (or, is scheduleable) it's not blocked. And then the async implementation can ensure your code cooperatively gives up the CPU for other code.

Re: Async hazard: MMAP is blocking IO

#46

Post by rbanffy »

magicalhippo wrote: post #25
icedchai wrote: post #22

Earlier quoted context omitted.

Do you consider reading from a normal array (one not backed by a memory mapped file) to also be blocking?

In the languages and platforms I use, absolutely yes. Do you have some examples where a normal memory read is async?

> Do you have some examples where a normal memory read is async?

This hints at a way to make it work, but would need the compiler (or explicit syntax) to make it clear you want to be able to switch to another task when the page fault triggers the disk read, and return to a blocking access that resolves the read from memory after the IO part is concluded.

It could look like a memory read but would include a preparation step.

Re: Async hazard: MMAP is blocking IO

#47

Post by loeg »

malkia wrote: post #35

WIth mmap you have to be prepared to handle unexpected page fault errors due to corrupted volume: Unlike standard read/write, where one can handle the issue, now it can happen anywhere the memory is mapped - your code, third party library, etc. It gets even unwieldy, and now you have to add additional tracking where access is to be expected. Blindly delegating mmap area to any code path that does not have such handli…

It is also the case on Linux/BSD.

Re: Async hazard: MMAP is blocking IO

#48

Post by loeg »

jcalvinowens wrote: post #40
malkia wrote: post #35

WIth mmap you have to be prepared to handle unexpected page fault errors due to corrupted volume: Unlike standard read/write, where one can handle the issue, now it can happen anywhere the memory is mapped - your code, third party library, etc. It gets even unwieldy, and now you have to add additional tracking where access is to be expected. Blindly delegating mmap area to any code path that does not have such handli…

On Linux, if you get a SIGBUS from poking a memory map that generally means you'd have certainly gotten -ENOMEM or -EIO during an equivalent sequence of syscalls (or been oom-killed, if you overcommit). Those are treated as fatal in the vast majority of programs, so dying to SIGBUS isn't meaningfully different for most usecases. By your logic, passing a file descriptor to a library is also "unwieldy", because the lib…

> On Linux, if you get a SIGBUS from poking a memory map that generally means you'd have certainly gotten -ENOMEM or -EIO during an equivalent sequence of syscalls (or been oom-killed, if you overcommit). Those are treated as fatal in the vast majority of programs, so dying to SIGBUS isn't meaningfully different for most usecases.

No. EIO is not instantly fatal in the same way that SIGBUS is. It allows for printing an error message, associating the error with some context of what failed, and either recovering in a degraded state or exiting cleanly. Doing any of this in a SIGBUS handler ranges from unwieldy to impossible.

Re: Async hazard: MMAP is blocking IO

#49

Post by a-dub »

rbanffy wrote: post #41
a-dub wrote: post #26

Earlier quoted context omitted.

it outsources buffer management and user thread i/o scheduling to the kernel. for some use cases it's a great way to simplify implementation or boost performance. for others it may not perform as well. the blog post points (in my mind) at some more general advice when programming which is not to mix and match paradigms unless you really know what you're doing. if you want to do user space async io, cool. if using ker…

Making it work asynchronously would require the compiler to split the memory access into two parts, a non-blocking IO dispatch and a blocking access to the mapped address. The OS would need to support that, however, and the language would need to keep track of what is a materialised array and what’s not.

as i understand, mmap is only efficient because it can leverage hardware support for trapping into the kernel when a page needs to be loaded to satisfy an access attempt.

i think adding software indirection to every access in the mapped region would be really slow.

i think a better answer would be to impose more structure on the planned memory access, then maybe given some constraints (like say, "this loop is embarrassingly parallel") the system could be smarter about working on the stuff in ram first while the rest is loaded in.

Re: Async hazard: MMAP is blocking IO

#50

Post by malkia »

jcalvinowens wrote: post #40
malkia wrote: post #35

WIth mmap you have to be prepared to handle unexpected page fault errors due to corrupted volume: Unlike standard read/write, where one can handle the issue, now it can happen anywhere the memory is mapped - your code, third party library, etc. It gets even unwieldy, and now you have to add additional tracking where access is to be expected. Blindly delegating mmap area to any code path that does not have such handli…

On Linux, if you get a SIGBUS from poking a memory map that generally means you'd have certainly gotten -ENOMEM or -EIO during an equivalent sequence of syscalls (or been oom-killed, if you overcommit). Those are treated as fatal in the vast majority of programs, so dying to SIGBUS isn't meaningfully different for most usecases. By your logic, passing a file descriptor to a library is also "unwieldy", because the lib…

Passing file handle is different - a well written library would check and handle errors, but pointer to buffer - and I don't expect a library to "try" and "catch" (even if it's possible) cases. Would strlen do this? No.
Post reply on HN