Async hazard: MMAP is blocking IO
huonw.github.io
Async hazard: MMAP is blocking IO
1–10 of 119 posts
Re: Async hazard: MMAP is blocking IO
#2No secret. Reading from memory is synchronous and always has been, at least in a normal computer. (Sometimes I think of how you could fit a fancy memory controller in a transport triggered architecture but that’s something different)
Re: Async hazard: MMAP is blocking IO
#3No secret. Reading from memory is synchronous and always has been, at least in a normal computer. (Sometimes I think of how you could fit a fancy memory controller in a transport triggered architecture but that’s something different)
On Linux, you might be able to use userfaultfd to make it async...
Re: Async hazard: MMAP is blocking IO
#4No secret. Reading from memory is synchronous and always has been, at least in a normal computer. (Sometimes I think of how you could fit a fancy memory controller in a transport triggered architecture but that’s something different)
By blocking they mean that it can take ballpark non-volatile storage times instead of ballpark RAM times
Re: Async hazard: MMAP is blocking IO
#5No secret. Reading from memory is synchronous and always has been, at least in a normal computer. (Sometimes I think of how you could fit a fancy memory controller in a transport triggered architecture but that’s something different)
By blocking they mean that it can take ballpark non-volatile storage times instead of ballpark RAM times
I believe they mean that since it bypasses the (Tokio) scheduler, so if you use it in async code you lose the main benefit of async code (namely, the scheduler is able to switch to some other task while waiting for IO to complete.).
Basically the same behavior you'd get if you called a blocking syscall directly.
Re: Async hazard: MMAP is blocking IO
#6No secret. Reading from memory is synchronous and always has been, at least in a normal computer. (Sometimes I think of how you could fit a fancy memory controller in a transport triggered architecture but that’s something different)
By blocking they mean that it can take ballpark non-volatile storage times instead of ballpark RAM times
GP is aware. mmap makes files act like memory. Memory is always synchronous, thus blocking, so mmaped files are always blocking. I'm surprised OP even found this surprising. It should be completely obvious.
Re: Async hazard: MMAP is blocking IO
#7No secret. Reading from memory is synchronous and always has been, at least in a normal computer. (Sometimes I think of how you could fit a fancy memory controller in a transport triggered architecture but that’s something different)
By blocking they mean that it can take ballpark non-volatile storage times instead of ballpark RAM times
Same thing would happen if your memory has been paged to disk too.
Re: Async hazard: MMAP is blocking IO
#8Earlier quoted context omitted.
By blocking they mean that it can take ballpark non-volatile storage times instead of ballpark RAM times
GP is aware. mmap makes files act like memory. Memory is always synchronous, thus blocking, so mmaped files are always blocking. I'm surprised OP even found this surprising. It should be completely obvious.
At first I thought the title meant the mmap() call itself blocks, which I figured could be slightly surprising. But it seems they're referring to I/O on the mapped file? I'm also baffled, how could it possibly not block?
Re: Async hazard: MMAP is blocking IO
#9I like this point - it's no secret that mmap can make memory access cost the same as an IO (swap can too) - but the interaction with async schedulers isn't immediately obvious. The cost can, sometimes, be even higher than this post says, because of write back behavior in Linux.
Mmap is an interesting tool for system builders. It's super powerful, and super useful. But it's also kind of dangerous because the gap between happy case and worst case performance is so large. That makes benchmarking hard, adds to the risk of stability bugs, and complicates taming tail latency. It's behavior also varies a lot between OSs.
It's also nice to see all the data in this post. Too many systems design conversations are just dueling assertions.
Re: Async hazard: MMAP is blocking IO
#10> This is thus a worst case, the impact on real code is unlikely to be quite this severe!
I think the actual worst-case would be to read the pages in a (pseudo-)random order.