31–40 of 182 posts
I've become convinced that there are very few, if any, reasons to MMAP a file on disk. It seems to simplify things in the common case, but in the end it adds a massive amount of unnecessary complexity.
Many general-purpose OS abstractions start leaking when you're working on systems-like software. You notice it when web servers are doing kernel bypass to for zero-copy, low-latency networking, or database engines throw away the kernel's page cache to implement their own file buffer.
With mmap, you get to avoid thinking about how much data to buffer at once, caching data to speed up repeated access, or shedding that cache when memory pressure is high. The kernel does all that. It may not do it in the absolute ideal way for your program but the benefit is you don't have to think about these logistics.
But if you're already writing intense systems code then you can probably do a better job than the kernel by optimizing for your use case.
Many general-purpose OS abstractions start leaking when you're working on systems-like software. You notice it when web servers are doing kernel bypass to for zero-copy, low-latency networking, or database engines throw away the kernel's page cache to implement their own file buffer.
Web servers doing kernel bypass for zero-copy networking? Do you have a specific example in mind? I'm curious.
You'll find DPDK mentioned a lot in the networking/HPC/data center literature. An example of a backend framework that uses DPDK is the seastar framework [2]. Also, I recently stumbled upon a paper for efficient RPC networks in data centers [3].
If you want to learn more, the p99 conference has tons of speakers talking about some interesting challenges in that space.
Earlier quoted context omitted.
mmap can be handy but usually is not a good idea when you care about ACID properties. So it tends to be most useful outside databases.
Can you give some examples where mmap is useful?
A very over-simplified and probably a bit incorrect description of what it did was to create a smaller version of the image - one that could fit in memory - by sub sampling every nth pixel, which was addressed via mmap.
It actually dealt with jpegs so I have no idea how that bit worked as they are not bitmaps.
Earlier quoted context omitted.
mmap can be handy but usually is not a good idea when you care about ACID properties. So it tends to be most useful outside databases.
Can you give some examples where mmap is useful?
Another technique that can only be done with mmap is to map two contiguous regions of virtual memory to the same underlying buffer. This allows you to use a ring buffer but only read from/write to what looks like a contiguous region of memory.
Earlier quoted context omitted.
Web servers doing kernel bypass for zero-copy networking? Do you have a specific example in mind? I'm curious.
Probably the most common example is sendfile() for writing file contents out to a socket without reading them into userspace: https://man7.org/linux/man-pages/man2/sendfile.2.html
Then I found out Apache supports it via the EnableSendfile directive. Nice.
>This directive controls whether httpd may use the sendfile support from the kernel to transmit file contents to the client. By default, when the handling of a request requires no access to the data within a file -- for example, when delivering a static file -- Apache httpd uses sendfile to deliver the file contents without ever reading the file if the OS supports it.
Earlier quoted context omitted.
Web servers doing kernel bypass for zero-copy networking? Do you have a specific example in mind? I'm curious.
Probably the most common example is sendfile() for writing file contents out to a socket without reading them into userspace: https://man7.org/linux/man-pages/man2/sendfile.2.html
Earlier quoted context omitted.
Probably the most common example is sendfile() for writing file contents out to a socket without reading them into userspace: https://man7.org/linux/man-pages/man2/sendfile.2.html
Isn't that the opposite? That is, bypassing user space, not kernel space?
I'm not very familiar with that though.
Earlier quoted context omitted.
Web servers doing kernel bypass for zero-copy networking? Do you have a specific example in mind? I'm curious.
The most common example is DPDK [1]. It's a framework for building bespoke networking stacks that are usable from userspace, without involving the kernel. You'll find DPDK mentioned a lot in the networking/HPC/data center literature. An example of a backend framework that uses DPDK is the seastar framework [2]. Also, I recently stumbled upon a paper for efficient RPC networks in data centers [3]. If you want to learn…
I do wonder what trend is going to win: bypass the kernel or embrace the kernel for everything?
The way I see it, latency decreases either way (as long as you don't have to switch back and forth between kernel and user space), but userspace seems better from a security standpoint.
Then again, everyone is doing eBPF, so probably the "embrace the kernel" approach is going to win. Who knows.
Earlier quoted context omitted.
Web servers doing kernel bypass for zero-copy networking? Do you have a specific example in mind? I'm curious.
Probably the most common example is sendfile() for writing file contents out to a socket without reading them into userspace: https://man7.org/linux/man-pages/man2/sendfile.2.html