Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

db.cs.cmu.edu

Post reply on HN
Print view View original

41–50 of 182 posts

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#41

Post by kentonv »

arter4 wrote: post #36
kentonv wrote: post #28

Earlier quoted context omitted.

Probably the most common example is sendfile() for writing file contents out to a socket without reading them into userspace: https://man7.org/linux/man-pages/man2/sendfile.2.html

Yes, I knew about sendfile() but I wasnt't aware of any web server using that (though I know Kafka uses it). Then I found out Apache supports it via the EnableSendfile directive. Nice. >This directive controls whether httpd may use the sendfile support from the kernel to transmit file contents to the client. By default, when the handling of a request requires no access to the data within a file -- for example, when d…

I'd expect most serious web servers support it. I've written one that does (workerd), it's not too hard.

That said, it's tricky to use if the server also does TLS termination... then you need kTLS, which is a much bigger can of worms.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#42

Post by pjdesno »

Not just databases - we ran into the same issues when we needed a high-performance caching HTTP reverse proxy for a research project. We were just going to drop in Varnish, which is mmap-based, but performance sucked and we had to write our own.

Note that Varnish dates to 2006, in the days of hard disk drives, SCSI, and 2-core server CPUs. Mmap might well have been as good or even better than I/O back then - a lot of the issues discussed in this paper (TLB shootdown overhead, single flush thread) get much worse as the core count increases.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#43

Post by hyc_symas »

This is a pretty old argument and IMO it's far out of date/obsolete.

Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern application/server environment, no user level process has accurate information about the total state of the machine, only the kernel (or hypervisor) does and it's an exercise in futility to try to manage paging etc at the user level.

As Dr. Michael Stonebraker put it: The Traditional RDBMS Wisdom is (Almost Certainly) All Wrong. https://slideshot.epfl.ch/play/suri_stonebraker (See the slide at 21:25 into the video). Modern DBMSs spend 96% of their time managing buffers and locks, and only 4% doing actual useful work for the caller.

Granted, even using mmap you still need to know wtf you're doing. MongoDB's original mmap backing store was a poster child for Doing It Wrong, getting all of the reliability problems and none of the performance benefits. LMDB is an example of doing it right: perfect crash-proof reliability, and perfect linear read scalability across arbitrarily many CPUs with zero-copy reads and no wasted effort, and a hot code path that fits into a CPU's 32KB L1 instruction cache.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#44

Post by nh2 »

arter4 wrote: post #36
kentonv wrote: post #28

Earlier quoted context omitted.

Probably the most common example is sendfile() for writing file contents out to a socket without reading them into userspace: https://man7.org/linux/man-pages/man2/sendfile.2.html

Yes, I knew about sendfile() but I wasnt't aware of any web server using that (though I know Kafka uses it). Then I found out Apache supports it via the EnableSendfile directive. Nice. >This directive controls whether httpd may use the sendfile support from the kernel to transmit file contents to the client. By default, when the handling of a request requires no access to the data within a file -- for example, when d…

Pretty much all modern Linux web servers support sendfile(). Examples:

* nginx: [1] * Haskell webserver module: [2] * caddy: [3]

[1]: https://nginx.org/en/docs/http/ngx_http_core_module.html#sen... [2]: https://hackage.haskell.org/package/warp-3.3.28/docs/Network... [3]: https://github.com/caddyserver/caddy/pull/5022

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#45

Post by cratermoon »

wood_spirit wrote: post #26
nemo44x wrote: post #13

Earlier quoted context omitted.

For documents it made access fast since there’s no joins, etc. that require paging from all over. The problem ended up being updates and compaction issues.

My memory is that the problem was ACID. The document stores didn’t promise to be reliable because apparently that didn’t scale. And there was a very well known cartoon video discussion about it with “web scale” and “just write to dev null” and other classics that became memes :)

Did you ever read Pat Helland's article, "Life Beyond Distributed Transactions: An apostate’s opinion" https://dl.acm.org/doi/10.1145/3012426.3025012? "This article explores and names some of the practical approaches used in the implementation of large-scale mission-critical applications in a world that rejects distributed transactions."

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#46

Post by Dwedit »

AnotherGoodName wrote: post #15
jasonhansel wrote: post #4

I've become convinced that there are very few, if any, reasons to MMAP a file on disk. It seems to simplify things in the common case, but in the end it adds a massive amount of unnecessary complexity.

Complexity? You mmap it in and then read the multi terrabyte file as if it was an array. The opposite with actual file io sucks in terms of complexity. I get that you can write bespoke code that performs better but mmap is a one liner to turn a file into an array.

Need to handle the exceptions/signals every time a disk read fails. With classic IO, you know when the read will happen. But with memory-mapped files, the exception can happen at any time you are reading from the memory range.

As for why disk reads fail, yes that's a thing. Less common on internal storage (bad sectors), but more common on removable USB devices or Network drives (especially on wifi).

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#47

Post by danappelxx »

hyc_symas wrote: post #43

This is a pretty old argument and IMO it's far out of date/obsolete. Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern a…

Who is deploying databases in containers?

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#49

Post by orbz »

danappelxx wrote: post #47
hyc_symas wrote: post #43

This is a pretty old argument and IMO it's far out of date/obsolete. Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern a…

Who is deploying databases in containers?

A disturbingly large number of deployments I’ve seen using Kubernetes or docker compose have databases deployed as such.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#50

Post by huahaiy »

danappelxx wrote: post #47
hyc_symas wrote: post #43

This is a pretty old argument and IMO it's far out of date/obsolete. Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern a…

Who is deploying databases in containers?

Embedded DB
Post reply on HN