51–60 of 137 posts
RavenDB's response to this paper: https://ayende.com/blog/196161-C/re-are-you-sure-you-want-to...
From that article: the whole fsyncgate thing seems like a pretty strong counterargument to "mmap adds more complexity than it removes": https://danluu.com/fsyncgate/
why settle for errno when you can have a segfault.
Yup. Why use the operating system's async I/O system when you can simply burn a thread and do blocking I/O? Been down that primrose path, have the road rash to prove it. mmap() is great until you realize that pretty much all you've avoided is some buffer management that you probably need to do anyway. The OS just doesn't have the information it needs to do a great (or even correct) job of caching database pages.
mmap isn't non-blocking; page faults are blocking, no different from a read or write to a (non-direct I/O) file using a syscall.
Until recently io_uring literally burned a thread (from a thread pool) for every read or write regular file operation, too. Though now it finally has hooks into the buffer cache so it can opportunistically perform the operation from the same thread that dequeued the command, pushing it to a worker thread if it would need to wait for a cache fault.[1]
[1] Technically the same behavior could be implemented in user space using userfaultfd, but the latency would likely be higher on faults.
Interesting parallels in this work to Tanenbaum's "RPC Considered Harmful"†; in both cases, you've got an abstraction that papers over a huge amount of complexity, and it ends up burning you because a lot of that complexity turns out to be pretty important and the abstraction has cost you control over it. † https://www.cs.vu.nl/~ast/Publications/Papers/euteco-1988.pd...
Whenever you need better performance and reliability, identify an abstraction beloved of CS professors, and bypass it.
When I last checked, libtorrent was utterly failing to use O_DIRECT semantics. I started making a patch, but there are several places that do file ops, and the main one was more complicated than I could afford to dive into at the time.
Earlier quoted context omitted.
You don't want the OS to take care of reading from disk and page caching/eviction. You want the DB itself to have explicit control over that, because the DB has information on access patterns and table format that the OS is not aware of. It is better equipped than the OS to anticipate what portions of tables/indices need to be cached in memory. It is better equipped to calculate when/where/what/how much to prefetch f…
The counterargument to this is that the kernel can make decisions based on nonlocal information about the system. If your database server is the only process in the system that is using significant memory, then sure, you might as well manage it yourself. But if there are multiple processes competing for memory, the kernel is better equipped to decide which processes' pages should be paged out or kept into memory.
Earlier quoted context omitted.
A user process doesn't have the information it needs to do a good job of coordinating updates from multiple writers to database pages and indices. With MMAP, writers have access to shared atomics which they can update using compare-exchange operations to prevent data races which would be common when using read() and write() without locks.
Are you saying that without mmap() there will be data races??
Suppose you have a big data file and want to mark which pages are occupied and which pages are free. Suppose a writer wants to read a bit from an index page to the stack to check whether a data page is occupied, modify the page bit in the stack to claim the data page if another process hasn't claimed it, and write the updated value back to memory to claim the data page to store the data value if another process hasn't claimed it.
If each process read()s the index bits, they can both see that page 2 bit in the index is unset and try to claim it, then write() back the updated index value. The updates to the index will collide, both writers will think the claimed page 2 when only one should have, and one of the data values written to that page will get lost.
Earlier quoted context omitted.
The counterargument to this is that the kernel can make decisions based on nonlocal information about the system. If your database server is the only process in the system that is using significant memory, then sure, you might as well manage it yourself. But if there are multiple processes competing for memory, the kernel is better equipped to decide which processes' pages should be paged out or kept into memory.
Generally for perf critical use cases you dedicate the machine to the database. This simplifies many things (avoiding having to reason about sharing, etc etc).
Earlier quoted context omitted.
Generally for perf critical use cases you dedicate the machine to the database. This simplifies many things (avoiding having to reason about sharing, etc etc).
This makes me wonder whether there would be value in an OS that is also a DBMS (or vice versa). In other words, if the DBMS has total control over the hardware, perhaps performance can be maximized without too much additional complexity.
Earlier quoted context omitted.
The really key part seems to be this: "If you aren’t using mmap, on the other hand, you still need to handle of all those issues" Which seems like a reasonable statement. Is it less work to make your own top-to-bottom buffer pool, and would that necessarily avoid similar issues? Or is it less work to use mmap(), but address the issues?
When I worked on/with BerkeleyDB in the late 90s we came to the conclusion that the various OS mmap() implementations had been tweaked/fixed to the point where they worked for the popular high profile applications (in those days: Oracle). So it can appear like everything is fine, but that probably means your code behaves the same way as .
This was back in the early 2000's, when having 4 gigabytes of RAM would be considered large. The "database server" was single threaded, and all changes were logged to a WAL-ish file before updating the mmap'd database. It was fun stuff to work on. It worked well, but it wasn't a general purpose DB.
The default Linux settings dealing with memory mapped files are pretty horrible. The observed poor performance is directly related to not configuring several very important kernel parameters.