This link should be to this page that includes more info and the accompanying video:
Are you sure you want to use MMAP in your database management system? [pdf]
41–50 of 137 posts
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#42Earlier quoted context omitted.
Yup. Why use the operating system's async I/O system when you can simply burn a thread and do blocking I/O? Been down that primrose path, have the road rash to prove it. mmap() is great until you realize that pretty much all you've avoided is some buffer management that you probably need to do anyway. The OS just doesn't have the information it needs to do a great (or even correct) job of caching database pages.
A user process doesn't have the information it needs to do a good job of coordinating updates from multiple writers to database pages and indices. With MMAP, writers have access to shared atomics which they can update using compare-exchange operations to prevent data races which would be common when using read() and write() without locks.
Are you saying that without mmap() there will be data races??
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#43This link should be to this page that includes more info and the accompanying video: https://db.cs.cmu.edu/mmap-cidr2022/
Thank you for sharing your DB course(s) videos on the YouTube. I'm a CMU staff member (Open Learning Initiative) that would never be able to enroll on-site, given likely my lower priority for getting a seat, but watching your videos online has been fantastic.
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#44How do you implement lockless atomic updates for multiple writers across multiple threads & processes without mmap? With mmap it is straight forward for processes to open persistent arrays of atomics as a file, and use compare and exchange operations to prevent data races when multiple threads or processes update the same page without any file locks, advisory locks, or mutexes. With manual read() and write() calls, t…
Why do you need lockless atomic updates to a file-backed memory area? Genuinely curious.
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#45Earlier quoted context omitted.
You don't want the OS to take care of reading from disk and page caching/eviction. You want the DB itself to have explicit control over that, because the DB has information on access patterns and table format that the OS is not aware of. It is better equipped than the OS to anticipate what portions of tables/indices need to be cached in memory. It is better equipped to calculate when/where/what/how much to prefetch f…
I'm the author (well, one of) RavenDB You are correct to an extent, but there are a few things yo noted. * you can design your system so the access pattern that the OS is optimized for matches your needs * you can use madvise() to give some useful hints * the amount of complexity you don't have to deal with is staggering
OTOH, if you care about that last 5 percent or so of performance there is the complexity that what the OS has optimized for might be different between different OS's (e.g., MacOS, Linux, FreeBSd, etc.) and indeed, might change between different versions of Linux, or even, in the case of buffered writeback, between different filesystems on the same version of Linux. This is probably historically one of the most important reasons why enterprise databases like Oracle DB, DB2, etc., have used direct I/O, and not buffered I/O or mmap.
Speaking as an OS developer, we're not going to try to optimize buffered I/O for a particular database. We'll be using becnhmarks like compilebench and postmark to optimize our I/O, and if your write patterns, or readahead patterns, or caching requirements, don't match those workloads, well.... sucks to be you.
I'll also point out that those big companies that actually pay the salarise of us file system developers (e.g., Oracle, Google, etc.) for the most part use Direct I/O for our performance critical workloads. If database companies that want to use mmap want to hire file system developers and contribute benchmarks and performance patches for ext4, xfs, etc., speaking as the ext4 maintainer, I'll welcome that, and we do have a weekly video conference where I'd love to have your engineers join to discuss your contributions. :-)
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#46This link should be to this page that includes more info and the accompanying video: https://db.cs.cmu.edu/mmap-cidr2022/
Thank you for sharing your DB course(s) videos on the YouTube. I'm a CMU staff member (Open Learning Initiative) that would never be able to enroll on-site, given likely my lower priority for getting a seat, but watching your videos online has been fantastic.
Since you have a CMU ID, AFAIK you should be able to enroll in Piazza in addition to following along with the assignments/projects (if you wanted to).
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#47Most of the times I used mmap I wasn't happy in the end.
I went through a phase when I thought it was fun to do extreme random access on image files, archives and things like that. At some point I think "I want to do this for a file I fetch over the network" and that needs a rewrite.
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#48The pragmatic consideration that usually influences the decision to use mmap() is the large discontinuity in skill and expertise required to replace it. Writing your own alternative to mmap() can be significantly superior in terms of performance and functionality, and often lends itself to a cleaner database architecture. However, this presumes a sufficiently sophisticated design for an mmap() replacement. The learni…
The original version of MongoDB used mmap, and I worked at a company that had a ton of issues with cache warmup and the cache getting trashed by competing processes. Granted this was a long time ago, but the main issue was the operating system's willingness to reallocate large swaths of memory from the address space to whatever process was asking for memory right now.
Once the working set got trashed, performance would go through the floor, and our app would slow to a crawl while the cache went through the warmup cycle.
Long story short, with that model, Mongo couldn't "own" the memory it was using, and this lead to chronic problems. Wiredtiger fixed this completely, but I still think this is a cautionary tale for anyone considering building a DB without a dedicated memory manager.
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#49The pragmatic consideration that usually influences the decision to use mmap() is the large discontinuity in skill and expertise required to replace it. Writing your own alternative to mmap() can be significantly superior in terms of performance and functionality, and often lends itself to a cleaner database architecture. However, this presumes a sufficiently sophisticated design for an mmap() replacement. The learni…
The original version of MongoDB used mmap, and I worked at a company that had a ton of issues with cache warmup and the cache getting trashed by competing processes. Granted this was a long time ago, but the main issue was the operating system's willingness to reallocate large swaths of memory from the address space to whatever process was asking for memory right now. Once the working set got trashed, performance wou…
[deleted]
Re: Are you sure you want to use MMAP in your database management system? [pdf]
#50Earlier quoted context omitted.
I'm the author (well, one of) RavenDB You are correct to an extent, but there are a few things yo noted. * you can design your system so the access pattern that the OS is optimized for matches your needs * you can use madvise() to give some useful hints * the amount of complexity you don't have to deal with is staggering
OTOH, if you care about that last 5 percent or so of performance there is the complexity that what the OS has optimized for might be different between different OS's (e.g., MacOS, Linux, FreeBSd, etc.) and indeed, might change between different versions of Linux, or even, in the case of buffered writeback, between different filesystems on the same version of Linux. This is probably historically one of the most import…
The key from my perspective is that I CAN design my access patterns to match what you'll optimized.
Another aspect to remember is that mmap being even possible for databases as the primary mechanism is quite new.
Go 15 years ago and you are in 32 bit land. That rule out mmap as your approach.
At this point, I might as well skip the OS and go direct IO.
As for differ OS behavior, I generally find that they all roughly optimize for the same thing.
I need best perf on Linux and Windows. Other systems I can get away with just being pretty good