Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are you sure you want to use MMAP in your database management system? [pdf]

db.cs.cmu.edu

Post reply on HN
Print view View original

21–30 of 137 posts

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#21

Post by rectang »

One possible advantage of using mmap over a buffer pool can be programmer ergonomics.

Reading data into a buffer pool in process RAM takes time to warm up, and the pool can only be accessed by a single process. In contrast, for an mmap-backed data structure, assuming that files are static once written (which can be the case for an multi-version concurrency control (MVCC) architecture), you open an mmap read-only connection from any process and the so long as the data is already in the OS cache, you get instant fast reads. This makes managing database connections much easier, since connections are cheap and the programmer can just open as many as they want whenever and wherever they want.

It is true that cache eviction strategy used by the OS is likely to be suboptimal. So if you're in a position to only run a single database process, you might decide to make different tradeoffs.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#22

Post by stingraycharles »

jandrewrogers wrote: post #16

The pragmatic consideration that usually influences the decision to use mmap() is the large discontinuity in skill and expertise required to replace it. Writing your own alternative to mmap() can be significantly superior in terms of performance and functionality, and often lends itself to a cleaner database architecture. However, this presumes a sufficiently sophisticated design for an mmap() replacement. The learni…

When you say “replacing mmap()”, could you elaborate a bit on it? The way you write it sounds like you’re describing a reimplementation of mmap() with the same API, while I believe the actual goal would be to completely rewrite the persistence and caching layer to be like a “real” database.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#23

Post by dicroce »

jandrewrogers wrote: post #16

The pragmatic consideration that usually influences the decision to use mmap() is the large discontinuity in skill and expertise required to replace it. Writing your own alternative to mmap() can be significantly superior in terms of performance and functionality, and often lends itself to a cleaner database architecture. However, this presumes a sufficiently sophisticated design for an mmap() replacement. The learni…

I have written a couple of mmap() based time series databases. In my case, these were databases for holding video. For my uses, mmap() has been great. I strongly agree with your comment. Maybe mmap() isn't the greatest, but it has worked for me.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#24

Post by masklinn »

matheusmoreira wrote: post #17
10000truths wrote: post #13

Earlier quoted context omitted.

You don't want the OS to take care of reading from disk and page caching/eviction. You want the DB itself to have explicit control over that, because the DB has information on access patterns and table format that the OS is not aware of. It is better equipped than the OS to anticipate what portions of tables/indices need to be cached in memory. It is better equipped to calculate when/where/what/how much to prefetch f…

> the DB has information on access patterns and table format that the OS is not aware of Aren't system calls such as madvise supposed to allow user space to let the kernel know precisely that information?

> precisely

Madvise is discussed in the paper, and it notes specifically that:

* madvise is not precise

* madvise is... an advice, which the system is completely free to disregard

* madvise is error-prone, providing the wrong hint can have dire consequences

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#25

Post by jandrewrogers »

tyingq wrote: post #8
assface wrote: post #2

RavenDB's response to this paper: https://ayende.com/blog/196161-C/re-are-you-sure-you-want-to...

The really key part seems to be this: "If you aren’t using mmap, on the other hand, you still need to handle of all those issues" Which seems like a reasonable statement. Is it less work to make your own top-to-bottom buffer pool, and would that necessarily avoid similar issues? Or is it less work to use mmap(), but address the issues?

Some issues with mmap() can be avoided entirely if you have your own buffer pool. Others are easier to handle because they are made explicit and more buffer state is exposed to the program logic. That's the positive side.

The downside is that writing an excellent buffer pool is not trivial, especially if you haven't done it before. There are many cross-cutting design concerns that have to be accounted for. In my experience, an excellent C++ implementation tends to be on the order of 2,000 lines of code -- someone has to write that. It also isn't simple code, the logic is relatively dense and subtle.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#26

Post by lmilcin »

stingraycharles wrote: post #22
jandrewrogers wrote: post #16

The pragmatic consideration that usually influences the decision to use mmap() is the large discontinuity in skill and expertise required to replace it. Writing your own alternative to mmap() can be significantly superior in terms of performance and functionality, and often lends itself to a cleaner database architecture. However, this presumes a sufficiently sophisticated design for an mmap() replacement. The learni…

When you say “replacing mmap()”, could you elaborate a bit on it? The way you write it sounds like you’re describing a reimplementation of mmap() with the same API, while I believe the actual goal would be to completely rewrite the persistence and caching layer to be like a “real” database.

The task is deceivingly simple.

You have a file and a bunch of memory and you need to make sure data is being moved from file to memory when needed and from memory to file when needed.

mmap() is one algorithm to do it, and the idea is that it is not necessarily the best one.

Knowing more about your data and application and needs should theoretically enable you to design an algorithm that will be more efficient at moving data back and forth.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#27

Post by jandrewrogers »

matheusmoreira wrote: post #17
10000truths wrote: post #13

Earlier quoted context omitted.

You don't want the OS to take care of reading from disk and page caching/eviction. You want the DB itself to have explicit control over that, because the DB has information on access patterns and table format that the OS is not aware of. It is better equipped than the OS to anticipate what portions of tables/indices need to be cached in memory. It is better equipped to calculate when/where/what/how much to prefetch f…

> the DB has information on access patterns and table format that the OS is not aware of Aren't system calls such as madvise supposed to allow user space to let the kernel know precisely that information?

The madvise() functions and similar are a blunt and imprecise instrument. The kernel is free to ignore them, and frequently does in practice. It also does not prevent the kernel from proactively doing things you don't want it to do with your buffer pool at the worst possible time.

A user space buffer pool gives you precise and deterministic control of many of these behaviors.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#28

Post by sprachspiel »

pizlonator wrote: post #19

This is a great write-up! Makes me wonder if there is an alternative universe in which there is a syscall with semantics similar to mmap that avoids these pitfalls. It's not like mmap's semantics are the only semantics that we could have for memory-mapped IO.

This would be exactly the kind of innovation we would need in computer science. Instead we often get stuck in local minima (in this case a 40-year old POSIX interface) without realizing how much pain this causes.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#29

Post by randbox »

How do you implement lockless atomic updates for multiple writers across multiple threads & processes without mmap?

With mmap it is straight forward for processes to open persistent arrays of atomics as a file, and use compare and exchange operations to prevent data races when multiple threads or processes update the same page without any file locks, advisory locks, or mutexes.

With manual read() and write() calls, the data may be overwritten by another writer before the update is committed.

Re: Are you sure you want to use MMAP in your database management system? [pdf]

#30

Post by ayende »

10000truths wrote: post #13
assface wrote: post #2

RavenDB's response to this paper: https://ayende.com/blog/196161-C/re-are-you-sure-you-want-to...

You don't want the OS to take care of reading from disk and page caching/eviction. You want the DB itself to have explicit control over that, because the DB has information on access patterns and table format that the OS is not aware of. It is better equipped than the OS to anticipate what portions of tables/indices need to be cached in memory. It is better equipped to calculate when/where/what/how much to prefetch f…

I'm the author (well, one of) RavenDB

You are correct to an extent, but there are a few things yo noted.

* you can design your system so the access pattern that the OS is optimized for matches your needs

* you can use madvise() to give some useful hints

* the amount of complexity you don't have to deal with is staggering

Post reply on HN