Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

db.cs.cmu.edu

Post reply on HN
Print view View original

81–90 of 182 posts

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#81

Post by hyc_symas »

gavinray wrote: post #78
hyc_symas wrote: post #71

Earlier quoted context omitted.

Correct, and thank you. I wrote LMDB, wrote a lot of OpenLDAP, and worked on BerkeleyDB for many years. And actually Andy Pavlo invited me to CMU to give a lecture on LMDB a few years back. https://www.youtube.com/watch?v=tEa5sAh-kVk Andy and I have had this debate going for a long time already.

Well, I eat my shorts. Isn't LMBD closer to an embedded key-value store than an RDBMS, though? Also there's a section in the paper that mentions it's single-writer.

Yes, LMDB is an embedded key/value store but it can be used as the backing store of any other DB model you care for. E.g. as a backend to MySQL, or SQLite, or OpenLDAP, or whatever.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#82

Post by crabbone »

kentonv wrote: post #8
Dwedit wrote: post #2

Memory-Mapped Files = access violations when a disk read fails. If you're not prepared to handle those, don't use memory-mapped files. (Access violation exceptions are the same thing that happens when you attempt to read a null pointer) Then there's the part with writes being delayed. Be prepared to deal with blocks not necessarily updating to disk in the order they were written to, and 10 seconds after the fact. Thi…

> Be prepared to deal with blocks not necessarily updating to disk in the order they were written to, and 10 seconds after the fact. This can make power failures cause inconsistencies. This is not specific to mmap -- regular old write() calls have the same behavior. You need to fsync() (or, with mmap, msync()) to guarantee data is on disk.

> This is not specific to mmap -- regular old write() calls have the same behavior.

This is not true. This depends on how the file was opened. You may request DIRECT | SYNC when opening and the writes are acknowledged when they are actually written. This is obviously a lot slower than writing to cache, but this is the way for "simple" user-space applications to implement their own cache.

In the world of today, you are very rarely writing to something that's not network attached, and depending on your appliance, the meaning of acknowledgement from write() differs. Sometimes it's even configurable. This is why databases also offer various modes of synchronization -- you need to know how your appliance works and configure the database accordingly.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#83

Post by crabbone »

afr0ck wrote: post #24
Dwedit wrote: post #2

Memory-Mapped Files = access violations when a disk read fails. If you're not prepared to handle those, don't use memory-mapped files. (Access violation exceptions are the same thing that happens when you attempt to read a null pointer) Then there's the part with writes being delayed. Be prepared to deal with blocks not necessarily updating to disk in the order they were written to, and 10 seconds after the fact. Thi…

Linux throws a SIGBUS. A process should anticipate such I/O failures by implementing a SIGBUS handler, especially a database server. For the second part of your comment, on Linux systems, there is the msync() system call that can be used to flush the page cache on demand.

> msync() system call that can be used to flush the page cache on demand.

for everyone, not just the file you mapped to memory. I.e. the guarantee is that your file will be written, but there's no way to do that w/o affecting others. This is not such a hot idea in an environment where multiple threads / processes are doing I/O.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#84

Post by jemfinch »

LAC-Tech wrote: post #63
gavinray wrote: post #61

Earlier quoted context omitted.

Out of curiosity, how many databases have you written? This is co-authored by Pavlo, Viktor Leiss, with feedback from Neumann. I'm sorry, but if someone on the internet claims to know better than those 3, you're going to need some monumental evidence of your credibility. Additionally, what you link here: > ... (See the slide at 21:25 into the video). Modern DBMSs spend 96% of their time managing buffers and locks, an…

This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).

"Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers" is already an appeal to authority in itself.

We shouldn't apply a higher bar to the counterargument than we applied to the argument in the first place.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#85

Post by whartung »

josephg wrote: post #20
formerly_proven wrote: post #14

Earlier quoted context omitted.

mmap can be handy but usually is not a good idea when you care about ACID properties. So it tends to be most useful outside databases.

Can you give some examples where mmap is useful?

It’s useful in a world of several processes sharing things. This is much less common today in a world of “single process” containers and VMs as well as monolithic processes using threads or async techniques.

However, Java can build a special library file of the core JRE classes that it can mmap into memory with the intent to speed up startup times, mostly for small Java programs.

Guile scheme will mmap files that have been compiled to byte code. You can visualize a contrived (especially today) scenario where Guile is used for CGI handlers, having the bulk of their code mapped, the overall memory impact of simultaneous handlers is much lower, as well as start up times.

The process model is less common today so the value of this goes down, but it can still have its place.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#86

Post by cratermoon »

wood_spirit wrote: post #52
cratermoon wrote: post #45

Earlier quoted context omitted.

Did you ever read Pat Helland's article, "Life Beyond Distributed Transactions: An apostate’s opinion" https://dl.acm.org/doi/10.1145/3012426.3025012 ? "This article explores and names some of the practical approaches used in the implementation of large-scale mission-critical applications in a world that rejects distributed transactions."

No I haven’t. Thanks for the interesting link :) Admittedly I live in a world where big distributed transactions are a given and work fine and sql speeds us up not slows us down. I’m guessing sql and acid scaled after all?

> I’m guessing sql and acid scaled after all?

Yes and no. Distributed transactions and two-phase commit have been superseded by things like Paxos and Raft, with a variety of consistency models, so the implementation is drastically different.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#87

Post by FridgeSeal »

crabbone wrote: post #69
spockz wrote: post #58

Earlier quoted context omitted.

Given the ability to deploy pods to dedicated nodes based on label selectors, what is the actual performance impact of running a database in a container on a bare metal host with mounted volume versus running that same process with say systemd on that same node? Basically, shouldn’t the overhead of running a container be minimal?

The problem is kubelet likes to spike in memory / CPU / network usage. It's not a well-behaved program to put alongside a database. It's not written with an eye for resource utilization. Also, it brings nothing of value to the table, but requires a lot of dance around it to keep it going. I.e. if you are a decent DBA, you don't have a problem setting up a node to run your database of choice, you would be probably opp…

I run DB’s on K8s, not because I don’t know what I’m doing, but because most of the trade offs are worth it.

If I run a db workload in K8s, it’s a tiny fraction of the operational overhead, and not a massively noticeable performance loss.

I would absolutely love a way to deploy and manage db’s as easily as K8s with fewer of the quite significant issues that have mentioned, so if you know of something that is better behaved around singular workloads, but keeps the simple deploys, the resiliency, the ease of networking and config deployments, the ease of monitoring, etc, I am all ears.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#88

Post by jandrewrogers »

LAC-Tech wrote: post #63
gavinray wrote: post #61

Earlier quoted context omitted.

Out of curiosity, how many databases have you written? This is co-authored by Pavlo, Viktor Leiss, with feedback from Neumann. I'm sorry, but if someone on the internet claims to know better than those 3, you're going to need some monumental evidence of your credibility. Additionally, what you link here: > ... (See the slide at 21:25 into the video). Modern DBMSs spend 96% of their time managing buffers and locks, an…

This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).

I think the real argument is more nuanced. Where you see mmap() fail badly on Linux, even for read-only workloads, is under a few specific conditions: very large storage volumes, highly concurrent access, non-trivial access patterns (e.g. high-dimensionality access methods). Most people do not operate data models under these conditions, but if you do then you can achieve large integer factor gains in throughput by not using mmap().

Interestingly, most of the reason for these problems has to do with theoretical limitations of cache replacement algorithms as drivers of I/O scheduling. There are alternative approaches to scheduling I/O that work much better in these cases but mmap() can’t express them, so in those cases bypassing mmap() offers large gains.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#89

Post by tayo42 »

pjdesno wrote: post #42

Not just databases - we ran into the same issues when we needed a high-performance caching HTTP reverse proxy for a research project. We were just going to drop in Varnish, which is mmap-based, but performance sucked and we had to write our own. Note that Varnish dates to 2006, in the days of hard disk drives, SCSI, and 2-core server CPUs. Mmap might well have been as good or even better than I/O back then - a lot of…

Varnish has a file system backed cache that depends on the page cache to keep it fast.

What did you differently in your custom one that was faster then varnish?

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#90

Post by crabbone »

FridgeSeal wrote: post #87
crabbone wrote: post #69

Earlier quoted context omitted.

The problem is kubelet likes to spike in memory / CPU / network usage. It's not a well-behaved program to put alongside a database. It's not written with an eye for resource utilization. Also, it brings nothing of value to the table, but requires a lot of dance around it to keep it going. I.e. if you are a decent DBA, you don't have a problem setting up a node to run your database of choice, you would be probably opp…

I run DB’s on K8s, not because I don’t know what I’m doing, but because most of the trade offs are worth it. If I run a db workload in K8s, it’s a tiny fraction of the operational overhead, and not a massively noticeable performance loss. I would absolutely love a way to deploy and manage db’s as easily as K8s with fewer of the quite significant issues that have mentioned, so if you know of something that is better b…

If you think that deploying anything with Kubernetes is simple... well, I have bad news for you.

It's simple, until you hit a problem. And then it becomes a lot worse than if you had never touched it. You are now in the stage of a person who'd never made backups and never had a failure that required them to restore from backups, and you are wondering why would anyone do it. Adverse events are rare, and you may go like this for years, or, perhaps the rest of your life... unfortunately, your experience will not translate into a general advice.

But, again, you just might be in the camp where performance doesn't matter. Nor does uptime matter, nor does your data have very high value... and in that case it's OK to use tools that don't offer any of that, and save you some time. But, you cannot advise others based on that perspective. Or, at least, not w/o mentioning the downsides.

Post reply on HN