Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

db.cs.cmu.edu

Post reply on HN
Print view View original

51–60 of 182 posts

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#51

Post by amluto »

dataflow wrote: post #27
josephg wrote: post #20

Earlier quoted context omitted.

Can you give some examples where mmap is useful?

If your data is likely to already be in the system cache, memory mapping can achieve zero copying of the data, whereas reading will perform at least one memcpy. So there can be a performance advantage depending on the usage pattern. Also, I've never tested this, but I believe mapped files will get flushed as long as the system stays running. So if you only need resilience against abnormal termination rather than syst…

> Also, I've never tested this, but I believe mapped files will get flushed as long as the system stays running. So if you only need resilience against abnormal termination rather than system crashes, it seems like a good option?

Linux will not lose data written to a MAP_SHARED mapping when the process crashes.

But! Linux will synchronously update mtime when starting to write to a currently write protected mapping (e.g. one which was just written out). This means (a) POSIX is violated (IMO) and (b) what should be a minor fault to enable writes turns into an actual metadata write, which can cause actual synchronous IO.

I have an ancient patch set to fix this, but I never got it all the way into upstream Linux.

What you can do is mmap a file on a tmpfs as long as you trust yourself to have some other reliable process handle the data even if your application terminates abnormally. This is awkward with a container solution if you need to survive termination of the entire container.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#52

Post by wood_spirit »

cratermoon wrote: post #45
wood_spirit wrote: post #26

Earlier quoted context omitted.

My memory is that the problem was ACID. The document stores didn’t promise to be reliable because apparently that didn’t scale. And there was a very well known cartoon video discussion about it with “web scale” and “just write to dev null” and other classics that became memes :)

Did you ever read Pat Helland's article, "Life Beyond Distributed Transactions: An apostate’s opinion" https://dl.acm.org/doi/10.1145/3012426.3025012 ? "This article explores and names some of the practical approaches used in the implementation of large-scale mission-critical applications in a world that rejects distributed transactions."

No I haven’t. Thanks for the interesting link :)

Admittedly I live in a world where big distributed transactions are a given and work fine and sql speeds us up not slows us down. I’m guessing sql and acid scaled after all?

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#54

Post by danappelxx »

orbz wrote: post #49
danappelxx wrote: post #47

Earlier quoted context omitted.

Who is deploying databases in containers?

A disturbingly large number of deployments I’ve seen using Kubernetes or docker compose have databases deployed as such.

IMO if you’re concerned about performance and yet are deploying databases this way — mmap should not even be on the radar.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#55

Post by wmf »

Dwedit wrote: post #2

Memory-Mapped Files = access violations when a disk read fails. If you're not prepared to handle those, don't use memory-mapped files. (Access violation exceptions are the same thing that happens when you attempt to read a null pointer) Then there's the part with writes being delayed. Be prepared to deal with blocks not necessarily updating to disk in the order they were written to, and 10 seconds after the fact. Thi…

I wonder how many apps don't handle errors from read() anyway.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#56

Post by tadfisher »

hyc_symas wrote: post #43

This is a pretty old argument and IMO it's far out of date/obsolete. Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern a…

Maybe someone should pull LMDB's mmap/paging system into a usable library. I'd love to use the k/v store part of course, but I keep hitting the default key size limitation and would prefer not to link statically.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#57

Post by charcircuit »

danappelxx wrote: post #54
orbz wrote: post #49

Earlier quoted context omitted.

A disturbingly large number of deployments I’ve seen using Kubernetes or docker compose have databases deployed as such.

IMO if you’re concerned about performance and yet are deploying databases this way — mmap should not even be on the radar.

How would containers even hurt performance? How does the database no longer having the ability to see other processes on the machine somehow make it slower?

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#58

Post by spockz »

orbz wrote: post #49
danappelxx wrote: post #47

Earlier quoted context omitted.

Who is deploying databases in containers?

A disturbingly large number of deployments I’ve seen using Kubernetes or docker compose have databases deployed as such.

Given the ability to deploy pods to dedicated nodes based on label selectors, what is the actual performance impact of running a database in a container on a bare metal host with mounted volume versus running that same process with say systemd on that same node? Basically, shouldn’t the overhead of running a container be minimal?

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#59

Post by benlivengood »

For all of its usefulness in the good old days of rusty disks I wonder if virtual memory is worth having for dedicated databases, caches, and storage heads. Avoiding TLB flushes entirely sounds like a huge win for massively multithreaded software and memory management in a large shared flat address space doesn't sound impossibly hard.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#60

Post by danappelxx »

charcircuit wrote: post #57
danappelxx wrote: post #54

Earlier quoted context omitted.

IMO if you’re concerned about performance and yet are deploying databases this way — mmap should not even be on the radar.

How would containers even hurt performance? How does the database no longer having the ability to see other processes on the machine somehow make it slower?

I’ll assume the worst case:

- lots of containers running on a single host

- containers are each isolated in a VM (aka virtualized)

- workloads are not homogenous and change often (your neighbor today may not be your neighbor tomorrow)

I believe these are fair assumptions if you’re running on generic infrastructure with kubernetes.

In this setup, my concerns are pretty much noisy neighbors + throttling. You may get latency spikes out of nowhere and the cause could be any of:

- your neighbor is hogging IO (disk or network)

- your database spawned too many threads and got throttled by CFS

- CFS scheduled your DBs threads on a different CPU and you lost your cache lines

In short, the DB does not have stable, predictable performance, which are exactly the characteristics you want it to have. If you ran the DB on a dedicated host you avoid this whole suite of issues.

You can alleviate most of this if you make sure the DB’s container gets the entire host’s resources and doesn’t have neighbors.

Post reply on HN