Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

db.cs.cmu.edu

Post reply on HN
Print view View original

71–80 of 182 posts

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#71

Post by hyc_symas »

LAC-Tech wrote: post #63
gavinray wrote: post #61

Earlier quoted context omitted.

Out of curiosity, how many databases have you written? This is co-authored by Pavlo, Viktor Leiss, with feedback from Neumann. I'm sorry, but if someone on the internet claims to know better than those 3, you're going to need some monumental evidence of your credibility. Additionally, what you link here: > ... (See the slide at 21:25 into the video). Modern DBMSs spend 96% of their time managing buffers and locks, an…

This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).

Correct, and thank you. I wrote LMDB, wrote a lot of OpenLDAP, and worked on BerkeleyDB for many years. And actually Andy Pavlo invited me to CMU to give a lecture on LMDB a few years back. https://www.youtube.com/watch?v=tEa5sAh-kVk

Andy and I have had this debate going for a long time already.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#72

Post by morelisp »

danappelxx wrote: post #47
hyc_symas wrote: post #43

This is a pretty old argument and IMO it's far out of date/obsolete. Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern a…

Who is deploying databases in containers?

I'm running prod databases in containers so the server infra team doesn't have to know anything about how that specific database works or how to upgrade it, they just need to know how to issue generic container start/stop commands if they want to do some maintenance.

(But just in containers, not in Kubernetes. I'm not crazy.)

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#73

Post by hyc_symas »

tadfisher wrote: post #56
hyc_symas wrote: post #43

This is a pretty old argument and IMO it's far out of date/obsolete. Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern a…

Maybe someone should pull LMDB's mmap/paging system into a usable library. I'd love to use the k/v store part of course, but I keep hitting the default key size limitation and would prefer not to link statically.

It wouldn't be much use without the B+tree as well; it's the B+tree's cache friendliness that allows applications to run so efficiently without the OS knowing any specifics of the app's usage patterns.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#74

Post by gavinray »

LAC-Tech wrote: post #63
gavinray wrote: post #61

Earlier quoted context omitted.

Out of curiosity, how many databases have you written? This is co-authored by Pavlo, Viktor Leiss, with feedback from Neumann. I'm sorry, but if someone on the internet claims to know better than those 3, you're going to need some monumental evidence of your credibility. Additionally, what you link here: > ... (See the slide at 21:25 into the video). Modern DBMSs spend 96% of their time managing buffers and locks, an…

This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).

The argument is that:

- Queries can trigger blocking page faults when accessing (transparently) evicted pages, causing unexpected I/O stalls

- mmap() complicates transactionality and error-handling

- Page table contention, single-threaded page eviction, and TLB shootdowns become bottlenecks

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#75

Post by hyc_symas »

crabbone wrote: post #68
danappelxx wrote: post #47

Earlier quoted context omitted.

Who is deploying databases in containers?

Nobody who matters. Those who do that don't know what they are doing (even if they outnumber the other side hundred to one, they "don't count" because they aren't aiming for good performance anyways). Well, maybe not quite... of course it's possible that someone would want to deploy a database in a container because of the convenience of assembling all dependencies in a single "package", however, they would never run…

>but I don't think GP meant "one process" literally. Neither that is realistic nor is it necessary.

The point was simply about other processes that could be competing for resources - CPU, memory, or I/O. It is expensive for a user-level process to perform accounting for all of these resources, and without such accounting you can't optimally allocate them.

If there are other apps that can suddenly spike memory usage then any careful buffer tuning you've done goes out the window. Likewise for any I/O scheduling you've done, etc.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#76

Post by gcoakes »

danappelxx wrote: post #60
charcircuit wrote: post #57

Earlier quoted context omitted.

How would containers even hurt performance? How does the database no longer having the ability to see other processes on the machine somehow make it slower?

I’ll assume the worst case: - lots of containers running on a single host - containers are each isolated in a VM (aka virtualized) - workloads are not homogenous and change often (your neighbor today may not be your neighbor tomorrow) I believe these are fair assumptions if you’re running on generic infrastructure with kubernetes. In this setup, my concerns are pretty much noisy neighbors + throttling. You may get la…

> - containers are each isolated in a VM (aka virtualized)

Why are you assuming containers are virtualized? Is there some container runtime that does that as an added security measure? I thought they all use namespaces on Linux.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#77

Post by crabbone »

charcircuit wrote: post #57
danappelxx wrote: post #54

Earlier quoted context omitted.

IMO if you’re concerned about performance and yet are deploying databases this way — mmap should not even be on the radar.

How would containers even hurt performance? How does the database no longer having the ability to see other processes on the machine somehow make it slower?

There are many "holes" in these containers.

1. fsync. You cannot "divide" it between containers. Whoever does it, stalls I/O for everyone else.

2. Context switches. Unless you do a lot of configurations outside of container runtime, you cannot ensure exclusive access to the number of CPU cores you need.

3. Networking has the same problem. You would either have to dedicate a whole NIC or SRI-OV-style virtual NIC to your database server. Otherwise just the amount of chatter that goes on through the control plane of something like Kubernetes will be a noticeable disadvantage. Again, containers don't help here, they only get in the way as to get that kind of exclusive network access you need more configuration on the host, and, possible an CNI to deal with it.

4. kubelet is not optimized to get out of your way. It needs a lot of resources and may spike, hindering or outright stalling database process.

5. Kubernetes sucks at managing memory-intensive processes. It doesn't work (well or at all) with swap (which, again, cannot be properly divided between containers). It doesn't integrate well with OOM killer (it cannot replace it, so any configurations you make inside Kubernetes are kind of irrelevant, because system's OOM killer will do how it pleases, ignoring Kubernetes).

---

Bottom line... Kubernetes is lame from infrastructure perspective. It's written for Web developers. To make things appear simpler for them, while sacrificing a lot of resources and hiding a lot of actual complexity... which is impossible to hide, and which, in an even of failure will come to bite you. You don't want that kind of program near your database.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#78

Post by gavinray »

hyc_symas wrote: post #71
LAC-Tech wrote: post #63

Earlier quoted context omitted.

This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).

Correct, and thank you. I wrote LMDB, wrote a lot of OpenLDAP, and worked on BerkeleyDB for many years. And actually Andy Pavlo invited me to CMU to give a lecture on LMDB a few years back. https://www.youtube.com/watch?v=tEa5sAh-kVk Andy and I have had this debate going for a long time already.

Well, I eat my shorts.

Isn't LMBD closer to an embedded key-value store than an RDBMS, though? Also there's a section in the paper that mentions it's single-writer.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#79

Post by hyc_symas »

gavinray wrote: post #74
LAC-Tech wrote: post #63

Earlier quoted context omitted.

This is a classic appeal to authority. Let's play the argument, not the man. (My understanding is that the GP wrote LMDB, works on openLDAP, and was a maintainer for BerkelyDB for a number of years. But even if he'd only written 'hello, world!' I'm much more interested in the specific arguments).

The argument is that: - Queries can trigger blocking page faults when accessing (transparently) evicted pages, causing unexpected I/O stalls - mmap() complicates transactionality and error-handling - Page table contention, single-threaded page eviction, and TLB shootdowns become bottlenecks

1 - for reading any uncached data, the I/O stalls are unavoidable. Whatever client requested that data is going to have to wait regardless.

2 - complexity? this is simply false. LMDB's ACID txns using MVCC are much simpler than any "traditional" approach.

3 - contention is a red herring since this approach is already single-writer, as is common for most embedded k/v stores these days. You lose more perf by trying to make the write path multi-threaded, in lock contention and cache thrashing.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#80

Post by danappelxx »

gcoakes wrote: post #76
danappelxx wrote: post #60

Earlier quoted context omitted.

I’ll assume the worst case: - lots of containers running on a single host - containers are each isolated in a VM (aka virtualized) - workloads are not homogenous and change often (your neighbor today may not be your neighbor tomorrow) I believe these are fair assumptions if you’re running on generic infrastructure with kubernetes. In this setup, my concerns are pretty much noisy neighbors + throttling. You may get la…

> - containers are each isolated in a VM (aka virtualized) Why are you assuming containers are virtualized? Is there some container runtime that does that as an added security measure? I thought they all use namespaces on Linux.

It’s becoming standard as a security measure. See: Kata containers, Firecracker VM
Post reply on HN