Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

db.cs.cmu.edu

Post reply on HN
Print view View original

91–100 of 182 posts

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#91

Post by crabbone »

hyc_symas wrote: post #70
crabbone wrote: post #65

Earlier quoted context omitted.

> your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. There's nothing special about kernel programmers. In fact, if I had to compare, I'd go with storage people being the more experienced / knowledgeable ones. They have a highly competitive environment, which requires a lot more understandin…

> There's nothing special about kernel programmers. Yes, that was a shorthand generalization for "people who've studied computer architecture" - which most application developers never have. > no DBA worth their salt would put database in the environment where it has to share resources with applications. Most applications today are running on smartphones/mobile devices. That means they're running with local embedded…

> Most applications today are running on smartphones/mobile devices.

That's patently false. There are about 8 bn. people. Even if everyone has a smartphone or two, it's nothing compared to the total of all devices that can be called "computer". I think that "smart TV" alone will beat the number of smartphones. But even that is a drop in a bucket when it comes to the total of running programs on Earth / its orbit.

But, that's beside the point. Smartphones aren't designed to run database servers. Even if they indeed were the majority, they'd still be irrelevant for this conversation because they are a wrong platform for deploying databases. In other words, it doesn't matter how people deploy databases to smartphones -- they have no hopes of achieving good performance, and whether they use mmap or not is of no consequences -- they've lost the race before they even qualified for it.

> LMDB

Are we talking about this? https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Databa... If so, this is irrelevant for databases in general.

> LMDB databases may have only one writer at a time

(Taken from the page above) -- this isn't a serious contender for database server space. It's a toy database. You shouldn't give general advice based on whatever this system does or doesn't.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#92

Post by FridgeSeal »

crabbone wrote: post #77
charcircuit wrote: post #57

Earlier quoted context omitted.

How would containers even hurt performance? How does the database no longer having the ability to see other processes on the machine somehow make it slower?

There are many "holes" in these containers. 1. fsync. You cannot "divide" it between containers. Whoever does it, stalls I/O for everyone else. 2. Context switches. Unless you do a lot of configurations outside of container runtime, you cannot ensure exclusive access to the number of CPU cores you need. 3. Networking has the same problem. You would either have to dedicate a whole NIC or SRI-OV-style virtual NIC to yo…

As these are obviously very real issues, and Kubernetes also isn’t going away imminently, how many of these can be fixed/improved with different design on the application front?

Would using direct-Io API’s fix most of the fsync issues? If workloads pin their stuff to specific cores can we incite some of the overhead here? (Assuming we’re only running a single dedicated workload + kubelet on the node).

> You would either have to dedicate a whole NIC or SRI-OV-style virtual NIC to your database server

Tbh I’ve no idea we could do this with commodity cloud servers, nor do I know how, but I’m terribly interested in knowing how, do you know if there’s like a “dummy’s guide to better networking”? Haha

> kubelet is not optimized to get out of your way...Kubernetes sucks at managing memory-intensive processes

Definitely agree on both these issues, I’ve blown up the kubelet by overallocating memory before, which basically borked the node until some watchdog process kicked in. Sounds like the better solution here is a kubelet rebuilt to operate more efficiently and more predictably? Is the solution a db-optimised kubelet/K8s?

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#93

Post by dang »

Related:

Are You Sure You Want to Use MMAP in Your Database Management System? [pdf] - https://news.ycombinator.com/item?id=31504052 - May 2022 (43 comments)

Are you sure you want to use MMAP in your database management system? [pdf] - https://news.ycombinator.com/item?id=29936104 - Jan 2022 (127 comments)

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#94

Post by lokar »

crabbone wrote: post #69
spockz wrote: post #58

Earlier quoted context omitted.

Given the ability to deploy pods to dedicated nodes based on label selectors, what is the actual performance impact of running a database in a container on a bare metal host with mounted volume versus running that same process with say systemd on that same node? Basically, shouldn’t the overhead of running a container be minimal?

The problem is kubelet likes to spike in memory / CPU / network usage. It's not a well-behaved program to put alongside a database. It's not written with an eye for resource utilization. Also, it brings nothing of value to the table, but requires a lot of dance around it to keep it going. I.e. if you are a decent DBA, you don't have a problem setting up a node to run your database of choice, you would be probably opp…

If you care about perf you would pin the kubelet and all other overhead workload to one core, and mask that off for your workload.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#95

Post by [deleted] »

hyc_symas wrote: post #70
crabbone wrote: post #65

Earlier quoted context omitted.

> your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. There's nothing special about kernel programmers. In fact, if I had to compare, I'd go with storage people being the more experienced / knowledgeable ones. They have a highly competitive environment, which requires a lot more understandin…

> There's nothing special about kernel programmers. Yes, that was a shorthand generalization for "people who've studied computer architecture" - which most application developers never have. > no DBA worth their salt would put database in the environment where it has to share resources with applications. Most applications today are running on smartphones/mobile devices. That means they're running with local embedded…

[deleted]

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#96

Post by kentonv »

crabbone wrote: post #82
kentonv wrote: post #8

Earlier quoted context omitted.

> Be prepared to deal with blocks not necessarily updating to disk in the order they were written to, and 10 seconds after the fact. This can make power failures cause inconsistencies. This is not specific to mmap -- regular old write() calls have the same behavior. You need to fsync() (or, with mmap, msync()) to guarantee data is on disk.

> This is not specific to mmap -- regular old write() calls have the same behavior. This is not true. This depends on how the file was opened. You may request DIRECT | SYNC when opening and the writes are acknowledged when they are actually written. This is obviously a lot slower than writing to cache, but this is the way for "simple" user-space applications to implement their own cache. In the world of today, you ar…

> This is not true. This depends on how the file was opened. You may request DIRECT | SYNC

Well sure, but 99.9% of people don't do that (and shouldn't, unless they really know what they are doing).

> In the world of today, you are very rarely writing to something that's not network attached, and depending on your appliance, the meaning of acknowledgement from write() differs.

What network-attached storage actually uses O_SYNC behavior without being asked? I'd be quite surprised if any did this as it would make typical workloads incredibly slow in order to provide a guarantee they didn't ask for.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#97

Post by lokar »

crabbone wrote: post #77
charcircuit wrote: post #57

Earlier quoted context omitted.

How would containers even hurt performance? How does the database no longer having the ability to see other processes on the machine somehow make it slower?

There are many "holes" in these containers. 1. fsync. You cannot "divide" it between containers. Whoever does it, stalls I/O for everyone else. 2. Context switches. Unless you do a lot of configurations outside of container runtime, you cannot ensure exclusive access to the number of CPU cores you need. 3. Networking has the same problem. You would either have to dedicate a whole NIC or SRI-OV-style virtual NIC to yo…

My background is more borg then k8s, but…

Alway allocate whole cores, just mask them off

Dedicate physical IO devices for sensitive workloads

You can have per cgroup swap if you want, but imo swap is not useful

I think all of this is possible in k8s

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#98

Post by jerrygenser »

hyc_symas wrote: post #43

This is a pretty old argument and IMO it's far out of date/obsolete. Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs. In the modern a…

> Taking full control of your I/O and buffer management is great if (a) your developers are all smart and experienced enough to be kernel programmers and (b) your DBMS is the only process running on a machine. In practice, (a) is never true, and (b) is no longer true because everyone is running apps inside containers inside shared VMs.

The article is about DBMS developers. For DBMS developers, "in practice" (a) and (b) are usually true I think.

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

#99

Post by jerrygenser »

wood_spirit wrote: post #26
nemo44x wrote: post #13

Earlier quoted context omitted.

For documents it made access fast since there’s no joins, etc. that require paging from all over. The problem ended up being updates and compaction issues.

My memory is that the problem was ACID. The document stores didn’t promise to be reliable because apparently that didn’t scale. And there was a very well known cartoon video discussion about it with “web scale” and “just write to dev null” and other classics that became memes :)

Document stores often are reliable and more fault tolerant. But yes they trade ACID.

There are some applications that require high throughput (usually write) but can be fine with read consistency.

Couple of examples - consumer facing comment systems where other users are OK to miss your comment by 30 seconds - timeseries logging where you are usually reading infrequently but writing very much in a denormalized format so joins aren't as critical

For general CRUD, ACID is important though.

Post reply on HN