Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

Post by colanderman » Thu, Jul 06, 2023, 7:17 PM UTC

ayende wrote: post #178

hyc_symas wrote: post #161

Earlier quoted context omitted.

Every HDD since the 1980s has guaranteed atomic sector writes: > Currently all hard drive/SSD manufacturers guarantee that 512 byte sector writes are atomic. As such, failure to write the 106 byte header is not something we account for in current LMDB releases. Also, failures of this type should result in ECC errors in the disk sector - it should be impossible to successfully read a sector that was written incorrectl…

Doesn't help when you work with pages :-) Assume 512 sectors ( I know those are rare ), but I don't think that there is any guarantees that 4KB page would be: * Written atomically * Written in a particular order

Even memory ordering guarantees within sector boundaries are sufficient, and something the kernel could provide on its own.

Post by afr0ck » Sun, Jul 09, 2023, 12:18 AM UTC

crabbone wrote: post #170

afr0ck wrote: post #169

Earlier quoted context omitted.

msync() affects only the pages that part of the mmap area you ask for in the arguments. From the man pages: > int msync(void addr[.length], size_t length, int flags); > msync() flushes changes made to the in-core copy of a file that was mapped into memory using mmap(2) back to the filesystem

No it doesn't. That's physically impossible. Read what you quoted -- it never says that it's going to do it only for the file in question. If you don't know why it's not possible, here's a simplified version of it: hardware protocols (s.a. SCSI) must have fixed size messages to fit them through the pipeline. I.e. you cannot have a message larger than the memory segment used for communication with the device, because…

I don't know what you're talking about, but msync() flushes only the pages in that range. The pages are in the page cache (on Linux, it's a per-file xarray [1] of pages). Once all the dirty pages in the range are located, they go through the filesystem to be mapped to block numbers and then submitted to the block layer to be written to the storage device. Only the disk blocks mapped to the pages in that range will be written.

Source: I'm a Linux kernel developer.

[1] https://docs.kernel.org/core-api/xarray.html

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)

Re: Are You Sure You Want to Use MMAP in Your Database Management System? (2022)