Use mmap with care • Hacker News Board

Re: Use mmap with care

#121

Post by 0xffff2 » Thu, May 02, 2019, 4:53 PM UTC

ben-schaaf wrote: post #48

Asooka wrote: post #14

Oh I see you didn't get to caveat 5: you can't read anything more complicated than raw bytes, i.e. chars, because of unaligned memory access errors. Let's say you mmap a file and do something like this: char *fileContents=...mmap etc...; int headerOffset=*(int*)fileContents; int *someListOfNumbers=(int*)(fileContents+headerOffset); int importantSum= someListOfNumbers[0] + someListOfNumbers[1] + someListOfNumbers[2] +…

> Oh I see you didn't get to caveat 5 > On x86 you'll get away with it... Guess we'll have to wait for the proliferation of a different architecture to encounter this one :)

Different architectures like ARM? Plenty of code will only ever run on an x86 chip, but a non-trivial amount will run on x86 and ARM at some point. I've certainly been bitten by this before.

Re: Use mmap with care

#122

Post by theoh » Thu, May 02, 2019, 4:59 PM UTC

rocqua wrote: post #84

Honestly, this reads like a thorough indictment of signals in user space. * Signal handlers are process global * Signal handlers need to be re-entrant safe Re-entrancy is painful but can be done, but process-global signal handlers means that pulling in a totally unrelated library can break your code. Moreover, it makes the combined use of certain libraries straight-up impossible. Similarly, it means that the use of l…

Previously: https://news.ycombinator.com/item?id=11899385

"I found a paragraph in this article http://www.linusakesson.net/programming/tty/ very apt at describing what Unix signals are like:

  In *The Hitchhiker's Guide to the Galaxy*, Douglas Adams 
  mentions an extremely dull planet, inhabited by a bunch of 
  depressed humans and a certain breed of animals with sharp
  teeth which communicate with the humans by biting them very
  hard in the thighs. This is strikingly similar to UNIX, in
  which the kernel communicates with processes by sending
  paralyzing or deadly signals to them."

Re: Use mmap with care

#123

Post by ajross » Thu, May 02, 2019, 5:01 PM UTC

beat wrote: post #108

The first serious bug I ever dealt with professionally was a result of the hazards of mmap(). This was 1995, and I was working on AIX with a system that used a series of shared memory buffers for IPC. It was originally written with shmat(), and on AIX (at least in those days), shmat was limited to three shared segments, so we had a lot of performance-wrecking blocking going on while waiting for the buffers to be clea…

Yeah. It's sometimes hard to remember just How Bad software was even in living memory. These days, we all just assume that the basics all work and are surprised to see bugs, which we vote up to the top of HN. But it wasn't always like that, things just failed in crazy ways, at all levels of the stack.

At the start of the dotcom boom, a server uptime measured in months was considered notable, and the idea of a client system lasting a week between some failure or voodoo reboot was laughable. Now... I dunno, when was the last time I bounced my phone? Genuinely can't remember.

Re: Use mmap with care

#124

Post by 0xffff2 » Thu, May 02, 2019, 5:02 PM UTC

saagarjha wrote: post #95

z3t4 wrote: post #88

Or you could stream parse the file. eg. parse it one chunk at a time.

That’s what they were doing before?

Before they were reading the whole file into memory and parsing it. Now, they are using mmap to read parts of the file into memory when they are used. If you're only doing a single linear pass through a file there is a third option; you could read part of the file in a single chunk at a time, parse that chunk, then read the next part of the file into the same chunk of memory.

I don't know enough about the process being discussed here to know whether it needs to do lookups at random offsets in the file, but iff it doesn't then the chunked read solution could be a simpler way to reduce memory usage.

Re: Use mmap with care

#125

Post by theoh » Thu, May 02, 2019, 5:10 PM UTC

samcday wrote: post #42

CoolGuySteve wrote: post #39

I feel like this is kind of a dumb design. You want to abstract two different kinds of file reader: an mmap reader and a regular reader. (And I would add a gz reader, personally). Then by inspecting the properties of the file, you can determine if it is local when opening, and if so, mmap the file. I say this because if the file is coming via the network or a FAT32 partition you’re not going to save much time with mm…

But now you have two completely independent code paths. Both of which will need to go through the same maturation phase that the ST folks evidently went through with mmap. And if the code needs to evolve for other reasons, potentially both of these paths will need some love too. Seems like the worst choice in a situation like this!

Butler Lampson talks about the general concept of an overlay in operating system design. It's a slightly different case, on the surface, but the possible application in this case is that if mmap fails, you probably should have a fallback code path that tries to access the file normally.

That's obviously not going to work for all applications. But it's something you consciously have to do anyway if mmap may not be available (e.g. when working across the network). The OS design should make it easy to move up and down in this hierarchy of access methods, but the application programmer still needs to know which "level" of access they have to the file.

There's work to be done here in improving the design of the OS interface. That would lift part of the the burden of maintaining multiple paths in the application code.

https://www.youtube.com/watch?v=TRLJ6XdmgnA&t=11m35s

slides: https://bwlampson.site/Slides/Hints%20and%20principles%20(HL...

Re: Use mmap with care

#126

Post by jfhufl » Thu, May 02, 2019, 5:13 PM UTC

thomasjudge wrote: post #99

gok wrote: post #92

It would be really nice if there was a POSIX equivalent to Foundation's NSDataReadingMappedIfSafe, which uses mmap() when the file isn't backed by NFS and falls back to read() when unsafe or not worth it.

Is there a way for a process to know what kind of filesystem something lives on, or is this just a developer-provided clue?

getmntent(3)

Re: Use mmap with care

#127

Post by dicroce » Thu, May 02, 2019, 5:17 PM UTC

I've successfully used mmap() a few times in the last few years... Luckily for me the use cases I've had weren't really subject to the same problems I've since read about here and elsewhere:

1) I'm always mmap()ing the whole file (and the files are power of 2 sized). 2) The files I'm mapping are stored on file systems I control (and so are never on NFS). 3) In one case, my use of mmap() is limited to read only.

Re: Use mmap with care

#128

Post by mcguire » Thu, May 02, 2019, 5:19 PM UTC

beat wrote: post #108

The first serious bug I ever dealt with professionally was a result of the hazards of mmap(). This was 1995, and I was working on AIX with a system that used a series of shared memory buffers for IPC. It was originally written with shmat(), and on AIX (at least in those days), shmat was limited to three shared segments, so we had a lot of performance-wrecking blocking going on while waiting for the buffers to be clea…

AIX 3.2.5, around the same time, had a different mmap issue.

If you mmap a file and write a byte into the mapped region, but beyond the end of the file, you get a signal; in handling the signal, you can extend the file. You also have

1. Made an error, as documented in the mmap man page,

2. Done something that works on JFS,

3. In fact, done what JFS does itself under the covers, and

4. Made a huge, huge mistake.

If you did this on a NFS filesystem, it deadlocks the filesystem. If you are using an automounter, it turns into a tar-baby: IBM's automountd was single threaded. Any attempt to touch that filesystem deadlocks. Any attempt to mount a new filesystem blocks waiting on automountd. The only solution was to reboot the system.

This is a bit of a problem if you have a database implementation class all trying this at the same time.

I reported this to IBM and their response was, "it's documented as not working, whaddya want?" I considered releasing it on BUGTRAQ, but I didn't. The AIX 4 multithready rewrite fixed it, BTW.

Re: Use mmap with care

#129

Post by mcguire » Thu, May 02, 2019, 5:20 PM UTC

dicroce wrote: post #127

I've successfully used mmap() a few times in the last few years... Luckily for me the use cases I've had weren't really subject to the same problems I've since read about here and elsewhere: 1) I'm always mmap()ing the whole file (and the files are power of 2 sized). 2) The files I'm mapping are stored on file systems I control (and so are never on NFS). 3) In one case, my use of mmap() is limited to read only.

mmap seems to be specifically designed to trigger the differences between NFS and Unix file system semantics.

Re: Use mmap with care

#130

Post by pcwalton » Thu, May 02, 2019, 5:21 PM UTC

ajross wrote: post #123

beat wrote: post #108

The first serious bug I ever dealt with professionally was a result of the hazards of mmap(). This was 1995, and I was working on AIX with a system that used a series of shared memory buffers for IPC. It was originally written with shmat(), and on AIX (at least in those days), shmat was limited to three shared segments, so we had a lot of performance-wrecking blocking going on while waiting for the buffers to be clea…

Yeah. It's sometimes hard to remember just How Bad software was even in living memory. These days, we all just assume that the basics all work and are surprised to see bugs, which we vote up to the top of HN. But it wasn't always like that, things just failed in crazy ways, at all levels of the stack. At the start of the dotcom boom, a server uptime measured in months was considered notable, and the idea of a client…

Yep, this is part of the reason why it drives me crazy when people say the '90s were the heyday of computing. Computers were awful back then.