Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use mmap with care

sublimetext.com

Post reply on HN
Print view View original

211–218 of 218 posts

Re: Use mmap with care

#211

Post by user5994461 »

bogomipz wrote: post #205
user5994461 wrote: post #184

Earlier quoted context omitted.

>>> One of the first rules in the marvelous book The Pragmatic Programmer is "Select() isn't broken". Yeah, but sometimes it is. How come you give this example just now? This can't be a coincidence. select() just caused us a major production outage. FYI: select() is broken on pretty much all Linux kernels up to very recent ones. Doesn't work when there are more than 1024 opened file descriptors, not a lot for a serve…

>"FYI: select() is broken on pretty much all Linux kernels up to very recent ones." Can you elaborate or might you have some links? What was the cause and resolution?

See first paragraph in section BUGS. http://man7.org/linux/man-pages/man2/select.2.html

If you google for select 1024 file descriptors, you will find a ton of issues affecting a ton of libraries. The correction seems to be using poll(), but in our case was to remove the routine that wasn't needed anymore.

Re: Use mmap with care

#212

Post by quotemstr »

gubbrora wrote: post #85
quotemstr wrote: post #21

There's also the matter of taking an implicit "system call" (via page fault) the first time your program touches a page that hasn't yet been faulted. This old myth that mmap is the fast and efficient way to do IO just won't die. mmap does have perfectly legitimate use cases (e.g., reducing anonymous commit charge) but you should try to make regular reads work first. That said , there's nothing wrong with mmap or SIGB…

Interesting but how will you determine not-null is the hot path? Getting it wrong coughs cost you

In the case of VMs, null means "throw NullPointerException". How often do you see that happening?

Re: Use mmap with care

#213

Post by smsm42 »

mcguire wrote: post #128
beat wrote: post #108

The first serious bug I ever dealt with professionally was a result of the hazards of mmap(). This was 1995, and I was working on AIX with a system that used a series of shared memory buffers for IPC. It was originally written with shmat(), and on AIX (at least in those days), shmat was limited to three shared segments, so we had a lot of performance-wrecking blocking going on while waiting for the buffers to be clea…

AIX 3.2.5, around the same time, had a different mmap issue. If you mmap a file and write a byte into the mapped region, but beyond the end of the file, you get a signal; in handling the signal, you can extend the file. You also have 1. Made an error, as documented in the mmap man page, 2. Done something that works on JFS, 3. In fact, done what JFS does itself under the covers, and 4. Made a huge, huge mistake. If yo…

NFS deadlocks are not unique to AIX. I've seen NFS in the state where any attempt to touch it locks up the process doing it (in the state where it can't even be killed - yes, the 'D' state) and if you need something from that process, good luck - reboot is pretty much the only way out. Nowdays, I think, NFS drivers in Linux became a bit saner but before that, if you get D's on NFS, you're out of luck.

Re: Use mmap with care

#214

Post by snarfy »

dullgiulio wrote: post #96
rocqua wrote: post #86

Earlier quoted context omitted.

Vim needs to do a lot more special stuff, because it has to handle random insertions and deletions. Doing those things to a file (mmaped or otherwise) requires moving all data after the edit. If I recall correctly, vim/vi uses a linked list of 'chunks' that is dynamically merged. For long files I would expect some form of lazy loading of chunks.

That's right. The key data structure is the rope: https://en.wikipedia.org/wiki/Rope_(data_structure)

Nice! I thought xi was the only editor that used ropes.

Re: Use mmap with care

#215

Post by [deleted] »

ben-schaaf wrote: post #180
0xffff2 wrote: post #121

Earlier quoted context omitted.

Different architectures like ARM? Plenty of code will only ever run on an x86 chip, but a non-trivial amount will run on x86 and ARM at some point. I've certainly been bitten by this before.

Sure, but my point is that both Sublime products only support x86 currently. There's likely a fair amount of other stuff that would break if we ported to ARM.

[deleted]

Re: Use mmap with care

#216

Post by gianpaj »

ben-schaaf wrote: post #117
daemin wrote: post #66

Earlier quoted context omitted.

I was using Sublime Merge on the weekend and after doing a few changes with my remotes using the normal command line, it exited (well crashed I assume). When things like this happen do you automatically get an error report? Because you mentioned that you use the error reporting library from Google in that article.

We do get crash reports. If you have some more details in relation to this it would be great to hear from you on our forums or on the issue tracker: https://github.com/SublimeTextIssues/Core

I believe the correct link for SublimeMerge GH issues is: https://github.com/sublimehq/sublime_merge

Re: Use mmap with care

#217

Post by speedplane »

beat wrote: post #108

The first serious bug I ever dealt with professionally was a result of the hazards of mmap(). This was 1995, and I was working on AIX with a system that used a series of shared memory buffers for IPC. It was originally written with shmat(), and on AIX (at least in those days), shmat was limited to three shared segments, so we had a lot of performance-wrecking blocking going on while waiting for the buffers to be clea…

> One of the first rules in the marvelous book The Pragmatic Programmer is "Select() isn't broken". Yeah, but sometimes it is. And if a junior programmer came to me today reporting a bug in OS memory management, my first response would be "What's wrong with your code, really?"

In my years of software development I had the fun but unfortunate experience of coming across not one, but two compiler bugs. The program would segfault on a seemingly innocuous line, but after looking a the assembly, it was clear the compiler was doing something screwy. These types of bugs are exceedingly rare, and they can be maddening to debug, but when you catch them you get a nice sense of satisfaction.

Re: Use mmap with care

#218

Post by techslave »

beat wrote: post #108

The first serious bug I ever dealt with professionally was a result of the hazards of mmap(). This was 1995, and I was working on AIX with a system that used a series of shared memory buffers for IPC. It was originally written with shmat(), and on AIX (at least in those days), shmat was limited to three shared segments, so we had a lot of performance-wrecking blocking going on while waiting for the buffers to be clea…

this is an example of where experience matters. kids these days don’t get to experience deep bugs like this. and they are the worse for it.
Post reply on HN