Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Something is deeply broken in OS X memory management

workstuff.tumblr.com

Post reply on HN
Print view View original

171–180 of 269 posts

Re: Something is deeply broken in OS X memory management

#171

Post by st3fan »

telemachos wrote: post #118
thought_alarm wrote: post #59

Earlier quoted context omitted.

Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. That's not swapping. That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login. Create a .hushlogin file in your home directory to prevent that.

Two other things that can make OSX terminal launching slow: 1. The default use of /usr/libexec/path_helper to manage your $PATH .[1] 2. An accumulation of log files in /var/log/asl .[2] For (1), I just edit /etc/profile and disable path_helper altogether. I set the PATH manually. (This also allows me to put /usr/local/bin before /usr/bin , which is my preference. I've never understood Apple's default settings for $PA…

time /usr/libexec/path_helper PATH="/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/share/python:/usr/local/sbin:/Users/stefan/bin"; export PATH;

real 0m0.004s user 0m0.001s sys 0m0.002s

Really? Are you sure path_helper slows things down?

Re: Something is deeply broken in OS X memory management

#172

Post by lattepiu »

Something is quite wrong indeed. I disabled the dynamic pager, and now my system is working as it's supposed to. Snappy and responsive.

I opened all of my apps, expecting it to crash miserably: instead, the system started paging as it should, stayed responsive (though slower), and promptly returned to normal once it regained memory.

I don't know what's going on, but I can definitely say that this is how I want my computer to work.

Re: Something is deeply broken in OS X memory management

#173

Post by georgemcbay »

andyzweb wrote: post #160
tomc1985 wrote: post #48

I did this for awhile and ran into an interesting (if fatal) edge case: I use a 1.5tb external drive formatted in exFAT to minimize cross-platform headaches, and whenever the drive is marked dirty (improper shutdown, eject, etc), OSX will run fsck_exfat on it before I can use it. fsck_exfat isn't a huge deal -- or wouldn't be, if it didn't have a nasty tendency to leak RAM... the moment you plug in, fsck_exfat's foot…

exFAT is Microsoft proprietary shit.

The problem isn't with exFAT, it is with fsck_exfat, which I'm fairly certain Microsoft didn't write.

chkdsk on Windows manages to clean exFAT volumes just fine without using up 8gb+ of memory.

Re: Something is deeply broken in OS X memory management

#174

Post by kfury »

nirvana wrote: post #149
kfury wrote: post #122

Thank god for this article. My wife is a photographer making heavy use of Lightroom on her 17" MBP and has been experiencing these exact problems for a year or two. We've tried everything to fix it, rebuilding the system from scratch, to no avail. She had 4 gigs of RAM which we recently upped to 8gigs which reduced the severity of the problem. I really, really hope this is something that gets fixed in Mountain Lion.…

She's not crazy, but she is running Adobe software on a machine without sufficient RAM. Adobe installs gods own cache of really crappy stuff that starts up at boot and who knows what kind of kexts they shove in there to make your machine unstable. I won't run any adobe software after I saw the abuse they did to my machine. Apple basically gets a free pass if you're running Adobe. This is a company that ships crap. Al…

Moving from Lightroom to Aperture's not a possibility, given the workflow, experience, and catalog data she's built up in Lightroom over the years.

Upgrading from 4 to 8 gigs last week helped a lot. I'd go to 16 except her MBP won't support it.

I'd love to get her on an SSD but she's on a 1tb drive now and it would be hard for her to try and fit into a 512gb SSD now (especially now that she's on the D800 with huge video files and 72mb raw photo files.

It's frustrating that it will work find some of the time and not others, implying that the problem could be fixed with better memory management. I do hope that a serious Adobe competitor arises to force Adobe to make its apps faster and more resource efficient.

Re: Something is deeply broken in OS X memory management

#175

Post by joelreymont »

eridius wrote: post #92
collint wrote: post #2

You can run `purge` in a terminal to free your 'inactive' ram. I've set up a cron job to purge frequently, keeps thing humming.

That's a terrible idea. `purge` is not a tool for freeing inactive ram. It's a tool for deleting the filesystem cache in order to simulate cold boot conditions for performance analysis. Yes, it has the side effect of reducing memory usage, inasmuch as it's throwing away stuff that was kept in memory. But that doesn't make it an appropriate tool to use.

From my days of optimizing Firefox startup on the Mac, I remember that this doesn't work 100%. The only way to ensure that caches are 100% purged is to unmount the filesystem.

Re: Something is deeply broken in OS X memory management

#176

Post by telemachos »

st3fan wrote: post #171
telemachos wrote: post #118

Earlier quoted context omitted.

Two other things that can make OSX terminal launching slow: 1. The default use of /usr/libexec/path_helper to manage your $PATH .[1] 2. An accumulation of log files in /var/log/asl .[2] For (1), I just edit /etc/profile and disable path_helper altogether. I set the PATH manually. (This also allows me to put /usr/local/bin before /usr/bin , which is my preference. I've never understood Apple's default settings for $PA…

time /usr/libexec/path_helper PATH="/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/share/python:/usr/local/sbin:/Users/stefan/bin"; export PATH; real 0m0.004s user 0m0.001s sys 0m0.002s Really? Are you sure path_helper slows things down?

> Are you sure path_helper slows things down?

I'm sure that it did, but not sure that it does. The code you ran isn't quite what /etc/profile does. Here's a run of that on an older machine where I work (see below on versions):

    $ time eval `/usr/libexec/path_helper -s`

    real	0m0.106s
    user	0m0.084s
    sys	        0m0.021s
However, looking at my current machine, I realize that /usr/libexec/path_helper is no longer a shell script at all. It's compiled. Running that same test on a newer machine, I get this:

    $ time eval `/usr/libexec/path_helper -s`

    real	0m0.004s
    user	0m0.001s
    sys  	0m0.003s
So, in a nutshell, I think you're right for current machines: The path_helper advice looks to be out of date. I can't edit my original answer any more, but thanks for making me rethink this (I've been annoyed by slow-opening terminals in OSX for years. Apparently, they worked on this part of the problem.)

Re: Something is deeply broken in OS X memory management

#177

Post by adavies42 »

thought_alarm wrote: post #59
carguy1983 wrote: post #51

I have an 8-way Xeon Mac Pro w/ 20GB of RAM, almost half of which is 'free' at any point during the day unless I'm doing something really out of the ordinary. Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. I really don't give a shit how it's not "technically" broken - that's broken from an experience point of view. And I haven't re-installed the OS (this was an…

Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. That's not swapping. That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login. Create a .hushlogin file in your home directory to prevent that.

relatedly, i call "`eval resize`" in my .zshrc--any idea why that takes ~5s if it's more than a minute or two since i last did it, but is instantaneous otherwise? (yes, i'm certain that it's that line that's taking time--if i start typing during the delay before the prompt shows up, i get a resize error about unrecognized characters and then a couple bits of random ANSI on my input.)

Re: Something is deeply broken in OS X memory management

#178

Post by kristinn »

acdha wrote: post #133
kristinn wrote: post #103

Earlier quoted context omitted.

I bought a brand new Macbook Pro, 15" with upgraded CPU, etc. with 4GB of RAM last year (2011 model). I didn't want to upgrade to a SSD, I wanted to do that later. After I upgraded to Lion (through the Mac Store) the machine is as speedy as my old Macbook Pro I bought in 2006 and is running Snow Leopard. It's horrible and it's so annoying and I'm pretty much really angry. If Apple will not fix this I will NEVER buy a…

Use something like fs_usage to see which of your programs is churning the disk. This is not the normal state of Lion; there are two system services (Spotlight and Time Machine) which can generate tons of background I/O through normal operation - I usually disable TM outright and make sure Spotlight ignores my development folder.

I'll check that out. Thank you for your suggestion. One thing I find strange though, this started happening after I upgraded to Lion. It wasn't like that when I was using Snow Leopard and both Time Machine and Spotlight exist in Snow Leopard.

Re: Something is deeply broken in OS X memory management

#179

Post by ricardobeat »

obituary_latte wrote: post #54
colomon wrote: post #43

Earlier quoted context omitted.

For what it's worth, I'm running Snow Leopard (10.6.8) and I see the same sort of terrible performance the article describes. Though I did improve matters drastically by telling mds not to index my (Linux) MP3 server and my Time Machine drive. That made it go from nearly unusable to just frequently annoying. The other crazy thing I've been seeing is it routinely takes Chrome minutes to shutdown -- in fact, pretty muc…

Ah yes. The dreaded Chrome renderer never enderer. I too have experienced the Chrome issue enough that I don't even try to close it normally anymore. Force quit is the only way I exit Chrome. Thankfully, the restore tabs functionality works well.

I'm glad to see I'm not the only one. This drives me crazy every time.

Re: Something is deeply broken in OS X memory management

#180

Post by n8gray »

telemachos wrote: post #176
st3fan wrote: post #171

Earlier quoted context omitted.

time /usr/libexec/path_helper PATH="/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/share/python:/usr/local/sbin:/Users/stefan/bin"; export PATH; real 0m0.004s user 0m0.001s sys 0m0.002s Really? Are you sure path_helper slows things down?

> Are you sure path_helper slows things down? I'm sure that it did , but not sure that it does . The code you ran isn't quite what /etc/profile does. Here's a run of that on an older machine where I work (see below on versions): $ time eval `/usr/libexec/path_helper -s` real 0m0.106s user 0m0.084s sys 0m0.021s However, looking at my current machine, I realize that /usr/libexec/path_helper is no longer a shell script…

You're not crazy -- path_helper used to be godawful slow. It used to be a shell script with exponential (IIRC) time complexity in the number of components in your path. It could very easily add a huge delay to your launch time.
Post reply on HN