Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Something is deeply broken in OS X memory management

workstuff.tumblr.com

Post reply on HN
Print view View original

251–260 of 269 posts

Re: Something is deeply broken in OS X memory management

#251

Post by saurik »

thought_alarm wrote: post #248
saurik wrote: post #222

Earlier quoted context omitted.

No. The sequence of events concerning this in login: 0) `login -pf` 1) quietlog = 0 2) if ("-q" in argv) quietlog = 1 3) if (!quietlog) getlastlogxbyname(&lastlog) 4) if (!quietlog) quietlog = access(".hushlogin") == 0 5) dolastlog(quietlog) -> 6) if (!quietlog) printf(lastlog) You can see from this that the "searching the system logs" (which, to be clear, is going to be really really fast: /var/run/utmpx is a small…

It does not use /var/run/utmpx anymore. Look at the code for getlastlogxbyname(). It does an ASL query for last login, and that's the source of the delay. http://www.opensource.apple.com/source/Libc/Libc-763.12/gen/...

As I stated, that cannot be the source of the delay, because getlastlogxbyname is called based on a check of quietlog before quietlog is updated to take into account .hushlogin. With the exception of step #7, all of this code is inside of a single function (main), which makes it very easy to verify that the sequence of events I'm describing is correct. (I will happily believe you, however, that getlastlogxbyname is internally now using something horrendously slow to look up that information.)

(edit: I have gone ahead and verified your statements regarding getlastlogxbyname now being based on ASL. Using that knowledge, and based on st3fan's comments about the output of dtrace, I then used dtruss to verify my own assertion regarding the order of events. The result: .hushlogin in fact only affects the output of "last login"; it does not keep login from getting that information in the first place with ASL. To keep it from doing so you must pass -q, something Terminal does not do.)

Re: Something is deeply broken in OS X memory management

#252

Post by saurik »

st3fan wrote: post #244
saurik wrote: post #222

Earlier quoted context omitted.

No. The sequence of events concerning this in login: 0) `login -pf` 1) quietlog = 0 2) if ("-q" in argv) quietlog = 1 3) if (!quietlog) getlastlogxbyname(&lastlog) 4) if (!quietlog) quietlog = access(".hushlogin") == 0 5) dolastlog(quietlog) -> 6) if (!quietlog) printf(lastlog) You can see from this that the "searching the system logs" (which, to be clear, is going to be really really fast: /var/run/utmpx is a small…

Saurik, using dtrace I do see something touching about 50 logs in /var/log/asl every time in login. I wonder where that comes from. I don't think it is getlastlogxbyname() or dolastlog().

As thought_alarm states, getlastlogxbyname may not be accessing utmpx anymore (I have not myself checked); however, the behavior of that function cannot be affected by .hushlogin, as it is called before .hushlogin is checked. All of this logic (excepting step #7) happens within a single function (main), so it is very simple to see the flow.

(edit: I have gone ahead and checked: thought_alarm is correct, in that getlastlogxbyname is now using ASL instead of utmpx; however, I have also verified my sequencing assertion with dtrace: .hushlogin has no effect on the usage of ASL, but manually passing -q to login does: it thereby cannot be the source of a .hushlogin-mediated delay.)

Re: Something is deeply broken in OS X memory management

#253

Post by batista »

ramblex wrote: post #239
9oliYQjP wrote: post #136

Earlier quoted context omitted.

You sir are a gentleman and scholar. As a laptop user with only 8 GB of RAM and a slow non-SSD hard disk, this one trick just made my day! Thanks.

Only 8GB? ha. I have 4GB and having any kind of flash player open in chrome means I have no RAM free.

Really? Because I have 2GB too and I can have several flash videos open and get along just fine with other programs.

Re: Something is deeply broken in OS X memory management

#254

Post by batista »

sciurus wrote: post #82
k-mcgrady wrote: post #62

Earlier quoted context omitted.

I'm running on a 160GB HDD (nearly full), 2GB ram, on a 4 year old MacBook. I always have iTunes, Xcode, Safari/Chrome running with multiple tabs, and a mail client (Sparrow). The system originally ran Leopard and I upgraded to SL and then Lion. I wouldn't say it's slow. It's definitely getting slower but that's expected since it's such an old system. Could it be that you're just expecting too much from your computer…

"It's definitely getting slower but that's expected since it's such an old system." I see this sentiment a lot, but I disagree with it. What are "iTunes, Xcode, Safari/Chrome, and a mail client" doing now that they weren't doing four years ago? Is it enough to justify their latest versions feeling less responsive than their versions from four years ago?

>I see this sentiment a lot, but I disagree with it. What are "iTunes, Xcode, Safari/Chrome, and a mail client" doing now that they weren't doing four years ago?

Lot's of things. XCode was rewritten and does live AST syntax completion, background compiles, etc.

Safari/Chrome have several more features --did Chrome even exist 4 years ago?

Re: Something is deeply broken in OS X memory management

#255

Post by batista »

merlish wrote: post #23
w0utert wrote: post #4

Not this again, we already went through this a few weeks ago. Back then, I thought the conclusion was that there is nothing broken about OS X memory management, and that with every 'fix' you come up with, you will just introduce another degenerate corner case. The same holds for any OS, trade-offs are made that may have some negative effect in some cases, to the benefit of the general cases. I don't recognize any of…

I see the same issues the guy in the article has seen. Basically, I have a total of 8GB of ram in this here mac mini, and when my desktop session gets heavy things start going south. For example: Let's take a large Chrome session (~150 tabs spread over several windows), an IDE open somewhere, Spotify, Steam and some background apps, and a small Windows VM. Generally, Activity Monitor would show that Chrome in this in…

>I see the same issues the guy in the article has seen. Basically, I have a total of 8GB of ram in this here mac mini, and when my desktop session gets heavy things start going south. For example: Let's take a large Chrome session (~150 tabs spread over several windows), an IDE open somewhere, Spotify, Steam and some background apps, and a small Windows VM.

150 tabs? A VM? An IDE?

That's just A LOT. Of course things will go south, what did you expect a magic machine that can run everything and whistle away hapilly with 0% load?

Re: Something is deeply broken in OS X memory management

#256

Post by batista »

seanalltogether wrote: post #45
obituary_latte wrote: post #22

Earlier quoted context omitted.

>I don't recognize any of his symptoms anyway, and my OS X computers get pretty RAM-heavy use, with almost always a linux VM open, XCode, Safari with ~10 tabs, iTunes with a few thousand songs, etc. Oh boy I wish I could say the same. Admittedly I don't shut down on a daily basis but this didn't used to be a problem in SL. FWIW I've an old tank of a tower that has a video card on its last leg.. shutting down invariab…

I had to shut down on a daily basis because of this problem with my system having only 4 GB of ram. There is definitely something broken in there. http://i.imgur.com/ohBEF.png

I don't see anything wrong in this picture. You still have free memory. Even if it didn't show free memory you would probably still be OK, because it would just be reserved for use

Re: Something is deeply broken in OS X memory management

#257

Post by batista »

nirvana wrote: post #149
kfury wrote: post #122

Thank god for this article. My wife is a photographer making heavy use of Lightroom on her 17" MBP and has been experiencing these exact problems for a year or two. We've tried everything to fix it, rebuilding the system from scratch, to no avail. She had 4 gigs of RAM which we recently upped to 8gigs which reduced the severity of the problem. I really, really hope this is something that gets fixed in Mountain Lion.…

She's not crazy, but she is running Adobe software on a machine without sufficient RAM. Adobe installs gods own cache of really crappy stuff that starts up at boot and who knows what kind of kexts they shove in there to make your machine unstable. I won't run any adobe software after I saw the abuse they did to my machine. Apple basically gets a free pass if you're running Adobe. This is a company that ships crap. Al…

>Apple basically gets a free pass if you're running Adobe. This is a company that ships crap.

Yes, millions of professional designers using Adobe software are idiots. You are just making BS claims with no support whatsoever. Try opening a huge image in Photoshop and any other editor and see which behaves better and faster.

The only "crap" stuff Adobe does is mostly whatever it acquired from Macromedia.

>I'd recommend getting rid of Lightroom and going to Aperture, or given aperture is a bit behind the curve, upgrading to 16GB of RAM and seeing what adobe-installed processes and KEXTS you can get rid of.

And I'd recommend not listening to BS anecodotal suggestions on the internets. Read a couple of professionally done reviews and benchmarks. All state that Lightroom is faster and more efficient that Aperture. Aperture got a little better in the last version, but still no match to Lightroom.

(I'm not bashing Apple, I like both. Things are what they are though, and yes I've tried both of them.

Thing is: working with freaking huge images, like hundreds of 16 megapixel RAW files, will be slow, whatever you use.

Re: Something is deeply broken in OS X memory management

#258

Post by batista »

kfury wrote: post #174
nirvana wrote: post #149

Earlier quoted context omitted.

She's not crazy, but she is running Adobe software on a machine without sufficient RAM. Adobe installs gods own cache of really crappy stuff that starts up at boot and who knows what kind of kexts they shove in there to make your machine unstable. I won't run any adobe software after I saw the abuse they did to my machine. Apple basically gets a free pass if you're running Adobe. This is a company that ships crap. Al…

Moving from Lightroom to Aperture's not a possibility, given the workflow, experience, and catalog data she's built up in Lightroom over the years. Upgrading from 4 to 8 gigs last week helped a lot. I'd go to 16 except her MBP won't support it. I'd love to get her on an SSD but she's on a 1tb drive now and it would be hard for her to try and fit into a 512gb SSD now (especially now that she's on the D800 with huge vi…

>Moving from Lightroom to Aperture's not a possibility, given the workflow, experience, and catalog data she's built up in Lightroom over the years.

Forget the advice. Lightroom is faster, as noted in every review of both programs. Try Aperture yourself with the demo to find out.

Working with 10+ megapixel images is always going to be slow, and with camera advances, it will get worse every time your wife gets a higher resolution camera --so comparing it with how it used to be when you have 6mp files is not exactly correct.

More memory and an SSD will definitely help.

Re: Something is deeply broken in OS X memory management

#259

Post by thought_alarm »

saurik wrote: post #251
thought_alarm wrote: post #248

Earlier quoted context omitted.

It does not use /var/run/utmpx anymore. Look at the code for getlastlogxbyname(). It does an ASL query for last login, and that's the source of the delay. http://www.opensource.apple.com/source/Libc/Libc-763.12/gen/...

As I stated, that cannot be the source of the delay, because getlastlogxbyname is called based on a check of quietlog before quietlog is updated to take into account .hushlogin. With the exception of step #7, all of this code is inside of a single function (main), which makes it very easy to verify that the sequence of events I'm describing is correct. (I will happily believe you, however, that getlastlogxbyname is i…

You're right. ASL is the source of the slowdown, but .hushlogin isn't actually doing anything to solve the problem.

The correct way to bypass the ASL query is to set Terminal to open shells with /bin/bash (or your shell of choice) instead of the default login shell. Terminal will still use /usr/bin/login to launch the shell, but it passes the -q switch to prevent the ASL query.

When I dug into the source code a couple of months ago, I inadvertently made both changes (Terminal settings and .hushlogin). Clearly it's the Terminal settings that solved the problem and not .hushlogin. Thanks for clearing it up.

Re: Something is deeply broken in OS X memory management

#260

Post by JoeAltmaier »

joe_the_user wrote: post #157
w0utert wrote: post #4

Not this again, we already went through this a few weeks ago. Back then, I thought the conclusion was that there is nothing broken about OS X memory management, and that with every 'fix' you come up with, you will just introduce another degenerate corner case. The same holds for any OS, trade-offs are made that may have some negative effect in some cases, to the benefit of the general cases. I don't recognize any of…

Not this again, we already went through this a few weeks ago. In- fricken -deed... We have been through this problem again and again and again, in different OSes, at different times and with different things triggering the various problems. It usually ends with a "neck-beard" saying with enough authority "look, really, they are doing it right even if it seems totally illogical to you and any brokenness is just your c…

If you're not oversubscribed, it shouldn't thrash. Isn't that the complaint? Then its a legitimate issue.
Post reply on HN