Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Something is deeply broken in OS X memory management

workstuff.tumblr.com

Post reply on HN
Print view View original

241–250 of 269 posts

Re: Something is deeply broken in OS X memory management

#241

Post by fredsted »

replax wrote: post #191
rohansingh wrote: post #60

Earlier quoted context omitted.

Yes. I reinstalled OS X on a couple machines do to various issues. I was extremely surprised at how nice the experience is. When I logged back in after the reinstall, Chrome even reopened with my old tabs in it.

Not sure if that is a good or a bad thing. Would really really annoy me, tough. If I reinstall an OS, I do it because I want a clean plate, everything back to "standard". Opening my old tabs in my browser tells me, that it kept various config/temp files, which is probably what I wanted to get rid of.

User and System files are separate in Unix.

It's a good thing.

Re: Something is deeply broken in OS X memory management

#242

Post by Intermernet »

joe_the_user wrote: post #157
w0utert wrote: post #4

Not this again, we already went through this a few weeks ago. Back then, I thought the conclusion was that there is nothing broken about OS X memory management, and that with every 'fix' you come up with, you will just introduce another degenerate corner case. The same holds for any OS, trade-offs are made that may have some negative effect in some cases, to the benefit of the general cases. I don't recognize any of…

Not this again, we already went through this a few weeks ago. In- fricken -deed... We have been through this problem again and again and again, in different OSes, at different times and with different things triggering the various problems. It usually ends with a "neck-beard" saying with enough authority "look, really, they are doing it right even if it seems totally illogical to you and any brokenness is just your c…

> the solutions to it that any of these OSes have are tuned-black-magic-split-the-difference-haphazard affairs.

I remember reading a book that was basically annotated Linux 2.4 kernel code (Brilliant book, taught me lots, now can't find it) that discussed the, very deliberately, sloppy method of allocating which tables to swap, and when. The justification was that, similar to scheduling process time-slices, the processing penalty of finely controlling every request outweighed the error correction penalty that was inherent in actual testing.

AFAIK the 2.6 and 3 series kernels have even more "tuned-black-magic" under the hood.

I'd hate to even think what my Windows 7 machine behind the scenes... I've often worried that a windows core dump might actually summon Demons from the Dungeon Dimensions (thanks TP)

Re: Something is deeply broken in OS X memory management

#243

Post by Intermernet »

boundlessdreamz wrote: post #215
sipefree wrote: post #186

Earlier quoted context omitted.

I've been trying to get people I know at Apple to start referring to it internally as Mountain Goat. It's working.

Are you implying that Mountain Lion is slow?

Mountain Goats rock! I reckon one might even get away from a Mountain Lion. Operative word .

Re: Something is deeply broken in OS X memory management

#244

Post by st3fan »

saurik wrote: post #222
thought_alarm wrote: post #59

Earlier quoted context omitted.

Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. That's not swapping. That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login. Create a .hushlogin file in your home directory to prevent that.

No. The sequence of events concerning this in login: 0) `login -pf` 1) quietlog = 0 2) if ("-q" in argv) quietlog = 1 3) if (!quietlog) getlastlogxbyname(&lastlog) 4) if (!quietlog) quietlog = access(".hushlogin") == 0 5) dolastlog(quietlog) -> 6) if (!quietlog) printf(lastlog) You can see from this that the "searching the system logs" (which, to be clear, is going to be really really fast: /var/run/utmpx is a small…

Saurik, using dtrace I do see something touching about 50 logs in /var/log/asl every time in login. I wonder where that comes from. I don't think it is getlastlogxbyname() or dolastlog().

Re: Something is deeply broken in OS X memory management

#245

Post by dspillett »

huhtenberg wrote: post #150
thought_alarm wrote: post #59

Earlier quoted context omitted.

Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. That's not swapping. That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login. Create a .hushlogin file in your home directory to prevent that.

So it takes 5 seconds to search the system logs for the date of the last login? How is this any less broken?

It isn't broken from a memory management point of view, which is what the post being discussed is talking about, I think is the point.

Re: Something is deeply broken in OS X memory management

#246

Post by dspillett »

carguy1983 wrote: post #51

I have an 8-way Xeon Mac Pro w/ 20GB of RAM, almost half of which is 'free' at any point during the day unless I'm doing something really out of the ordinary. Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. I really don't give a shit how it's not "technically" broken - that's broken from an experience point of view. And I haven't re-installed the OS (this was an…

> Yet it still swaps to disk ALL THE TIME

Is it actively paging to disk at times when there is plenty of free RAM? A common complaint on Linux is "I've closed a lot of stuff but it is still using swap" because even if the pages are read back into RAM when next used they are kept in the swap area in case they need to go out again (that way they don't need to be written unless changed, saving some I/O).

Under Linux you can see how much is found in both RAM and on disk with the "SwapCached" entry in /proc/meminfo - it won't stop counting those pages as present in the swap areas until either it runs out of never used swap space so needs to overwrite then to page out other pages or the page is changes in memory (at which point the copy on disk is stale so can not be reused without being updated anyway).

> and a new Terminal.app window can take up to 5 seconds to open.

Have you monitored system activity at such times to see where the delay is? While it could be due to unnecessary disk I/O it could also be elsewhere such as delayed DNS lookups if anything in your profile scripts does such a thing and there is an issue with your connectivity or DNS configuration.

(I'm not an OS X user and never have been so sorry if these thoughts are irrelevant - but I'm guessing memory management in OS X is similar enough for knowledge of how Linux plays the game not to be completely useless)

Re: Something is deeply broken in OS X memory management

#247

Post by commononsense »

It really doesn't matter if it's broken. Apple already has your money.

Food for thought: When something like this happens to a free OS, like NetBSD, to which Mr. Metzger has been a longtime contributor, it usually gets fixed rather quickly.

I haven't run with disk swap in many years. With FreeBSD, you need to at least configure swap on a /dev/md, but it doesn't need a real disk. The FreeBSD kernel may panic otherwise. With NetBSD you don't even need a md. It is nearly impossible to get the kernel to panic from memory starvation.

These are "free" systems.

What did you pay for your MacBook Pro and it's amazing GUI?

Re: Something is deeply broken in OS X memory management

#248

Post by thought_alarm »

saurik wrote: post #222
thought_alarm wrote: post #59

Earlier quoted context omitted.

Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. That's not swapping. That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login. Create a .hushlogin file in your home directory to prevent that.

No. The sequence of events concerning this in login: 0) `login -pf` 1) quietlog = 0 2) if ("-q" in argv) quietlog = 1 3) if (!quietlog) getlastlogxbyname(&lastlog) 4) if (!quietlog) quietlog = access(".hushlogin") == 0 5) dolastlog(quietlog) -> 6) if (!quietlog) printf(lastlog) You can see from this that the "searching the system logs" (which, to be clear, is going to be really really fast: /var/run/utmpx is a small…

It does not use /var/run/utmpx anymore.

Look at the code for getlastlogxbyname(). It does an ASL query for last login, and that's the source of the delay.

http://www.opensource.apple.com/source/Libc/Libc-763.12/gen/...

Re: Something is deeply broken in OS X memory management

#249

Post by mistercow »

mlreed328 wrote: post #208
mistercow wrote: post #93

Earlier quoted context omitted.

Well, I suppose it seems sillier on a laptop than on a desktop.

or a server...

Well sure, but I wasn't saying that it should be default behavior for Mac OS X server or server-aimed n*x distros. I don't think that most people are using non-server Mac OS X as a remote server, and those who are could override the default.

Re: Something is deeply broken in OS X memory management

#250

Post by napierzaza »

ramblex wrote: post #239
9oliYQjP wrote: post #136

Earlier quoted context omitted.

You sir are a gentleman and scholar. As a laptop user with only 8 GB of RAM and a slow non-SSD hard disk, this one trick just made my day! Thanks.

Only 8GB? ha. I have 4GB and having any kind of flash player open in chrome means I have no RAM free.

I disabled flash a long time ago. The hard part is trying to download Youtube videos to disk since I had buffering (via flash or html5).
Post reply on HN