Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Something is deeply broken in OS X memory management

workstuff.tumblr.com

Post reply on HN
Print view View original

101–110 of 269 posts

Re: Something is deeply broken in OS X memory management

#101

Post by burgerbrain »

thought_alarm wrote: post #59
carguy1983 wrote: post #51

I have an 8-way Xeon Mac Pro w/ 20GB of RAM, almost half of which is 'free' at any point during the day unless I'm doing something really out of the ordinary. Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. I really don't give a shit how it's not "technically" broken - that's broken from an experience point of view. And I haven't re-installed the OS (this was an…

Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. That's not swapping. That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login. Create a .hushlogin file in your home directory to prevent that.

> "That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login."

Am I missing something? `w` on my linux system takes well below one second:

  [burgerbrain@eeepc] ~ % time w   
  14:11:18 up 19:41,  6 users,  load average: 0.27, 0.10, 0.14
  USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
  REDACTED
  w  0.01s user 0.05s system 61% cpu 0.096 total
I can't imagine why anything like that would ever take anywhere near 5 seconds.

Re: Something is deeply broken in OS X memory management

#102

Post by cgs »

microtonal wrote: post #72
mahyarm wrote: post #52

Earlier quoted context omitted.

Why don't you spend $140 and add an SSD to the pro? You can keep the HDD for an extra $80 with a cd drive bay HDD caddy.

Doesn't the latter void warranty?

Not in my experience. I did exactly that (install SSD in cd drive bay, and use orig hd for a second drive), and when I had a fan issue it was not a problem.

Re: Something is deeply broken in OS X memory management

#103

Post by kristinn »

w0utert wrote: post #4

Not this again, we already went through this a few weeks ago. Back then, I thought the conclusion was that there is nothing broken about OS X memory management, and that with every 'fix' you come up with, you will just introduce another degenerate corner case. The same holds for any OS, trade-offs are made that may have some negative effect in some cases, to the benefit of the general cases. I don't recognize any of…

I bought a brand new Macbook Pro, 15" with upgraded CPU, etc. with 4GB of RAM last year (2011 model). I didn't want to upgrade to a SSD, I wanted to do that later. After I upgraded to Lion (through the Mac Store) the machine is as speedy as my old Macbook Pro I bought in 2006 and is running Snow Leopard. It's horrible and it's so annoying and I'm pretty much really angry. If Apple will not fix this I will NEVER buy a machine from them again. The only reason I'm buying this overpriced hardware is because I like working on OSX and I don't want to struggle setting up a "Hackingtosh". You probably won't define this as a bug because you're not experiencing it yourself. Maybe it's not an implementation bug but it surely is a bug from the user's perspective.

edit: Just for the record. I'm not doing some kind of heavy processing work. Most of the time I've got one Chrome window open with some tabs, email client, macvim and iTerm2. It's not like I'm doing some heavy work. I'm not even running a VM.

Re: Something is deeply broken in OS X memory management

#104

Post by CGamesPlay »

burgerbrain wrote: post #101
thought_alarm wrote: post #59

Earlier quoted context omitted.

Yet it still swaps to disk ALL THE TIME and a new Terminal.app window can take up to 5 seconds to open. That's not swapping. That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login. Create a .hushlogin file in your home directory to prevent that.

> "That delay is /usr/bin/login searching the system logs so that it can display the date and time of your last login." Am I missing something? `w` on my linux system takes well below one second: [burgerbrain@eeepc] ~ % time w 14:11:18 up 19:41, 6 users, load average: 0.27, 0.10, 0.14 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT REDACTED w 0.01s user 0.05s system 61% cpu 0.096 total I can't imagine why anything like that…

That shows your currently active login, which is of course in main memory. Your most recent login may well no longer be active, and so you have to search the log files.

Re: Something is deeply broken in OS X memory management

#105

Post by illicium »

There's a nice script[1] for tweaking OS X's dynamic pager settings to reduce the system's swappiness that helps a bit. Incidentally, if you have both an SSD and HDD installed, you can use it to move the swapfiles to the HDD to reduce wear.

[1]: http://dropsafe.crypticide.com/article/3848

Re: Something is deeply broken in OS X memory management

#106

Post by mikeash »

microtonal wrote: post #18
mikeash wrote: post #6

No actual data, barely any technical discussion at all, mention of "the garbage collection algorithm" which most likely isn't even being used by most of the apps running, capped by a total cargo-cult solution... and this is #1 on the front page?

The post is (too) light on information, but I don't dismiss it immediately. Perry Metzger was very much involved with NetBSD (maybe he still is) as a developer. He might know a thing or two about memory management ;).

Metzger's post is much more subdued and interesting. IMO it would have made a much better submission. Link for the curious (it's linked from this article):

https://plus.google.com/116685507294337280246/posts/camYp28M...

Re: Something is deeply broken in OS X memory management

#107

Post by RickHull »

munin wrote: post #34
kevingadd wrote: post #30

Earlier quoted context omitted.

Yeah, I'm sure if you just read the Darwin kernel there's a '#define USE_LOTS_OF_MEMORY 1' in there that you can change. The class of problems described in the original post are not the sort of thing you 'just find' by glancing at kernel source code. The problems described sound like they could be an issue of poorly tuned heuristics/thresholds, or necessitate some extra machinery inside the OS X memory manager that i…

yes I know, that's the point of my question ;) everyone is all positive about "open source" until they have to dive into a few millions lines of complicated system-level C code and then...

> yes I know, that's the point of my question ;)

It's a facile point.

> everyone is all positive about "open source" until they have to dive into a few millions lines of complicated system-level C code and then...

Does anyone doubt that 99% of open source users never read a line of the source code which they are using? The point is, they have the opportunity to, and more importantly, the 1% (or whatever) with the skills and resources are able to actually do something about it.

If you don't have the ability to change or examine the source code, then there is little incentive to do any runtime analysis which might illuminate the problem.

Re: Something is deeply broken in OS X memory management

#108

Post by k-mcgrady »

sciurus wrote: post #82
k-mcgrady wrote: post #62

Earlier quoted context omitted.

I'm running on a 160GB HDD (nearly full), 2GB ram, on a 4 year old MacBook. I always have iTunes, Xcode, Safari/Chrome running with multiple tabs, and a mail client (Sparrow). The system originally ran Leopard and I upgraded to SL and then Lion. I wouldn't say it's slow. It's definitely getting slower but that's expected since it's such an old system. Could it be that you're just expecting too much from your computer…

"It's definitely getting slower but that's expected since it's such an old system." I see this sentiment a lot, but I disagree with it. What are "iTunes, Xcode, Safari/Chrome, and a mail client" doing now that they weren't doing four years ago? Is it enough to justify their latest versions feeling less responsive than their versions from four years ago?

I think that as the developers are developing them on better hardware they pay less attention to performance. For example you wouldn't expect the lastest build of iTunes, or OS X to run on hardware for 10 years ago. As system specs improve developers seem to pay less attention to performance & file size. It seems crazy to me that modern web browsers are over 50mb in size. But as hard drive size isn't really a constraint anymore the developers don't optimize that aspect as much.

Re: Something is deeply broken in OS X memory management

#109

Post by whateverer »

mikeash wrote: post #6

No actual data, barely any technical discussion at all, mention of "the garbage collection algorithm" which most likely isn't even being used by most of the apps running, capped by a total cargo-cult solution... and this is #1 on the front page?

Yes, I agree. Way better than most HN articles.

Re: Something is deeply broken in OS X memory management

#110

Post by mikeash »

viraptor wrote: post #100
mikeash wrote: post #6

No actual data, barely any technical discussion at all, mention of "the garbage collection algorithm" which most likely isn't even being used by most of the apps running, capped by a total cargo-cult solution... and this is #1 on the front page?

Garbage collection does not have to mean GC as we know it from Java. It's not that the objects created by the app are garbage collected. The whole memory management in modern systems resembles a GC (it pretty much is a GC). Just instead of managing liveness of the objects, you manage the block mapping. Sometimes you have to write them back to the disk, sometimes you have no memory left and you have to swap them out..…

When he says "the garbage collection algorithm may require that all of a program’s data be in physical RAM before collection can happen," it sure sounds like he's talking about in-app heap collection. Does "may require that all of a program's data be in physical RAM" really apply to the kernel level of memory management? It makes no sense to me when applied that way.
Post reply on HN