Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Help, Linux ate my RAM

linuxatemyram.com

Post reply on HN
Print view View original

31–40 of 106 posts

Re: Help, Linux ate my RAM

#31

Post by illumin8 »

ivan78 wrote: post #18
illumin8 wrote: post #11

I've tried to educate Oracle DBAs on why top is wrong and their memory really isn't being used. It's painful and they often refuse to believe that I know what I'm talking about, and that they should use the free -m command to see what memory is actually available for use. Is there any particular reason why Oracle DBAs are less likely to believe this? Perhaps it's because most of them grew up in legacy UNIX environmen…

As a longtime Linux user/admin and beginning Oracle DBA I now know that all memory should be occupied by Oracle, not by OS. :-) Serious mode on: I'm sure it's a big problem to be narrow expert. They can be brilliant specialist in their field, but one step aside and they are absolutely helpless.

Thanks for the great comment. I really wish more people like yourself at least understood Linux memory management. I think it will truly help you to be an exceptional DBA. I can also understand how an expert DBA might not know or care too much about the OS underneath his software, although I would argue that if you understand the OS fundamentals, you will be that much better at whatever specialty you have.

Re: Help, Linux ate my RAM

#32

Post by Steve_Baker »

francoisdevlin wrote: post #24
krobertson wrote: post #19

Earlier quoted context omitted.

I think this is a pretty common misconception overall. Working at startups, often find devs with multiple hats sometimes doing ops tasks. Seen many who hop on a system trying to diagnose some issue, fire up top and proclaim "OMG, the problem is we're running out of memory!" 2nd is explaining virtual/resident set size.

Okay, I'm one of those devs. What is this virtual/resident set size thing you're talking about? EDIT: Thank you for all the helpful responses!

The virtual size of a program includes all shared objects (shared libraries, shared copy on write memory pages, the read only executable pages and other shared memory) that a process uses in addition to the memory that only that particular process is using (its resident set size.) Shared objects like shared libraries can be mapped into memory by multiple processes and thus don't use any additional physical memory for each additional process that uses it.

The RSS thus usually indicates the amount of heap and stack a process is using that is unique to it.

Re: Help, Linux ate my RAM

#33

Post by tiles »

jff wrote: post #17
mwexler wrote: post #9

firefoxatemyram.com is still available. Perhaps we can put a site up there as well.

Yep, firefox is currently 555 MB resident and using 1.5 GIGABYTES of virtual memory space. Goddamn, firefox, you are a pig . Saddest thing? I've got gmail, github, and maybe 10 static pages open. Linux just looks like it ate your RAM. Firefox straight up does eat it.

Am I missing something? Does Firefox not use unused RAM for cache in a similar manner as Linux uses unused RAM?

Re: Help, Linux ate my RAM

#34

Post by illumin8 »

francoisdevlin wrote: post #24
krobertson wrote: post #19

Earlier quoted context omitted.

I think this is a pretty common misconception overall. Working at startups, often find devs with multiple hats sometimes doing ops tasks. Seen many who hop on a system trying to diagnose some issue, fire up top and proclaim "OMG, the problem is we're running out of memory!" 2nd is explaining virtual/resident set size.

Okay, I'm one of those devs. What is this virtual/resident set size thing you're talking about? EDIT: Thank you for all the helpful responses!

Short and overly simple answer: If a program allocates 2GB of RAM, it might not need to use it all right away. The kernel pretends like it just gave the program 2GB of RAM, but it doesn't really need to flush all those unused pages of cache yet. It can still use them for other things like disk cache, and when the program tries to access those pages then the kernel will dynamically page them in and out as needed. This explains why a program might have 2GB virtual but only 128MB is resident in actual physical memory pages.

This is overly simple and there are a lot of nuances such as shared memory segments, etc.

Re: Help, Linux ate my RAM

#35

Post by jws »

nandemo wrote: post #16
jws wrote: post #6

Earlier quoted context omitted.

Programmers need to remember that users are like this: (from an Apple discussion thread) … [periodically]'installd' begins using 100% of all 4 CPU cores, my fan goes full speed and the whole computer gets very hot. Seems to happen before Software Update checks for updates. I usually go to the terminal and kill the 'installd' process, which reduces the fan speed and heat to normal within a minute. I wonder if this guy…

I'm a programmer but not a Mac user. I fail to see what is wrong with the bug report above. Care to explain?

"installd" is a completely undocumented (outside of Apple) daemon that is somehow involved in installing your software (it runs while installing App store programs) and checking to see what updates you might need. It has no user visible existence. No one would ever know about it unless they ran "ps" or something similar. They will find no hint at what it does, or if killing it is safe.

Re: Help, Linux ate my RAM

#37

Post by paxswill »

nandemo wrote: post #16
jws wrote: post #6

Earlier quoted context omitted.

Programmers need to remember that users are like this: (from an Apple discussion thread) … [periodically]'installd' begins using 100% of all 4 CPU cores, my fan goes full speed and the whole computer gets very hot. Seems to happen before Software Update checks for updates. I usually go to the terminal and kill the 'installd' process, which reduces the fan speed and heat to normal within a minute. I wonder if this guy…

I'm a programmer but not a Mac user. I fail to see what is wrong with the bug report above. Care to explain?

installd is part of framework that handles installed packages. It's fired up as part of the normal software update process (although it really shouldn't be taking a core for itself). Killing it would probably abort the software update process.

Re: Help, Linux ate my RAM

#38

Post by Legion »

krobertson wrote: post #19
illumin8 wrote: post #11

I've tried to educate Oracle DBAs on why top is wrong and their memory really isn't being used. It's painful and they often refuse to believe that I know what I'm talking about, and that they should use the free -m command to see what memory is actually available for use. Is there any particular reason why Oracle DBAs are less likely to believe this? Perhaps it's because most of them grew up in legacy UNIX environmen…

I think this is a pretty common misconception overall. Working at startups, often find devs with multiple hats sometimes doing ops tasks. Seen many who hop on a system trying to diagnose some issue, fire up top and proclaim "OMG, the problem is we're running out of memory!" 2nd is explaining virtual/resident set size.

2 minute fix: get people to use htop, not top.

Re: Help, Linux ate my RAM

#39

Post by gaius »

illumin8 wrote: post #11

I've tried to educate Oracle DBAs on why top is wrong and their memory really isn't being used. It's painful and they often refuse to believe that I know what I'm talking about, and that they should use the free -m command to see what memory is actually available for use. Is there any particular reason why Oracle DBAs are less likely to believe this? Perhaps it's because most of them grew up in legacy UNIX environmen…

I've been an Oracle DBA for 15 years, and no-one uses top for that, as it's well known not to account in any sort of meaningful way for the way Oracle uses shared memory. The only thing it's useful for is seeing which of the sysadmin's Perl scripts is chewing the CPU.

Re: Help, Linux ate my RAM

#40

Post by maushu »

francoisdevlin wrote: post #24
krobertson wrote: post #19

Earlier quoted context omitted.

I think this is a pretty common misconception overall. Working at startups, often find devs with multiple hats sometimes doing ops tasks. Seen many who hop on a system trying to diagnose some issue, fire up top and proclaim "OMG, the problem is we're running out of memory!" 2nd is explaining virtual/resident set size.

Okay, I'm one of those devs. What is this virtual/resident set size thing you're talking about? EDIT: Thank you for all the helpful responses!

I'm a developer too and this is what I understand (might be wrong):

- Virtual memory is basically "abstract memory" that is linked (mapped) to RAM or HDD, this means that by accessing this "memory" you might actually be accessing the HDD and, because of this, larger than physical ram.

- Virtual set size is the allocated virtual memory (the above) to the process.

- Resident set size is the allocated physical (RAM) memory to the process.

- Shared memory is memory that is shared on multiple processes, meaning that if you have 10 processes using 10mb of resident memory each and have 2mb of shared memory each, the total of resident memory used is not 100mb but 82mb.

Post reply on HN