Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Help, Linux ate my RAM

linuxatemyram.com

Post reply on HN
Print view View original

61–70 of 106 posts

Re: Help, Linux ate my RAM

#61

Post by henrikschroder »

ivan78 wrote: post #50
ivan78 wrote: post #30

Earlier quoted context omitted.

Well, if you don't understand it, you must read some basic book about modern computer architecture. Short answer: It is possible to reserve memory address space without assigning actual physical memory. As your program runs it can dynamically assign physical memory pages to its address space when it needs it. Again - you must read a book if you want to understand it.

Why downvoting? It may be not the best explanation, but IMO the closest to the way a "developer" should understand it. On Linux a virtual memory is committed with mmap call. Check this for details: http://stackoverflow.com/questions/2782628/any-way-to-reserv... This is basically how shared libraries and other shared objects are attached to your process.

> Why downvoting?

> you must read a book if you want to understand it.

"Go read a book" is not a very constructive or helpful answer to a question that - as it turns out - could be answered with a brief comment.

Re: Help, Linux ate my RAM

#62

Post by covati »

ajross wrote: post #25
krobertson wrote: post #19

Earlier quoted context omitted.

I think this is a pretty common misconception overall. Working at startups, often find devs with multiple hats sometimes doing ops tasks. Seen many who hop on a system trying to diagnose some issue, fire up top and proclaim "OMG, the problem is we're running out of memory!" 2nd is explaining virtual/resident set size.

That's depressing. Developers are the ones who have to understand this. In IT it's common to find people (even "DBAs") who have made a career of following procedures someone else wrote.

I don't think that is necessarily true. Most modern programming languages, as well as the relative cheap nature of RAM, have rendered a true understanding of the inner workings of RAM less important.

Would they be better at their job if they did? Probably. But do they have to? I'm not so sure anymore.

Re: Help, Linux ate my RAM

#63

Post by ivan78 »

seclorum wrote: post #52
illumin8 wrote: post #11

I've tried to educate Oracle DBAs on why top is wrong and their memory really isn't being used. It's painful and they often refuse to believe that I know what I'm talking about, and that they should use the free -m command to see what memory is actually available for use. Is there any particular reason why Oracle DBAs are less likely to believe this? Perhaps it's because most of them grew up in legacy UNIX environmen…

Teach them to interpret and understand cat /proc/meminfo .. one of the most interesting things you can do with this is pipe it into gnuplot and watch it over time as things happen. Try it sometimes .. you might get through to one or two.

If you want to be more enterprisey, you can pipe it to SNMP counter and then draw a graph with your Network Monitoring System. It is much more convenient if you have more that few servers.

Re: Help, Linux ate my RAM

#64

Post by scott_s »

ArbitraryLimits wrote: post #58
scott_s wrote: post #48

Earlier quoted context omitted.

Modern web browser are complicated pieces of software that are also the nexus for most people's interactions with their computer. tiles' point is correct: Firefox will tend to cache things much like the OS itself. We should expect it to use a lot of memory. If your system performance has not suffered, there is no problem.

I'm sorry, but this reply doesn't make any sense to me. If applications have to cache something, they should use the file system, which would not affect their resident memory at all. Maybe I'm just old school, but where I come from operating systems handle the memory hierarchy, not applications.

They cache it in memory to avoid latency. Writing to and reading from disk increases latency. Even crossing the kernel boundary will increase latency.

Modern web browsers have architectures similar to OSes at this point - because they have requirements similar to OSes. I think it's natural that they will take on some of the same responsibilities.

Re: Help, Linux ate my RAM

#65

Post by ajross »

covati wrote: post #62
ajross wrote: post #25

Earlier quoted context omitted.

That's depressing. Developers are the ones who have to understand this. In IT it's common to find people (even "DBAs") who have made a career of following procedures someone else wrote.

I don't think that is necessarily true. Most modern programming languages, as well as the relative cheap nature of RAM, have rendered a true understanding of the inner workings of RAM less important. Would they be better at their job if they did? Probably. But do they have to? I'm not so sure anymore.

Sure, but we're not talking about the "narrow, career day-job programmers" here. We're talking about the "wear multiple hats and diagnose problems wherever they lie" set. Those people shouldn't be diagnosing problems with their own ignorance. At the very least they should understand what they don't know.

Re: Help, Linux ate my RAM

#66

Post by ivan78 »

ivan78 wrote: post #50
ivan78 wrote: post #30

Earlier quoted context omitted.

Well, if you don't understand it, you must read some basic book about modern computer architecture. Short answer: It is possible to reserve memory address space without assigning actual physical memory. As your program runs it can dynamically assign physical memory pages to its address space when it needs it. Again - you must read a book if you want to understand it.

Why downvoting? It may be not the best explanation, but IMO the closest to the way a "developer" should understand it. On Linux a virtual memory is committed with mmap call. Check this for details: http://stackoverflow.com/questions/2782628/any-way-to-reserv... This is basically how shared libraries and other shared objects are attached to your process.

to henrikschroder: I expected that this reaction was caused by mentioning a book. It is my deepest belief that questions like memory management can not be answered with brief comment. You must read at least a basic book if you want to understand it in depth.

Re: Help, Linux ate my RAM

#67

Post by krobertson »

ajross wrote: post #65
covati wrote: post #62

Earlier quoted context omitted.

I don't think that is necessarily true. Most modern programming languages, as well as the relative cheap nature of RAM, have rendered a true understanding of the inner workings of RAM less important. Would they be better at their job if they did? Probably. But do they have to? I'm not so sure anymore.

Sure, but we're not talking about the "narrow, career day-job programmers" here. We're talking about the "wear multiple hats and diagnose problems wherever they lie" set. Those people shouldn't be diagnosing problems with their own ignorance. At the very least they should understand what they don't know.

Agreed. It can sometimes be a side effect of small companies, or young developers living the startup lifestyle. Not saying its bad or the root of the problem, just an observation.

More so, I think it comes more with modern languages that don't force you to manage memory anymore. When you don't need to alloc/free everything you're using, it is easy to get lazy. Factor in never generation who've never worked outside a managed memory language. There are minute details about retaining references, having multiple copies of the same data, or other leaks (file descriptors?).

Re: Help, Linux ate my RAM

#68

Post by dfc »

click170 wrote: post #4

It's cute that this pops up every few years, and I think it points to a steady (if slow) attraction of new users to Linux.

It makes me feel old. As soon as I saw the headline I knew what the story was going to be. Another fresh wave of young faces learning about memory...I just wish they'd stay off the lawn.

Re: Help, Linux ate my RAM

#69

Post by jws »

CJefferson wrote: post #53
jws wrote: post #6

Earlier quoted context omitted.

Programmers need to remember that users are like this: (from an Apple discussion thread) … [periodically]'installd' begins using 100% of all 4 CPU cores, my fan goes full speed and the whole computer gets very hot. Seems to happen before Software Update checks for updates. I usually go to the terminal and kill the 'installd' process, which reduces the fan speed and heat to normal within a minute. I wonder if this guy…

I have seen installd peg my CPU to 100% and have killed it. What would you suggest I do? One of the most annoying features of modern OSes is when some system process just decides to start going wild, eating memory and CPU. Often I find reinstalling is the only way to fix such things.

Let it finish. It has some thinking to do, maybe crypto checksums? Maybe just a really inefficient corner case on an algorithm. There's nothing wrong with letting your CPU work. if running at 100% makes your machine flakey, it is broken.

If your foreground performance is being impacted too severely (and I haven't seen this from installd, I just noticed and researched installd while removing Mac Keeper (malware) from my wife's laptop) then reboot. It's extreme, but it has the best chance of getting your processes shut down cleanly as opposed to a kill where you could nail a process in the middle of a state that really does not want to persist. Programmers are a lazy sort. They won't consider the effect of termination at each point in their program. You are hunting for bugs using your live system as bait if you kill a program.

Re: Help, Linux ate my RAM

#70

Post by ootachi »

ineedtosleep wrote: post #49
jff wrote: post #17

Earlier quoted context omitted.

Yep, firefox is currently 555 MB resident and using 1.5 GIGABYTES of virtual memory space. Goddamn, firefox, you are a pig . Saddest thing? I've got gmail, github, and maybe 10 static pages open. Linux just looks like it ate your RAM. Firefox straight up does eat it.

Why is Firefox always mentioned in this context? Sure they have had a history of it, but lately they've been fine and if one compares the combined spawned processes of Chrome, Chrome typically has more memory consumption.

HN's pro-Google bias.
Post reply on HN