Tired of explaining Linux's seemingly obtuse memory usage to confused newbies? • Hacker News Board

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#71

Post by vacri » Wed, Feb 02, 2011, 4:07 AM UTC

jerf wrote: post #54

rufugee wrote: post #45

Earlier quoted context omitted.

Care to explain why this is so?

Stop measuring your swap space in terms of disk space, start imagining it in terms of "amount of time it takes to fill up". At a nice round 20MB/sec write, it would take a solid 400 seconds to use that much swap at full speed. In reality, you can't fill it that fast either, because it's seeking, also yanking other stuff out to run your other processes, and then sticking the stuff back in to run those other processes,…

Not a memory expert myself, but a friend of mine dismisses overly large swap as consuming more RAM to handle it's addressing - that is, the more swap you have, the less of your RAM you can access.

I'm not sure what the consumption rate is, but he won't create general servers with more than 2GB swap (if that), no matter how much RAM the system has.

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#72

Post by Nick_C » Wed, Feb 02, 2011, 5:49 AM UTC

js2 wrote: post #70

spudlyo wrote: post #65

You can't disable disk caching. The only reason anyone ever wants to disable disk caching is because they think it takes memory away from their applications, which it doesn't! Disk cache makes applications load faster and run smoother, but it NEVER EVER takes memory away from them! Therefore, there's absolutely no reason to disable it! Never say never. You can and should disable/purge linux buffer caching in certain…

Funny, I just noticed this in the Mac OS X rsync man page for the very first time: --cache Apple specific option to enable filesystem caching of rsync file i/o Otherwise fcntl(F_NOCACHE) is used to limit memory growth.

There was a patch for all architectures, but it got dropped after 3.07 (current is 3.09). Andrew wrote "I don't feel this patch is general enough nor straight-forward enough to make it into rsync."

See these for more: http://insights.oetiker.ch/linux/fadvise/ https://bugzilla.samba.org/show_bug.cgi?id=7004

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#73

Post by nwmcsween » Wed, Feb 02, 2011, 8:02 AM UTC

requinot59 wrote: post #51

nwmcsween wrote: post #46

Earlier quoted context omitted.

Swap is mainly an emergency backing, i have nagios message me whenever swap hits x amount depending on the server, it's better than oom killer picking off processes.

It might be better to use proper rlimits, and not let a mad process gone wild triggers the OOM. Swap just delays the problem.

The problem with set number limits is that changes in software and hardware require changes in limits - it's easier to just watch swap and get notified.

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#74

Post by yason » Wed, Feb 02, 2011, 8:39 AM UTC

colanderman wrote: post #52

requinot59 wrote: post #38

Earlier quoted context omitted.

> I often just turn swap off entirely. And you're smart to do so. Swap is useless for 99% of end-user systems. ChromeOS doesn't use any swap partition, for instance. As for hibernation, it's not available if you use a crypted swap, which is wise to use (at least on a laptop). As a result, I, too, disabled swap completely.

Swap still provides one important function: it allows large, inactive, long-running processes to be moved from RAM to make room for more caches and buffers. That said, if you never fill your RAM with buffers/cache, then of course swap makes no sense.

On the other hand, one could say that moving a large process from RAM to swap certainly makes it nearly inactive and long-running...

Memory is cheap. I'd rather pay a little more and have the long-running processes stay in memory than worry about my more active processes ending up in swap accidentally. If I wish to reserve 1GB for these long-running processes in RAM instead of swap, I can still win by buying 2GB more RAM.

Besides there's always some disk-backing you can't generally avoid: pages containing read-only executables can be purged from memory when unused and re-read from the original .so or binary when needed. This is something that would never go to swap anyway.