Live data from Hacker News

Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

linuxatemyram.com

51–60 of 74 posts

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#51
post #30

Earlier quoted context omitted.

I often just turn swap off entirely. A normal workload on my 4GB linux machine will frequently fail to fill even 2GB of buffers . At the moment my machine has still managed to fail to fill 800MB of my RAM with anything and I'm running two virtual machines and doing my usual work load. I do have a swap partition but only so I can hibernate.

Swap is mainly an emergency backing, i have nagios message me whenever swap hits x amount depending on the server, it's better than oom killer picking off processes.

It might be better to use proper rlimits, and not let a mad process gone wild triggers the OOM. Swap just delays the problem.

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#52
post #30

Earlier quoted context omitted.

I often just turn swap off entirely. A normal workload on my 4GB linux machine will frequently fail to fill even 2GB of buffers . At the moment my machine has still managed to fail to fill 800MB of my RAM with anything and I'm running two virtual machines and doing my usual work load. I do have a swap partition but only so I can hibernate.

> I often just turn swap off entirely. And you're smart to do so. Swap is useless for 99% of end-user systems. ChromeOS doesn't use any swap partition, for instance. As for hibernation, it's not available if you use a crypted swap, which is wise to use (at least on a laptop). As a result, I, too, disabled swap completely.

Swap still provides one important function: it allows large, inactive, long-running processes to be moved from RAM to make room for more caches and buffers.

That said, if you never fill your RAM with buffers/cache, then of course swap makes no sense.

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#53
post #42

Earlier quoted context omitted.

> I often just turn swap off entirely. And you're smart to do so. Swap is useless for 99% of end-user systems. ChromeOS doesn't use any swap partition, for instance. As for hibernation, it's not available if you use a crypted swap, which is wise to use (at least on a laptop). As a result, I, too, disabled swap completely.

"And you're smart to do so." Finally, that's the first comment I get, instead of someone extolling the virtues of swap and babbling warmed-over 1990s rules of thumb about "twice the RAM is the recommended size of your swap file" as if I'm going to wait for even 200MB of swap to fill up before flipping out and killing the offending process, let alone 8GB.

>Finally, that's the first comment I get, instead of someone extolling the virtues of swap and babbling warmed-over 1990s rules of thumb about "twice the RAM is the recommended size of your swap file" as if I'm going to wait for even 200MB of swap to fill up before flipping out and killing the offending process, let alone 8GB.

Those people have doubtless misunderstood the point of swap. You should have a swapfile/partition because it allows allocated but currently unused memory (from an application which keeps data hanging around which is not needed for most of it's working life, or an application which simply leaks) to be dumped to long term storage, thus freeing memory for its real use: page cache. Sweet, sweet page cache.

I'm always happy to see a few tens, even a couple of hundred MB of swap in use, because it means that some application had some unused data hanging around for so long that to leave it there would mean my machine having to read from disk more frequently, which would be Bad.

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#54
post #45

Earlier quoted context omitted.

> I often just turn swap off entirely. And you're smart to do so. Swap is useless for 99% of end-user systems. ChromeOS doesn't use any swap partition, for instance. As for hibernation, it's not available if you use a crypted swap, which is wise to use (at least on a laptop). As a result, I, too, disabled swap completely.

Care to explain why this is so?

Stop measuring your swap space in terms of disk space, start imagining it in terms of "amount of time it takes to fill up". At a nice round 20MB/sec write, it would take a solid 400 seconds to use that much swap at full speed. In reality, you can't fill it that fast either, because it's seeking, also yanking other stuff out to run your other processes, and then sticking the stuff back in to run those other processes, so it's really a mess; the real amount of time it could take just to make initial real use of 8GB of swap could be days, no joke. (In other words, I don't care if you have a 80MB/s drive, it really doesn't matter much, plus swap usage tends to be full of seeks anyhow so it's not like you're going to get 20MB/s either.) Given that a modern system is, relatively speaking, brought to a near-complete halt by being in swap, what possible process are you going to run where you are willing to put up with your system being brought to a halt for even tens of minutes at a time, let alone the hours or days it'll take to fully utilize 8GB?

Clearly, you don't need and basically can't use an 8GB swap partition. So, how long are you willing to put up with? That will vary, but let's say 10 seconds before you're "flipping out and killing processes". That's a 200MB swap partition. But... that's only 5% of the size of your RAM! If that's the difference between a process completing or not, you've probably already lost. Or you should just kill Firefox.

On a 4GB system, the most likely reason a process is pushing you into swap is that it is in an infinite allocation loop, and all having 4GB of swap does is make your system crawl that much more before the process dies.

This is a result of RAM sizes increasing far faster than hard drive write speed has. When I had 32MB of RAM, it made sense to have some swap. I could swap out, say, 16MB of unused executable pages (bits of windows, bits of drivers I'm not using, bits of the massive Office suite I'm not using, etc) and get that much more working set, and this could happen in a reasonable time; the system choked for a couple of seconds but recovered in a stable manner. Now swapping out 16MB of executable is a joke. SSDs may change the balance, but these balances have been out of whack for a long time, I rather suspect that even with an SSD it won't be worth swap. Especially since by the time SSDs are truly common 8GBs of RAM may be entry-level because, well, why not? (Poking at Best Buy really quickly, at the $500 line you get 4GB for laptops, a little over $300 for desktops, coming down fast. I'm not sure they have anything less than 2GB now and even that is really into "don't use swap" for the average user.)

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#55
post #35
post #33

It looks like gnome-system-monitor reports the "correct" amount of free RAM. I'd imagine most newbies would use that rather than popping open a terminal and typing "free," right?

You'd be surprised. Some users who use Linux consider themselves to be pretty adept, and I can easily see this problem being an issue for people who perceive themselves to be clever.

[deleted]

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#56
post #35
post #33

It looks like gnome-system-monitor reports the "correct" amount of free RAM. I'd imagine most newbies would use that rather than popping open a terminal and typing "free," right?

You'd be surprised. Some users who use Linux consider themselves to be pretty adept, and I can easily see this problem being an issue for people who perceive themselves to be clever.

I wouldn't say that only "clever" users would rather use `free`, since most Linux installations don't have GNOME or X11 installed to begin with.

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#57
post #20
post #8

Wouldn't it make more sense to just include the RAM used for disk-caching when calculating free memory? After all, the whole reason it's being used for disk-caching in the first place is that it's free! The Linuxoids that know what this even means can always explicitly look for the line that says how much memory is being used for caching...

It costs time to zero pages; disk cache pages are not zeroed. Not having a source of zero pages will add latency to large memory allocation requests. (The pages must be zeroed to avoid the security problem of snooping on data you might not have access to.)

Exactly the kind of thing users not familiar with the way Linux handles memory DON'T consider when they check how much free memory they have. If you're the sort of person concerned with the overhead of zeroing pages then you can look at more detailed memory-usage info to find the details you're looking for.

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#58

This is misleading, as the kernel will often swap out processes rather than shrink the disk cache. Its tendency to do this is controlled by the vm.swappiness parameter, which by default heavily favors swapping: http://kerneltrap.org/node/3000 Decreasing this value will favor reducing the cache size and will tend to increase responsiveness. You can set it in the /etc/sysctl.conf file. Low values (≤ 5) are especially g…

Is there a way to decrease the swappiness of OS X?

Re: Tired of explaining Linux's seemingly obtuse memory usage to confused newbies?

#59
Hmm, this situation is like owning an electric vehicle. The manufacturers are passionate about technology, and they invented a flywheel that can store massive amounts of energy, giving your vehicle incredible transient acceleration response. They ship many vehicles with this new feature, but unfortunately, they forgot to account for what this would do to the battery charge display after starting. As a result, their reps keep on having to explain that the vehicles are not rapidly leaking charge shortly after starting -- instead that energy is being placed in the flywheel where it can greatly increase the car's responsiveness.

Whoops!

The difference between the scenario above and Linux's disk cache? Techies blame the former on the car company but the latter on the end-users.

Post reply on HN