Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In defence of swap: common misconceptions

chrisdown.name

Post reply on HN
Print view View original

121–130 of 151 posts

Re: In defence of swap: common misconceptions

#121

Post by mikekchar »

quotemstr wrote: post #62

Very few people on this thread read and understood the article. The point isn't working with data sets larger than RAM. The point is making better use of the RAM you do have by taking pages you'll almost never touch and spilling them to disk so that there's more room in RAM for pages you will touch. Banning swap is like making self-storage companies illegal and forcing everyone to hold all possessions in their homes.…

Feel free to explain it to me.

" Under no/low memory contention

[...]

Without swap: We cannot swap out rarely-used anonymous memory, as it’s locked in memory. While this may not immediately present as a problem, on some workloads this may represent a non-trivial drop in performance due to stale, anonymous pages taking space away from more important use."

Now imagine that I have no memory contention. In other words I've got 8 Gigs of memory and I have never run out of memory. The OOM killer has never run. I've never even come close. How exactly is this representing a non-trivial drop in performance?

To be fair, if I put some of my long running processes into swap, I could cache more files, but I really don't see how this represents a statistically significant improvement. I honestly can't think of anything else.

If you sometimes run out of memory (or even get close), then you should have some swap. This seems fairly obvious to me. Relying on the OOM killer to "clean things up" is pretty dubious. But was there every any serious argument to do this? I've literally never heard of that before.

I'd be very happy to hear something enlightening about this, but I didn't see anything in the article (perhaps I missed it).

Re: In defence of swap: common misconceptions

#122

Post by cdown »

viraptor wrote: post #120

I feel like the real metric is like to see from swap usage is: how much time did I spend waiting to swap-in a page, how much extra cache pages it gave me, and what's the cache hit ratio. If the big purpose is to allow those extra few pages to be available, then either it's worth doing or not - there should be an objective way to look at this. Unfortunately only the second and third part is easily available. The first…

> how much time did I spend waiting to swap-in a page

You can do this with eBPF/BCC by using funclatency (https://github.com/iovisor/bcc/blob/master/tools/funclatency...) to trace swap related kernel calls. It depends on exactly what you want, but take a look at mm/swap.c and you'll probably find a function which results in the semantics you want.

Re: In defence of swap: common misconceptions

#123

Post by cdown »

mikekchar wrote: post #121
quotemstr wrote: post #62

Very few people on this thread read and understood the article. The point isn't working with data sets larger than RAM. The point is making better use of the RAM you do have by taking pages you'll almost never touch and spilling them to disk so that there's more room in RAM for pages you will touch. Banning swap is like making self-storage companies illegal and forcing everyone to hold all possessions in their homes.…

Feel free to explain it to me. " Under no/low memory contention [...] Without swap: We cannot swap out rarely-used anonymous memory, as it’s locked in memory. While this may not immediately present as a problem, on some workloads this may represent a non-trivial drop in performance due to stale, anonymous pages taking space away from more important use." Now imagine that I have no memory contention. In other words I'…

I mean, it really depends on your application how non-trivial the performance improvement will be, but this statement isn't theoretical -- memory bound systems are a major case where being able to transfer out cold pages to swap can be a big win. In such systems, having optimal efficiency is all about having this balancing act between overall memory use without causing excessive memory pressure -- swap can not only reduce pressure, but often is able to allow reclaiming enough pages that we can increase application performance when memory is the constraining factor.

Re: In defence of swap: common misconceptions

#124

Post by jcelerier »

mikekchar wrote: post #114
crististm wrote: post #107

Earlier quoted context omitted.

In 2005 I was able to run Linux on 512MB RAM _without_ swap (on purpose - every day) without issues. Today it will bark at me on 8GB of RAM for not having swap enabled.

I'm running on an 8GB Linux box without swap and never even come close to running out of memory. If I don't have any VMs running, then it's pretty unusual for me to use much more than 1-2 gigs. It's interesting... one of my colleagues has serious problems with performance because he keeps running out of memory -- and I don't think he's doing anything unusual. I think there is something wrong with some of the major di…

> I'm running on an 8GB Linux box without swap and never even come close to running out of memory. If I don't have any VMs running, then it's pretty unusual for me to use much more than 1-2 gigs. It's interesting... one of my colleagues has serious problems with performance because he keeps running out of memory -- and I don't think he's doing anything unusual.

64gb here and 40gb used. Firefox alone uses 2 gigs with a mere ~30 tabs.

Re: In defence of swap: common misconceptions

#125

Post by Neil44 »

The system should be tuned so that under excessive pressure from requests it starts to turn requests away before running out of RAM. Having a small ammount of swap will let you get closer to the limit of RAM use without risking OOM killing something and getting the system into an undefined state. Also you can swap a fair bit of the system you don't use and leave it there, giving you more RAM to use for processing requests.

Re: In defence of swap: common misconceptions

#126

Post by gizzlon »

jcelerier wrote: post #124
mikekchar wrote: post #114

Earlier quoted context omitted.

I'm running on an 8GB Linux box without swap and never even come close to running out of memory. If I don't have any VMs running, then it's pretty unusual for me to use much more than 1-2 gigs. It's interesting... one of my colleagues has serious problems with performance because he keeps running out of memory -- and I don't think he's doing anything unusual. I think there is something wrong with some of the major di…

> I'm running on an 8GB Linux box without swap and never even come close to running out of memory. If I don't have any VMs running, then it's pretty unusual for me to use much more than 1-2 gigs. It's interesting... one of my colleagues has serious problems with performance because he keeps running out of memory -- and I don't think he's doing anything unusual. 64gb here and 40gb used. Firefox alone uses 2 gigs with…

I run KDE, Firefox, Slack (browser based app), Chromium, VS Code (browser based app) (plus Gvim, shell etc) on Arch and it's currently steady at 3.9 GB (of 16)

Edit: Also be sure it's actually used: https://www.linuxatemyram.com/

Re: In defence of swap: common misconceptions

#127

Post by MaxBarraclough »

quotemstr wrote: post #66
_9jgl wrote: post #65

Earlier quoted context omitted.

The issue is that the current OOM killer doesn't support this usage at all. To extend the analogy: what do you do if grandma comes and fills your house with stuff? You need space to work, so you go and drop it off at the self storage place, but what if she just keeps filling your house up? The OOM killer will do absolutely nothing until both your house and the whole self storage place are totally full. By that point,…

Well, it doesn't help that when grandma calls and asks whether you have room for more stuff, the Linux kernel responds on your behalf, "Yes, of course I have room. I live in a TARDIS." And you then do all driving to the self-storage facility to maintain the illusion as long as you can. I really don't like overcommit. Anyway, I agree with you that this behavior is annoying, but I think it ought to be possible to fix i…

The analogy is now getting in the way, rather than helping to clarify.

Re: In defence of swap: common misconceptions

#128

Post by clarry »

Anderkent wrote: post #11
kstenerud wrote: post #4

So, in other words, if you have enough memory for your workload that you won't run out, there's no benefit to having swap space (i.e. you've wasted money on memory you don't need). But if you DO have swap space, there won't be a performance hit (at least not under Linux) because it will only swap out some rarely used pages and then sit there doing nothing. So, in the general case, it's better to have it and not need…

> So, in other words, if you have enough memory for your workload that you won't run out, there's no benefit to having swap space (i.e. you've wasted money on memory you don't need). No, that's the opposite. If you have enough memory for your workload that you won't run out, swapping lets you use more memory for disk cache (instead of keeping unaccessed anonymous pages in real ram). Unless by "won't run out" you mean…

> If you have enough memory for your workload that you won't run out, swapping lets you use more memory for disk cache

Except that it doesn't happen in practice, on my systems anyway. If you have plenty of memory, you can keep all your programs in it and as much as the system wants to cache and still not run out.

The theory says that swap effectively buys you some memory to spend on more important things (than what the system chooses to page out). So does buying more memory.

> Unless by "won't run out" you mean "never have to throw away a disk cache page", which seems very unrealistic.

I have an instance of top running on my desktops & laptops all the time. I never see cache using up all of the memory.

                  total        used        free      shared  buff/cache   available
    Mem:           7848        1523        5827          53         497        6033

Re: In defence of swap: common misconceptions

#129

Post by tmyklebu »

dozzie wrote: post #108
tmyklebu wrote: post #91

Earlier quoted context omitted.

> Very few people on this thread read and understood the article. Hmm. I read the article and I think I understood it. However, in my experience, you run out of RAM if and only if your working set is too big. In my experience, all involved find it desirable to reduce the size of the working set as quickly as possible. Your experience seems to differ. > The point isn't working with data sets larger than RAM. The point…

> (1) The kernel can predict which pages the user will "almost never touch." > I disagree with (1). [...] The kernel does not have sufficiently detailed knowledge of the process's purpose or access patterns. You're in for quite a surprise, particularly on desktop. I have a number of processes with some pages swapped out, and I see no impact on interacting with the said processes. Firefox, gDesklets, a volume changer,…

> You're in for quite a surprise, particularly on desktop. I have a number of processes with some pages swapped out, and I see no impact on interacting with the said processes. Firefox, gDesklets, a volume changer, and several instances of rxvt are among them.

Is an appreciable amount of RAM freed up here? I was under the impression that Firefox churned through whatever it allocated (garbage-collected Javascript VM) and rxvt had a very small footprint, most of which is code shared among all of your rxvt instances.

>> I also disagree with (2). The consequences of getting these predictions wrong seem to be very bad. > > Only in the case of repeated mispredictions, which only happens if you really have low RAM and are on a good way to invoke OOM killer anyway.

Even if things were as rosy as you suggest, isn't that my point? Better that the OOM killer cleans something up than I bounce the machine and clean everything up. That said, the OOM killer won't necessarily run anytime soon:

I just spun up a VM with 1GB of memory and 1GB of swap. 'time ssh guest echo hello' from the host usually takes anywhere from 140ms to 1.2s. I wrote a C program that allocates a gigabyte (in two 512MB pieces) and churns it through swap by writing to random bytes. 'time ssh guest echo hello' now takes 4-8 seconds. The oom killer didn't run once in the five minutes I ran the swap-churning process. Setting /proc/sys/vm/swappiness to 0 didn't change the symptoms; 'time ssh guest echo hello' still takes 4-8 seconds. This is on Linux 4.9.65.

If I crank the number of churning threads up from 1 to 8, the one 'time ssh guest echo hello' I tried took 33 seconds. I am not patient enough to see what happens with 64 churning threads, which is entirely reasonable, but I would expect the latency involved in rescuing the machine to cause any reasonable administrator to simply bounce it.

In this workload, the kernel is consistently failing to predict which pages are unimportant; the mispredictions are expensive; the RAM saved by swapping out bash, sshd, and killall (or whatever) is negligible; and the important work of allowing remote login to diagnose and clean up the mess is held up unconscionably long to make room for what, in practical instances, is a user error.

I did a 'swapoff -a' and ran the same C program and it gets killed almost immediately.

Re: In defence of swap: common misconceptions

#130

Post by clarry »

Anderkent wrote: post #7
loeg wrote: post #5

All of this boils down to "because buying more disk is cheaper than buying more RAM" and "avoid memory contention." The author discusses the situation as if the quantity of RAM is fixed and swap can be added (or not). But that isn't the only possibility — you can also add more RAM (it's just expensive). For the same number of GB of RAM+swap vs just RAM, there is no reason to prefer the option with swap.

Sure, but why wouldn't you run some swap with your much bigger ram anyway? In the end the core idea is: sometimes you have anonymous memory that is accessed so rarely that you'd rather have an extra disk cache page. If you assume that the kernel is not paging out memory that you actually use when not under pressure, swap doesn't hurt you.

> Sure, but why wouldn't you run some swap with your much bigger ram anyway?

If you don't need it, you don't need it. The other question is: how much swap exactly should I have? And why wouldn't I just add that much RAM instead?

> In the end the core idea is: sometimes you have anonymous memory that is accessed so rarely that you'd rather have an extra disk cache page.

That's the theory. In practice I always have more than enough RAM for all the cached pages the system wants to cache. On my laptop right now (booted today), I have 500MB of cached pages and 5.8 gigabytes of free memory. On my server (booted 499 days ago) I have 700MB of cached pages and 6 gigabytes of free memory.

If I were running out of memory [be it for cache or applications], I'd prefer to have more RAM than add swap. Yes, I keep calling it emergency memory.

> If you assume that the kernel is not paging out memory that you actually use when not under pressure, swap doesn't hurt you.

1) Bad assumption 2) it doesn't help you either, so why bother? Actually I might have a use for that disk space. In that case the swap just hurts.

Post reply on HN