Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Origin of the rule that swap size should be 2x of the physical memory

retrocomputing.stackexchange.com

Post reply on HN
Print view View original

41–50 of 78 posts

Re: Origin of the rule that swap size should be 2x of the physical memory

#41

Post by rmunn »

nwallin wrote: post #33
pezezin wrote: post #5

Earlier quoted context omitted.

Zero. My office workstation has 48 GB of RAM, my home computer has 64 (I went a bit overboard). I have very bad memories of swap thrashing and the computer becoming totally unresponsive until I forced a reset; if I manage to fill up so much RAM, I very much prefer the offending process to die instead of killing the whole computer.

I have 64GB of RAM and 16GB of swap. Swap is small enough it can't get really out of hand. I have memories from like 20 years ago that even when I had plenty of RAM, and plenty of it was free, I would get random OOM killer events relatively regularly. Adding just a tiny bit of swap made that stop happening. I'm like 90% sure at this point it's just a stupid superstition I carry. But I'm not gonna stop doing it even t…

Same here, though I settled on 32GB of swap because I have a 4TB SSD (caught a good sale on a Samsung EVO SSD at Newegg). But whenever I run `top`, I constantly see:

    MiB Swap:  32768.0 total,  32768.0 free,      0.0 used.
I could safely get away with 4GB of swap, and see no difference.

Re: Origin of the rule that swap size should be 2x of the physical memory

#42

Post by bandrami »

bandrami wrote: post #13

I've had arguments with people about this for 20 years now and the most compelling case I heard involved the price of storage vs the price of RAM in the mid to late 1990s, and that this 2x represented an optimal use of money in designing a system at that point in time.

Also I still put my swap partition at the end of the drive out of habit because you used to want to do that for performance reasons when it was stored on a spinning magnet.

Re: Origin of the rule that swap size should be 2x of the physical memory

#43

Post by anyfoo »

fluoridation wrote: post #34
anyfoo wrote: post #28

Earlier quoted context omitted.

I don't think that's correct. Having swap still allows you to page out rarely-used pages from RAM, and letting that RAM be used for things that positively impact performance, like caching actually used filesystem objects. Pages that are backed by disk (e.g. files) don't need that, but anonymous memory that e.g. has only been touched once and then never even read afterwards should have a place to go as well. Also, wit…

Every time I've ran out of physical memory on Linux I've had to just reboot the machine, being unable to issue any kind of commands by input devices. I don't know what it is, but Linux just doesn't seem to be able to deal with that situation cleanly.

The mentioned situation is not running out of memory, but being able to use memory more efficiently.

Running out of memory is a hard problem, because in some ways we still assume that computers are turing machines with an infinite tape. (And in some ways, theoretically, we have to.) But it's not clear at all which memory to free up (by killing processes).

If you are lucky, there's one giant with tens of GB of resident memory usage to kill to put your system back into a usable state, but that's not the only case.

Re: Origin of the rule that swap size should be 2x of the physical memory

#44

Post by cr125rider »

Dylan16807 wrote: post #29
charcircuit wrote: post #22

Earlier quoted context omitted.

Today's swap also is not preallocated by the user. It is entirely handled by the OS itself. If it needs swap space to hibernate it will go ahead and allocate it itself.

It does? Last I checked Linux doesn't do dynamic swap sizes, and while Windows has dynamic swap sizes it has a separate big non-dynamic file for hibernation. I have no idea what MacOS does.

You guys are getting Linux to hibernate!?

Re: Origin of the rule that swap size should be 2x of the physical memory

#45

Post by fluoridation »

anyfoo wrote: post #43
fluoridation wrote: post #34

Earlier quoted context omitted.

Every time I've ran out of physical memory on Linux I've had to just reboot the machine, being unable to issue any kind of commands by input devices. I don't know what it is, but Linux just doesn't seem to be able to deal with that situation cleanly.

The mentioned situation is not running out of memory, but being able to use memory more efficiently. Running out of memory is a hard problem, because in some ways we still assume that computers are turing machines with an infinite tape. (And in some ways, theoretically, we have to.) But it's not clear at all which memory to free up (by killing processes). If you are lucky, there's one giant with tens of GB of residen…

Windows doesn't do that, though. If a process starts thrashing the performance goes to shit, but you can still operate the machine to kill it manually. Linux though? Utterly impossible. Usually even the desktop environment dies and I'm left with a blinking cursor.

What good is it to get marginally better performance under low memory pressure at the cost of having to reboot the machine under extremely high memory pressure?

Re: Origin of the rule that swap size should be 2x of the physical memory

#46

Post by anyfoo »

Bender wrote: post #18

Managed over 50k servers with zero swap. Set overcommit ratio to 0, min_free configured based on a Redhat formula and had application teams keep some memory free. Adjust oom scores at application startup especially for database servers where panic is set to 0. Servers ranged from 144GB ram to 3TB ram and that memory is heavily utilized. On servers meant to be stateless app and web servers panic was set to 2 to reboot…

There's a chance that those servers might run more efficiently with some swap space, for the reasons mentioned many times in this thread. Swap space is not just for overcommitting.

Re: Origin of the rule that swap size should be 2x of the physical memory

#47

Post by justinsaccount »

antongribok wrote: post #40

I think more people should know about the existence of ZRAM on modern Linux distributions. It's really changed the way I look at swap configs. ZRAM is a compressed block device that is stored in RAM. It's great! Previously, if I ever had high memory pressure situations, I really dreaded the slowdowns. Now, with swap sitting on top of /dev/zram0 it's a completely different experience. I have ZRAM enabled on all of my…

One interesting thing with zram (which OS X also does by default) is that certain memory leaks... effectively don't. I have a little raspberry pi where I have zram enabled. If I make a string in python and keep appending 'a's to it, eventually zram just soaks it up:

  $ zramctl
  NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
  /dev/zram0 zstd          3.8G  2.3G 13.2M 17.2M       4 [SWAP]
2.3GB of 'a's that gets compressed down to 20MB.

Re: Origin of the rule that swap size should be 2x of the physical memory

#48

Post by anyfoo »

fluoridation wrote: post #45
anyfoo wrote: post #43

Earlier quoted context omitted.

The mentioned situation is not running out of memory, but being able to use memory more efficiently. Running out of memory is a hard problem, because in some ways we still assume that computers are turing machines with an infinite tape. (And in some ways, theoretically, we have to.) But it's not clear at all which memory to free up (by killing processes). If you are lucky, there's one giant with tens of GB of residen…

Windows doesn't do that, though. If a process starts thrashing the performance goes to shit, but you can still operate the machine to kill it manually. Linux though? Utterly impossible. Usually even the desktop environment dies and I'm left with a blinking cursor. What good is it to get marginally better performance under low memory pressure at the cost of having to reboot the machine under extremely high memory pres…

In my experience the situations where you run into thrashing are rather rare nowadays. I personally wouldn't give up a good optimization for the rare worst case. (There's probably some knobs to turn as well, but I haven't had the need to figure that out.)

Re: Origin of the rule that swap size should be 2x of the physical memory

#49

Post by Bender »

anyfoo wrote: post #46
Bender wrote: post #18

Managed over 50k servers with zero swap. Set overcommit ratio to 0, min_free configured based on a Redhat formula and had application teams keep some memory free. Adjust oom scores at application startup especially for database servers where panic is set to 0. Servers ranged from 144GB ram to 3TB ram and that memory is heavily utilized. On servers meant to be stateless app and web servers panic was set to 2 to reboot…

There's a chance that those servers might run more efficiently with some swap space, for the reasons mentioned many times in this thread. Swap space is not just for overcommitting.

The theories are repeated a often but I have never seen any empirical data to back it up assuming one is setting the options I mentioned. These anecdotes usually come from servers with default settings and no attempt to tune them for the intended workloads and no capacity planning for application resources. Even OS maintainers are starting to recognize this and have created daemons such as tuned for the people that never touch settings. The next evolution will be dynamic adjustments from continuous bpf traces. I just keep it simple and avoid the circular arguments all together.

Re: Origin of the rule that swap size should be 2x of the physical memory

#50

Post by anyfoo »

Sohcahtoa82 wrote: post #15

None of the answers are satisfying to me, tbh. I install more RAM so I can swap less . If I have 8 GB, then the 2x rule means I should have a 16 GB swap file, giving me 24 GB of total memory to work with. If I then stumble upon a good deal on RAM and upgrade to 32 GB, then if I never had memory problems with 24 GB, then I should be able to completely disable paging and not have a problem. But instead, the advice woul…

I'm getting tired of typing this, but swap space is not just to increase available virtual memory. If you upgrade from 8 GB to 24 GB, then with proper swap space usage, you have 16 GB that could be used for additional disk cache.

Sure, you're still better off with 24 GB overall compared to 8GB+swap whether you add swap to your 24 GB or not, but swap can still make things more better.

(That says nothing about whether the 2x rule is still useful though, I have no idea.)

Post reply on HN