Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Origin of the rule that swap size should be 2x of the physical memory

retrocomputing.stackexchange.com

Post reply on HN
Print view View original

11–20 of 78 posts

Re: Origin of the rule that swap size should be 2x of the physical memory

#11

Post by quotemstr »

pezezin wrote: post #5
LowLevelKernel wrote: post #2

Curious, How much swap have you personally allocated on your personal setup?

Zero. My office workstation has 48 GB of RAM, my home computer has 64 (I went a bit overboard). I have very bad memories of swap thrashing and the computer becoming totally unresponsive until I forced a reset; if I manage to fill up so much RAM, I very much prefer the offending process to die instead of killing the whole computer.

It's funny how people think they're disabling swapping just because they don't have a swap file. Where do you think mmap()-ed file pages go? Your machine can still reclaim resident file-backed pages (either by discarding them if they're clear or writing them to their backing file if dirty) and reload them later. That's.... swap.

Instead of achieving responsiveness by disabling swap entirely (which is silly, because everyone has some very cold pages that don't deserve to be stuck in memory), people should mlockall essential processes, adjust the kernel's VM swap propensity, and so on.

Also, I wish we'd just do away with the separation between the anonymous-memory and file-backed memory subsystems entirely. The only special about MAP_ANONYMOUS should be that its backing file is the swap file.

Re: Origin of the rule that swap size should be 2x of the physical memory

#12

Post by quotemstr »

dirk94018 wrote: post #10

Early BSD VM pre-allocated swap backing for every anonymous page — you couldn't allocate virtual memory without a swap slot reserved for it, even if the page was never paged out. When a process forks, the child needed swap reservations for the parent's entire address space (before exec replaces it). A large process forking temporarily needs double its swap allocation. If your working set is roughly equal to physical…

TBF, I think overcommit was and remains an ugliness in how we manage memory. I wish we'd solved the fork commit-charge-spike issue by encouraging vfork (and later, posix_spawn) more heavily, not by making the OS lie about the availability of memory.

The ship's long sailed though, so even I run with overcommit enabled and only grumble about what might have been.

Re: Origin of the rule that swap size should be 2x of the physical memory

#13

Post by bandrami »

I've had arguments with people about this for 20 years now and the most compelling case I heard involved the price of storage vs the price of RAM in the mid to late 1990s, and that this 2x represented an optimal use of money in designing a system at that point in time.

Re: Origin of the rule that swap size should be 2x of the physical memory

#14

Post by linsomniac »

pezezin wrote: post #5
LowLevelKernel wrote: post #2

Curious, How much swap have you personally allocated on your personal setup?

Zero. My office workstation has 48 GB of RAM, my home computer has 64 (I went a bit overboard). I have very bad memories of swap thrashing and the computer becoming totally unresponsive until I forced a reset; if I manage to fill up so much RAM, I very much prefer the offending process to die instead of killing the whole computer.

I did similar with my 32GB laptop, but it was fairly flaky for ~4 years and I just recently put 48GB of swap on and it's been so much better. It's using over 20GB of the swap. The are cases in Linux where running without swap results in situations very similar to swapping too much.

Re: Origin of the rule that swap size should be 2x of the physical memory

#15

Post by Sohcahtoa82 »

None of the answers are satisfying to me, tbh.

I install more RAM so I can swap less. If I have 8 GB, then the 2x rule means I should have a 16 GB swap file, giving me 24 GB of total memory to work with. If I then stumble upon a good deal on RAM and upgrade to 32 GB, then if I never had memory problems with 24 GB, then I should be able to completely disable paging and not have a problem. But instead, the advice would be to increase my paging file to 64 GB!?

It doesn't make any sense. At all.

Re: Origin of the rule that swap size should be 2x of the physical memory

#16

Post by kgwxd »

void-star wrote: post #9
petcat wrote: post #3

The OP clearly states that he wants to know the earliest origin of the rule , and the only answers he gets are people giving their own opinions on how much swap space you should have. Too bad because it's an interesting question that I would also like to know the answer to.

Nope. Those are not the only answers I am seeing. I’m still curious though. 2x was nice because nobody really questioned it. Now that we have there doesn’t seem to be one “answer”. This is a fun/interesting question that comes up every now and then here and elsewhere :-) I suspect someone smarter than me about system tuning will have a much smarter and nuanced answer than “just use 2x”

I thought the modern advice was you don't need it at all. No more spinning disks, so the there's no speed gain using the inner-most ring, and modern OSes manage memory in more advanced, and dynamic ways. That's what I choose to believe anyway, I don't need anymore hard choices when setting up Linux :)

Re: Origin of the rule that swap size should be 2x of the physical memory

#17

Post by stock_toaster »

LowLevelKernel wrote: post #2

Curious, How much swap have you personally allocated on your personal setup?

On systems with 32/64/128 GB of ram, I'll typically have a 1GB or 2GB swap. Just so that the system can page out here and there to run optimally. Depending on the system, swap is typically either empty or just has a couple hundred MB kicking around.

Re: Origin of the rule that swap size should be 2x of the physical memory

#18

Post by Bender »

Managed over 50k servers with zero swap. Set overcommit ratio to 0, min_free configured based on a Redhat formula and had application teams keep some memory free. Adjust oom scores at application startup especially for database servers where panic is set to 0.

Servers ranged from 144GB ram to 3TB ram and that memory is heavily utilized. On servers meant to be stateless app and web servers panic was set to 2 to reboot on oom which mostly occurred in the performance team that were constantly load testing hardware and apps and a few dev machines were developers were not sharing nicely. Engineered correctly OOM will be very rare and this only gets better with time as applications have more controls over memory allocation and other tools like namespaces/cgroups. Java will always leak, just leave more room for it.

Re: Origin of the rule that swap size should be 2x of the physical memory

#19

Post by retrac »

quotemstr wrote: post #11
pezezin wrote: post #5

Earlier quoted context omitted.

Zero. My office workstation has 48 GB of RAM, my home computer has 64 (I went a bit overboard). I have very bad memories of swap thrashing and the computer becoming totally unresponsive until I forced a reset; if I manage to fill up so much RAM, I very much prefer the offending process to die instead of killing the whole computer.

It's funny how people think they're disabling swapping just because they don't have a swap file. Where do you think mmap()-ed file pages go? Your machine can still reclaim resident file-backed pages (either by discarding them if they're clear or writing them to their backing file if dirty) and reload them later. That's.... swap. Instead of achieving responsiveness by disabling swap entirely (which is silly, because e…

The ghost of Multics walks yet the page tables awaiting recorporealization.

Re: Origin of the rule that swap size should be 2x of the physical memory

#20

Post by debazel »

stock_toaster wrote: post #17
LowLevelKernel wrote: post #2

Curious, How much swap have you personally allocated on your personal setup?

On systems with 32/64/128 GB of ram, I'll typically have a 1GB or 2GB swap. Just so that the system can page out here and there to run optimally. Depending on the system, swap is typically either empty or just has a couple hundred MB kicking around.

On what OS are you using these settings? I found that Windows will refuse to allocate more virtual memory when the commit charge hit the max RAM size even if there is plenty of physical memory left to use.

I have 64 GiB of RAM and programs would start to crash at only 25 GiB of physical memory usage in some workloads because of high commit charge. I had to re-enable a 64 GiB SWAP file again just to be able to actually use my RAM.

My understanding is that Linux will not crash on the allocation and instead crash when too much virtual memory becomes active instead. Not sure how Mac handles it.

Post reply on HN