Live data from Hacker News

In defence of swap: common misconceptions

chrisdown.name

31–40 of 151 posts

Re: In defence of swap: common misconceptions

#31
post #16

Earlier quoted context omitted.

Modern AWS instance types are EBS-only.

With the exception of the High IO types (I2/I3). They still get it and the newer instances get NVMe SSDs. In other words they are making it a feature of certain types that would benefit from it.

For example, F1 instances have NVMe local instance storage.

https://aws.amazon.com/ec2/instance-types/f1/

Re: In defence of swap: common misconceptions

#32
We've disabled swap (by not configuring a swap partition) on every server we've deployed since 2009. It's a little irritating to have to manually remove the swap partition from various Linux' "server" default install options even today. Of course this means I'm still installing on bare metal that I own, so...dinosaur.

Re: In defence of swap: common misconceptions

#33

I recently reenabled swap on my Windows machine due to frequent OOM, even with 16GB of RAM while playing Overwatch and browsing on Firefox. It seems like both of these programs allocate vast swaths of memory but then do not actually use that memory very heavily. After I turned swap back on, I did not notice any degradation in performance but my system stability skyrocketed.

Windows has vastly different policies for RAM allocation and commit than Linux. Windows basically does not overcommit while Linux systems not only does it by default but quite depend on it for various loads to work properly. In consequence, the userspace has a tendency to handle RAM differently, but there is no magic: if programs are trying to allocate twice the amount of RAM and then only use half of it, Windows with a swap that can be large enough will work perfectly while without swap the allocation will fail. Under Linux, the situation is less clear: without swap it will succeed (well it depends on the fine details of overcommit that are selected, but you get the idea) although if you really then use all that RAM, the OOM killer will start to more or less "randomly" kill "any" process to cope with the lack of RAM (as a last resort measure though; the caches and buffers are flushed before, etc.)

Re: In defence of swap: common misconceptions

#34
> Under temporary spikes in memory usage > With swap: We’re more resilient to temporary spikes, but in cases of severe memory starvation, the period from memory thrashing beginning to the OOM killer may be prolonged. We have more visibility into the instigators of memory pressure and can act on them more reasonably, and can perform a controlled intervention.

Somehow that doesn't resonate with my experience. I tend to remember the cases where I can't even SSH into the box, because the fork in sshd takes minutes, as does spawning the login shell.

I'd really like some way to have swap, but still loosen the OOM killer on the biggest memory hog when the system slows down to a crawl. I haven't found that magic configuration yet.

Re: In defence of swap: common misconceptions

#35

Swap was a great idea, but its time is gone. Swap doesn't make sense anymore, hard drives have not scaled and kept up with the improvements in RAM. In the Pentium 1 era EDO RAM maxed out at 256MB/s and hard disk xfer was 10MB/s. Common RAM size was 16MB. In today's era DDR4 maxes out at 32GB/s and hard disk xfer is 500 MB/s. Common RAM size is 16GB. RAM xfer rate has grown is 320x. RAM capacity has grown 100x. Disk x…

The purpose of swap is not running things out of it. The purpose is shoveling unused data out of memory. And for that it doesn't need to be particularly fast. Swap has a lot less purpose in a world without memory leaks and extraneous functions. But in practice it's quite good at getting several gigabytes of unnecessary data out of the way, so ram can be used properly. Swap, well-used, should only take up a few percen…

This reply was a much better explanation than the entire original article, or to summarize “Because memory leaks”.

Re: In defence of swap: common misconceptions

#36
In my experience, a misbehaving linux system that's out of RAM and has swap to spare will be unusably slow. The process of switching to a tty, logging in, and killing whatever the offending process is can easily take a good 15 minutes. Xorg will just freeze. Oh, and hopefully you know what process it is, else good luck running `top`.

Until this is fixed, I'll just keep running my systems with very small amounts of swap (say, 512MB in a system with 16GB of RAM). I'd rather the OOM killer kick in than have to REISUB or hold down the power button.

Some benchmarks with regards to the performance claims would be nice.

Re: In defence of swap: common misconceptions

#38
In defence against swap on my personal computer:

-PCs have a lot of RAM now

-When you allocate that much memory it's usually a bug in your own code like a size_t that overflowed. I never saw programs I would actually want to use try to allocate that much

-When using swap instead of ram, everything becomes so slow that you're screwed anyway. The UI doesn't even respond fast enough to kill whatever tries to use all that memory.

-How common is a situation where you need more memory than your ram size yet less than ram+swap size in a useful way? Usually if something needs a lot, it's really lot (and as mentioned above not desirable)

-Added complexity of making extra partition

-Added complexity if you want to use full disk encryption

-I do the opposite of using disk as ram: I put /tmp in a ramdisk of a few gigs

-Disks are slow and fast ssd's are expensive so you would't want to sacrifice their space (maybe if this changes some day...)

Re: In defence of swap: common misconceptions

#39

Swap was a great idea, but its time is gone. Swap doesn't make sense anymore, hard drives have not scaled and kept up with the improvements in RAM. In the Pentium 1 era EDO RAM maxed out at 256MB/s and hard disk xfer was 10MB/s. Common RAM size was 16MB. In today's era DDR4 maxes out at 32GB/s and hard disk xfer is 500 MB/s. Common RAM size is 16GB. RAM xfer rate has grown is 320x. RAM capacity has grown 100x. Disk x…

There have been times when I have needed to do something which requires more memory than is physically available on the machine. Without swap, those tasks would have been literally impossible to do without upgrading the machine. With swap, it just takes a little longer - but still much shorter than it would take to order more RAM sticks.

One of those times was on a 512MB RAM VPS, where I needed to compile something - you don't want a 512MB VPS to do a lot of compilation, but in that one instance, I was very glad I could easily just make a swap file and get on with it. The other time was on my laptop with 8 GB of RAM.

Also, even ignoring the times where swap makes possible a task which would otherwise have been impossible, you flat out ignored the content of the article. Did you even read it? If you have a long running task which allocates a lot of memory, then proceeds to only very rarely use that memory (or maybe it only needs that memory when it shuts down, or just forgot to free that memory), swap allows the system to swap out that memory and instead do something useful with it, like caching files or not killing processes. It doesn't matter that the disk is slower than RAM, if the swapped-out memory is rarely or never accessed.

Re: In defence of swap: common misconceptions

#40
post #6

I have an older Chromebook (c720), which is really quite memory starved (2GB RAM), and have experienced ChromeOS completely frying the SSD simply through prolonged tab-heavy swapping. Now, I've replaced the SSD and installed a non-Google Linux distro, and would like to limit the amount of swapping Firefox can do. I had been planning to simply use cgroups' memory features to limit the amount of memory consumed by Fire…

Firefox has a couple options which may help you get by. I can't guarantee these will fix everything but they are worth experimenting with. about:memory has various options, including a 'minimize memory usage' button and profiling tools. about:preferences has Privacy & Security > Cached Web Content > Override automatic cache management (select and set at 500MB, 1GB, or whatever works best).

Yep, you can adjust the max amount of cache it will store in ram and disk(separately).
Post reply on HN