Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In defence of swap: common misconceptions

chrisdown.name

Post reply on HN
Print view View original

61–70 of 151 posts

Re: In defence of swap: common misconceptions

#61

Post by ibiza »

Count me among the believers in running w/ swap. Here's all it takes to provide Linux with a little swap space:

  fallocate -l 8G /swapfile
  chmod 0600 /swapfile
  mkswap /swapfile
  swapon /swapfile
Add an entry in /etc/fstab & you're done. "This little trick" made all the difference on a compute cluster I managed, where each node contained 96G of RAM. It's much more pleasant to monitor swap usage than the OOMKiller and related kernel parameters.

Re: In defence of swap: common misconceptions

#62

Post by quotemstr »

Very few people on this thread read and understood the article. The point isn't working with data sets larger than RAM. The point is making better use of the RAM you do have by taking pages you'll almost never touch and spilling them to disk so that there's more room in RAM for pages you will touch.

Banning swap is like making self-storage companies illegal and forcing everyone to hold all possessions in their homes. Sure, you'd be able to get to grandma's half broken kitschy dog coaster that you can't bring yourself to throw away, but you'd also be harder to harder to fit and find your own stuff, the stuff you need all the time.

If you find yourself driving to and from the self storage place every day, you probably need a bigger home. But self storage is plenty useful even if you almost never visit it.

Re: In defence of swap: common misconceptions

#63

Post by tmyklebu »

Dylan16807 wrote: post #55
tmyklebu wrote: post #47

Earlier quoted context omitted.

> The purpose of swap is not running things out of it. The purpose is shoveling unused data out of memory. And for that it doesn't need to be particularly fast. You can't detect a priori whether data is "unused." If you guess wrong a few times in a row, you get the familiar pattern where your Linux box is unresponsive to everything and needs to be bounced. If you could detect whether data is rarely used, swap still i…

> Applications can mmap() a file and use that region for "rarely used data" if such is known in advance. They could, but that's a lot like just making swap be manual. > Extraneous functions should be backed by the executable in the common case. I don't mean the code itself, I mean all the data it builds up for something that isn't needed.

> They could, but that's a lot like just making swap be manual.

Sure. Isn't that a good thing? Rarely-used data can be swapped out to storage allocated for the purpose, as you desire, and too-large working sets don't have to hose the machine.

> I don't mean the code itself, I mean all the data it builds up for something that isn't needed.

I guess I don't often run programs that waste large amounts of memory for no reason. Either my working set fits or it's too large, and swap only matters if it's too large.

Is Java the typical beneficiary here?

Re: In defence of swap: common misconceptions

#64

Post by adrianmonk »

CoconutPilot wrote: post #28

Swap was a great idea, but its time is gone. Swap doesn't make sense anymore, hard drives have not scaled and kept up with the improvements in RAM. In the Pentium 1 era EDO RAM maxed out at 256MB/s and hard disk xfer was 10MB/s. Common RAM size was 16MB. In today's era DDR4 maxes out at 32GB/s and hard disk xfer is 500 MB/s. Common RAM size is 16GB. RAM xfer rate has grown is 320x. RAM capacity has grown 100x. Disk x…

Swap may or may not be obsolete, but I don't find this particular argument on the subject convincing.

When dealing with swap, continuous transfer rate of the hard disk is not the relevant metric; seek time is.

In my experience, when the system starts needing to read a lot of pages back in from swap, it tends to do so in more or less random order. It reads a small amount of data, then it seeks to a new position, then reads another, and so on.

(I also find the argument unconvincing for a separate reason: I think hard drives are somewhat obsolete. Even my home system has flash instead of swap. And flash has a massively better seek time.)

Re: In defence of swap: common misconceptions

#65

Post by _9jgl »

quotemstr wrote: post #62

Very few people on this thread read and understood the article. The point isn't working with data sets larger than RAM. The point is making better use of the RAM you do have by taking pages you'll almost never touch and spilling them to disk so that there's more room in RAM for pages you will touch. Banning swap is like making self-storage companies illegal and forcing everyone to hold all possessions in their homes.…

The issue is that the current OOM killer doesn't support this usage at all.

To extend the analogy: what do you do if grandma comes and fills your house with stuff? You need space to work, so you go and drop it off at the self storage place, but what if she just keeps filling your house up?

The OOM killer will do absolutely nothing until both your house and the whole self storage place are totally full. By that point, you've spent a huge amount of time just driving to and from self storage, so you haven't had time to do any actual work; it would probably have been better to tell grandma that you don't want any more stuff once she filled up your house for the first time.

Re: In defence of swap: common misconceptions

#66

Post by quotemstr »

_9jgl wrote: post #65
quotemstr wrote: post #62

Very few people on this thread read and understood the article. The point isn't working with data sets larger than RAM. The point is making better use of the RAM you do have by taking pages you'll almost never touch and spilling them to disk so that there's more room in RAM for pages you will touch. Banning swap is like making self-storage companies illegal and forcing everyone to hold all possessions in their homes.…

The issue is that the current OOM killer doesn't support this usage at all. To extend the analogy: what do you do if grandma comes and fills your house with stuff? You need space to work, so you go and drop it off at the self storage place, but what if she just keeps filling your house up? The OOM killer will do absolutely nothing until both your house and the whole self storage place are totally full. By that point,…

Well, it doesn't help that when grandma calls and asks whether you have room for more stuff, the Linux kernel responds on your behalf, "Yes, of course I have room. I live in a TARDIS." And you then do all driving to the self-storage facility to maintain the illusion as long as you can. I really don't like overcommit.

Anyway, I agree with you that this behavior is annoying, but I think it ought to be possible to fix it (e.g., with memory cgroups or something like Android's lmkd) without giving up on the idea of spilling infrequently-accessed private dirty pages to disk.

Re: In defence of swap: common misconceptions

#67

Post by vardump »

_9jgl wrote: post #65
quotemstr wrote: post #62

Very few people on this thread read and understood the article. The point isn't working with data sets larger than RAM. The point is making better use of the RAM you do have by taking pages you'll almost never touch and spilling them to disk so that there's more room in RAM for pages you will touch. Banning swap is like making self-storage companies illegal and forcing everyone to hold all possessions in their homes.…

The issue is that the current OOM killer doesn't support this usage at all. To extend the analogy: what do you do if grandma comes and fills your house with stuff? You need space to work, so you go and drop it off at the self storage place, but what if she just keeps filling your house up? The OOM killer will do absolutely nothing until both your house and the whole self storage place are totally full. By that point,…

Well, just don't allow overcommit, problem solved.

Re: In defence of swap: common misconceptions

#68

Post by sigstoat »

tmyklebu wrote: post #52
al2o3cr wrote: post #48

Earlier quoted context omitted.

hard disk xfer is 500 MB/s You need better disks. The one in the laptop I'm typing this on can do 2GB/s, which greatly weakens your comparison.

If I may ask, which disks are you using?

the highest end NVMe SSDs can sustain those multi-gigabyte/second reads. https://smile.amazon.com/Samsung-950-PRO-Internal-MZ-V5P512B... for instance.

not sure even they could pull off those speeds with random reads, though.

Re: In defence of swap: common misconceptions

#69

Post by loeg »

dingaling wrote: post #46
loeg wrote: post #5

All of this boils down to "because buying more disk is cheaper than buying more RAM" and "avoid memory contention." The author discusses the situation as if the quantity of RAM is fixed and swap can be added (or not). But that isn't the only possibility — you can also add more RAM (it's just expensive). For the same number of GB of RAM+swap vs just RAM, there is no reason to prefer the option with swap.

> you can also add more RAM (it's just expensive) Or just impossible Many laptops still have remarkably low maximum-RAM limits. The ones I have here ( Dell SME & corporate types ) are 4GB and 8GB. I live in constant fear of the solid-blue disk-access lamp. And when I boot-up after a swap-thrash I am scolded for an unclean shutdown :(

You have not given an actual example of "impossible." A computer that can hold more RAM than 4-8 GB is possible to obtain, at some cost. You may object to the cost ("expensive"), but it is not impossible.

Re: In defence of swap: common misconceptions

#70

Post by userbinator »

kzrdude wrote: post #50
cosarara97 wrote: post #36

In my experience, a misbehaving linux system that's out of RAM and has swap to spare will be unusably slow. The process of switching to a tty, logging in, and killing whatever the offending process is can easily take a good 15 minutes. Xorg will just freeze. Oh, and hopefully you know what process it is, else good luck running `top`. Until this is fixed, I'll just keep running my systems with very small amounts of sw…

What function does the 0.5GB swap have?

A last-ditch safety buffer, to induce the slowdown so that you'll recognise that RAM is running low and hopefully prevent from actually completely running out.
Post reply on HN