Earlier quoted context omitted.
The sane thing is to have swap enabled. Having swap "disabled" forces your system to swap out executables to disk, since these are likely the only memory-mapped files you have. So, if your memory fills up, you get catastrophic thrashing of the instruction cache. If you're lucky, you really go over available memory, and the OOMKiller kills some random process. But if you're not, your system will keep chugging along at…
People keep saying this, yet infinite real-world experience shows that systems perform far better if the OOM Killer actually gets to kill something, which is only possible with swap disabled. In my experience, the OOM killer picks the right target first maybe 70% of the time, and the rest of the time it kills some other large process and allows enough progress for the blameworthy process to either complete or get OOM…
Conversely, with a few GB of old data paged out to disk, even to a slow HDD, there is going to be much, much less thrashing going on. Chances are, the system will work pretty normally, since it's most likely that memory that isn't being used at all is what will get swapped out, so it's unlikely to need to be swapped in any time soon. The spike that caused you to go over your normal memory usage will die down, memory will get freed naturally, and worse you'll see is that some process will have a temporary spike in latency soem time later when it actually needs those swapped out pages.
Now, if the spike is too large to fit even in RAM + swap, the OOMKiller will still run and the system will recover that way.
The only situation where you'll get in the state you are describing is if veitually all of your memory pages are constantly getting read and written to, so that the VMM can't evict any "stale" pages to swap. This should be a relatively rare occurrence, but I'm sure there are workloads where this happens, and I agree that in those cases, disabling swap is a good idea.
> If you aren't checking `malloc` failure you have UB, but hopefully you will just crash (killing processes is a good thing when the system fundamentally can't fulfill everything you're asking of it!), and there's a pretty good chance the process that gets killed is blameworthy.
This is a very optimistic assumption. Crashing is about as likely as some kind of data corruption for these cases. Not to mention, crashing (or getting OOMKilled, for that matter) are very likely to cause data loss - a potentially huge issue. If you can avoid the situation altogether, that's much better. Which means overprovisioning and enabling some amount of swap if your workload is of a nature that doesn't constantly churn the entire working memory.
> "Swap isn't even that slow for SSDs" is part of the problem. All developers should be required to use an HDD with full-disk encryption, so that they stop papering over their performance bugs.
You're supposed to design software for the systems you actually yarget, not some lowest common denominator. If you're targeting use cases where the software will be deployed on 5400 RPM HDDs with full disk encryption at rest running on an Intel Celeron CPU with 512 MB of RAM, then yes, design your system for those constraints. Disable swap, overcommit too, probably avoid any kind of VM technology, etc.
But don't go telling people who are designing for servers running on SSDs to disable swap because it'll make the system unusably slow - it just won't.