This seems to always be explained in a confusing and sometimes wrong manner, so I may as well also try to explain it in a confusing and sometimes wrong manner too.
Both Windows and Linux allow applications to map memory in an uncommitted state. However, in Linux, uncommitted memory can magically become committed by simply trying to use it, whereas on Windows you have to first actually commit it with a separate API call.
You might wonder why you would even bother allocating uncommitted memory on Windows if you have to explicitly commit it later. Simple: just to reserve a contiguous slice of address space for later.
The consequence of this is that on Windows an allocation failure usually occurs at an API call where it can return a failure, and indeed does. On Linux though, an allocation can fail during a page fault that is entirely transparent to the application. So instead, the default behavior is to allow overcommit, where applications are allowed to commit more memory than the system actually has, under the assumption that in many cases it won't all be in use at once. Instead of an application hitting an error or crashing on a failed allocation, if there is no memory left, a process simply hangs.
Here's my take on why this is the way it is:
- Because programs are written without any knowledge of what memory pages are committed, applications will allocate physical memory pages transparently even if they didn't malloc or mmap anything. Because of that, when an allocation fails under memory pressure, it's pretty likely the first page to stall on overcommit isn't really related to the actual memory pressure.
- To try to mitigate this, during high memory pressure situations, mechanisms like the OOM killer have to score tasks based on several factors to try to guess which ones are causing the problem.
- When you have something like make -j running, it is pretty easy for it to choose Firefox instead of the 200 instances of Clang, since individually none of those Clang instances are really using much memory. This, of course, doesn't fix the bleeding, it just kills your browser. There are ways to mitigate this problem, but they are not often implemented and nothing will ever be perfect.
One may wonder if it is worth the trouble... Probably not, but it does have its advantages, particularly when it comes to databases and caches, which can aggressively reserve memory and mmap files and heavily lean on demand paging for memory management. It's just that in this case, managing memory pressure becomes somewhat heuristics-based, and tools like cgroups are often desired to isolate failure domains and control memory allocations in production systems.
This is a tough problem to solve. Many improvements to the Linux OOM situation have been made, from simply fixing problems that made the wedging even worse, to tweaking the OOM scoring, to usermode daemons like systemd-oomd that try to catch memory pressure earlier. However, so far this is a solidly unsolved problem - the same exact wedging is still possible today.
As hopeless as it seems, I am still open-minded here. The Linux desktop moves slow, but nonetheless it seems pretty good at overcoming challenging obstacles.