While I have had my time fighting the OOM killer, I believe overcommit would have always won. To torture the metaphor a bit more, airlines have OOF mechanism - they just eject the overcommitted passengers before the plane takes off. A passenger buying a ticket is malloc(), but passengers don't always utilize the seat (use the memory). Normally this works out fine, but occasionally, there are too many passengers. Than…
In this worldview, malloc is like me buying a plane ticket at the counter for a specific flight that's going to leave soon. I'd be really annoyed if I were bumped off a flight I just paid for (and would've rather been told "that flight is full, try again later" (malloc returns NULL)). This is, for example what Windows does. Under memory pressure, it'll say to applications, "hey no I'm not in a giving mood for memory right now" (and will sometimes bump the size of the pagefile if configured to do this, but only up to a point).
The thought behind this is that well... applications have to handle malloc returning NULL anyway. Whether that's calling abort and giving up is one matter, another might be to retry the allocation at a later time (maybe after Windows has bumped the pagefile size), another might be to handle an error using some preallocated buffer or whatever.
