For once, Microsoft's decision to just not do overcommit in Windows seems sensible
PostgreSQL and the OOM killer: Why we use strict memory overcommit
21–30 of 140 posts
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#22I have disabled overcommit both on Windows and on Linux. I hate having random programs being killed. Unfortunately, many programs commit 2x memory than they actually use. Often I see ~32GB committed and ~16GB resident.
how exactly did you disabled it on Windows? I dont think it has an option for that.
I don't think it has overcommit at all, at least that's the default. That would be why you don't have Windows OOM killer stories.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#23Earlier quoted context omitted.
how exactly did you disabled it on Windows? I dont think it has an option for that.
Settings -> View advanced system settings -> Performance (Settings) -> Advanced -> Virtual memory (Change...) -> No paging file
This is almost always a bad idea.
If no memory is available where a page file would make a difference, this leads to application crashes instead. A crash is (usually) worse than paging.
Certain applications, Photoshop being the historical example, will outright fail to run with no page file present.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#24I have disabled overcommit both on Windows and on Linux. I hate having random programs being killed. Unfortunately, many programs commit 2x memory than they actually use. Often I see ~32GB committed and ~16GB resident.
Does this result in programs more frequently erroring/crashing because they can't allocate? I don't know how well many of the programs I frequently use on my desktop (Firefox, GNOME desktop, JVM + IntelliJ, Slack, etc.) handle allocation failures. I'm not sure they would do much better than crash, but I know the default OOM killer settings work well for me. About once a year a real runaway process (usually a throwawa…
> Does this result in programs more frequently erroring/crashing because they can't allocate?
I run Firefox, VSCodium with LSP, Discord, Signal and there's still space left for a game like CS2. I'm not a heavy user by any means.
> I'm not sure they would do much better than crash
I have yet to see a program that silently handles allocation failures and doesn't crash. These days everything is coded to crash if no memory :(
> About once a year a real runaway process (usually a throwaway program I'm working on) gets OOM-killed
In my case it killed system critical processes with no way to recover. With disabled overcommit, it freezes for a while (usually for a minute or two), I close some random program of my choosing and then see in Resource Monitor what's eating my ram.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#25Earlier quoted context omitted.
how exactly did you disabled it on Windows? I dont think it has an option for that.
Settings -> View advanced system settings -> Performance (Settings) -> Advanced -> Virtual memory (Change...) -> No paging file
That's disabling swap, not overcommit. Windows doesn't overcommit. It's one of the reason why it handles low memory situations so much more gracefully than Linux.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#26Earlier quoted context omitted.
Settings -> View advanced system settings -> Performance (Settings) -> Advanced -> Virtual memory (Change...) -> No paging file
That's disabling swap, not overcommit. Windows doesn't overcommit. It's one of the reason why it handles low memory situations so much more gracefully than Linux.
^
The purpose of the system commit limit and commit charge is to track all uses of these resources to ensure they are never overcommitted — that is, that there is never more virtual address space defined than there is space to store its contents, either in RAM or in backing store (on disk).
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#27Earlier quoted context omitted.
how exactly did you disabled it on Windows? I dont think it has an option for that.
I don't think it has overcommit at all, at least that's the default. That would be why you don't have Windows OOM killer stories.
The reason you hear less about Window's OOM killer is simply because it works well.
The Linux Kernel OOM killer kills random things. Userspace OOM killers are meant to improve this, and they work well in a server situation when you already know in advance what is likely to go haywire and what is safe to kill. But they don't work well on desktop (some of them are improving but it doesn't seem to be a priority).
The Windows OOM killer by comparison usually kills something sensible (i.e. the program that is actually using all the memory), and asks the user for permission before killing it (when possible). You do see a lot of memes of situations where it fails.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#28Earlier quoted context omitted.
Settings -> View advanced system settings -> Performance (Settings) -> Advanced -> Virtual memory (Change...) -> No paging file
This is almost always a bad idea. If no memory is available where a page file would make a difference, this leads to application crashes instead. A crash is (usually) worse than paging. Certain applications, Photoshop being the historical example, will outright fail to run with no page file present.
> this leads to application crashes instead
Same happens if the page file is full. In that case, why don't those programs use disk directly instead?
No such problem would've ever occured if programs hadn't allocated more than they actually use.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#29Earlier quoted context omitted.
I don't think it has overcommit at all, at least that's the default. That would be why you don't have Windows OOM killer stories.
The reason you hear less about Window's OOM killer is simply because it works well. The Linux Kernel OOM killer kills random things. Userspace OOM killers are meant to improve this, and they work well in a server situation when you already know in advance what is likely to go haywire and what is safe to kill. But they don't work well on desktop (some of them are improving but it doesn't seem to be a priority). The Wi…
damn, good observation, when my data analysis python script goes wrong and allocates 24 GB of RAM on a 32 GB computer, it crashes (gets killed) with "out of memory" error. I've never seen something else getting killed
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#30Earlier quoted context omitted.
This is almost always a bad idea. If no memory is available where a page file would make a difference, this leads to application crashes instead. A crash is (usually) worse than paging. Certain applications, Photoshop being the historical example, will outright fail to run with no page file present.
> this leads to application crashes instead Same happens if the page file is full. In that case, why don't those programs use disk directly instead? No such problem would've ever occured if programs hadn't allocated more than they actually use.
Your argument falls flat when a page file can be multi-GB and automatically grow. And if your application admin was competent, memory monitoring would be part of the application monitoring stack.
An application that grows in such a way (besides having backing stores for memory-mapped files, as well) will often perform so poorly that it requires addressing (adding RAM, looking for application faults, etc).
A page file is insurance, one that can last you much longer than available system memory.