Earlier quoted context omitted.
128GB of system memory?
That might remove a reason to not do it
Am I missing some sort of common-sense explanation of why page files are so great? I have enough system memory that I shouldn't need to spill it to disk.
PostgreSQL and the OOM killer: Why we use strict memory overcommit
131–140 of 140 posts
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#132Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#133Earlier quoted context omitted.
That might remove a reason to not do it
Am I missing some sort of common-sense explanation of why page files are so great? I have enough system memory that I shouldn't need to spill it to disk.
You have a lot of unused anonymous memory that is better swapped out and used as a file cache. Specifically for Linux swap, see https://chrisdown.name/2018/01/02/in-defence-of-swap.html
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#134Earlier quoted context omitted.
Am I missing some sort of common-sense explanation of why page files are so great? I have enough system memory that I shouldn't need to spill it to disk.
You have a lot of unused anonymous memory that is better swapped out and used as a file cache. Specifically for Linux swap, see https://chrisdown.name/2018/01/02/in-defence-of-swap.html
I don't understand how swap helps when you already have enough memory though. The article seems to say it's important to be able to swap out anonymous pages, but why? Especially when you aren't under memory contention? Just because memory isn't needed yet doesn't mean it's a benefit to get rid of it. It takes time to swap it to disk and then time to get it back. Why is that so important? Why would the kernel turn off things like compression otherwise? Makes no sense to me. No swap = punish the user by being less efficient on purpose? That's no argument for swap at all.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#135Earlier quoted context omitted.
Not before userspace software is fixed, it doesn't.
You can't fix the fact that you can't predict the future. The software will always allocate more memory than it needs because it can't predict its future resource usage, so limiting memory allocation with strict overcommit is meaningless.
I disagree... it should not allocate much (like 2x) more than it needs right now...
(allocating virtual memory, but not committing is different, and should be handled with MAP_NORESERVE (or similiar)).
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#136Earlier quoted context omitted.
The reason you hear less about Window's OOM killer is simply because it works well. The Linux Kernel OOM killer kills random things. Userspace OOM killers are meant to improve this, and they work well in a server situation when you already know in advance what is likely to go haywire and what is safe to kill. But they don't work well on desktop (some of them are improving but it doesn't seem to be a priority). The Wi…
> The Linux Kernel OOM killer kills random things. By default, the Linux kernel kills the largest process in the system (unless OOM adjust was applied).
A better default would be to first kill the processes with largest over-commit, then other if needed.
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#137Earlier quoted context omitted.
That might remove a reason to not do it
Am I missing some sort of common-sense explanation of why page files are so great? I have enough system memory that I shouldn't need to spill it to disk.
No, that's not the common sense you are missing ;-)
Your system your rules. But you twiddled a knob to appease your tweak imp, you didn't like the new behaviour, so you called it "fucking stupid". My experience is that the windows vmm is a very high quality component, so simple heuristics tells me PEBCAK.
I can think of two different possible reasons why memory compression might require a page file. Until you understand the technical reasoning, you don't know whether the design is stupid or clever. So taking a strong position is _eliding over a core tenet of wisdom_
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#138Earlier quoted context omitted.
Am I missing some sort of common-sense explanation of why page files are so great? I have enough system memory that I shouldn't need to spill it to disk.
No, that's not the common sense you are missing ;-) Your system your rules. But you twiddled a knob to appease your tweak imp, you didn't like the new behaviour, so you called it "fucking stupid". My experience is that the windows vmm is a very high quality component, so simple heuristics tells me PEBCAK. I can think of two different possible reasons why memory compression might require a page file. Until you underst…
> But you twiddled a knob to appease your tweak imp
I ran out of disk space because Windows decided my page file should suddenly be 64gb. With no space left to save my work, I found out there's no way to shrink the page file without rebooting. Of course the next thing I did was try to cast it off.
Maybe it was my skill issue for not expecting a sudden 64gb file there... or my skill issue for not choosing a nonzero size after?
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#139Earlier quoted context omitted.
> The Linux Kernel OOM killer kills random things. By default, the Linux kernel kills the largest process in the system (unless OOM adjust was applied).
A better default would be to first kill the processes with largest over-commit, then other if needed.
Why do you think it is a better default? What does it solve?
Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit
#140Earlier quoted context omitted.
No, that's not the common sense you are missing ;-) Your system your rules. But you twiddled a knob to appease your tweak imp, you didn't like the new behaviour, so you called it "fucking stupid". My experience is that the windows vmm is a very high quality component, so simple heuristics tells me PEBCAK. I can think of two different possible reasons why memory compression might require a page file. Until you underst…
> But you twiddled a knob to appease your tweak imp I ran out of disk space because Windows decided my page file should suddenly be 64gb. With no space left to save my work, I found out there's no way to shrink the page file without rebooting. Of course the next thing I did was try to cast it off. Maybe it was my skill issue for not expecting a sudden 64gb file there... or my skill issue for not choosing a nonzero si…
That's a good reason, I might do the same. I certainly used to twiddle that setting trying to make space for Sim City 2000 on an 80MB disk.
I see a major footgun in memory compression that Linux doesn't care about - it'll just oomkill your shit, hence TFA. But footguns are not acceptable in mass-market software. Apple also removes them.
It's easy to look at the half-baked ui libraries and fucking Teams and fucking SharePoint and conclude that Microsoft engineers are stupid. But, for your interest: https://blogsystem5.substack.com/p/windows-nt-vs-unix-design