Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL and the OOM killer: Why we use strict memory overcommit

ubicloud.com

Post reply on HN
Print view View original

111–120 of 140 posts

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#111

Post by psd1 »

LoganDark wrote: post #94
alexrp wrote: post #92

Earlier quoted context omitted.

> Both Windows and macOS do so much better out of the box for essentially any workload. We test FreeBSD, Linux, macOS, NetBSD, OpenBSD, and Windows in Zig's CI fleet. Of these, Windows is the only OS that we've had to configure with swap double the size of physical RAM to not hit completely unjustifiable OOMs. By "unjustifiable", I mean that we're not even close to actually running out of physical memory (let alone s…

Windows seems to work a lot better with a 16MB page file for whatever reason, just because it refuses to enable memory compression without it. Fucking stupid

My curiosity is piqued. What reason did you have to disable the page file?

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#112

Post by baq »

pyinstallwoes wrote: post #90
baq wrote: post #45

Linux vm defaults are legit insane in 2026. - system dies under memory pressure (regardless of swapping, actually not having swap makes it worse which should be common knowledge by now) - system dies under disk pressure even if there are tons of free memory (this one is fun to diagnose) - system can technically not die, but render itself useless (or worse) under memory pressure by the oom killer - memory compression…

What are better defaults for 2026+?

start with a couple gigs of swap, fixed size sub-gig dirty buffers, enable zram, enable an oom daemon with some sane default configuration like 'kill X, wayland, sshd and shells last and browsers and node processes first'

for some more user sanity huge ram consumers under memory pressure should be suspended, paged out as much as possible and the user notified (macOS does it right)

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#113

Post by IshKebab »

man8alexd wrote: post #109
IshKebab wrote: post #104

Earlier quoted context omitted.

If a memory allocations fails with strict mode then you'll get a null pointer returns and some kind of crash or panic (in code that doesn't handle it properly). If it fails with the default mode the whole process will get killed by the OS. Is that really much better?

OOM is better. If a program doesn't handle ENOMEM properly, then its state is unpredictable and can lead to data corruption.

I don't think it's as simple as that. Killing a program can have unintended consequences too, e.g. corrupting files they were in the middle of writing.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#114

Post by 0x1d7 »

alexrp wrote: post #92
baq wrote: post #45

Linux vm defaults are legit insane in 2026. - system dies under memory pressure (regardless of swapping, actually not having swap makes it worse which should be common knowledge by now) - system dies under disk pressure even if there are tons of free memory (this one is fun to diagnose) - system can technically not die, but render itself useless (or worse) under memory pressure by the oom killer - memory compression…

> Both Windows and macOS do so much better out of the box for essentially any workload. We test FreeBSD, Linux, macOS, NetBSD, OpenBSD, and Windows in Zig's CI fleet. Of these, Windows is the only OS that we've had to configure with swap double the size of physical RAM to not hit completely unjustifiable OOMs. By "unjustifiable", I mean that we're not even close to actually running out of physical memory (let alone s…

Windows will autogrow the page file (*swap file is separate from a page file in Windows; swap is exclusively for UWP apps) as needed.

It sounds like the application wanted to allocate contiguous regions of memory when none were available. That's a typical indicator of an 'early' OOM condition.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#115

Post by 0x1d7 »

man8alexd wrote: post #88
0x1d7 wrote: post #84

Earlier quoted context omitted.

Which, by default, is dumb for a presumably interactive system. Photoshop (or equiv) is going to be my "largest" process on a system. Because it's the one I'm interacting with. Don't kill what I'm using.

You can always tell the system not to kill your Photoshop (or whatever else) by setting the OOM Score Adjust. This mechanism has existed for almost two decades and systemd has supported it for over a decade.

Yes, you can but the fact that I have to do it makes for a poor system.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#116

Post by 0x1d7 »

frollogaston wrote: post #99
nok22kon wrote: post #29

Earlier quoted context omitted.

damn, good observation, when my data analysis python script goes wrong and allocates 24 GB of RAM on a 32 GB computer, it crashes (gets killed) with "out of memory" error. I've never seen something else getting killed

That's what I've seen Linux do too. What's different about how Windows chooses a process to kill?

Windows doesn't kill.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#117

Post by LoganDark »

psd1 wrote: post #111
LoganDark wrote: post #94

Earlier quoted context omitted.

Windows seems to work a lot better with a 16MB page file for whatever reason, just because it refuses to enable memory compression without it. Fucking stupid

My curiosity is piqued. What reason did you have to disable the page file?

128GB of system memory?

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#118

Post by frollogaston »

man8alexd wrote: post #106
frollogaston wrote: post #100

Earlier quoted context omitted.

Why are programs even allocating memory that they don't use?

Because no one can predict the future, and they don't know how many resources they will need.

I mean they can just malloc when they need more, what am I missing? Unless this is about JVMs which might preallocate a ton for their heaps and not use it

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#119

Post by baq »

LoganDark wrote: post #94
alexrp wrote: post #92

Earlier quoted context omitted.

> Both Windows and macOS do so much better out of the box for essentially any workload. We test FreeBSD, Linux, macOS, NetBSD, OpenBSD, and Windows in Zig's CI fleet. Of these, Windows is the only OS that we've had to configure with swap double the size of physical RAM to not hit completely unjustifiable OOMs. By "unjustifiable", I mean that we're not even close to actually running out of physical memory (let alone s…

Windows seems to work a lot better with a 16MB page file for whatever reason, just because it refuses to enable memory compression without it. Fucking stupid

It might be shocking but so does Linux (though 16MB won’t cut it) and all other OSes

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#120

Post by swappie »

alexrp wrote: post #92
baq wrote: post #45

Linux vm defaults are legit insane in 2026. - system dies under memory pressure (regardless of swapping, actually not having swap makes it worse which should be common knowledge by now) - system dies under disk pressure even if there are tons of free memory (this one is fun to diagnose) - system can technically not die, but render itself useless (or worse) under memory pressure by the oom killer - memory compression…

> Both Windows and macOS do so much better out of the box for essentially any workload. We test FreeBSD, Linux, macOS, NetBSD, OpenBSD, and Windows in Zig's CI fleet. Of these, Windows is the only OS that we've had to configure with swap double the size of physical RAM to not hit completely unjustifiable OOMs. By "unjustifiable", I mean that we're not even close to actually running out of physical memory (let alone s…

When will people learn?

NT is optimized, since day one, to swap.

This is a feature, not a bug.

Post reply on HN