PostgreSQL and the OOM killer: Why we use strict memory overcommit • Hacker News Board

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#41

Post by __s » Fri, Jul 03, 2026, 6:44 PM UTC

szmarczak wrote: post #24

sterwill wrote: post #10

Earlier quoted context omitted.

Does this result in programs more frequently erroring/crashing because they can't allocate? I don't know how well many of the programs I frequently use on my desktop (Firefox, GNOME desktop, JVM + IntelliJ, Slack, etc.) handle allocation failures. I'm not sure they would do much better than crash, but I know the default OOM killer settings work well for me. About once a year a real runaway process (usually a throwawa…

> Does this result in programs more frequently erroring/crashing because they can't allocate? I run Firefox, VSCodium with LSP, Discord, Signal and there's still space left for a game like CS2. I'm not a heavy user by any means. > I'm not sure they would do much better than crash I have yet to see a program that silently handles allocation failures and doesn't crash. These days everything is coded to crash if no memo…

> I have yet to see a program that silently handles allocation failures and doesn't crash

Postgres handles allocation failures

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#42

Post by fpoling » Fri, Jul 03, 2026, 6:48 PM UTC

wongarsu wrote: post #21

For once, Microsoft's decision to just not do overcommit in Windows seems sensible

Microsoft just followed VAX/VMS that does not overcommit. And there is a noise on Linux mail lists to implement process builder pattern which VAX had like 50 years ago…

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#43

Post by dang » Fri, Jul 03, 2026, 6:53 PM UTC

furkansahin wrote: post #11

ozgune wrote: post #6

(Ozgun from Ubicloud) I agree with the blog post's technical contents, but I feel we came across too strong in the title. For Ubicloud as a managed Postgres provider, we use strict memory overcommit. Our experience with operating Postgres at scale taught us that it's better to enable this than going with the defaults. However, I can see many other scenarios, where using strict memory overcommit would have unanticipat…

(Furkan, submitter) Hmm, I haven’t thought about that. I updated the title to better reflect Ubicloud Postgres' position.

(previous title was "PostgreSQL and the OOM Killer: Why You Must Use Strict Memory Overcommit", if anyone is wondering.) Thanks for updating it here as well!

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#44

Post by fpoling » Fri, Jul 03, 2026, 7:03 PM UTC

sterwill wrote: post #10

szmarczak wrote: post #3

I have disabled overcommit both on Windows and on Linux. I hate having random programs being killed. Unfortunately, many programs commit 2x memory than they actually use. Often I see ~32GB committed and ~16GB resident.

Does this result in programs more frequently erroring/crashing because they can't allocate? I don't know how well many of the programs I frequently use on my desktop (Firefox, GNOME desktop, JVM + IntelliJ, Slack, etc.) handle allocation failures. I'm not sure they would do much better than crash, but I know the default OOM killer settings work well for me. About once a year a real runaway process (usually a throwawa…

Java allowed to handle out-of-memory rather well if one wanted to even 25 years ago. Basically one allocated a buffer on a startup taking 5% of memory that the application was supposed to use and made all threads to catch the oom exception. When handling that the buffer would be released, GC would be forced and a special flag would be set asking app to cancel any memory-intensive tasks until enough memory would be released and the buffer can be allocated again. It worked extremely well.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#45

Post by baq » Fri, Jul 03, 2026, 7:03 PM UTC

Linux vm defaults are legit insane in 2026.

- system dies under memory pressure (regardless of swapping, actually not having swap makes it worse which should be common knowledge by now)

- system dies under disk pressure even if there are tons of free memory (this one is fun to diagnose)

- system can technically not die, but render itself useless (or worse) under memory pressure by the oom killer

- memory compression of any sort is not enabled

- ...

Both Windows and macOS do so much better out of the box for essentially any workload.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#46

Post by man8alexd » Fri, Jul 03, 2026, 7:04 PM UTC

Mode 0 (Heuristic) is described incorrectly. All this complex heuristic was removed almost a decade ago. Currently, the kernel refuses a single allocation that exceeds the physical memory. That is all.

The article ignores the proper modern solution to prevent OOM killing of critical processes - OOM Score Adjust.

Tuning CommitLimit manually is an archaic, imprecise, and error-prone way to handle memory limits, only suitable for single-process workloads that can handle ENOMEM properly. It completely ignores dynamic file page cache memory allocation. You still can get OOM if you get unusually high file activity. On the other hand, under low file activity, it wastes memory on the same page cache, because it can't be reclaimed without memory pressure, and memory pressure can't be created because workload hits ENOMEM earlier. Don't use strict overcommit.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#47

Post by IshKebab » Fri, Jul 03, 2026, 7:15 PM UTC

There's so much great stuff here.

First, Linux's default memory management strategy is bonkers. OOM killing rarely actually works in my experience, at least on desktop. It takes ages to kick in and usually the system just freezes and you have to hard reboot. I've experienced this on every Linux system I've used, even my current one with 128GB of RAM and 64GB of swap, so don't say "it works for me". Windows and Mac do not have this issue at all, so clearly it's possible to do it better.

Has anyone tried using strict overcommit on desktop Linux?

Second, this bug is a great counterpoint to those annoying people who naysay Rust with "but not all bugs are memory safety bugs, what about logic bugs? huh?". Rust code would not have had this bug.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#48

Post by koverstreet » Fri, Jul 03, 2026, 7:19 PM UTC

baq wrote: post #45

Linux vm defaults are legit insane in 2026. - system dies under memory pressure (regardless of swapping, actually not having swap makes it worse which should be common knowledge by now) - system dies under disk pressure even if there are tons of free memory (this one is fun to diagnose) - system can technically not die, but render itself useless (or worse) under memory pressure by the oom killer - memory compression…

The mm people are increasingly hostile to any method of handling OOMs (like, just failing the allocation) besides the OOM killer - it's become very dominated by the hyperscalars and cloud vendors.

Working around mm nuttiness is a frequent source of frustration.

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#49

Post by leononame » Fri, Jul 03, 2026, 7:35 PM UTC

fpoling wrote: post #40

leononame wrote: post #4

This has bitten me multiple times. The problem I have is that at work we deploy the application (written in Go) and PostgreSQL on the same machine. The backend app allocates a lot of virtual memory, and initially we had overcommit to 0 (heuristic). This caused crashes on big queries in PostgreSQL and we set it to 2. The whole system became a bit unstable because the backend would still allocate a lot of virtual memor…

You can also use cgroup to set allocation policy for a particular program.

Interesting idea. How does that work in practice? If I've got 64GiB RAM, and PostgreSQL has 32GiB memory usage, and Go has 32GiB of memory.

If the database requests more memory, it gets ENOMEM, but if the backend app requests more memory, it does get some more because it can overcommit?

Sounds dangerous, if the go program then writes to the overcommitted memory, you'd still trigger the OOM killer, right?

Re: PostgreSQL and the OOM killer: Why we use strict memory overcommit

#50

Post by man8alexd » Fri, Jul 03, 2026, 7:38 PM UTC

IshKebab wrote: post #47

There's so much great stuff here. First, Linux's default memory management strategy is bonkers. OOM killing rarely actually works in my experience, at least on desktop. It takes ages to kick in and usually the system just freezes and you have to hard reboot. I've experienced this on every Linux system I've used, even my current one with 128GB of RAM and 64GB of swap, so don't say "it works for me". Windows and Mac do…

Yes, many have tried to use strict overcommit on the desktop. It is a good footgun. https://unix.stackexchange.com/a/797888/1027