Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is an OOM kill count in Linux

medium.com

Post reply on HN
Print view View original

51–60 of 66 posts

Re: There is an OOM kill count in Linux

#51

Post by kaba0 »

GTP wrote: post #32
magicalhippo wrote: post #16

Earlier quoted context omitted.

This is one area where Windows is far superior to Linux as a desktop experience. By accident, I once wrote an infinite loop that just allocated a bunch of memory. As I ran the program my Windows system quickly became laggy and unresponsive. However, not completely. With a bit of patience I got Task Manager up and managed to kill the process and the system was back to normal within a minute. On Linux, I experience the…

On the other hand, at least in my experience, Windows becomes laggy much sooner than Linux. IMO which of the two behaviors is better really depends on the use case.

I don’t think you can make an apples to oranges comparison here, there are too many differences. But sure, linux is mostly server-workload oriented.

Re: There is an OOM kill count in Linux

#52

Post by samus »

gnramires wrote: post #23
magicalhippo wrote: post #16

Earlier quoted context omitted.

This is one area where Windows is far superior to Linux as a desktop experience. By accident, I once wrote an infinite loop that just allocated a bunch of memory. As I ran the program my Windows system quickly became laggy and unresponsive. However, not completely. With a bit of patience I got Task Manager up and managed to kill the process and the system was back to normal within a minute. On Linux, I experience the…

Indeed! I wonder why the system can't just stop giving the process memory when it's starting to become critical? (say, you're over 95% memory used) I really think a more general solution is treating memory (and CPU time as welL!) as a scarce resource, and programs should either: (1) Deal with having severely denied resources (default behavior); (2) Use a communication protocol to negotiate memory with the system. Neg…

This is already the case. Processes have to ask politely for memory.

The OS could really just say no. But that would commit memory to a process even though it might not actually end up using it, making it unavailable for processes that need it right now. Also, memory allocation latency is higher since the OS has to really do all the bookkeeping up front. Therefore, most Linux distributions by default optimistically grant allocation requests even though there might not be enough memory available yet.

The big disadvantage is that the OS might be unable to actually grant memory. The process then gets OOM killed. This is sort of fine on servers since users usually get assigned limits. But there is something left to be desired for single-user systems.

Processes can at any time return memory to the OS when they don't need it. Usually they do that, but this has the disadvantage that they have to ask the OS again for it, which is quite slow compared to keeping it around.

Few applications are actually able to just release memory on request. It's critical user and internal application data after all. Databases and applications with garbage collectors come closest.

Re: There is an OOM kill count in Linux

#53

Post by samus »

kaba0 wrote: post #50
roblabla wrote: post #34

Earlier quoted context omitted.

> Maybe this is reinventing the wheel? I believe something like that exists on mobile platforms for memory, at least on iOS you get a message (applicationDidReceiveMemoryWarning:) when the system is memory starved and wants your app to free memory. If you don't release enough memory and the memory pressure doesn't go down, the system will start killing apps.

Aren’t it simply more common for apps to go into a “save state and basically exit” mode, from which they are expected to recover without any fuss? Like, ios is very aggressive in “killing” background apps, and since this is the general consensus, apps just behave nicely the majority of time.

Also on Android there are explicit guidelines that processes have to be prepared to be paused, hibernated, or killed at any time. Data in memory is not safe in any way.

Re: There is an OOM kill count in Linux

#54

Post by samus »

psyclobe wrote: post #36
magicalhippo wrote: post #16

Earlier quoted context omitted.

This is one area where Windows is far superior to Linux as a desktop experience. By accident, I once wrote an infinite loop that just allocated a bunch of memory. As I ran the program my Windows system quickly became laggy and unresponsive. However, not completely. With a bit of patience I got Task Manager up and managed to kill the process and the system was back to normal within a minute. On Linux, I experience the…

I’ve noticed this too, make -j and boom hard lock only option is to reboot. Really annoying l! Windows kinda deals with it but it has other issues; I have also noticed this behavior on MacOS.

It should be possible to avoid it by using ulimit to set memory and subprocess limits.

Re: There is an OOM kill count in Linux

#55

Post by mgdlbp »

contrarian1234 wrote: post #7

Out of curiosity, what do people recommend for OOM these days on Linux? My number one recurring issue on Ubuntu is me not noticing the memory usage, running out of memory and locking up my system (forcing a hard reboot and loss of unsaved work) EDIT: I'd preferably like an OOM handler that would freeze my system and pop up a little menu from which I could select which process to nuke

Tangentially, an issue with the desktop use case is that the main process of a long-running browser is often the first to be killed, instead of one of its content processes, which it can survive losing. This is the case since 2019 as the kernel no longer prefers killing children.[0]

As far as I know, unless a browser manages its oom_score_adj, which none do, a wrapper is insufficient as a workaround and a daemon would have to do so on its behalf, because the value is inherited when forking.

[0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

Re: There is an OOM kill count in Linux

#56

Post by manymatter »

gkbrk wrote: post #20
manymatter wrote: post #19

Earlier quoted context omitted.

What if the thing you kill is in the critical stack to saving your work? If it isn't I don't really understand why you would be swapping it in a lot I would view the OOm solution as a compute as cattle thinb, but here we are talking about a user desktop where the user can take the best action for themselves once they realize there's a problem.

> If it isn't I don't really understand why you would be swapping it in a lot. The user doesn't decide which processes are swapped in. If the process gets CPU time and tries to access its data, that data will get swapped in. > We are talking about a user desktop where the user can take the best action for themselves once they realize there's a problem. You can't do that with swap, because once you realize there's a p…

This sounds nothing like the gradual leak problem described.. An OOM killer is great once you actually run out of resources, removing virtual resources to always run out and play roulette is using it as a fad hammer.

Re: There is an OOM kill count in Linux

#58

Post by Always_Anon »

lokar wrote: post #47
ynik wrote: post #21

Earlier quoted context omitted.

That only works if the system is accessing mostly anonymous pages. If the load on the system is accessing plenty of mmapped code/data pages, it can still trash those even if swap is disabled. I've still seen systems hanging for 10+ minutes without recovering even though swap was already disabled. The Linux kernel OOM killer only acts if there's nothing left that can be discarded, which often happens way too late to s…

This is why on production systems important binaries should mlock all of their code pages into ram at startup.

[dead]

Re: There is an OOM kill count in Linux

#59

Post by mattpallissard »

contrarian1234 wrote: post #7

Out of curiosity, what do people recommend for OOM these days on Linux? My number one recurring issue on Ubuntu is me not noticing the memory usage, running out of memory and locking up my system (forcing a hard reboot and loss of unsaved work) EDIT: I'd preferably like an OOM handler that would freeze my system and pop up a little menu from which I could select which process to nuke

Wrap everything in systemd-run. I documented some of it here on my shitty site. Some of the memory limits have changed over time, notably the browsers unit

https://pallissard.net/2022/06/27/limiting_application_resou...

Edit: I like your user space oomd idea though, that would be killer

Re: There is an OOM kill count in Linux

#60

Post by alexeiz »

contrarian1234 wrote: post #7

Out of curiosity, what do people recommend for OOM these days on Linux? My number one recurring issue on Ubuntu is me not noticing the memory usage, running out of memory and locking up my system (forcing a hard reboot and loss of unsaved work) EDIT: I'd preferably like an OOM handler that would freeze my system and pop up a little menu from which I could select which process to nuke

I'm on openSUSE TW, and I considered systemd-oom, but the setup/configuration procedure for it is pretty elaborate. So I opted for earlyoom, which worked just a fine and required no manual configuration whatsoever.

In my opinion, if your distro already integrates systemd-oom, like Fedora, it's easier to just use that. Otherwise earlyoom is the next best choice.

Also it's recommended to enable zram. It's much faster than a swap partition or file. The combination of earlyoom and zram made my system much more responsive when it's running short of memory.

Post reply on HN