Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vm.overcommit_memory=2 is the right setting for servers

ariadne.space

Post reply on HN
Print view View original

111–120 of 150 posts

Re: Vm.overcommit_memory=2 is the right setting for servers

#111

Post by zrm »

themafia wrote: post #101
zrm wrote: post #79

Earlier quoted context omitted.

Which is a major way turning off overcommit can cause problems. The expectation for disabling it is that if you request memory you're going to use it, which is frequently not true. So if you turn it off, your memory requirements go from, say, 64GB to 512GB. Obviously you don't want to have to octuple your physical memory for pages that will never be used, especially these days, so the typical way around that is to al…

> your memory requirements go from, say, 64GB to 512GB. Then your memory requirements always were potentially 512GB. It may just happen to be even with that amount of allocation you may only need 64GB of actual physical storage; however, there is clearly a path for your application to suddenly require 512GB of storage. Perhaps when it's under an attack or facing misconfigured clients. If your failure strategy is "jus…

> Then your memory requirements always were potentially 512GB. It may just happen to be even with that amount of allocation you may only need 64GB of actual physical storage; however, there is clearly a path for your application to suddenly require 512GB of storage.

If an allocator unconditionally maps in 512GB at once to minimize expensive reallocations, that doesn't inherently have any relationship to the maximum that could actually be used in the program.

Or suppose a generic library uses buffers that are ten times bigger than the maximum message supported by your application. Your program would deterministically never access 90% of the memory pages the library allocated.

> If your failure strategy is "just let the server fall over under pressure" then this might be fine for you.

The question is, what do you intend to happen when there is memory pressure?

If you start denying allocations, even if your program is designed to deal with that, so many others aren't that your system is likely to crash, or worse, take a trip down rarely-exercised code paths into the land of eldritch bugs.

Re: Vm.overcommit_memory=2 is the right setting for servers

#112

Post by johncolanduoni »

jclulow wrote: post #107
johncolanduoni wrote: post #104

Earlier quoted context omitted.

No non-embedded libc will actually return NULL. Very, very little practical C code actually relies only on specified behavior of the spec and will work with literally any compliant C compiler on any architecture, so I don’t find this particularly concerning. Usefully handling allocation errors is very hard to do well, since it infects literally every error handling path in your codebase. Any error handling that calls…

> No non-embedded libc will actually return NULL This is just a Linux ecosystem thing. Other full size operating systems do memory accounting differently, and are able to correctly communicate when more memory is not available.

There are functions on many C allocators that are explicitly for non-trivial allocation scenarios, but what major operating system malloc implementation returns NULL? MSVC’s docs reserve the right to return NULL, but the actual code is not capable of doing so (because it would be a security nightmare).

Re: Vm.overcommit_memory=2 is the right setting for servers

#113

Post by im3w1l »

Bjartr wrote: post #97
sidewndr46 wrote: post #58

Earlier quoted context omitted.

It's been a while but while I agree the man page says that, my limited understanding was the typical libc on linux won't really return NULL under any sane scenario. Even when the memory can't be backed

But why would you want to violate the docs on something as fundamental as malloc? Why risk relying on implementation specific quirks in the first place?

Because it's orders of magnitudes easier not to handle it. It's really as simple as that.

Re: Vm.overcommit_memory=2 is the right setting for servers

#114

Post by adastra22 »

dasil003 wrote: post #54
dbdr wrote: post #20

Earlier quoted context omitted.

> exceed swap + a configurable amount (default is 50%) of physical RAM Naive question: why is this default 50%, and more generally why is this not the entire RAM, what happens to the rest?

Not sure if I understand your question but nothing "happens to the rest", overcommitting just means processes can allocate memory in excess of RAM + swap. The percentage is arbitrary, could be 50%, 100% or 1000%. Allocating additional memory is not a problem per se, it only becomes a problem when you try to actually write (and subsequently read) more than you have.

They’re talking about the never-overcommit setting.

Re: Vm.overcommit_memory=2 is the right setting for servers

#115

Post by b112 »

MrDrMcCoy wrote: post #93
Tuna-Fish wrote: post #31

Earlier quoted context omitted.

The fundamental problem is that your machine is running software from a thousand different projects or libraries just to provide the basic system, and most of them do not handle allocation failure gracefully. If program A allocates too much memory and overcommit is off, that doesn't necessarily mean that A gets an allocation failure. It might also mean that code in library B in background process C gets the failure,…

There's also cgroup resource controls to separately govern max memory and swap usage. Thanks to systemd and systemd-run, you can easily apply and adjust them on arbitrary processes. The manpages you want are systemd.resource-control and systemd.exec. I haven't found any other equivalent tools that expose these cgroup features to the extent that systemd does.

I really dislike systemd, and its monolithic mass of over-engineered, all encompassing code. So I have to hang a comment here, showing just how easy this is to manage in a simple startup script. How these features are always exposed.

Taken from a SO post:

  # Create a cgroup
  mkdir /sys/fs/cgroup/memory/my_cgroup
  # Add the process to it
  echo $PID > /sys/fs/cgroup/memory/my_cgroup/cgroup.procs
  
  # Set the limit to 40MB
  echo $((40 \* 1024 \* 1024)) > /sys/fs/cgroup/memory/my_cgroup/memory.limit_in_bytes
Linux is so beautiful. Unix is. Systemd is like a person with makeup plastered 1" thick all over their face. It detracts, obscures the natural beauty, and is just a lot of work for no reason.

Re: Vm.overcommit_memory=2 is the right setting for servers

#116

Post by Asmod4n »

There are some situations where you can somewhat handle malloc returning NULL.

One would be where you have frequent large mallocs which get freed fast. Another would be where you have written a garbage collected language in C/C++.

When calling free, delete or letting your GC do that for you the memory isn't actually given back immediately, glibc has malloc_trim(0) for that, which tries it's best to give back as much unused memory to the OS as possible.

Then you can retry your call to malloc and see if it fails and then just let your supervisor restart your service/host/whatever or not.

Re: Vm.overcommit_memory=2 is the right setting for servers

#117

Post by masklinn »

loeg wrote: post #63
mahkoh wrote: post #38

Earlier quoted context omitted.

The point of disabling overcommit, as per the article, is that all pages in virtual memory must be backed by physical memory at all times. Therefore all virtual memory must reserve physical memory at the time of the fork call, even if the contents of the pages only get copied when they are touched.

Surely e.g. shared memory segments that are mapped by multiple processes are not double-counted? So it's only COW memory in particular that gets this treatment? Linux could just not do that.

COW is not shared memory, it’s an optimised copy. There is no way to guarantee that the optimisation will hold forever thus it is a form of overcommit (and indeed the reason most unices overcommit in the first place): most children will not touch most of the virtual memory they inherited but any can, so if you require precise memory accounting you have to account for that in the same way you account for large anonymous maps.

Re: Vm.overcommit_memory=2 is the right setting for servers

#118

Post by masklinn »

simscitizen wrote: post #105

There's already a popular OS that disables overcommit by default (Windows). The problem with this is that disallowing overcommit (especially with software that doesn't expect that) can mean you don't get anywhere close to actually using all the RAM that's installed on your system.

Windows also splits memory allocations between allocating the virtual space and committing real memory. So you can allocate a large VM when you need one and use piecemeal commits within that space.

POSIX not so much.

Re: Vm.overcommit_memory=2 is the right setting for servers

#119

Post by machinationu »

simscitizen wrote: post #105

There's already a popular OS that disables overcommit by default (Windows). The problem with this is that disallowing overcommit (especially with software that doesn't expect that) can mean you don't get anywhere close to actually using all the RAM that's installed on your system.

I am regularly getting close to 100% RAM usage on Windows doing data processing in Python/numpy

Re: Vm.overcommit_memory=2 is the right setting for servers

#120

Post by minitech »

silon42 wrote: post #14
Animats wrote: post #6

Setting 2 is still pretty generous. It means "Kernel does not allow allocations that exceed swap + (RAM × overcommit_ratio / 100)." It's not a "never swap or overcommit" setting. You can still get into thrashing by memory overload. We may be entering an era when everyone in computing has to get serious about resource consumption. NVidia says GPUs are going to get more expensive for the next five years. DRAM prices ar…

For me, on the desktop, thrashing overload is the most common way the Linux system effectively crashes... (I've left it overnight a few times, sometimes it recovered, but not always). I'm not disabling overcommit for now, but maybe I should.

SysRq+F to trigger the OOM killer manually might help (has to be enabled with the kernel.sysrq sysctl, see https://docs.kernel.org/admin-guide/sysrq.html#how-do-i-enab...).
Post reply on HN