Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vm.overcommit_memory=2 is the right setting for servers

ariadne.space

Post reply on HN
Print view View original

101–110 of 150 posts

Re: Vm.overcommit_memory=2 is the right setting for servers

#101

Post by themafia »

zrm wrote: post #79
otabdeveloper4 wrote: post #52

Earlier quoted context omitted.

> because of fork and because of the way thread stacks are allocated For modern (post-x86_64) memory allocators a common strategy is to allocate hundreds of gigabytes of virtual memory and let the kernel handle deal with actually swapping in physical memory pages upon use. This way you can partition the virtual memory space into arenas as you like. This works really well.

Which is a major way turning off overcommit can cause problems. The expectation for disabling it is that if you request memory you're going to use it, which is frequently not true. So if you turn it off, your memory requirements go from, say, 64GB to 512GB. Obviously you don't want to have to octuple your physical memory for pages that will never be used, especially these days, so the typical way around that is to al…

> your memory requirements go from, say, 64GB to 512GB.

Then your memory requirements always were potentially 512GB. It may just happen to be even with that amount of allocation you may only need 64GB of actual physical storage; however, there is clearly a path for your application to suddenly require 512GB of storage. Perhaps when it's under an attack or facing misconfigured clients.

If your failure strategy is "just let the server fall over under pressure" then this might be fine for you.

Re: Vm.overcommit_memory=2 is the right setting for servers

#102

Post by kccqzy »

otabdeveloper4 wrote: post #52
EdiX wrote: post #48

This is completely wrong. First, disabling overcommit is wasteful because of fork and because of the way thread stacks are allocated. Sorry, you don't get exact memory accounting with C, not even Windows will do exact accounting of thread stacks. Secondly, memory is a global resource so you don't get local failures when it's exhausted, whoever allocates first after memory has been exhausted will get an error they mig…

> because of fork and because of the way thread stacks are allocated For modern (post-x86_64) memory allocators a common strategy is to allocate hundreds of gigabytes of virtual memory and let the kernel handle deal with actually swapping in physical memory pages upon use. This way you can partition the virtual memory space into arenas as you like. This works really well.

Yeah but these only request address space. The memory is neither readable nor writable until a subsequent mprotect call. Ideally reserving address space only shouldn’t be counted as overcommit.

Re: Vm.overcommit_memory=2 is the right setting for servers

#103

Post by minitech »

inkyoto wrote: post #94
barchar wrote: post #90

Earlier quoted context omitted.

Unless you reserve on fork you're still over committing because after the fork writes to basically any page in either process will trigger memory commitment. Thread stacks come up because reserving them completely ahead of time would incur large amounts of memory usage. Typically they start small and grow when you touch the guards. This is a form of overcommit. Even windows dynamically grows stacks like this

> […] because after the fork writes to basically any page in either process will trigger memory commitment. This is largely not true for most processes. For a child process to start writing into its own data pages en masse, there has to exist a specific code path that causes such behaviour. Processes do not randomly modify their own data space – it is either a bug or a peculiar workload that causes it. You would have…

> This is largely not true for most processes.

> In most scenarios, forking a process has a negligible effect on the overall memory consumption in the system.

Yes, that’s what they’re getting at. It’s good overcommitment. It’s still overcommitment, because the OS has no way of knowing whether the process has the kind of rare path you’re talking about for the purposes of memory accounting. They said that disabling overcommit is wasteful, not that fork is wasteful.

Re: Vm.overcommit_memory=2 is the right setting for servers

#104

Post by johncolanduoni »

LordGrey wrote: post #53
sidewndr46 wrote: post #45

Earlier quoted context omitted.

Do any of the settings actually result in "malloc" or a similar function returning NULL?

malloc() and friends may always return NULL. From the man page: If successful, calloc(), malloc(), realloc(), reallocf(), valloc(), and aligned_alloc() functions return a pointer to allocated memory. If there is an error, they return a NULL pointer and set errno to ENOMEM. In practice, I find a lot of code that does not check for NULL, which is rather distressing.

No non-embedded libc will actually return NULL. Very, very little practical C code actually relies only on specified behavior of the spec and will work with literally any compliant C compiler on any architecture, so I don’t find this particularly concerning.

Usefully handling allocation errors is very hard to do well, since it infects literally every error handling path in your codebase. Any error handling that calls a function that might return an indirect allocation error needs to not allocate itself. Even if you have a codepath that speculatively allocates and can fallback, the process is likely so close to ruin that some other function that allocates will fail soon.

It’s almost universally more effective (not to mention easier) to keep track of your large/variable allocations proactively, and then maintain a buffer for little “normal” allocations that should have an approximate constant bound.

Re: Vm.overcommit_memory=2 is the right setting for servers

#105

Post by simscitizen »

There's already a popular OS that disables overcommit by default (Windows). The problem with this is that disallowing overcommit (especially with software that doesn't expect that) can mean you don't get anywhere close to actually using all the RAM that's installed on your system.

Re: Vm.overcommit_memory=2 is the right setting for servers

#106

Post by barchar »

inkyoto wrote: post #96
barchar wrote: post #90

Earlier quoted context omitted.

Unless you reserve on fork you're still over committing because after the fork writes to basically any page in either process will trigger memory commitment. Thread stacks come up because reserving them completely ahead of time would incur large amounts of memory usage. Typically they start small and grow when you touch the guards. This is a form of overcommit. Even windows dynamically grows stacks like this

Also, > Thread stacks come up because reserving them completely ahead of time would incur large amounts of memory usage. Typically they start small and grow when you touch the guards. This is a form of overcommit. Ahead of the time memory reservation entails a page entry being allocated in the process’s page catalogue («logical» allocation), and the page «sits» dormant until it is accessed and causes a memory access…

I am aware reserving excess memory doesn't commit said memory. But it does reserve memory, which is what we were talking about. The point was that because you can have a lot of threads and restricting reserved stacks to some small value is annoying all systems overcommit stack. Windows initially commits some memory (reserving space in the page file/ram) for each but will dynamically commit more when you touch the guard. This is overcommit. Linux does similarly.

Idle threads do increase the amount of committed stack. Once their stack grows it stays grown, it's not common to unmap the end and shrink the stacks. In a system without overcommit these stacks will contribute to total reserved phys/swap in the child, though ofc the pages will be cow.

> Windows employs a distinct process/thread design, making the UNIX concept of a process foreign. Threads are the primary construct in Windows and the kernel is highly optimised for thread management rather than processes. Cygwin has outlined significant challenges in supporting fork(2) semantics on Windows and has extensively documented the associated difficulties. However, I am veering off-topic.

The nt kernel actually works similarly to Linux w.r.t. processes and threads. Internally they are the same thing. The userspace is what makes process creation slow. Actually thread creation is also much slower than on Linux, but it's better than processes. Defender also contributes to the problems here.

Windows can do cow mappings, fork might even be implementable with undocumented APIs. Exec is essentially impossible though. You can't change the identity of a process like that without changing the PID and handles.

Fun fact: the clone syscall will let you create a new task that both shares VM and keeps the same stack as the parent. Chaos results, but it is fun. You used to be able to share your PID with the parent too, which also caused much destruction.

Re: Vm.overcommit_memory=2 is the right setting for servers

#107

Post by jclulow »

johncolanduoni wrote: post #104
LordGrey wrote: post #53

Earlier quoted context omitted.

malloc() and friends may always return NULL. From the man page: If successful, calloc(), malloc(), realloc(), reallocf(), valloc(), and aligned_alloc() functions return a pointer to allocated memory. If there is an error, they return a NULL pointer and set errno to ENOMEM. In practice, I find a lot of code that does not check for NULL, which is rather distressing.

No non-embedded libc will actually return NULL. Very, very little practical C code actually relies only on specified behavior of the spec and will work with literally any compliant C compiler on any architecture, so I don’t find this particularly concerning. Usefully handling allocation errors is very hard to do well, since it infects literally every error handling path in your codebase. Any error handling that calls…

> No non-embedded libc will actually return NULL

This is just a Linux ecosystem thing. Other full size operating systems do memory accounting differently, and are able to correctly communicate when more memory is not available.

Re: Vm.overcommit_memory=2 is the right setting for servers

#108

Post by Skunkleton »

Over commit is a design choice, and it is a design choice that is pretty core to Linux. Basic stuff like fork(), for example, gets wasteful when you don't over commit. Less obvious stuff like buffer caches also get less effective. There are certainly places where you would rather fail at allocation time, but that isn't everywhere and it doesn't belong as a default.

Re: Vm.overcommit_memory=2 is the right setting for servers

#109

Post by barchar »

minitech wrote: post #103
inkyoto wrote: post #94

Earlier quoted context omitted.

> […] because after the fork writes to basically any page in either process will trigger memory commitment. This is largely not true for most processes. For a child process to start writing into its own data pages en masse, there has to exist a specific code path that causes such behaviour. Processes do not randomly modify their own data space – it is either a bug or a peculiar workload that causes it. You would have…

> This is largely not true for most processes. > In most scenarios, forking a process has a negligible effect on the overall memory consumption in the system. Yes, that’s what they’re getting at. It’s good overcommitment. It’s still overcommitment, because the OS has no way of knowing whether the process has the kind of rare path you’re talking about for the purposes of memory accounting. They said that disabling ove…

Yep. If you aren't overcommitting on fork it's quite wasteful, and if you are overcommitting on fork then you've already given up on not having to handle oom conditions after malloc has returned.

Re: Vm.overcommit_memory=2 is the right setting for servers

#110

Post by agilob »

An aircraft company discovered that it was cheaper to fly its planes with less fuel on board. The planes would be lighter and use less fuel and money was saved. On rare occasions however the amount of fuel was insufficient, and the plane would crash. This problem was solved by the engineers of the company by the development of a special OOF (out-of-fuel) mechanism. In emergency cases a passenger was selected and thrown out of the plane. (When necessary, the procedure was repeated.) A large body of theory was developed and many publications were devoted to the problem of properly selecting the victim to be ejected. Should the victim be chosen at random? Or should one choose the heaviest person? Or the oldest? Should passengers pay in order not to be ejected, so that the victim would be the poorest on board? And if for example the heaviest person was chosen, should there be a special exception in case that was the pilot? Should first class passengers be exempted? Now that the OOF mechanism existed, it would be activated every now and then, and eject passengers even when there was no fuel shortage. The engineers are still studying precisely how this malfunction is caused.

https://lwn.net/Articles/104185/

Post reply on HN