Malloc Never Fails (2012) • Hacker News Board

Re: Malloc Never Fails (2012)

#101

Post by twic » Mon, Jun 10, 2019, 2:35 PM UTC

mannykannot wrote: post #74

twic wrote: post #57

Earlier quoted context omitted.

It's a shame that turning off overcommit is done at the system level, rather than the malloc call level. Some applications, and perhaps some allocations within otherwise naive applications, might prefer to have an allocation fail early, where failure could be sensibly handled.

I think that would ultimately have undesirable consequences. If this were an option on a malloc call, any such call would have to reserve the allocated memory at that point, reducing the usefulness of overcommit for other processes. This would set up a 'tragedy of the commons' scenario, where every application developer defensively uses this feature because other applications are using it.

You could mitigate that by treating these allocations conservatively: they only succeed if they can reserve the memory they want and there is 'enough' uncommitted memory left to cover all as-yet-uncommitted normal allocations. So, it would fail long before a normal allocation would.

I don't know quite what 'enough' means, but this whole business is already a rat's nest of heuristics, so one more should fit in nicely.

Re: Malloc Never Fails (2012)

#102

Post by detaro » Mon, Jun 10, 2019, 2:35 PM UTC

cheerlessbog wrote: post #98

dooglius wrote: post #94

Earlier quoted context omitted.

The OOM killer can strike at any time, regardless of the mapping options you've set, even if you aren't explicitly or implicitly allocating new memory. What I'd like, and what I think twic is asking for, is a way to "opt-out" of the overcommit paradigm at a process level, so that allocations by a given process reserve memory and may fail, but in return the kernel promises not to OOM kill it.

Most of my programming experience is in Windows. Can someone briefly explain why it doesn't seem to need an OOM killer? Is it happier to page? Or it always commits on allocation? Or something else.

AFAIK windows lets whatever application asks for memory and can't get it crash, whereas the OOM killer tries to be intelligent about what gets killed.

Re: Malloc Never Fails (2012)

#103

Post by [deleted] » Mon, Jun 10, 2019, 2:43 PM UTC

mehrdadn wrote: post #65

brianfen wrote: post #64

Earlier quoted context omitted.

The malloc() function requests memory from the kernel using mmap(). If mmap() returns successfully, it means that the kernel is saying that the memory is allocated. So from malloc()'s perspective the memory allocation has been successful and it must return a non-null pointer to the caller. The standard does not require malloc() to distrust what the kernel said and try to actually write to that memory to double-check…

The standard literally could not care less if there's a kernel underneath. It doesn't care how malloc is implemented or what the kernel, if any, looks like "from its perspective". The perspective that has relevance is that of the caller of malloc. From your own quotes: > The pointer returned if the allocation succeeds is suitably aligned so that [...] > If the space cannot be allocated, a null pointer is returned. If…

[deleted]

Re: Malloc Never Fails (2012)

#104

Post by dooglius » Mon, Jun 10, 2019, 2:47 PM UTC

mehrdadn wrote: post #27

Translation: This is a euphemism for saying Linux blatantly violates the language specification with no remorse. (Yes, I realize Linux is the kernel, etc.)

The C spec actually states at the beginning, "This International Standard does not specify ... the size or complexity of a program and its data that will exceed the capacity of any specific data-processing system or the capacity of a particular processor;"

Re: Malloc Never Fails (2012)

#105

Post by steveklabnik » Mon, Jun 10, 2019, 2:54 PM UTC

mmillin wrote: post #73

pornel wrote: post #68

This assumption lead to Rust's standard library not having a way to catch allocation failures (which is only now being rectified, and only partially). It's very Linux-centric and presumes a certain config+usage pattern. Not true on Windows. Not quite true on macOS. Not true in WASM. Definitely not true on embedded platforms.

Is there somewhere to read about how rust is tackling this problem?

A short overview:

1. Rust the language knows nothing about allocation. If you care about this behavior, it mostly limits the code of others' that you can use, but you can always write your own versions of things that respect fallible allocations.

2. Rust's standard library assumes memory is infallible. This is partially because it's a good default, and partially because our allocator API was not ready yet.

3. We've been working on the allocator API.

4. We have a rough plan for parameterizing data structures over allocators.

5. If this topic is of interest to you, https://github.com/rust-lang/wg-allocators is where to get involved.

Re: Malloc Never Fails (2012)

#106

Post by unilynx » Mon, Jun 10, 2019, 3:18 PM UTC

lmm wrote: post #84

unilynx wrote: post #66

Earlier quoted context omitted.

It's certainly more robust, but a lot of overhead for what should be a very uncommon situation. In eg an application server, requests themselves may be independent, but still share a lot of cached data - and as long as locking overhead doesn't overwhelm you, a single process is still the fastest way to share data between (worker) threads.

If that data is read-only then a fork()ed child process will share it just as efficiently as a thread would. If your threads need to share interleaved modifications to some common state then that approach isn't viable. But in that case it will be very hard to "roll back"/"skip over" a failed request, as it's difficult to be confident that you haven't corrupted that shared data in a way that will cause the same proble…

fork only works if the central cache never updates. Eg a cache of compiled bytecode for scripts gets filled as those scripts are requested. A cache of database results fills as those results are requested.

I'm not sure why skipping over failed requests is hard. VMs associated with the request are aborted, the client gets a 500/503 and can retry later. Incomplete data doesn't get added to the caches at all, so no corruption of shared data.

Re: Malloc Never Fails (2012)

#107

Post by [deleted] » Mon, Jun 10, 2019, 3:23 PM UTC

caf wrote: post #86

joosters wrote: post #61

Earlier quoted context omitted.

n.b. turning over-commit off comes with its own set of problems, such as causing programs to fail long before all memory has really been exhausted. For example, fork() will have to ensure that there is enough memory for a complete copy of the running process. If you have a large process, e.g. using 4GB of a 8GB machine, then fork() won't be able to run, even if you just want to fork and run a tiny program. With over-…

Or if you're just forking to exec a tiny program, use vfork().

[deleted]

Re: Malloc Never Fails (2012)

#108

Post by Paul-ish » Mon, Jun 10, 2019, 3:35 PM UTC

When I use Python sometimes I run into a MemoryError when working with large datasets. How does the Python runtime know I am out of memory if the kernel won't tell it. Does it try a write and catch the signal?

Re: Malloc Never Fails (2012)

#109

Post by 95014_refugee » Mon, Jun 10, 2019, 3:37 PM UTC

dragontamer wrote: post #83

jvanderbot wrote: post #80

What bothers me is that I read this train wreck without any red flags until I saw his correction at the end. Even the headline was wrong given the article, which itself was wrong. I really should have coffee before HN

What exactly is "wrong" about the article? Linux's Overcommit behavior is non-obvious to many programmers. Its one of those issues that very few programmers I've come across in the workplace understand properly. This blogpost properly understands the issues associated with Overcommit, and have done some preliminary investigations that describe the behavior. Its a really good blogpost. The general point of the blogpos…

What's "wrong" about the article is that the author has no idea what malloc(3) does, or about the wide range of execution environments C programs find themselves in and the different ways that the API maps to those environments.

More generally, the article does nothing to educate the public or move the "debate" about overcommit in any sort of useful direction.

Re: Malloc Never Fails (2012)

#110

Post by KirinDave » Mon, Jun 10, 2019, 3:43 PM UTC

philpem wrote: post #28

I saw the title and thought "oh heck this isn't right at all"... Set VM Overcommit to zero on an embedded system with no swap (a Raspberry Pi will do nicely). Write a C program that malloc()'s all the RAM. Watch malloc start to fail when you hit the RAM limit and the kernel has dumped all the I/O cache it can.

I think the author uses a hyperbolistic title, but if you read the article it s addressed that there are ways it can fail.

The larger point is that very few users of malloc understand its semantics, and in fact you can't know exactly how malloc will behave without knowing things about the runtime configuration of the system (as opposed to the hardware availability as many people like to think the simple case is).