Live data from Hacker News

Malloc Never Fails (2012)

scvalex.net

121–130 of 165 posts

Re: Malloc Never Fails (2012)

#121
post #92

Earlier quoted context omitted.

I guess you pre-allocate these UI elements when you start your application.

What if malloc fails at that point because some other application (we're in a preemptive multitasking OS) decided to gobble up all RAM?

If your application fails at start-up, you don't need to worry about saving state and graceful failure.

Re: Malloc Never Fails (2012)

#122
post #68

This assumption lead to Rust's standard library not having a way to catch allocation failures (which is only now being rectified, and only partially). It's very Linux-centric and presumes a certain config+usage pattern. Not true on Windows. Not quite true on macOS. Not true in WASM. Definitely not true on embedded platforms.

Attempting to catch allocation failures is the stupidest feature of C++ and the main reason that people turn off exceptions in that language.

Re: Malloc Never Fails (2012)

#123
Interesting topic with many things to say about, but wrong content. The point is that in most C programs, it is not worth to handle OOM errors, because what you can do during OOM is of very little value, on the other hand handling OOM correctly is very hard. However you can't do this in libraries, because you don't know how the library is going to be used. So for instance in order to make my Radix tree library resistant to OOM failures, I had to write a specific fuzzy test that used a malloc failing with a given probability, and check if the tree is sane after some stress-work with such malloc. In general the complexity of handling OOM in complex programs that deal with complex data structures is not often recognized.

Re: Malloc Never Fails (2012)

#124

When I use Python sometimes I run into a MemoryError when working with large datasets. How does the Python runtime know I am out of memory if the kernel won't tell it. Does it try a write and catch the signal?

Perhaps any write can fail, and if one does then it triggers some code which frees some pre-reserved space which allows the interpreter to continue, and properly unwind the stack.

Re: Malloc Never Fails (2012)

#125

What bothers me is that I read this train wreck without any red flags until I saw his correction at the end. Even the headline was wrong given the article, which itself was wrong. I really should have coffee before HN

What exactly is "wrong" about the article? Linux's Overcommit behavior is non-obvious to many programmers. Its one of those issues that very few programmers I've come across in the workplace understand properly. This blogpost properly understands the issues associated with Overcommit, and have done some preliminary investigations that describe the behavior. Its a really good blogpost. The general point of the blogpos…

What's wrong with the article is that malloc is not even a feature of Linux. There's mmap and brk, both of which have documented failure modes.

Re: Malloc Never Fails (2012)

#126
post #88

Earlier quoted context omitted.

If you take a look at the Linux kernel, a hugely complex and sophisticated piece of software, you'll find that it gracefully handles allocation failures.

The kernel's "graceful handling" of allocation failures includes killing unrelated processes in an attempt to free some memory.

Also just deadlocking. The kernel's behavior in the face of real memory pressure is actually quite poor. It's unsafe to run a Linux box out of memory.

Re: Malloc Never Fails (2012)

#127

Earlier quoted context omitted.

> It's worth pointing out that if you turn overcommit off ... you will, in fact, get this guarantee But many Linux applications assume the overcommit behavior. Its far easier to just "go with the flow" or "When in Rome...". Overcommit is the programming culture of Linux and should be assumed when writing Linux apps.

That has not been my experience. I disable overcommit on my machines and it has been considerably nicer to have apps exit due to malloc failing than random processes getting OOM-killed.

My experience with the oom-killer has been consistently terrible. Stuff like the X server getting killed taking down the whole user session instead of whatever was actually using the most memory. Probably happening because it's the parent process to everything else, but it's still a mind-blowingly dumb way to handle the low memory condition.

Re: Malloc Never Fails (2012)

#128

malloc fails with ulimit

Very relevant, because containers are often under what basically is ulimit. Not that you can do much when malloc fails...

When a memory control group runs out of space the process allocating just gets unilaterally killed with no notification. ulimit instead makes brk or mmap return a failure code, which the application could attempt to handle.

Re: Malloc Never Fails (2012)

#129
post #114

Earlier quoted context omitted.

> Unless you are working in something like a medical or nuclear device, it is not worth to bother with such things. Where "something like a medical or nuclear device" implies a piece of software that is expected to work correctly. In other words, you should always ensure that failures resulting from malloc(3) returning an error (and all other errors) are suitably contained.

No, i did not implied that, please do not put words in my mouth. My implication was software that if it fails it will kill people. Otherwise if a program crashes because of a situation happens once every 100000000 runs, not only is worthless to worry about it, it actually is preferable to not do that as to keep the codebase clean and hence easier to maintain for bugs that actually do affect people.

Keeping the codebase clean is why I prefer that allocation errors throw an exception rather than returning NULL.

Re: Malloc Never Fails (2012)

#130

How does one guarantee that allocated memory is real? Can you easily wrap malloc to zero/poke the memory in a way that catches all exceptions and guarantees yay or nay ?

You must use mlock call if you want to guarantee that no later action can suffer a major page fault.
Post reply on HN