Live data from Hacker News

Malloc Never Fails (2012)

scvalex.net

51–60 of 165 posts

Re: Malloc Never Fails (2012)

#51
post #26
post #16

Earlier quoted context omitted.

This is true, in my experience. If you keep your program fairly simple, implement it in plain C, avoid recursion, and keep these requirements in mind from the start, then it is possible to write a program that fails gracefully when it runs out of memory. Some old-school Unix daemons do this. But for most software it's impractical, so you might as well define your own xmalloc() that calls abort() if malloc() returns z…

An exception-safe C++ program can do the same through a std::bad_alloc exception. If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine. (I usually connected other out-of-resource conditions such…

> If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine.

If you have a system that handles independent requests, it might be more robust to model each request as its own process, and then simply allow those processes to crash if they find themselves in an unexpected condition (and handle process crashes).

Re: Malloc Never Fails (2012)

#52
post #36

Earlier quoted context omitted.

In practice i haven't seen any application to gracefully handle out of memory situations. Even back in Windows 3.x days when running out of memory was common, most applications simply hanged or crashed (which also took the entire GUI with them) and those that didn't often had a "no memory, kthxbye" dialog that shut down the application. A very few rare cases would try to display some "sorry, no memory, close some pro…

Keep in mind that Windows 3.x wasn't really the pinnacle of computing in its days. PCs were relatively irrelevant compared to the systems everyone apart from a mom and pop shop ran their business on. So the typical behavior you saw was a combined effect of physical constraints and lack of skill or care.

No it wasn't which is exactly why i brought it up (well that, and it was the most popular - despite its shortcomings).

Re: Malloc Never Fails (2012)

#53

Earlier quoted context omitted.

Raspberry Pis almost always run 32 bit OSes, so that's a common place to encounter 32-bit.

I had always thought they'd be running a 64-bit OS, given that the new hardware is 64-bit. TIL that Raspbian is still 32-bit!

Unless you need the address space, there is little gain with a 64 bit OS. The wider pointers use more RAM and eat up more of the available memory bandwidth and caches. So not swotching to 64 bits is likely the best use if the hardware.

Re: Malloc Never Fails (2012)

#54
post #36

Earlier quoted context omitted.

In practice i haven't seen any application to gracefully handle out of memory situations. Even back in Windows 3.x days when running out of memory was common, most applications simply hanged or crashed (which also took the entire GUI with them) and those that didn't often had a "no memory, kthxbye" dialog that shut down the application. A very few rare cases would try to display some "sorry, no memory, close some pro…

> what would you do if you run out of memory while making the fourth button in a toolbar? Rollback what I’d created so far, evict caches, ask malloc to trim, try again, if that failed roll back again and ask the user to reduce the volume of application data open by closing views or documents or whatever before they try again. But yeah it’s a lot of engineering.

> ask the user

You just failed to create a toolbar button, how are you going to ask a user do something if you already failed to create a tiny UI element?

Re: Malloc Never Fails (2012)

#55
post #36

Earlier quoted context omitted.

> Not that you can do much when malloc fails... Tell the user they can’t do that operation? Use on-disk storage instead? Re-use memory you already have (such as evicting a cache and taking the memory it already had allocated)? Abandon what you were doing if it was only an optimisation and wasn’t essential (such as allocating memory as part of a speculative execution)? Run a garbage collector and try again? Lots of op…

In practice i haven't seen any application to gracefully handle out of memory situations. Even back in Windows 3.x days when running out of memory was common, most applications simply hanged or crashed (which also took the entire GUI with them) and those that didn't often had a "no memory, kthxbye" dialog that shut down the application. A very few rare cases would try to display some "sorry, no memory, close some pro…

https://www.sqlite.org/malloc.html is a welcome contrast to the typical behavior you're describing, but I agree that such measures are rare. (and, not an application: though it provides the mechanisms for applications to follow its lead.)

Re: Malloc Never Fails (2012)

#56
post #36

Earlier quoted context omitted.

> Not that you can do much when malloc fails... Tell the user they can’t do that operation? Use on-disk storage instead? Re-use memory you already have (such as evicting a cache and taking the memory it already had allocated)? Abandon what you were doing if it was only an optimisation and wasn’t essential (such as allocating memory as part of a speculative execution)? Run a garbage collector and try again? Lots of op…

In practice i haven't seen any application to gracefully handle out of memory situations. Even back in Windows 3.x days when running out of memory was common, most applications simply hanged or crashed (which also took the entire GUI with them) and those that didn't often had a "no memory, kthxbye" dialog that shut down the application. A very few rare cases would try to display some "sorry, no memory, close some pro…

> In practice i haven't seen any application to gracefully handle out of memory situations

I have seen code hit an allocation failure, bubble the error up the stack, which causes various pieces of code along the stack to free their heap allocations, all the while the process is able to log what is going and keep running.

I have even seen this happen when the allocation that failed was tiny, less than 100 bytes.

Re: Malloc Never Fails (2012)

#57
post #7

The title here is just blatantly false; there are certainly some scenarios in which it won't fail, but many in which it will. The author is aware of this since he fixes the claim toward the end: > To clarify, the surprising behaviour malloc has does not mean we should ignore its return value. We just need to be careful because malloc returning successfully does not always mean that we can use the requested memory. It…

It's a shame that turning off overcommit is done at the system level, rather than the malloc call level. Some applications, and perhaps some allocations within otherwise naive applications, might prefer to have an allocation fail early, where failure could be sensibly handled.

Re: Malloc Never Fails (2012)

#58

Earlier quoted context omitted.

Which part of the specification does this violate?

Like, every part? It doesn't actually allocate the space requested, it doesn't indicate failure by returning NULL (or by any other reliable means for that matter)...

You can't be serious about "every part". This definitely does not violate section 1.1 which only talks about the scope of the standard. It also definitely does not violate 1.2 which talks about what is not within the scope of the standard.

So once again, can you cite the exact section number from the standard that you think is being violated here?

Re: Malloc Never Fails (2012)

#59

Earlier quoted context omitted.

Like, every part? It doesn't actually allocate the space requested, it doesn't indicate failure by returning NULL (or by any other reliable means for that matter)...

I had a look, and the C standard doesn't actually say anything about indicating failure. Here is all it says about malloc: > The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate. > The malloc function returns either a null pointer or a pointer to the allocated space. I think it's certainly debatable whether overcommitting while lazily allocating counts as…

> I had a look, and the C standard doesn't actually say anything about indicating failure.

> The malloc function returns either a null pointer or a pointer to the allocated space.

Uhm, either malloc returns a pointer to the allocated space, or it returns null. I don't see what's so complicated about this. There's no provision for "return a non-null pointer to unallocated space".

Re: Malloc Never Fails (2012)

#60

Earlier quoted context omitted.

Like, every part? It doesn't actually allocate the space requested, it doesn't indicate failure by returning NULL (or by any other reliable means for that matter)...

You can't be serious about "every part". This definitely does not violate section 1.1 which only talks about the scope of the standard. It also definitely does not violate 1.2 which talks about what is not within the scope of the standard. So once again, can you cite the exact section number from the standard that you think is being violated here?

See reply to sibling comment.
Post reply on HN