Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malloc Never Fails (2012)

scvalex.net

Post reply on HN
Print view View original

141–150 of 165 posts

Re: Malloc Never Fails (2012)

#141

Post by asveikau »

gmueckl wrote: post #95
saagarjha wrote: post #72

Earlier quoted context omitted.

It’s a toss up, I think: actual results depend on the task at hand. 64-bit has the benefit of more, wider registers, which can speed up certain tasks.

I stand corrected. I missed the fact that ARM also introduced new registers in AArch64. I thought that such a change only happened in amd64.

The other thing about amd64 is that amd64 code can safely ditch old features like x87 floating point, which is much slower than more recent features. Also, function calls will more frequently pass by register. And I can think of one OS specific feature that is better on amd64: when MS ported to amd64 they took it as an opportunity to speed up the ABI for Structured Exception Handling (SEH). The x86 one incurred a time cost even for code that did not throw.

Of course none of this applies to arm/aarch64.

Re: Malloc Never Fails (2012)

#142

Post by amelius »

lalaithion wrote: post #136
amelius wrote: post #124

Earlier quoted context omitted.

Perhaps any write can fail, and if one does then it triggers some code which frees some pre-reserved space which allows the interpreter to continue, and properly unwind the stack.

I believe that the python interpreter creates a MemoryError at startup, and throws that when you run out of memory.

Ok, but while unwinding a try/except block, the interpreter might need some extra memory, I suppose, depending on the code in the except clause.

Re: Malloc Never Fails (2012)

#143

Post by shereadsthenews »

dragontamer wrote: post #139
shereadsthenews wrote: post #125

Earlier quoted context omitted.

What's wrong with the article is that malloc is not even a feature of Linux. There's mmap and brk, both of which have documented failure modes.

Most typical programmers will be using malloc, or some mechanism built on top of malloc (C++ new). MMap and brk are useful for certain situations (explicitly getting Huge Pages), but I don't think that the typical programmer necessarily needs to know about those. Since glibc malloc is built on top of mmap and brk, I think your distinction is mostly academic. For any programmer using Linux and glibc... malloc's failur…

It is a mischaracterization to say that C++ operator new is built on top of malloc. If your new is overridden by, say, tcmalloc, your program will never call malloc.

Re: Malloc Never Fails (2012)

#144

Post by mehrdadn »

klodolph wrote: post #79
mehrdadn wrote: post #63

Earlier quoted context omitted.

> If you have some specific part of the C standard in mind please do tell See here: https://news.ycombinator.com/item?id=20145604 Also note the POSIX standard: Upon successful completion with size not equal to 0, malloc() shall return a pointer to the allocated space. If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned. Otherwise, it shall return a null…

Allocated "space", i.e., virtual memory space. Malloc allocates virtual memory space, not physical RAM.

C has no notion of virtual memory. If memory is allocated for you that means you can read and write to it.

Re: Malloc Never Fails (2012)

#145

Post by mehrdadn »

dooglius wrote: post #104
mehrdadn wrote: post #27

Translation: This is a euphemism for saying Linux blatantly violates the language specification with no remorse. (Yes, I realize Linux is the kernel, etc.)

The C spec actually states at the beginning, "This International Standard does not specify ... the size or complexity of a program and its data that will exceed the capacity of any specific data-processing system or the capacity of a particular processor;"

Yes and that's why malloc can return null.

Re: Malloc Never Fails (2012)

#146

Post by mehrdadn »

klodolph wrote: post #81
mehrdadn wrote: post #65

Earlier quoted context omitted.

The standard literally could not care less if there's a kernel underneath. It doesn't care how malloc is implemented or what the kernel, if any, looks like "from its perspective". The perspective that has relevance is that of the caller of malloc. From your own quotes: > The pointer returned if the allocation succeeds is suitably aligned so that [...] > If the space cannot be allocated, a null pointer is returned. If…

The space IS allocated, it's just that this space (virtual memory) is not backed by physical RAM.

Try telling that to the C standard. It has no notion of virtual or physical memory. If it's allocated that means you can read/write to it, end of story.

Re: Malloc Never Fails (2012)

#147

Post by [deleted] »

dragontamer wrote: post #82
dooglius wrote: post #7

The title here is just blatantly false; there are certainly some scenarios in which it won't fail, but many in which it will. The author is aware of this since he fixes the claim toward the end: > To clarify, the surprising behaviour malloc has does not mean we should ignore its return value. We just need to be careful because malloc returning successfully does not always mean that we can use the requested memory. It…

> It's worth pointing out that if you turn overcommit off ... you will, in fact, get this guarantee But many Linux applications assume the overcommit behavior. Its far easier to just "go with the flow" or "When in Rome...". Overcommit is the programming culture of Linux and should be assumed when writing Linux apps.

[deleted]

Re: Malloc Never Fails (2012)

#148

Post by vbezhenar »

unilynx wrote: post #66
lmm wrote: post #51

Earlier quoted context omitted.

> If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine. If you have a system that handles independent requests, it might be more robust to model each request as its own process, and then simply a…

It's certainly more robust, but a lot of overhead for what should be a very uncommon situation. In eg an application server, requests themselves may be independent, but still share a lot of cached data - and as long as locking overhead doesn't overwhelm you, a single process is still the fastest way to share data between (worker) threads.

You can use pool of processes, so you don't have to launch new process for every request. Also you can and should use shared memory for shared data. In the end it's almost like threads, but with more safety.

Re: Malloc Never Fails (2012)

#149

Post by kazinator »

You can arrange for malloc to fail. If overcommit is disabled with the right sysctl parameters, then mmap will fail if there isn't enough physical memory to materialize the entire mapping. Even under overcommit, very large malloc requests that translate directly to large mmap requests can fail with a null return.

Re: Malloc Never Fails (2012)

#150

Post by kazinator »

mehrdadn wrote: post #27

Translation: This is a euphemism for saying Linux blatantly violates the language specification with no remorse. (Yes, I realize Linux is the kernel, etc.)

> Linux blatantly violates the language specification with no remorse.

We may be able to argue that it doesn't. ISO C says in 1. Scope, this:

This International Standard does not specify

[...]

— the size or complexity of a program and its data that will exceed the capacity of any specific data-processing system or the capacity of a particular processor.

It seems that these weasel words have an interpretation that can be bent around overcommitted memory allocation.

Post reply on HN