Why malloc always does more than I asked for?
ssenthilnathan3.github.io
Why malloc always does more than I asked for?
1–10 of 52 posts
Re: Why malloc always does more than I asked for?
#2I've read some allocator walkthroughs before but I thought that this line stood out:
"to get individual free() working, the allocator needs to remember something about every allocation it handed out. and that’s the moment metadata stops being optional."
That's just a very nice distillation of an important concept.
Re: Why malloc always does more than I asked for?
#3I've read some allocator walkthroughs before but I thought that this line stood out: "to get individual free() working, the allocator needs to remember something about every allocation it handed out. and that’s the moment metadata stops being optional." That's just a very nice distillation of an important concept.
glad you found it useful :)
Re: Why malloc always does more than I asked for?
#4[deleted]
Re: Why malloc always does more than I asked for?
#5I've read some allocator walkthroughs before but I thought that this line stood out: "to get individual free() working, the allocator needs to remember something about every allocation it handed out. and that’s the moment metadata stops being optional." That's just a very nice distillation of an important concept.
This becomes particularly important when the allocation and metadata cannot (or should not) be stored in the same address space. An example of this is allocating memory on GPUs.
The conventional approach for allocating memory on GPUs for games and other applications is to use a real-time allocator such as TLSF. However, it is not usually discussed that TLSF is real-time because it stores metadata in-band. It is possible to create a variant of TLSF that preserves its real-time properties while storing metadata out-of-band, but this requires careful consideration.
Re: Why malloc always does more than I asked for?
#6I've read some allocator walkthroughs before but I thought that this line stood out: "to get individual free() working, the allocator needs to remember something about every allocation it handed out. and that’s the moment metadata stops being optional." That's just a very nice distillation of an important concept.
Oh? How do you think munmap does it?
If you can convince the caller to keep track of that metadata themselves you obviously don’t need to. That can be important.
Something I noticed is that _very often_ the code that is calling malloc(n) is keeping track of n somehow for its own reasons (bounds checking, grow/gap pointers, etc) so merging the value halves stack churn and it’s an easy win.
Re: Why malloc always does more than I asked for?
#7I've read some allocator walkthroughs before but I thought that this line stood out: "to get individual free() working, the allocator needs to remember something about every allocation it handed out. and that’s the moment metadata stops being optional." That's just a very nice distillation of an important concept.
In the other words, free() is a flawed API. :-)
Re: Why malloc always does more than I asked for?
#8> so alloc() doesn’t just need to hand back a pointer. it needs to hand back a pointer that’s correctly aligned for whatever type the caller is about to store there.
Malloc doesn't know the required alignment (because has no idea what the type is, everything is cast through void). So all malloc implementations have a minimum alignment guarantee. Typically 16 bytes these days on x86, as that means even 128bit SSE values will end up aligned by default.
You couldn't go below the sizeof(void ) anyway, the backpointer needs to aligned too.
The padding only happens when you use memalign or aligned_malloc to specify a much larger alignment.
Re: Why malloc always does more than I asked for?
#9I've read some allocator walkthroughs before but I thought that this line stood out: "to get individual free() working, the allocator needs to remember something about every allocation it handed out. and that’s the moment metadata stops being optional." That's just a very nice distillation of an important concept.
Oh? How do you think munmap does it? If you can convince the caller to keep track of that metadata themselves you obviously don’t need to. That can be important. Something I noticed is that _very often_ the code that is calling malloc(n) is keeping track of n somehow for its own reasons (bounds checking, grow/gap pointers, etc) so merging the value halves stack churn and it’s an easy win.
That's why C23 introduce free_sized [0].
Re: Why malloc always does more than I asked for?
#10Why bother with dynamic padding and a back pointer? That wastes at least 8 bytes.
You might as well always align to 8 bytes and make your header a multiple of 8.