Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stupid Smart Pointers in C

blog.kevinalbs.com

Post reply on HN
Print view View original

121–130 of 174 posts

Re: Stupid Smart Pointers in C

#121

Post by legohead »

LegionMammal978 wrote: post #116
legohead wrote: post #113

Earlier quoted context omitted.

When something would normally be freed, it calls the memory manager's version of free(), which zeroes the memory and adds it back into the appropriate available list.

Isn't that just stacking your own allocator on top of the libc allocator, the same way the libc allocator is stacked on top of the OS's page mappings? It's often a sensible idea, of course, but I wouldn't describe it as "never freeing things", just substituting libc's malloc()/free() for your own. It's not like the libc allocator is doing something radically different from keeping lists of available and allocated mem…

libc allocator is considered "slow" for the extra overhead it has. I'll just leave it at that as you can GPT the rest. I did some of my own testing back in the days I wrote C, and mine was significantly faster than malloc/free.

It's also a way, I suppose, to ensure your program has the memory it needs, depending on how you design it. Mine basically allocates all the expected memory I need at the start, while still being able to grab more if it needs. This was more of an issue back when you shared a server with lots of others.

Re: Stupid Smart Pointers in C

#122

Post by LegionMammal978 »

legohead wrote: post #121
LegionMammal978 wrote: post #116

Earlier quoted context omitted.

Isn't that just stacking your own allocator on top of the libc allocator, the same way the libc allocator is stacked on top of the OS's page mappings? It's often a sensible idea, of course, but I wouldn't describe it as "never freeing things", just substituting libc's malloc()/free() for your own. It's not like the libc allocator is doing something radically different from keeping lists of available and allocated mem…

libc allocator is considered "slow" for the extra overhead it has. I'll just leave it at that as you can GPT the rest. I did some of my own testing back in the days I wrote C, and mine was significantly faster than malloc/free. It's also a way, I suppose, to ensure your program has the memory it needs, depending on how you design it. Mine basically allocates all the expected memory I need at the start, while still be…

Of course the libc allocator is slow, it's designed as a one-size-fits-all solution that's poorly tuned for most workloads and often doesn't get substantially updated for decades. By writing your own, you aren't avoiding an allocator, you're just avoiding the libc implementation of an allocator.

(In fact, most libcs let you just redefine the malloc(), free(), and realloc() functions to point to your own allocator instead of the default one, so you don't have to rewrite all the functions calling them. E.g., the mimalloc allocator [0] can be configured as a drop-in replacement for the libc allocator.)

[0] https://github.com/microsoft/mimalloc

Re: Stupid Smart Pointers in C

#123

Post by EPWN3D »

masklinn wrote: post #97
EPWN3D wrote: post #71

Earlier quoted context omitted.

defer is nice, but I really want the cleanup attribute since it could in theory by applied to the return type of a function. In other words you could have malloc return a pointer with the cleanup attribute that automatically frees it at end of scope if it's non-NULL. (And if you want to persist the pointer just assign to a different variable and zero out the one malloc gave you.)

> In other words you could have malloc return a pointer with the cleanup attribute that automatically frees it at end of scope if it's non-NULL. That is not, as far as I know, how __attribute__((cleanup)) works. It just invokes the callback when the value goes out of scope. So you can't have malloc return an implicitly cleanup'd pointer unless malloc is a macro, in which case you can do the same with a defer block.

Yeah that's why I said "in theory". You can't do that with the attribute today, but it'd be possible to add that behavior to a standardized variant of it.

Re: Stupid Smart Pointers in C

#125

Post by LegionMammal978 »

knorker wrote: post #118
legohead wrote: post #113

Earlier quoted context omitted.

When something would normally be freed, it calls the memory manager's version of free(), which zeroes the memory and adds it back into the appropriate available list.

You're just describing every allocator in the world, except many (most?) skip the zeroing part. libc already does that. What is it that yours is adding? I'd say 25 years ago you could write your own naive allocator, make just a couple of assumptions for your use case, and beat libc. But no more. One of the selling points of Java in the 90s was the compacting part. Because in the 90s fragmentation was a much bigger pr…

> Today the libc allocators have advanced by maybe tens of thousands of PhDs worth of theory and practice.

Have they, though? Looking at the blame on glibc's malloc.c, the most substantial change in the last 10 years has been the addition of memory tagging. Apart from that, I mainly just see a bunch of small tweaks to the tcache, fastbins, etc., and the basic logic is broadly the same as it was 20 years ago. Similarly, the MallocInternals wiki page [0] appears mostly as it did in 2016, except for some more explanations of the tcache added in 2018.

I can easily believe that lots of work has been done on malloc/free-style allocators, but from what I've seen of most libc allocators, they hardly stand at the forefront of this work. (Except for the ones that just vendor some standalone allocator library and keep it up to date. But that describes neither Windows nor Linux.)

And of course, if you're writing your own allocator, you can relax some of the constraints, e.g., you can drop most of the multithreading support and not have to fool around with locks and shared arenas and whatnot.

[0] https://sourceware.org/glibc/wiki/MallocInternals

Re: Stupid Smart Pointers in C

#126

Post by cryptonector »

rwmj wrote: post #5

Really, don't do this, it's a portability and safety nightmare (aside from C not being memory safe already). C programmers are better off with either of these two techniques: * Use __attribute__((cleanup)). It's available in GCC and Clang, and we hope will be added to the C spec one day. This is widely used by open source software, eg. in systemd. * Use a pool allocator like Samba's talloc ( https://talloc.samba.org/…

Sure, but TFA is a very clever hack -- disgusting, yes, impractical and non-portable, true, subject to sad limitations -of course-, but genius and entertaining.

The "tl;dr" summary is that `free_on_exit()` replaces the caller's return address with a trampoline that calls a `do_exit()` function that frees all the memory marked to be freed "on exit", and it uses its own stack of cleanup handler closures so-to-speak. When returning all the allocations are freed and then the original return address is returned to.

Re: Stupid Smart Pointers in C

#127

Post by cryptonector »

jay-barronville wrote: post #124

Like many C hacks, this is a fun one, but hacks like this should absolutely be avoided for any serious C code–the magic is simply not worth the potential bugs!

PuTTY is written in a Duff's Device co-routine hack on steroids that I'm sure someone at one point said "should absolutely be avoided for any serious C code–the magic is simply not worth the potential bugs!"! And PuTTY is no small program, and it's very successful.

One person's hack-that-should-be-avoided-at-all-costs can be someone else's secret sauce.

Re: Stupid Smart Pointers in C

#128

Post by forrestthewoods »

rwmj wrote: post #5

Really, don't do this, it's a portability and safety nightmare (aside from C not being memory safe already). C programmers are better off with either of these two techniques: * Use __attribute__((cleanup)). It's available in GCC and Clang, and we hope will be added to the C spec one day. This is widely used by open source software, eg. in systemd. * Use a pool allocator like Samba's talloc ( https://talloc.samba.org/…

> Use __attribute__((cleanup)). It's available in GCC and Clang

Really, don’t do this, it’s a portability nightmare. If you’re going to write C stick to code that easy to run under MSVC.

Re: Stupid Smart Pointers in C

#129

Post by cryptonector »

PeterWhittaker wrote: post #83

1) 2018 2) I recently discovered the implementation of free_on_exit won't work if called directly from main if gcc aligns the stack. In this case, main adds padding between the saved eip and the saved ebp, (example). I think this can be fixed some tweaking, and will update this article when it is fixed. I do not believe the article was updated, suggesting that the "tweaking" was far more complex than the author expec…

Right, and then there's interactions with StackGuard-like features.

Re: Stupid Smart Pointers in C

#130

Post by dima55 »

torlok wrote: post #104
Tewboo wrote: post #15

Smart pointers in C often feel like trying to force a square peg into a round hole. They’re powerful, but without native language support like C++, they can lead to more complexity than they solve.

I've heard enough "C is superior to C++" arguments from game developers who then go and use header structs for inheritance, or X macros, enums, and switch statements for virtual functions, to know that more complexity isn't an issue as long as people feel clever and validated.

X macros can do tons of stuff C++ can't. And if you stick with C, you avoid mountains of C++ problems, even with all the stuff you're talking about.
Post reply on HN