Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Untangling Lifetimes: The Arena Allocator

rfleury.com

Post reply on HN
Print view View original

81–90 of 94 posts

Re: Untangling Lifetimes: The Arena Allocator

#81

Post by rfleury »

omnicognate wrote: post #9
kosolam wrote: post #3

Anyone read it all and can tldr any interesting insights?

It's a good technical explanation of how arena allocation (a useful and easily googleable memory management technique) works in practice, in C, along with: * A lot of arguing that all other ways of dealing with the problem (garbage collectors, RAII, etc) are misguided and make for worse programs. This is a lot more controversial than "arena allocation is useful" but I have some sympathy for it. He's not alone in thin…

> A general implication that he came up with this stuff himself, which he didn't.

How in the world did you get this from the article? Of course I didn't invent it, nor did I ever claim (or imply) that.

Re: Untangling Lifetimes: The Arena Allocator

#82

Post by rfleury »

SleepyMyroslav wrote: post #24

For me as non native speaker it is interesting choice of words with 'Untangling'. While arenas achieve the goal to simplify lifetime management by reducing number of different lifetimes they do it by effectively 'entangling' lifetimes. If arenas are reused they make UAF detection a problem that needs special care. Performance is usually great so it is used everywhere not only in gamedev.

A fair point on the title! I think you can see it from both perspectives. When I chose the title, I was thinking of "untangling" in this sense: In the traditional C program that overuses malloc/free, you end up in a forest of dynamic lifetimes all "tangled together". So, arenas are useful in "untangling those", in the same way that you might untangle cables for your PC by bundling them together.

Re: Untangling Lifetimes: The Arena Allocator

#83

Post by omnicognate »

rfleury wrote: post #81
omnicognate wrote: post #9

Earlier quoted context omitted.

It's a good technical explanation of how arena allocation (a useful and easily googleable memory management technique) works in practice, in C, along with: * A lot of arguing that all other ways of dealing with the problem (garbage collectors, RAII, etc) are misguided and make for worse programs. This is a lot more controversial than "arena allocation is useful" but I have some sympathy for it. He's not alone in thin…

> A general implication that he came up with this stuff himself, which he didn't. How in the world did you get this from the article? Of course I didn't invent it, nor did I ever claim (or imply) that.

I don't think you intended to create that impression, but I didn't go looking for it either. It's created by phrasing like

> In this post, I’ll be presenting an alternative to the traditional strategy of manual memory management that I’ve had success with: the arena allocator.

and

> My approach, on the other hand, is this: instead of assuming that malloc and free were the correct low-level operations, we can change the memory allocation interface...

Together with a complete absence of any reference to the history of the technique or any of the people besides you involved in its development.

It's not an accusation, but it's a clarification worth making as I genuinely think someone without knowledge of the subject could come away thinking you were presenting your own innovation.

Re: Untangling Lifetimes: The Arena Allocator

#84

Post by feoren »

rfleury wrote: post #79
feoren wrote: post #78

> But don’t worry, kiddo; in next class, you can return to your “safe” and “managed” padded-room languages where bugs and instabilities are “impossible” (or so they claim). Nobody has ever claimed this, ever, making this a major strawman. Does the author also consider everyone a childish padded-room pussy if they like seatbelts in cars and safety-related infrastructure on highways? The intro to an article should be t…

You are exactly the kind of person the intro was intended to filter out. Congratulations on your tantrum. > But not our Great Prophet Author. He knows the world is a Simple Place where Government Bad and Garbage Collection Bad and Universities Bad and Everyone Else Stupid and Everything Is Easy If You're Not An Idiot and Nothing Changes and Users Are Stupid and Programmers Are Stupid and Educators Are Stupid and ever…

The main problem with being an arrogant, condescending, narrow-minded asshole who thinks the world is a simple place and you're smarter than everyone else is actually ironically rather simple: you're wrong. About so, so, so many things in this world. And the problem with being proud of it, as you clearly are, is that you deprive yourself of the chance of ever being any less wrong.

But you go on intentionally filtering out professional software engineers with over a decade of experience if they have even the slightest difference of opinion about the subjects you write about. Clearly they have nothing to offer you: you already know everything anyway.

Re: Untangling Lifetimes: The Arena Allocator

#85

Post by rfleury »

feoren wrote: post #84
rfleury wrote: post #79

Earlier quoted context omitted.

You are exactly the kind of person the intro was intended to filter out. Congratulations on your tantrum. > But not our Great Prophet Author. He knows the world is a Simple Place where Government Bad and Garbage Collection Bad and Universities Bad and Everyone Else Stupid and Everything Is Easy If You're Not An Idiot and Nothing Changes and Users Are Stupid and Programmers Are Stupid and Educators Are Stupid and ever…

The main problem with being an arrogant, condescending, narrow-minded asshole who thinks the world is a simple place and you're smarter than everyone else is actually ironically rather simple: you're wrong . About so, so, so many things in this world. And the problem with being proud of it, as you clearly are, is that you deprive yourself of the chance of ever being any less wrong. But you go on intentionally filteri…

> Clearly they have nothing to offer you: you already know everything anyway.

I definitely don't know everything, and I am always learning from real professionals with real experience. But it's certainly true that you don't have anything to offer to me.

Re: Untangling Lifetimes: The Arena Allocator

#86

Post by abnercoimbre »

sfpotter wrote: post #25

This article is about 10 times longer than it needs to be. Also no need for the dripping condescension. I’m even among the most receptive to what he’s trying to say, but got exhausted after about 20% of the article and had to give up.

We're having Ryan at Handmade Seattle [0] this November to discuss memory management strategies (with other engineers across the memory-safety spectrum.)

You might prefer a public, conversational setting at the conference instead of a personal blog post. That said the article made useful technical points. And he wasn't hurling expletives or denigrating any individual.

[0] https://handmade-seattle.com/

Re: Untangling Lifetimes: The Arena Allocator

#87

Post by trap_goes_hot »

IshKebab wrote: post #4
kosolam wrote: post #3

Anyone read it all and can tldr any interesting insights?

This came up before. TL;DR: If you use arena allocators everywhere then you won't have any memory errors! I think the idea is a really interesting one but it would be better if he didn't wrap it in typical "I don't make mistakes" hubris. I mean I guess at least this time he has some more reason than "because I'm not stupid" but I think he's still wrong. His technique probably reduces the chance of memory errors, but…

As with everything, you need to use the tool for use-cases that it was designed for.

A downside of RAII is the hidden cost of unwinding, and predictability over CPU usage. If you have a resource that supports complicated internal transformations, and memory states, then you need an equally complicated destructor that can handle all those different states. If you're not careful with this you can get CPU spikes at scope-exits.

Re: Untangling Lifetimes: The Arena Allocator

#88

Post by adrian_b »

pixelfarmer wrote: post #58
adrian_b wrote: post #15

Earlier quoted context omitted.

On Linux and other UNIX-like operating systems, that would be mmap with the MAP_ANONYMOUS flag.

With Linux+glibc, malloc() makes use of different syscalls. Beside mmap(), there is also brk() in use: https://man7.org/linux/man-pages/man2/sbrk.2.html The big question about all that is when this memory is actually being released back to the OS, because by default it isn't really. That also means you can trigger OOM faults despite the process having free memory even (which the OS sees differently).

brk/sbrk are obsolete legacy functions.

Memory allocated with mmap is released with munmap.

brk/sbrk can increase the size of the data segment, thus allocating memory, but they can also decrease the size of the data segment, freeing the memory.

Whether calling the function free of the standard C library results sometimes in also invoking munmap or brk/sbrk to release memory to the operating system is obviously implementation dependent.

When malloc/free are bypassed and you get memory from the OS directly with mmap, to be used by a custom memory allocator, e.g. an arena allocator, then it is up to you to call munmap when the memory is no longer needed.

Re: Untangling Lifetimes: The Arena Allocator

#89

Post by jcparkyn »

doesntmeananyth wrote: post #43
kitd wrote: post #26

The author may be interested in Zig, where allocators are first-class components of the system. https://ziglang.org/documentation/0.9.1/#Choosing-an-Allocat... There is even an arena allocator provided (amongst others) by the std lib.

You may also be interested in this comment thread from the article page, where the author explains his opinion on Zig's approach to allocators: https://www.rfleury.com/p/untangling-lifetimes-the-arena-all...

I don't use zig, but it seems to me that functions can choose exactly which type of allocator they want to accept (just by changing the type of the allocator parameter), making his point invalid.

Re: Untangling Lifetimes: The Arena Allocator

#90

Post by quickthrower2 »

zozbot234 wrote: post #12
quickthrower2 wrote: post #10

Is there an equivalent insight but instead of C it is web dev and instead of Malloc it is JQuery and GC being React? In other words is there an elegant way to build a complex SPA with vanilla JS

> In other words is there an elegant way to build a complex SPA with vanilla JS Svelte and SolidJS. They add a compilation step that turns fully abstracted SPA framework code into vanilla JS doing efficient DOM operations. If you wanted something even closer to the speed of C, you could imagine a framework using WASM to generate arbitrarily complex raw HTML and add it to the DOM in a single operation. This would basi…

Thanks! Solid looks quite nice, looking at the docs it is like React but by making state getting a function, and tracking deps automatically, you change the perspective so that you don't need multiple renders (at least in the code you write, not sure about under the hood).
Post reply on HN