Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Untangling Lifetimes: The Arena Allocator

rfleury.com

Post reply on HN
Print view View original

31–40 of 94 posts

Re: Untangling Lifetimes: The Arena Allocator

#31

Post by GoblinSlayer »

saurik wrote: post #29
GoblinSlayer wrote: post #18

Earlier quoted context omitted.

It's a C++ myth that files and memory are the same problem, in reality files are an easier problem, cf java.

?! Java doesn't offer a reasonable solution for files... it is presumed they can be closed by the garbage collector calling "finalizers", but that results in problems such as holding file locks long after they should have been released (which is a correctness problem) and running out of file handles (a limited resource that Java does not track pressure against) due to this mechanism having no time horizon. The result…

There's no limit to perfection, but java approach is reasonable, because it works in practice.

Re: Untangling Lifetimes: The Arena Allocator

#32

Post by Joker_vD »

ordu wrote: post #17
Joker_vD wrote: post #13

Earlier quoted context omitted.

> The other issue he doesn't address is resource allocation. It's fine to just drop the memory of a lot of things I think there is a fundamental difference between memory and most other resources: (process's) memory is a) effectively an internal detail while other resources are externally visible (open files/sockets/etc); b) as almost a consequence, it's pretty much anonymous: when you ask for 1 MiB of memory, you as…

So do you argue that memory is different from point of view of resourse allocation/deallocation because it is much harder to exhaust memory than file descriptors, tcp ports and such like?

No, I am arguing that memory is (roughly speaking) an internal, private resource unlike most other things (files/sockets/pipes/processes/etc), and it's intended: other resources has state outside of the process which is used to communicate with other processes, or with specific hardware (e.g. GPU) or other machines. When you close() a file/socket, it's visible outside of the process, the other side receives inotify()/epoll() message, when a (child) process exits, its status and exit code change and SIGCHLD get sent, etc. Also, anonymity: if you need a socket connected to remote example.com:443, you want it to connected to example.com:443, not to something else; when you open a file/pipe, you want it to be able to communicate with someone else (in case of file, maybe with someone else from the past/future) — note how deeply this notion of sharing is built-in into file API by considering how difficult it was to design secure API for temp files which are files one doesn't want to share with anyone. But when you free() a block of memory... nothing outside of the process gets notified, really.

And that's why automated memory management (garbage collection) is a success story while attempts to blindly apply it to other resources failed: memory is different. Freeing other resources sends a very clear message to the external environment, including other processes, which is used for IPC while freeing memory doesn't.

Yes, in the end, memory in process is implemented by (shared) physical memory but modern OSes go to ridiculous length to support the illusion that it's unlimited and private. Yes, there is shared memory mappings which I explicitly leave outside the argument: those are externally visible, nameable, and hard to manage with GC.

Re: Untangling Lifetimes: The Arena Allocator

#33

Post by vitiral »

sfpotter wrote: post #25

This article is about 10 times longer than it needs to be. Also no need for the dripping condescension. I’m even among the most receptive to what he’s trying to say, but got exhausted after about 20% of the article and had to give up.

As someone already sold on arenas I appreciated both the depth and battle story style. Perhaps the author is mis-targeting: they are preaching to the choir instead of converting the sinners =]

Re: Untangling Lifetimes: The Arena Allocator

#34

Post by vitiral »

GoblinSlayer wrote: post #18
DougBTX wrote: post #7

The author should take another look at RIIA. The first big difference is that it works with non-memory resources too, for example open/close for files. The second is that it makes heap allocated data feel a lot like stack allocated data, since the resources are ultimately owned by variables on the stack. With “move semantics” resources can be passed as arguments and returned by functions too, so the example of lifeti…

It's a C++ myth that files and memory are the same problem, in reality files are an easier problem, cf java.

There is a separate solution for files: defer. Defer can be implemented trivially in assembly by putting the deferred block on the return stack (then the defer block simply returns). This glosses over handling the local stack, but it's not impossible. It would be a relatively minor feature to add to C IMO

Re: Untangling Lifetimes: The Arena Allocator

#35

Post by vitiral »

SleepyMyroslav wrote: post #24

For me as non native speaker it is interesting choice of words with 'Untangling'. While arenas achieve the goal to simplify lifetime management by reducing number of different lifetimes they do it by effectively 'entangling' lifetimes. If arenas are reused they make UAF detection a problem that needs special care. Performance is usually great so it is used everywhere not only in gamedev.

They are untangled from each other.

UAF would be the same difficulty as allocated arrays I would think?

Re: Untangling Lifetimes: The Arena Allocator

#36

Post by vitiral »

andreareina wrote: post #21

Isn't stack/arena allocation still vulnerable to dangling pointers?

Of course, it's still manual memory and pointer management after all. You could manually assign a pointer to 0xDEADBEEF if you want, or you could tell the arena to pop data that you are still using.

The point is not perfection, it is simplicity and mapping tools to requirements.

Re: Untangling Lifetimes: The Arena Allocator

#37

Post by omginternets »

DougBTX wrote: post #7

The author should take another look at RIIA. The first big difference is that it works with non-memory resources too, for example open/close for files. The second is that it makes heap allocated data feel a lot like stack allocated data, since the resources are ultimately owned by variables on the stack. With “move semantics” resources can be passed as arguments and returned by functions too, so the example of lifeti…

Could someone please explain the usefulness of RAII? As I understand it, RAII refers to the initialization semantics of certain languages, most famously C++. If I remember correctly, it effectively states that variable declarations amount to reserving memory resources and nothing else; memory is not zeroed, for instance.

My question is: what makes this so useful? Looking at wikipedia, the focus seems to be on lifetime guarantees, but I'm having trouble understanding how these are useful in practice. Contrasting this with a non-RAII language like Go might be helpful.

Re: Untangling Lifetimes: The Arena Allocator

#38

Post by jcelerier »

GoblinSlayer wrote: post #18
DougBTX wrote: post #7

The author should take another look at RIIA. The first big difference is that it works with non-memory resources too, for example open/close for files. The second is that it makes heap allocated data feel a lot like stack allocated data, since the resources are ultimately owned by variables on the stack. With “move semantics” resources can be passed as arguments and returned by functions too, so the example of lifeti…

It's a C++ myth that files and memory are the same problem, in reality files are an easier problem, cf java.

cf Java? Both in it and c# forgetting to close handles has been one of the most common mistakes I've seen. It pretty much never happens on c++.

Re: Untangling Lifetimes: The Arena Allocator

#39

Post by sfpotter »

vitiral wrote: post #33
sfpotter wrote: post #25

This article is about 10 times longer than it needs to be. Also no need for the dripping condescension. I’m even among the most receptive to what he’s trying to say, but got exhausted after about 20% of the article and had to give up.

As someone already sold on arenas I appreciated both the depth and battle story style. Perhaps the author is mis-targeting: they are preaching to the choir instead of converting the sinners =]

If they’re preaching, it’s quite a long and boring sermon, IMO ;-) Battle stories should be exciting

Re: Untangling Lifetimes: The Arena Allocator

#40

Post by williamcotton »

creata wrote: post #30
williamcotton wrote: post #27

Earlier quoted context omitted.

As a C programmer who uses memory arenas quite frequently can you expand on what you know about Rust’s current or future support for such a feature? I’d like to start writing new projects in Rust and I’d like to continue to use this approach to memory management!

You can of course use arenas in Rust just like in C. I think the parent was talking about support for using arenas as the memory for stuff like built-in containers (dynamic arrays, hash maps, etc.): https://github.com/rust-lang/wg-allocators

Thank you! Yeah, something like bumpalo would work just fine right now but for the future this is a great interface:

https://docs.rs/bumpalo/latest/bumpalo/#nightly-rust-allocat...

  use bumpalo::Bump;
  
  let bump = Bump::new();
  
  let mut v = Vec::new_in(&bump);
  v.push(0);
  v.push(1);
  v.push(2);
Post reply on HN