Live data from Hacker News

Golang's big miss on memory arenas

avittig.medium.com

91–100 of 157 posts

Re: Golang's big miss on memory arenas

#91
>If you choose lower-level languages like Rust, your team will spend weeks fighting the borrow checker, asynchronicity, and difficult syntax.

It's interesting the author decides to describe Rust in this way, but then spends the next 90% of the article lambasting the Go authors for having the restraint to not turn Go into Rust.

Arenas are simple to write, and if you need one, there are a lot of implementations available. If you want the language to give you complete flexibility on memory allocations then Go is the wrong language to use. Rust and Zig are right there, you pay upfront from that power with "difficult syntax".

Re: Golang's big miss on memory arenas

#92
post #50

A better route for something like Go IMO is to move to a compacting collector, this would allow them to move to a bump allocator like Java for super fast allocations and would make deallocation effectively "free" by only moving live objects. They may need to make it generational so they aren't constantly moving long lived objects, but that is a memory vs cpu trade off (could be one more GC flag?). If I recall, the pr…

Go exposes raw pointers to the programmer, and its current GC is entirely non-moving. Even excluding cgo, I think a moving one would probably break real programs that rely on pointer values.

Re: Golang's big miss on memory arenas

#93
I think the overall sentiment with this post is sound, but arenas aren't the answer to Go's performance challenges. From my perspective, possibly in an effort to keep the language simple, Go's designers didn't care about performance. 'let the GC handle it' was the philosophy and as a result you see poor design choices all the way through the standard library. And the abstracting everything through interfaces then compounds the issue because the escape compiler can't see through the interface. The standard library is just riddled with unnecessary allocations. Just look at the JSON parser for instance and the recent work to improve it.

There is some interesting proposals on short term allocations, being able to specify that a local allocation will not leak.

Most recently, I've been fighting with the ChaCha20-Poly1305 implementation because someone in their 'wisdom' added a requirement for contiguous memory for the implementation, including extra space for a tag. Both ChaCha20 and Poly1305 are streaming algorithms, but the go authors decide 'you cannot be trusted' - here's a safe one-shot interface for you to use.

Go really needs a complete overhaul of their Standard Library to fix this, but I can't see this ever getting traction due to the focus on not breaking anything.

Go really is a great language, but should include performance / minimise the GC burden as a key design consideration for it's APIs.

Re: Golang's big miss on memory arenas

#94
Implicit context [1] was one of the coolest features of a programming language I’ve ever seen that no one has ever implemented. And I’m really not sure why. Not just Go but most languages have this context passing problem with varying degrees of solution quality, making this implicit and built in could have opened up so many possibilities, more than just arenas.

[1]: https://youtu.be/ciGQCP6HgqI

Re: Golang's big miss on memory arenas

#95
post #54
post #53

Earlier quoted context omitted.

Basically what Odin does yea?

Hadn’t known about Odin but yes! > Operations such as new, free and delete by default will use context.allocator, which can be overridden by the user. When an override happens all called procedures will inherit the new context and use the same allocator. https://pkg.odin-lang.org/core/mem/

This sounds a lot like having an allocator as a parameter object a la Scheme[0]. Really cool!

[0] https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-6.html#...

Re: Golang's big miss on memory arenas

#96
This does make me appreciate some of the decisions that Zig has made, about passing allocators explicitly and also encouraging the use of the ArenaAllocator for most programs.

Since Zig built up the standard library where you always pass an allocator, they avoided the problem that the article mentions, about trying to retrofit Go's standard library to work with an arena allocator.

Although, that's not the case for IO in Zig. The most recent work has actually been reworking the standard library to be where you explicitly pass IO like you pass an allocator.

But it's still a young language so it's still possible to rework it.

I really do enjoy using the arena allocator. It makes things really easy, if your program follows a cyclical pattern where you allocate a bunch of memory and then when you're done just free the entire arena

Re: Golang's big miss on memory arenas

#97
post #93

I think the overall sentiment with this post is sound, but arenas aren't the answer to Go's performance challenges. From my perspective, possibly in an effort to keep the language simple, Go's designers didn't care about performance. 'let the GC handle it' was the philosophy and as a result you see poor design choices all the way through the standard library. And the abstracting everything through interfaces then com…

I agree about nearly all of this, but in my fantasy I think the 'unsafe' library should be how to break the abstraction layer and adjust things directly when a good language model isn't provided.

JSON's just a nightmare though. The inane legacy of UCS2 / UTF16 got baked into Unicode 8, and UTF16 escapes into JSON.

Re: Golang's big miss on memory arenas

#98

Man this person is mediocre at best. You can do fully manual memory management in Go if you want. The runtime is full of tons of examples where they have 0-alloc, Pools, ring buffers, Assembly, and tons of other tricks. If you really want an arena like behavior you could allocate a byte slice and use unsafe to cast it to literally any type. But like… the write up completely missed that manual memory management exists…

> You can do fully manual memory management in Go if you want. The runtime is full of tons of examples where they have 0-alloc, Pools, ring buffers, Assembly, and tons of other tricks.

The runtime only exposes a small subset of what it uses internally and there's no stable ABI for runtime internals. If you're lucky to get big enough and have friends they might not break you, some internal linkage is being preserved, but in the general case for a general user, nope. Updates might make your code untenable.

> If you really want an arena like behavior you could allocate a byte slice and use unsafe to cast it to literally any type.

AIUI the prior proposals still provided automated lifetime management, though that's related to various of the standing concerns, so you can't match that from "userspace" of go, finalizers don't get executed on a deterministic schedule. Put simply: that's not the same thing.

As someone else points out this is also much more fraught with error than just typing what you described. On top of the GC issue pointed out already, you'll also hit memory model considerations if you're doing any concurrency, which if you actually needed to do this surely you are. Once you're doing that you'll run into the issue, if you're trying to compete with systems languages, that Go only provides a subset of the platform available memory model, in the simplest form it only offers acq/rel atomic semantics. It also doesn't expose any notion of what thread you're running on (which can change arbitrarily) or even which goroutine you're running on. This limits your design space quite significantly at the bounds your performance for high frequency small region operations. I'd actually hazard an educated guess that an arena written as you casually suggest would perform extremely poorly at any meaningful scale (lets say >=32 cores, still fairly modest).

> You could argue that C++ RAII overhead is “bounded performance” compared to C. Or that C’s stack frames are “bounded performance” compared to a full in-register assembly implementation of a hot loop. > But that’s bloody stupid. Just use the right tool for the job and know where the tradeoffs are, because there’s always something. The tradeoff boundary for an individual project or person is just arbitrary.

Sure, reducto ad absurdum, though I typically would optimize against the (systems language) compiler long before I drop to assembly, it's 2025 systems compilers are great and have many optimizations, intrinsics and hints.

> Man this person is mediocre at best.

Harsh, I think the author is fine really. I think their most significant error isn't in missing or not discussing difficult other things they could do with Go, it's seemingly being under the misconception prior to the Arena proposal that Go actually cedes control for lower level optimization. It doesn't, and it never has, and it likely never will (it will gain other semi-generalized internal optimizations over time, lots of work goes into that).

In some cases you can hack some in on your own, but Go is not well placed as a "systems language" if you mean by that something like "competitive efficiency at upper or lower bound scale tasks", it is much better placed as a framework for writing general purpose servers at middle scales. It's best placed on systems that don't have batteries, and that have plenty of ram. It'll provide you with a decent opportunity to scale up and then out in that space as long as you pay attention to how you're doing along the way. It'll hurt if you need to target state of the art efficiency at extreme ends, and very likely block you wholesale.

I'm glad Go folks are still working on ideas to try to find a way for applications to get some more control over allocations. I'm also not expecting a solution that solves my deepest challenges anytime soon though. I think they'll maybe solve some server cases first, and that's probably good, that's Go's golden market.

Re: Golang's big miss on memory arenas

#99
post #92
post #50

A better route for something like Go IMO is to move to a compacting collector, this would allow them to move to a bump allocator like Java for super fast allocations and would make deallocation effectively "free" by only moving live objects. They may need to make it generational so they aren't constantly moving long lived objects, but that is a memory vs cpu trade off (could be one more GC flag?). If I recall, the pr…

Go exposes raw pointers to the programmer, and its current GC is entirely non-moving. Even excluding cgo, I think a moving one would probably break real programs that rely on pointer values.

Yes, there's a case to be made that exposing "real" pointers in a GC'd language was a substantial mistake, but I guess it simplified _some_ parts of FFI. The trade-off so far maybe is fine, but it is a shame that there are certain things that can't be done without introducing new substantial costs. Maybe the compiler could learn to do something suuuper clever like recognize when pointers are being used non-transparently and automatically pin those, seems fraught with potential error though, trivial example being stuff like &a[0] (that ones easier to catch, others might not be).

Re: Golang's big miss on memory arenas

#100
post #25

My guess is that when you measure, an arena is not worth the trouble when you run a generational GC, which essentially uses an arena for the eden space already. And if you have an arena, it's probably very short lived and would otherwise live entirely in eden.

Go's GC is not generational.

It's not, but joining the two comments together sync.Pool is often close to what you want for a subset of cases, and it's sort of a locality biased generational storage (without actually providing you strong long-term guarantees that it is that).
Post reply on HN