If you are worrying about cache structure latencies in Go, maybe you should just be using Rust or Zig instead that implicitly handle this better.
CPU cache-friendly data structures in Go
21–30 of 88 posts
Re: CPU cache-friendly data structures in Go
#22Most of this should be handled by the compiler already. But it is only 2025, I guess we're just not ready for it.
Are you thinking of some sort of annotation the compiler could read and handle?
Because if a compiler starts automatically padding all my structures to put all of the members on their own cache line I'm going to be quite peeved. It would be easy for it to do, yes, but it would be wrong 99%+ of the time.
A far more trenchant complaint is that Go won't automatically sort struct members if necessary to shrink them and you have to apply a linter to get that at linting time if you want it.
Re: CPU cache-friendly data structures in Go
#23Most of this should be handled by the compiler already. But it is only 2025, I guess we're just not ready for it.
or at least Linter should catch this https://golangci-lint.run/docs/linters/
I think it's really beyond the power of a linter to understand when this would matter. It'd be issuing warnings on almost every struct out there saying "these two members share a cache line" which you almost never care about.
Re: CPU cache-friendly data structures in Go
#24If you are worrying about cache structure latencies in Go, maybe you should just be using Rust or Zig instead that implicitly handle this better.
Not necessarily: you can go quite far with Go alone. It also makes it trivial to run "green threads" code, so if you need both (decent) performance and easy async code then Go still might be a good fit. Despite Go being pretty high level GC language on the surface it actually allows you to control stuff like struct layout, CPU affinity, etc, which typically matter more for performance than just a programming language of choice. There's a reason why e.g. VictoriaMetrics is still in Go even though they could've easily chosen any other language too
Re: CPU cache-friendly data structures in Go
#25Source code of the benchmarks? At least, the False Sharing and AddVectors trick don't work on my computer. (I only benchmarked the two. The "Data-Oriented Design" trick is a joke to me, so I stopped benchmarking more.) And I never heard of this following trick. Can anyone explain it? // Force 64-byte alignment for cache lines type AlignedBuffer struct { _ [0]byte // Magic trick for alignment data [1024]float64 } Mayb…
I can't find any claim anywhere else about the [0]byte trick, and in fact my own experiments in the playground show that it doesn't do anything.
If you embed an AlignedBuffer in another struct type, with smaller fields in front of it, it doesn't get 64-byte alignment.
If you directly allocate an AlignedBuffer (as a stack var or with new), it seems to end up page-aligned (the allocator probably has size classes) regardless of the presence of the [0]byte field.
https://go.dev/play/p/Ok7fFk3uhDn
Example output (w is a wrapper, w.b is the field in the wrapper, x is an allocated int32 to try to push the heap base forward, b is an allocated AlignedStruct):
&w = 0xc000126000
&w.b = 0xc000126008
&x = 0xc00010e020
&b = 0xc000138000
Re: CPU cache-friendly data structures in Go
#26Really cool article this is the kind of stuff I still come to HN for.
Re: CPU cache-friendly data structures in Go
#27If you are sweating this level of performance, are larger gains possible by switching to C, C++, Rust? How is Rust for micro-managing memory layouts?
Re: CPU cache-friendly data structures in Go
#28Most of this should be handled by the compiler already. But it is only 2025, I guess we're just not ready for it.
I'm not sure if golang has the same fundamental issues in common use, but in e.g. C you don't want the compiler reordering your structs or adding arbitrary padding because that makes it incompatible with other in-memory representations - e.g. if you're using shared memory with another process that hasn't received the same optimizations, if you're loading raw data into memory/using mmap, etc.
Likewise, one of the examples is moving from an array of structs to a struct of arrays; that's a lot more complex of a code reorganization than you'd want a compiler doing.
It would be good to have a static analyzer that could suggest these changes, but, at least in many cases, you don't want them done automatically.
Re: CPU cache-friendly data structures in Go
#29Most of this should be handled by the compiler already. But it is only 2025, I guess we're just not ready for it.
Do compiler ( GCC / llvm ) actually do that?
Re: CPU cache-friendly data structures in Go
#30Most of this should be handled by the compiler already. But it is only 2025, I guess we're just not ready for it.
Not really, virtually all these patterns involve tradeoffs that require understanding the data access patterns. I don't want my compiler adding more padding than bare minimum to every struct. I don't want it transforming an AoS to SoA when I choose AoS to match data access patterns. And so on... At best Go could add some local directives for compiling these optimizations, but these code changes are really minimal any…
I could imagine some kind of compiler declaration in C that would do something like specify break points - sort of like page breaks - for structs, or tell the compiler to automatically pad structs out so that components are on page boundaries, cache line boundaries, etc. Sort of "If we're not properly aligned, add whatever padding you think is best here".
I guess this is largely provided by std::hardware_destructive_interference_size in C++17, but I'm not sure if there are other language equivalents.
https://en.cppreference.com/w/cpp/thread/hardware_destructiv...