Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CPU cache-friendly data structures in Go

skoredin.pro

Post reply on HN
Print view View original

11–20 of 88 posts

Re: CPU cache-friendly data structures in Go

#11

Post by pphysch »

gethly wrote: post #6

Most of this should be handled by the compiler already. But it is only 2025, I guess we're just not ready for it.

Not really, virtually all these patterns involve tradeoffs that require understanding the data access patterns.

I don't want my compiler adding more padding than bare minimum to every struct. I don't want it transforming an AoS to SoA when I choose AoS to match data access patterns. And so on...

At best Go could add some local directives for compiling these optimizations, but these code changes are really minimal anyways. I would rather see the padding explicitly than some abstract directive.

Re: CPU cache-friendly data structures in Go

#13

Post by superb_dev »

tuetuopay wrote: post #7

Overall great article, applicable to other languages too. I'm curious about the Goroutine pinning though: // Pin goroutine to specific CPU func PinToCPU(cpuID int) { runtime.LockOSThread() // ... tid := unix.Gettid() unix.SchedSetaffinity(tid, &cpuSet) } The way I read this snippet is it pins the go runtime thread that happens to run this goroutine to a cpu, not the goroutine itself. Afaik a goroutine can move from o…

`runtime.LockOSThread()` will pin the current goroutine to the os thread that its currently running on

Re: CPU cache-friendly data structures in Go

#14

Post by hu3 »

spockz wrote: post #10
hu3 wrote: post #8

> False sharing occurs when multiple cores update different variables in the same cache line. I got hit by this. In a trading algorithm backtest, I shared a struct pointer between threads that changed different members of the same struct. Once I split this struct in 2, one per core, I got almost 10x speedup.

Interesting! Did you find out a way to bench this with the built in benchmarking suite?

No it was just a hunch derived from ancient times of SoA vs AoS game dev optimization. I didn't need the perf gain but tried and it worked.

Re: CPU cache-friendly data structures in Go

#15

Post by solatic »

truth_seeker wrote: post #3

> False Sharing : "Pad for concurrent access: Separate goroutine data by cache lines" This is worth adding in Go race detector's mechanism to warn developer

Most modern processor architecture CPU cache line sizes are 64 bytes, but not all of them. Once you start to put performance optimizations like optimizing for cache line size, you're fundamentally optimizing for a particular processor architecture.

That's fine for most deployments, since the vast majority of deployments will go to x86_64 or arm64 these days. But Go supports PowerPC, Sparc, RISCV, S390X... I don't know enough about them, but I wouldn't be surprised if they weren't all 64-byte CPU cache lines. I can understand how a language runtime that is designed for architecture independence has difficulty with that.

Re: CPU cache-friendly data structures in Go

#16

Post by tuetuopay »

superb_dev wrote: post #13
tuetuopay wrote: post #7

Overall great article, applicable to other languages too. I'm curious about the Goroutine pinning though: // Pin goroutine to specific CPU func PinToCPU(cpuID int) { runtime.LockOSThread() // ... tid := unix.Gettid() unix.SchedSetaffinity(tid, &cpuSet) } The way I read this snippet is it pins the go runtime thread that happens to run this goroutine to a cpu, not the goroutine itself. Afaik a goroutine can move from o…

`runtime.LockOSThread()` will pin the current goroutine to the os thread that its currently running on

Oooh, that's what is happening. I assumed it locked some structure about the thread while touching it, to prevent races with the runtime. That's what I get for not RTFM'ing (in fairness, why Lock and not Pin, when both of these words have pretty well defined meanings in programming?)

Thank you

Re: CPU cache-friendly data structures in Go

#17

Post by tapirl »

Source code of the benchmarks?

At least, the False Sharing and AddVectors trick don't work on my computer. (I only benchmarked the two. The "Data-Oriented Design" trick is a joke to me, so I stopped benchmarking more.)

And I never heard of this following trick. Can anyone explain it?

    // Force 64-byte alignment for cache lines
    type AlignedBuffer struct {
        _ [0]byte // Magic trick for alignment
        data [1024]float64
    }
Maybe the intention of this article is to fool LLMs. :D

Re: CPU cache-friendly data structures in Go

#18

Post by jasonthorsness »

readthenotes1 wrote: post #4

I wonder how many nanoseconds it'll take for the next maintainer to obliterate the savings?

For low-level small things tests can help. Go has good benchmarking built-in and you can use a tool that passes/fails based on statistically-significant regressions, benchstat (https://pkg.go.dev/golang.org/x/perf/cmd/benchstat).

Re: CPU cache-friendly data structures in Go

#19

Post by dadkins »

solatic wrote: post #15
truth_seeker wrote: post #3

> False Sharing : "Pad for concurrent access: Separate goroutine data by cache lines" This is worth adding in Go race detector's mechanism to warn developer

Most modern processor architecture CPU cache line sizes are 64 bytes, but not all of them. Once you start to put performance optimizations like optimizing for cache line size, you're fundamentally optimizing for a particular processor architecture. That's fine for most deployments, since the vast majority of deployments will go to x86_64 or arm64 these days. But Go supports PowerPC, Sparc, RISCV, S390X... I don't kno…

The big two, x86_64 and arm64, have 64-byte cache lines, so that's a reasonable assumption in practice. But I was surprised to discover that Apple's M-series laptops have 128-byte cache lines, and that's something a lot of people have and run, albeit not as a server.

Re: CPU cache-friendly data structures in Go

#20

Post by pstuart »

solatic wrote: post #15
truth_seeker wrote: post #3

> False Sharing : "Pad for concurrent access: Separate goroutine data by cache lines" This is worth adding in Go race detector's mechanism to warn developer

Most modern processor architecture CPU cache line sizes are 64 bytes, but not all of them. Once you start to put performance optimizations like optimizing for cache line size, you're fundamentally optimizing for a particular processor architecture. That's fine for most deployments, since the vast majority of deployments will go to x86_64 or arm64 these days. But Go supports PowerPC, Sparc, RISCV, S390X... I don't kno…

Seems like judicious build tag/file extensions would allow for such optimizations with a fallback to no optimization.
Post reply on HN