Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NUMA: Cores, memory, and the distance between them

edera.dev

Post reply on HN
Print view View original

31–36 of 36 posts

Re: NUMA: Cores, memory, and the distance between them

#31

Post by itemize123 »

jpecar wrote: post #14

I'm baffled by the fact that NUMA is still an issue in 2026. My impression is that this was all solved back in dotcom era already on those big SUNs. At least in HPC we solved this already in mid 2000s. Why is supposedly modern world still wasting time on this? Kernel these days exposes just about everything you would ever want to know about a system topology and every runtime should be making use of that information.…

what do you mean by this is solved?

Re: NUMA: Cores, memory, and the distance between them

#32

Post by inkyoto »

jpecar wrote: post #14

I'm baffled by the fact that NUMA is still an issue in 2026. My impression is that this was all solved back in dotcom era already on those big SUNs. At least in HPC we solved this already in mid 2000s. Why is supposedly modern world still wasting time on this? Kernel these days exposes just about everything you would ever want to know about a system topology and every runtime should be making use of that information.…

Sun was one of the few RISC design houses who stubbornly resisted NUMA in favour of SMP, which they have perfected in the hardware and in the software (Solaris). The «Solaris internals» book discusses the subject of SMP vs NUMA vs ccNUMA in the «Parallel systems architectures» section (3.2).

You might be thinking SGI who went in big on NUMA. IBM and HP have also built ccNUMA systems (e.g. HP Superdome and Superdome 2).

Re: NUMA: Cores, memory, and the distance between them

#33

Post by nanibot »

lukax wrote: post #2

NUMA can cause really crappy performance. We deployed a Go based LLM gateway in Kubernetes deployed on a server with hundreds of CPU cores. We didn't explicitly set GOMAXPROCS so Go runtime scheduled goroutines over different CPUs and it constantly used 200% CPU and GC was causing latency spikes. Then we set GOMAXPROCS 8 and all performance issues went away. Until recently Kubernetes didn't work well with NUMA.

I don't see how GOMAXPROCS alone can help here though. You would have to use Topology Manager (single-node policy) to avoid cross-NUMA allocations. This is in addition to other managers - Memory and CPU Manager.

CPU Pinning (via CPU Manager's Static Policy) will also be required to ensure your processes don't just get a CFS quota/share but are actually pinned onto specific CPU cores.

Re: NUMA: Cores, memory, and the distance between them

#34

Post by pjmlp »

lukax wrote: post #2

NUMA can cause really crappy performance. We deployed a Go based LLM gateway in Kubernetes deployed on a server with hundreds of CPU cores. We didn't explicitly set GOMAXPROCS so Go runtime scheduled goroutines over different CPUs and it constantly used 200% CPU and GC was causing latency spikes. Then we set GOMAXPROCS 8 and all performance issues went away. Until recently Kubernetes didn't work well with NUMA.

Some lessons are never lost it seems, back when Windows NT was recent, we had to lock threads/processes to specific CPUs on SMP machines (affinity), exactly for similar reasons.

Re: NUMA: Cores, memory, and the distance between them

#35

Post by sharts »

Why use NUMA if most of the time to get max performance you need locality and thus end up pinning things to a specific node to avoid the cost of crossing nodes. Moreover, you almost always must set this manually.

Wouldn’t it just be easier to skip it all together and just use single node architectures for much less cost and time?

Post reply on HN