Measuring CPU core-to-core latency

It looks kinda like the color scales are normalized to just-this-CPU's latency? It would be neater if the scale represented the same values among CPUs. Or rather, it would be neat if there were an additional view for this data that could make it easier to compare among them.

I think the differences are really interesting to consider. What if the scheduler could consider these designs when weighing how to schedule each task? Either statically or somehow empirically? I think I've seen sysfs info that describes the cache hierarchies, so maybe some of this info is available already. That nest [1] scheduler was recently shared on HN, I suppose it may be taking advantage of some of these properties.

[1] https://dl.acm.org/doi/abs/10.1145/3492321.3519585

Re: Measuring CPU core-to-core latency

#6

Post by SkipperCat » Sun, Sep 18, 2022, 7:30 PM UTC

fideloper wrote: post #3

Because I’m ignorant: What are the practical take aways from this? When is a cpu core sending a message to another core?

In HFT, we typically pin processes to run on a single isolated core (on a multicore machine). That allows the process to avoid a lot of kernel and other interrupts which could cause the process to not operate in a low latency manner.

If we have two of these processes, each on separate cores, and they occasionally need to talk to each other, then knowing the best choice of process/core location can keep the system operating in the lowest latency setup.

So, an app like this could be very helpful for determining where to place pinned processes onto specific cores.

There's also some common rules-of-thumb such as, don't put pinned processes that need to communicate on cores that are separated by the QPI, that just adds latency. Make sure if you're communicating with a NIC to find out which socket has the shortest path on the PCI bus to that NIC and other fun stuff. I never even thought about NUMA until I started to work with folks in HFT. It really makes you dig into the internals of the hardware to squeeze the most out of it.

Re: Measuring CPU core-to-core latency

#7

Post by ls65536 » Sun, Sep 18, 2022, 8:08 PM UTC

SkipperCat wrote: post #6

fideloper wrote: post #3

Because I’m ignorant: What are the practical take aways from this? When is a cpu core sending a message to another core?

In HFT, we typically pin processes to run on a single isolated core (on a multicore machine). That allows the process to avoid a lot of kernel and other interrupts which could cause the process to not operate in a low latency manner. If we have two of these processes, each on separate cores, and they occasionally need to talk to each other, then knowing the best choice of process/core location can keep the system ope…

In general this makes sense, but I think you need to be careful in some cases where the lowest latency between two logical "cores" is likely to be between those which are SMT siblings on the same physical core (assuming you have an SMT-enabled system). These logical "cores" will be sharing much of the same physical core's resources (such as the low-latency L1/L2 and micro-op caches), so depending on the particular workload, pinning two threads to these two logical "cores" could very well result in worse performance overall.

Re: Measuring CPU core-to-core latency

#8

Post by rigtorp » Sun, Sep 18, 2022, 8:14 PM UTC

I have something similar but in C++: https://github.com/rigtorp/c2clat

Re: Measuring CPU core-to-core latency

#9

Post by slabity » Sun, Sep 18, 2022, 8:28 PM UTC

ls65536 wrote: post #7

SkipperCat wrote: post #6

Earlier quoted context omitted.

In HFT, we typically pin processes to run on a single isolated core (on a multicore machine). That allows the process to avoid a lot of kernel and other interrupts which could cause the process to not operate in a low latency manner. If we have two of these processes, each on separate cores, and they occasionally need to talk to each other, then knowing the best choice of process/core location can keep the system ope…

In general this makes sense, but I think you need to be careful in some cases where the lowest latency between two logical "cores" is likely to be between those which are SMT siblings on the same physical core (assuming you have an SMT-enabled system). These logical "cores" will be sharing much of the same physical core's resources (such as the low-latency L1/L2 and micro-op caches), so depending on the particular wo…

SMT is usually disabled in these situations to prevent it from being a concern.

Re: Measuring CPU core-to-core latency

#10

Post by suprjami » Sun, Sep 18, 2022, 8:30 PM UTC

SkipperCat wrote: post #6

fideloper wrote: post #3

Because I’m ignorant: What are the practical take aways from this? When is a cpu core sending a message to another core?

In HFT, we typically pin processes to run on a single isolated core (on a multicore machine). That allows the process to avoid a lot of kernel and other interrupts which could cause the process to not operate in a low latency manner. If we have two of these processes, each on separate cores, and they occasionally need to talk to each other, then knowing the best choice of process/core location can keep the system ope…

I'm surprised how much crossing NUMA nodes can affect performance. We've seen NICs halve their throughout with (intentionally) wrong setups.

I think of NUMA nodes as multiple computers which just happen to share a common operating system.