Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring CPU core-to-core latency

github.com

Post reply on HN
Print view View original

51–60 of 96 posts

Re: Measuring CPU core-to-core latency

#52

Post by bhedgeoser »

On a 5950x, the latencies for core 0 are very high if SMT is enabled, I wonder why that is?

         0       1  
    0
    1   26±0
    2   26±0    17±0
    3   27±0    17±0
    4   32±0    17±0
    5   29±0    19±0
    6   32±0    18±0
    7   31±0    17±0
    8  138±1    81±0
    9  138±1    83±0
   10  139±1    80±0
   11  136±1    84±0
   12  134±1    83±0
   13  137±1    80±0
   14  136±1    84±0
   15  139±1    84±0
   16   16±0    16±0
   17   28±0     8±0
   18   33±0    17±0
   19   29±0    16±0
   20   28±0    17±0
   21   29±0    19±0
   22   32±0    18±0
   23   31±0    17±0
   24  137±1    81±0
   25  140±1    79±0
   26  143±1    80±0
   27  138±1    82±0
   28  139±1    82±0
   29  139±1    81±0
   30  142±1    82±0
   31  142±1    84±0

Re: Measuring CPU core-to-core latency

#54

Post by bee_rider »

lucb1e wrote: post #39
SkipperCat wrote: post #6

Earlier quoted context omitted.

In HFT, we typically pin processes to run on a single isolated core (on a multicore machine). That allows the process to avoid a lot of kernel and other interrupts which could cause the process to not operate in a low latency manner. If we have two of these processes, each on separate cores, and they occasionally need to talk to each other, then knowing the best choice of process/core location can keep the system ope…

What kind of "talking" are we talking about? I thought most IPC works somehow via shared memory under the hood rather than CPU cores actually communicating, how would you even do that?

"Shared memory" is really more of a description of the memory model that is exposed to the programmer, rather than the hardware.

Under the hood, there are caches -- sometimes memory addresses live in a cache above you because you put them there, sometimes they live in a cache above you because a neighboring core that shares your cache put them there, sometimes they live in RAM, sometimes they live in another cache on your chip and you have to ask for them through the on-chip network. The advice I have been given (as a non-HFT guy) is just to try not to mess around to much with the temporal locality, pin threads to cores, and let the hardware handle the rest.

Re: Measuring CPU core-to-core latency

#55

Post by snvzz »

>This software is licensed under the MIT license

Maybe consider including a MIT license in the repository.

Legally, that's a bit more sane than having a line in the readme.

In practice, github will recognize your license file and show the license in the indexes an d in the right column of your repository's main page.

Re: Measuring CPU core-to-core latency

#56

Post by mrtksn »

jtorsella wrote: post #28

If anyone is interested, here are the results on my M1 Pro running Asahi Linux: Min: 48.3 Max: 175.0 Mean: 133.0 I’ll try to copy the exact results once I have a browser on Asahi, but the general pattern is most pairs have >150ns and a few (0-1; 2-3,4,5; 3-4,5; 4-5; 6-7,8,9; 7-8,9; 8-9) are faster at about 50ns. Edit: The results from c2clat (a little slower, but the format is nicer) are below. CPU 0 1 2 3 4 5 6 7 8…

M1 on Macbook air on macOS Ventura beta 7:

min/max/mean - 34.6ns/57.7ns/39.1ns

       0       1       2       3       4       5       6       7   
  0
  1   58±6 
  2   48±6    46±6 
  3   39±6    37±6    36±6 
  4   41±6    38±6    42±6    38±6 
  5   35±6    37±6    37±6    36±6    36±6 
  6   38±6    39±6    36±6    40±6    40±6    38±6 
  7   39±6    38±6    36±6    37±6    38±6    36±6    38±6

Re: Measuring CPU core-to-core latency

#59

Post by Veliladon »

jeffbee wrote: post #25

Fails to build from source with Rust 1.59 so I tried the C++ `c2clat` from elsewhere in the thread. Quite interesting on Alder Lake, because the quartet of Atom cores has uniform latency (they share an L2 cache and other resources) while the core-to-core latency of the Core side of the CPU varies. Note that the way these are logically numbers is 0,1 are SMT threads of the first core and so forth through 14-15. 16-19…

It's probably the ring bus topology. As you pointed out E cores are directly connected to L2 and only one stop away from each other on the ring bus. P cores can be up to 6 stops away from each other because there's 4 on each side, an E core cluster on each side, and a system agent (PCIe/DRAM) or iGPU stop in between each side.

Re: Measuring CPU core-to-core latency

#60

Post by nviennot »

snvzz wrote: post #55

>This software is licensed under the MIT license Maybe consider including a MIT license in the repository. Legally, that's a bit more sane than having a line in the readme. In practice, github will recognize your license file and show the license in the indexes an d in the right column of your repository's main page.

Okay, I've added a file
Post reply on HN