Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

thehftguy.com

Post reply on HN
Print view View original

51–58 of 58 posts

Re: The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

#51

Post by mwpmaybe »

8fingerlouie wrote: post #32

I'm not well enough versed in the Linux kernel to comment on the post, but as a funny anecdote, 15 years ago, we were developing a system that relied heavily on using as many cores as possible simultaneously. Performance was critical, and we had beefy developer machines (for the time), all with 8 cores. Development was done in C++, and as the project progressed the system performed very well, and more than exceeded o…

It's amazing how much "optimization" you can achieve on modern systems by simply fencing process(es) to run within a cache region, or within a NUMA node. Even my homelab server with a P-core cluster and two E-cores clusters benefits massively from some simple cpusets to keep each process running within a cluster to mitigate context switching. Each P-cores has its own L2 cache, but E-core clusters share L2. Unfortunately all three clusters share a LLC, so there's only so much you can do.

Re: The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

#52

Post by mwpmaybe »

menaerus wrote: post #48
nathants wrote: post #42

Earlier quoted context omitted.

afaik not for my use case. i need low latency with no variability. you only get that staying within a single cpu cluster.

This all comes down to the cache coherency protocols. And it's not surprising that you see increased latency with zen4 microarchitecture because, as one of the parent commenters already said, it's almost as if you're running a NUMA architecture within a single physical chip. When dealing with NUMA we know that cross-socket, or in this case cross-CCD, latencies are always higher than the ones within the same socket or…

It makes Ryzen 9 processers great for budget VFIO workstations because you can run an OS (either host and guest or two guests) on each CCD.

Re: The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

#53

Post by jaimex2 »

skilled wrote: post #6

I'm a bit disappointed and slightly annoyed that this was published without a comment from any of the Kernel maintainers.

Really all that will happen is the site will get black listed as a credible source.

It won't be appearing again in my feeds that's for sure.

Re: The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

#54

Post by IYasha »

globular-toast wrote: post #39

I used to work on a supercomputer with 128 cores on it that ran Linux (I actually seem to remember it had 256 cores, but someone said the kernel had a limit of 128). This was less than 15 years ago. There were surely many systems just like that one. Does that mean the kernel had been patched? But nobody thought to push that patch upstream? Reading the other comments here it seems the title is stupid and wrong and my…

You didn't even have to have a supercomputer as common Opteron systems pretty much had more than 8 cores. So, someone would definitely notice that. Especially AMD. :)

Re: The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

#55

Post by rosencrantz »

The statement looks very misleading or even fraudulent. I used a system with 192 cores often and with GNU parallel, it did not stagnate at 8 parallel tasks for simple demonstrations. If we're talking about a case where 8 is intentionally the maximum (it's possible that some tasks should not parallelize more than 8), then the statement is misleading as well, since it gives the wrong impression. I have the service tag and the output from nproc and the exact version of everything where I used 192 CPUs. I suppose pseudoscience will always return and claim that the statement is true anyway, no matter what observations the rest of the world can give. There is pseudoscience forever who always says that the rest of the world is misunderstood.

Re: The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

#57

Post by sliken »

nathants wrote: post #16

this may not be an error. i’m doing high performance gamedev on a 7950x with smt off. 16 cores. i can only use 8 to run the game. mutex and other synchronization gets highly variable latency if the first 8 cores try to talk to the second 8. i hadn’t heard of this before i started this game, but apparently it’s well known, and nobody makes a chip with more than 8 cores that can have low variability synchronization. li…

I have the 7900 (non-x), I find it helpful to think of it as a dual socket 6 core system. For latency sensitive things like many games, it's best to pin it to a single socket/half the cores. Once you do that the other half can handle everything else. That way jitter stays low, performance is consistent, caches are warm, etc.

Re: The Linux kernel has been accidentally hardcoded to a maximum of 8 cores

#58

Post by nathants »

sliken wrote: post #57
nathants wrote: post #16

this may not be an error. i’m doing high performance gamedev on a 7950x with smt off. 16 cores. i can only use 8 to run the game. mutex and other synchronization gets highly variable latency if the first 8 cores try to talk to the second 8. i hadn’t heard of this before i started this game, but apparently it’s well known, and nobody makes a chip with more than 8 cores that can have low variability synchronization. li…

I have the 7900 (non-x), I find it helpful to think of it as a dual socket 6 core system. For latency sensitive things like many games, it's best to pin it to a single socket/half the cores. Once you do that the other half can handle everything else. That way jitter stays low, performance is consistent, caches are warm, etc.

i’m pleased with 7950x, it was just a surprise. when benchmarking the game engine, massive latency variance due to background jobs using all cores.

when i compile, all 16 cores are used.

when playing, core0 is main thread, core1 is physx, core2 is network, core3-7 are background jobs used by main thread.

i’m not sure more than 8 cpu cores can be meaningfully used by any current game engine tech.

when i run the server locally, i reduce usage to 4 cores so client and server don’t contend. not much impact.

Post reply on HN