%CPU utilization is a lie
brendanlong.com
%CPU utilization is a lie
1–10 of 176 posts
Re: %CPU utilization is a lie
#2The way they refer to cores in their system is confusing and non-standard. The author talks about a 5900X as a 24 core machine and discusses as if there are 24 cores, 12 of which are piggybacking on the other 12. In reality, there are 24 hyperthreads that are pretty much pairwise symmetric that execute on top of 12 cores with two sets of instruction pipeline sharing same underlying functional units.
Re: %CPU utilization is a lie
#3How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS.
It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.
Re: %CPU utilization is a lie
#4How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.
It has a lot to do with your workload as well as if not moreso than the chip architecture.
The primary trade-off is the cache utilization when executing two sets of instruction streams.
Re: %CPU utilization is a lie
#5How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.
For whatever it’s worth, operational database systems (many users/connections, unpredictable access patterns) are beneficiaries of modern hyperthreading.
I’m familiar with one such system where the throughput benefit is ~15%, which is a big deal for a BIOS flag.
IBM’s POWER would have been discontinued a decade ago were it not for transactional database systems, and that architecture is heavily invested in SMT, up to 8-way(!)
Re: %CPU utilization is a lie
#6How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.
It has a lot to do with your workload as well as if not moreso than the chip architecture. The primary trade-off is the cache utilization when executing two sets of instruction streams.
That's likely the primary factor, but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU.
Re: %CPU utilization is a lie
#7Utilization is not a lie, it is a measurement of a well-defined quantity, but people make assumptions to extrapolate capacity models from it, and that is where reality diverges from expectations.
Hyperthreading (SMT) and Turbo (clock scaling) are only a part of the variables causing non-linearity, there are a number of other resources that are shared across cores and "run out" as load increases, like memory bandwidth, interconnect capacity, processor caches. Some bottlenecks might come even from the software, like spinlocks, which have non-linear impact on utilization.
Furthermore, most CPU utilization metrics average over very long windows, from several seconds to a minute, but what really matters for the performance of a latency-sensitive server happens in the time-scale of tens to hundreds of milliseconds, and a multi-second average will not distinguish a bursty behavior from a smooth one. The latter has likely much more capacity to scale up.
Unfortunately, the suggested approach is not that accurate either, because it hinges on two inherently unstable concepts
> Benchmark how much work your server can do before having errors or unacceptable latency.
The measurement of this is extremely noisy, as you want to detect the point where the server starts becoming unstable. Even if you look at a very simple queueing theory model, the derivatives close to saturation explode, so any nondeterministic noise is extremely amplified.
> Report how much work your server is currently doing.
There is rarely a stable definition of "work". Is it RPS? Request cost can vary even throughout the day. Is it instructions? Same, the typical IPC can vary.
Ultimately, the confidence intervals you get from the load testing approach might be as large as what you can get from building an empirical model from utilization measurement, as long as you measure your utilization correctly.
Re: %CPU utilization is a lie
#8This hits so close to home. I once tried to explain to a manager that a server at 60% utilization had zero room left, and they looked at me like I had two heads. I wish I had this article back then!
Re: %CPU utilization is a lie
#9Earlier quoted context omitted.
It has a lot to do with your workload as well as if not moreso than the chip architecture. The primary trade-off is the cache utilization when executing two sets of instruction streams.
That's likely the primary factor, but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU.
>but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU.
That doesn't make any sense. Disabling SMT likely saves negligible amount of power, but disables any performance to be gained from the other thread. If there's thermal budget available, it's better to spend it by shoving more work onto the second thread than to leave it disabled. If anything, due to voltage/frequency curves, it might even be better to run your CPU at lower clocks but with SMT enabled to make up for it (assuming it's amenable to your workloads), than it is to run with SMT disabled.
Re: %CPU utilization is a lie
#10Earlier quoted context omitted.
It has a lot to do with your workload as well as if not moreso than the chip architecture. The primary trade-off is the cache utilization when executing two sets of instruction streams.
That's likely the primary factor, but then there's thermal throttling as well. You can't run all of the logic units flat out on a bunch of models of CPU.
May be true for FMA or AVX2 or similar stuff. Outside vector units that sounds implausible. Obviously multi core thermal throttling is a thing but that would by far dominate. Hyperthreading should have minimal impact there.