Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

%CPU utilization is a lie

brendanlong.com

Post reply on HN
Print view View original

21–30 of 176 posts

Re: %CPU utilization is a lie

#21

Post by BrendanLong »

tgma wrote: post #2

The way they refer to cores in their system is confusing and non-standard. The author talks about a 5900X as a 24 core machine and discusses as if there are 24 cores, 12 of which are piggybacking on the other 12. In reality, there are 24 hyperthreads that are pretty much pairwise symmetric that execute on top of 12 cores with two sets of instruction pipeline sharing same underlying functional units.

Thanks for the feedback. I think you're right, so I changed a bunch of references and updated the description of the processor to 12 core / 24 thread. In some cases, I still think "cores" is the right terminology though, since my OS (confusingly) reports utilization as-if I had 24 cores.

Re: %CPU utilization is a lie

#22

Post by sroussey »

tgma wrote: post #2

The way they refer to cores in their system is confusing and non-standard. The author talks about a 5900X as a 24 core machine and discusses as if there are 24 cores, 12 of which are piggybacking on the other 12. In reality, there are 24 hyperthreads that are pretty much pairwise symmetric that execute on top of 12 cores with two sets of instruction pipeline sharing same underlying functional units.

Will be interesting when (if?) Intel ships software defined cores which are the logical inverse of hyper threading.

Instead of having a big core with two instruction pipelines sharing big ALUs etc, they have two (or more) cores that combine resources and become one core.

Almost the same, yet quite different.

https://patents.google.com/patent/EP4579444A1/en

Re: %CPU utilization is a lie

#23

Post by tom_ »

hinkley wrote: post #3

How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.

Total throughout has always seemed better with it switched on for me, even for stuff that isn't hyper threading friendly. You get a free 10% at least.

Re: %CPU utilization is a lie

#24

Post by twoodfin »

jiggawatts wrote: post #18
twoodfin wrote: post #5

Earlier quoted context omitted.

For whatever it’s worth, operational database systems (many users/connections, unpredictable access patterns) are beneficiaries of modern hyperthreading. I’m familiar with one such system where the throughput benefit is ~15%, which is a big deal for a BIOS flag. IBM’s POWER would have been discontinued a decade ago were it not for transactional database systems, and that architecture is heavily invested in SMT, up to…

I've noticed an overreliance on throughput as measured during 100% load as the performance metric, which has resulted in hardware vendors "optimising to the test" at the expense of other, arguably more important metrics. For example: single-user latency when the server is just 50% loaded.

That’s more than fair.

In the system I’m most familiar with, however, the benefits of hyperthreading for throughput extend to the 50-70% utilization band where p99 latency is not stressed.

Re: %CPU utilization is a lie

#25

Post by toast0 »

duped wrote: post #11
hinkley wrote: post #3

How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.

For me today it's definitely a pessimation because I have enough well-meaning applications that spawn `nproc` worker threads. Which would be fine if they're the only process running, but they're not.

Wouldn't that be about the same badness without hyperthreads? If you're oversubscribed, there might be some benefit to having fewer tasks, but maybe you get some good throughput with two different application's threads running on opposite hyperthreads.

Re: %CPU utilization is a lie

#26

Post by tom_ »

twoodfin wrote: post #5
hinkley wrote: post #3

How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.

For whatever it’s worth, operational database systems (many users/connections, unpredictable access patterns) are beneficiaries of modern hyperthreading. I’m familiar with one such system where the throughput benefit is ~15%, which is a big deal for a BIOS flag. IBM’s POWER would have been discontinued a decade ago were it not for transactional database systems, and that architecture is heavily invested in SMT, up to…

Why do they need so many threads? This really feels like they just designed the cpu poorly, in that it can't extract enough parallelism out of the instruction stream already.

(Intel and AMD stopped at 2! Apparently more wasn't worth it for them. Presumably because the cpu was doing enough of the right thing already.)

Re: %CPU utilization is a lie

#27

Post by toast0 »

hinkley wrote: post #3

How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.

Going from 1 core to 2 hyperthreads was a big bonus in interactivity. But I think it was easy to get early systems to show worse throughput.

I think there's two kinds of loads where hyperthreads aren't more likely to hurt than help. If you've got a tight loop that uses all the processor execution resources, you're not gaining anything by splitting that in two, it just makes things harder. Or if your load is mostly bound by memory bandwidth without a lot of compute... having more threads probably means you're that much more oversubscribed on i/o and caching.

But a lot of loads are grab some stuff from memory and then do some compute, rinse and repeat. There's a lot of potential for idle time while waiting on a load, being able to run something else during that time makes a lot of sense.

It's worth checking how your load performs with hyperthreads off, but I think default on is probably the right choice.

Re: %CPU utilization is a lie

#28

Post by [deleted] »

judge123 wrote: post #8

This hits so close to home. I once tried to explain to a manager that a server at 60% utilization had zero room left, and they looked at me like I had two heads. I wish I had this article back then!

[deleted]

Re: %CPU utilization is a lie

#29

Post by sroussey »

BrendanLong wrote: post #21
tgma wrote: post #2

The way they refer to cores in their system is confusing and non-standard. The author talks about a 5900X as a 24 core machine and discusses as if there are 24 cores, 12 of which are piggybacking on the other 12. In reality, there are 24 hyperthreads that are pretty much pairwise symmetric that execute on top of 12 cores with two sets of instruction pipeline sharing same underlying functional units.

Thanks for the feedback. I think you're right, so I changed a bunch of references and updated the description of the processor to 12 core / 24 thread. In some cases, I still think "cores" is the right terminology though, since my OS (confusingly) reports utilization as-if I had 24 cores.

Eh, what’s a thread really? It’s a term for us humans.

The difference between two threads and one core or two cores with shared resources?

Nothing is really all that neat and clean.

It more of a 2 level NUMA type architecture with 2 sets of 6 SMP sets of 2.

The scheduler may look at it that way (depending), but to the end user? Or even to most of the system? Nah.

Re: %CPU utilization is a lie

#30

Post by hinkley »

duped wrote: post #11
hinkley wrote: post #3

How many times has hyperthreading been an actual performance benefit in processors? I cannot count how many times an article has come out saying you'll get better performance out of your by turning off hyperthreading in the BIOS. It's gotta be at least 2 out of every 3 chip generations going back to the original implementation, where you're better off without it than with.

For me today it's definitely a pessimation because I have enough well-meaning applications that spawn `nproc` worker threads. Which would be fine if they're the only process running, but they're not.

I wrote a little tool for our services that could do basic expression based off of nproc based on an environment variable at startup time.

You could do one thread for every two cores, three threads for every 2 cores, one thread per core ± 1, or both (2n + 1).

Unfortunately the sweet spot based on our memory usage always came out to 1:1, except for a while when we had a memory leak that was surprisingly hard to fix, and we ran n - 1 for about 4 months while a bunch of work and exploratory testing were done. We had to tune in other places to maximize throughput.

Post reply on HN