Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

%CPU utilization is a lie

brendanlong.com

Post reply on HN
Print view View original

141–150 of 176 posts

Re: %CPU utilization is a lie

#141

Post by kristopolous »

bionsystem wrote: post #77
kristopolous wrote: post #52

Tried to explain this in a job interview 5 years ago. They thought I was a bullshitter

Happened to me on a different topic, felt bad for way too long ; in hindsight I'm pretty sure I dodged a bullet.

This was the same interview where some guy was asking me about "big-o" - like the thing that you teach 19 year olds and I was saying that parallelization matters, i/o matters, quantization matters, whether you can run it on the GPU, these all matter.

The simple "big-o" number doesn't account for whether you need to pass terabytes over the bus for every operation - and on actual computers moving around terabytes, I know, shockingly, this affects performance.

And if you have a dual epyc board with 1,024 threads, being able to parallelize a solution and design things for cache optimization, this isn't meaningless.

It's a weak classifier - if you really think I'm going to be doing a lexical sort in like O(n^3) like some kind of clown, I don't know what you're hiring here.

Found out later he scored me "2/5".

Alright, cool.

Re: %CPU utilization is a lie

#142

Post by morning-coffee »

The lie is that hyper thread "cores" are equal to real "cores". Maybe this is what happens when an over 20-year old technology (hack) becomes ubiquitous and gets forgotten about? (We have to rediscover why our performance measurements don't seem to make sense?)

The other thing I think we have a hard time visualizing is that processor is only either executing (100%) or its waiting to execute (0%) and that happens over varying timescales... so trying to assign a % in between inherently means you're averaging over some arbitrary timescale...

Re: %CPU utilization is a lie

#143

Post by SAI_Peregrinus »

ot wrote: post #129
SAI_Peregrinus wrote: post #127

Earlier quoted context omitted.

That's all true, and the % part is still a lie. As you note, CPU utilization isn't linear, and percentages are linear measures. CPU utilization isn't a lie, % CPU utilization is.

It is a linear percentage of the amount of time the CPU is not idle. It is not linear in the amount of useful work, but that's not what "utilization" means. The lie is the assumption that CPU time is linear in useful work, but that has nothing to do with the definition of utilization, it's just something that people sometimes naively believe. > CPU utilization isn't a lie, % CPU utilization is What do you mean by thi…

Utilization can never reach 100%, since not all of the components of the CPU (or even one core) can actually be in use at once. Quite a few are shared between operations and thus mutually exclusive.

Re: %CPU utilization is a lie

#144

Post by colejohnson66 »

inetknght wrote: post #124
PunchyHamster wrote: post #101

Earlier quoted context omitted.

except it doesn't really tell you much, because having some parts of CPUs underutilized doesn't mean adding load will utilize them. Like if load underutilizes floating point units and you have nothing else that uses them

> Like if load underutilizes floating point units This is why I sigh really hard when people talk about some measurement of FLOPS per second, as if it's the only thing that matters. It matters. Perhaps it matters a lot for specific workloads. But most general workloads are integer-based.

But *sparkle emoji* AI *sparkle emoji*

Re: %CPU utilization is a lie

#145

Post by kqr »

saagarjha wrote: post #118
kqr wrote: post #76

It might be a lie, but it surely is a practical one. In my brief foray into site reliability engineering I used CPU utilisation (of CPU-bofund tasks) with queueing theory to choose how to scale servers before big events. The %CPU suggestions ran contrary to (and were much more conservative than) the "old wisdom" that would otherwise have been used. It worked out great at much lower cost than otherwise. What I'm tryin…

40% seems quite lightly utilized tbh

It depends on how variable the load is, compared to how fast the servers can scale up and down, etc. I often have as a rule of thumb to have enough headroom to be able to deal with twice the load while staying within a triple of the response time. You can solve the equations for your specific case, but eyeballing graphs such as [1] I end up somewhere in the area of 40 %.

The important part is of course to ask yourself the question "how much increased load may I need to handle, and how much can I degrade system performance in doing so?" You may work in an industry that only ever sees 10 % additional load at timescales where scaling is unfeasible, and then you can pick a significantly higher normal utilisation level. Or maybe you're in an industry where you cannot degrade performance by more than 10 % even if hit by five times the load – then you need a much, much more conservative target for utilisation.

[1]: https://erikbern.com/assets/wait-time-2.png

Re: %CPU utilization is a lie

#146

Post by kqr »

PunchyHamster wrote: post #101
gblargg wrote: post #80

Earlier quoted context omitted.

Like measuring RMS of an AC voltage by running it through a heating element: https://wikipedia.org/wiki/True_RMS_converter#Thermal_conver...

except it doesn't really tell you much, because having some parts of CPUs underutilized doesn't mean adding load will utilize them. Like if load underutilizes floating point units and you have nothing else that uses them

Also there's dark silicon to consider – the CPU simply cannot for thermal reasons run power to all parts of itself at the same time.

Re: %CPU utilization is a lie

#147

Post by freehorse »

saagarjha wrote: post #116
freehorse wrote: post #114

Author discovers that performance does not scale proportionally to %CPU utilisation, and gets instead to the conclusion that %CPU utilisation is a lie. There are many reasons for the lack of a proportional relationship, even in the case where you do not have hyperthreading or downclocking (in which cases you just need to interpret %CPU utilisation in that context, rather than declare it "a lie"). Even in apple silico…

Apple silicon downclocks quite a lot especially if you have a passively cooled machine

With the exception of macbook air that has passive cooling nothing as aggressive as "turbo" modes, and ime it is relatively hard to get to thermal limits just with cpu in general for the devices I have used. Most other manufacturers nowadays officially advertise boosted single core clock speeds that are much higher and lower when more cores are used at the same time. Thermal limits, in contrast, are much more circumstantial.

Re: %CPU utilization is a lie

#148

Post by codedokode »

A worse lie is memory usage reporting, I think in every major OS it is understated and misreported. In case with Linux, I wanted to know who is using memory, and tried to add PSS values for every process, I never got back the total memory usage. In case with Windows/Mac I judge by screenshot of their tools which show unrealistically small values.

As for the article, the slowdown can be also caused by increased use of shared resources like caches, TLBs, branch predictors.

Re: %CPU utilization is a lie

#149

Post by [deleted] »

kccqzy wrote: post #88
BrendanLong wrote: post #17

Earlier quoted context omitted.

A big part of this is that CPU utilization metrics are frequently averaged over a long period of time (like a minute), but if your SLO is 100 ms, what you care about is whether there's any ~100 ms period where CPU utilization is at 100%. Measuring p99 (or even p100) CPU utilization can make this a lot more visible.

If your SLO is 100 ms you need far more granular measurement periods than that. You should measure the p99 or p100 utilization for every 5-ms interval or so.

[deleted]

Re: %CPU utilization is a lie

#150

Post by hinkley »

Ambroisie wrote: post #136
hinkley wrote: post #12

Earlier quoted context omitted.

You also want to hit him with queueing theory. Up to a hair over 60% utilization the queuing delays on any work queue remain essentially negligible. At 70 they become noticeable, and at 80% they've doubled. And then it just turns into a shitshow from there on. The rule of thumb is 60% is zero, and 80% is the inflection point where delays go exponential. The biggest cluster I ran, we hit about 65% CPU at our target P9…

Do you have a link to a more in-depth analysis of the queuing theory for these numbers?

I can picture charts from various treatments in my head but none of the names stick.

I really should have a favorite couple of links or books but unfortunately I do not. I will put that on my todo list.

The magic search terms are “queue size/length”, “utilization”.

Post reply on HN