The Great CPU Stagnation

From what I understand while these two do contribute a lot to power usage, they don't really contribute that much to heating by themselves? Leakage should happen all the same in a processor that's completely idle and those typically don't heat up much. For higher clock speeds specifically I still don't see how lower resistance isn't key.

> I think this is part of why GPU power is skyrocketing and why Apple, Qualcomm, and the like are trying to shift towards services.

IMHO it's only a very small part of why GPU power consumption is going up. The main reason is the completely unnecessary chase for the performance crown.

From personal testing: my GPU manages to get 95% of its peak performance while being power limited to 80%. So the in order to squeeze the last 5% of performance out of the device, 20% more power is pushed through it. It stays above 99% peak performance while being power limited to ~87%.

But even just looking at the raw numbers paints a different picture. About 12 years ago, a high-end GPU (e.g. GTX 480) had a power draw of 250W at a theoretical peak FP32 performance of 1,345 GFLOPS. This year's RTX 4070 has a theoretical peak performance of 29.15 TFOPS at 200W, so we went from 5.38 GFLOPS/W to 145.75 GFLOPS/W in 12 years - a 27x improvement in efficiency and a ~22x improvement in raw performance.

Now let's compare that to the numbers from a decade ago: a GTX 580 from 2010 had a power rating of 244W at 49.41 GTexel/s. A Geforce2 Ultra from 2000 used about 10W at 2.0 GTexel/s. So we went from 0.2 GTexel/s/W to - you've guessed it - 0.2 GTexel/s/W, so same efficiency with a ~27x increase in performance over a decade, though the efficiency is only a guess, since neither GFLOPS nor official power draw figures are readily available for 2000-era hardware.

Fast forward a few years so we can get reliable power draw numbers and comparable performance in GFLOPS, we have the high end GeForce 8800 GTX at 155W for 345.6 GFLOPS in 2006. Ten years later, the comparable model would have been the GTX 1080 from 2016 with 180W at 8.873 TFLOPS. So 2.2 GFLOPS/W versus 49.3 GFLOPS/W or a 22x increase in efficiency and a ~26x increase in performance over the course of a decade.

So during the past 23 years, power efficiency steadily improved, while raw performance increase also showed no signs of slow down in the GPU space. This is given the same generous time frames, to account for the occasional generational leap.

I'm not convinced this is the right criteria to compare these processors (core count and purchase cost).

When looking at these high core count processors, the typical use case is for a server in a data centre, and these sorts of applications run 24/7 and the cost of power is a massive part of the TCO. I think you have to address power per gflop when evaluating performance for these parts, as this is the criteria they were designed against.

I think the processors are costed in consideration of the TCO of a 2U dual socket machine with a 2-3 year expected lifespan. They will be designed and costed to show year on year improvements.

Oh, and i'm not sure inflation was included as it will be relevant over the timescales involved.

In fact the CPU stagnation is worse. In contrast to the shrinking chip size and higher frequencies of past scaling laws, CPU core count does not easily translate to performance gains. It needs specialized and careful designed software to tap the capability.

In fact if CPU core count did translate more easily to performance gains I think already with the existing CPU's we'd have a fairly signficant one-time boost.

Maybe somebody has statistical survey of how much of the existing deployed CPU core count is typically used?

I think two major revolutions would be optical and reversible computing. The former would significantly shrink the heat generated which is a huge bottleneck but is very hard to build generic computing out of and expensive. The latter would basically result in computing obtaining a new theoretical lower bound on energy required but is purely research with no known approaches for actually building the things.

Asynchronous clockless designs might also drastically cut the power budget but those have failed to find adoption for some reason.