Why has CPU frequency ceased to grow? (2014) • Hacker News Board

Re: Why has CPU frequency ceased to grow? (2014)

#201

Post by jacksmith21006 » Wed, Feb 21, 2018, 5:14 PM UTC

BenoitP wrote: post #111

mjfl wrote: post #8

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm. I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardw…

I'd say exploiting parallelism is not the only way at all. Parallelism is only one way to compute differently. Specialization of hardware to specific workloads will explode in the next years as we can't rely anymore on Moore's law. This will happen on RISC-V, IMHO. We already have these: * Rendering, medium precision mathematics: GPU * Low precision mathematics: TPU * Software Defined Networking: Microsoft is deployi…

There is a great paper from Jeff Dean on using the TPUs for more traditional CS functions.

https://research.google.com/pubs/pub46518.html The Case for Learned Index Structures - Research at Google

Re: Why has CPU frequency ceased to grow? (2014)

#202

Post by darkmighty » Wed, Feb 21, 2018, 5:19 PM UTC

jedbrown wrote: post #177

Notably, single-thread performance of code that is not friendly to vectorization has not stagnated despite stagnant clock frequency. Indeed, SPECint performance continues to grow exponentially, albeit more slowly since ~2004. https://www.karlrupp.net/2018/02/42-years-of-microprocessor-...

> Indeed, SPECint performance continues to grow exponentially, albeit more slowly since ~2004

Interesting chart! However that looks like moving goalposts. If you allow functions to still be exponentials when the rate repeatedly diminishes, any monotonic function can be an exponential: f(x) = x is an "exponential" with diminishing growth rate log(x). That curve looks approximately logistic to me, as most technologies usually do.

Actually I believe speculative execution theoretically allows increasing linearly single-threaded performance for exponentially more multi-threaded performance -- so a transistor price Moore's law (if not power/density) should allow a continued linear single-threaded growth. The problem is that without power (inverse) scaling, this would also cost exponentially more power. It seems the slight increase in power visible in that chart could account for a fraction of increased single-threaded performance (other factors would be improved efficiency and architectural gains); expect those factors to also stall soon (which would fulfill the "logistic prophecy").

Re: Why has CPU frequency ceased to grow? (2014)

#203

Post by inspector-g » Wed, Feb 21, 2018, 5:24 PM UTC

IIAOPSW wrote: post #140

slivym wrote: post #7

Seems like an incredibly long winded way of saying 'To go faster you either need to split up each instruction into lots of parts or increase the voltage for the transisters. We've split the instructions as much as we can, and power consumption is proportional to Voltage cubed, so it's not a scalable plan.'

More importantly than mere power consumption, we don't have a way to remove the waste heat generated. Dennards law (like Moore's law but for power consumption per transistor) ended 10 years ago. Exactly the same time clock speeds stopped improving. There are actually a few computers out there that run around 10Ghz but they all have impractical cooling systems. If there ever were a return to exponential scaling, we wo…

For those who are curious, see the Wikipedia article on Landauer's Principle [1]

I hadn't heard of this before, but it sounds like we are a long way off from reaching the limit; as the article states, modern computers use millions of times more energy than what Landauer's Principle implies is the lowest possible amount.

1: https://en.wikipedia.org/wiki/Landauer%27s_principle

Re: Why has CPU frequency ceased to grow? (2014)

#204

Post by steveklabnik » Wed, Feb 21, 2018, 5:25 PM UTC

coldtea wrote: post #200

Thaxll wrote: post #188

Earlier quoted context omitted.

For once there is no consensus how to do parallelism an concurrency in Rust since none of the library are matures.

Don't most people opt for something called Tokyo? I don't do Rust much, but even I have heard of it by now...

Tokio for IO concurrency, Rayon for data parallelism.

Re: Why has CPU frequency ceased to grow? (2014)

#205

Post by kickopotomus » Wed, Feb 21, 2018, 5:26 PM UTC

Symmetry wrote: post #185

nullbyte wrote: post #182

Earlier quoted context omitted.

How do you know all this stuff?

Got my master's in it, before ending up in sensors then robotics instead. And a continued interest, I guess. Here are some free relevant courses. You might have to go back and take the pre-reqs. https://ocw.mit.edu/courses/electrical-engineering-and-compu... https://ocw.mit.edu/courses/electrical-engineering-and-compu...

Thank you for this! I have been looking for IC design MOOCs for awhile.

Re: Why has CPU frequency ceased to grow? (2014)

#206

Post by coolsunglasses » Wed, Feb 21, 2018, 5:45 PM UTC

brazzy wrote: post #193

floatboth wrote: post #178

Earlier quoted context omitted.

And in Haskell https://www.stackage.org/haddock/lts-10.6/parallel-3.2.1.1/C... And in .NET https://docs.microsoft.com/en-us/dotnet/api/system.threading... And in Java https://blog.oio.de/2016/01/22/parallel-stream-processing-in... Rayon is notable because Rust is a native-compiled, no-GC language.

To me, the "If it’s not thread-safe to do, then it won’t compile" part sounds rather more notable.

That's true of Haskell for the parallelism faculties. It's also true for STM. You can still get stuck or have race conditions with the other concurrency abstractions but it's less common than I encountered elsewhere. I didn't have problems resulting from mutable shared state that was aliased across threads and wasn't supposed to be, it's always explicit.

Re: Why has CPU frequency ceased to grow? (2014)

#207

Post by AnimalMuppet » Wed, Feb 21, 2018, 5:49 PM UTC

logicallee wrote: post #51

guys, I'll be honest: I found it really odd that the article didn't talk about the speed of light, and die size constraints. (c / 4 ghz = 7.49 cm only, if you double that frequency you have half that size to put components it between any two clock ticks.) But there are limits to my hubris - this is on intel.com, so I'm going to go with "I'm the one missing something". Is the speed of light and number of transistors y…

To amplify what pcj50 said: The speed of light is a very real constraint. It's just not the constraint that people are hitting, because there are other constraints you hit first (capacitance, heating, etc).

I recall reading clear back in the 1970s that IBM mainframes were trying to do a dual processor setup. This wasn't multiple cores on one die, this was separate physical boxes. And they were having trouble because they wanted them to operate in sync (in the sense of presenting one image to the OS and applications), but they were more than a foot apart, and they were operating at sub-nanosecond frequencies. For them, the speed of light was definitely a constraint. Even if they got around all the electrical stuff, the speed of light still put a limit on how "in sync" those two CPUs could be.

Re: Why has CPU frequency ceased to grow? (2014)

#208

Post by mcbits » Wed, Feb 21, 2018, 5:53 PM UTC

kowdermeister wrote: post #36

mtgx wrote: post #28

Earlier quoted context omitted.

TeraHertz CPUs would be nice.

Crank them up a little more and they will glow due to reaching visible light territory :)

Perhaps leading to little pixel-sized photovoltaic cores that absorb light on one side, do a bit of computation at 400-800 THz and emit light on the other side, merging the display with the CPU. I've heard that VR goggles with 8K resolution per eye would be approaching the limit of the human eye's resolving power, which would be about 66 million cores.

Re: Why has CPU frequency ceased to grow? (2014)

#209

Post by ajross » Wed, Feb 21, 2018, 6:02 PM UTC

fulafel wrote: post #180

ajross wrote: post #176

Earlier quoted context omitted.

> I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system There has. It's called a GPU. Things like OpenCL and CUDA are the new languages.

But it would be much better if a single language + hardware system emerged, rather than a multitude of mutually incompatible hardware systems and languages. With the current fragmented and sometimes proprietary forest of programming platforms, an application needs to be quite specialized to warrant investment in GPU compute outside the original niche of graphics acceleration. There are other giant problems too, after…

> But it would be much better if a single language + hardware system emerged

There will, but it takes time. The world of scalar hardware in the 1970's was no less fragmented. Honestly most of the incompatibilities in the SIMD world at this point are bugs and not fundamental problems. The vector world has settled on a broad architecture at this point for most things.

Re: Why has CPU frequency ceased to grow? (2014)

#210

Post by tarlinian » Wed, Feb 21, 2018, 6:08 PM UTC

tagrun wrote: post #142

I understand that companies heavily invested in silicon would like to portray it that way, but CPU frequencies haven't ceased to grow. DARPA manufactured a THz transistor made of InP back in 2014. Silicon isn't the only semiconductor in nature, and others are actively being researched. Also, "when you increase the frequency you increase the power" (which is their argument) doesn't explain why they can't increase the…

Transistor switching frequency has almost nothing to do with processor clock speeds which are almost entirely limited by wire RC delay. You will get increased drive currents by switching to higher mobility materials but the performance improvement over strained silicon isn't that large.