Why has CPU frequency ceased to grow? (2014) • Hacker News Board

Re: Why has CPU frequency ceased to grow? (2014)

#121

Post by rbanffy » Wed, Feb 21, 2018, 1:39 PM UTC

BenoitP wrote: post #111

mjfl wrote: post #8

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm. I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardw…

I'd say exploiting parallelism is not the only way at all. Parallelism is only one way to compute differently. Specialization of hardware to specific workloads will explode in the next years as we can't rely anymore on Moore's law. This will happen on RISC-V, IMHO. We already have these: * Rendering, medium precision mathematics: GPU * Low precision mathematics: TPU * Software Defined Networking: Microsoft is deployi…

Still, if you have only two floats to add and make a decision based on the result, a GPU or a TPU will not help you.

Hashing, maybe sorting, blitting and some math could be performed at the on-module DRAM controller level even, without data crossing over the slow DDR4 bus or mangling the CPU caches.

I'd love, in fact, to explore such an architecture in a simulator. What would happen to CPU performance if, say, hashes could be computed without reading the data, memory be cleared without zeroes hitting the bus or some SIMD operations be conducted on the memory.

Edit: clarify the processing could be done on the module side of the memory bus.

Re: Why has CPU frequency ceased to grow? (2014)

#122

Post by ta2384428 » Wed, Feb 21, 2018, 1:48 PM UTC

VHRanger wrote: post #104

quadcore wrote: post #10

Earlier quoted context omitted.

https://golang.org/ In case you dont know, Golang goroutines are a marvel of parallelism. They are coroutines which are dispatched on a few OS threads. So you can use 100% of a multi-cores CPU and yet, spawn, say, 10K of those light threads without worrying about context switches PLUS have them all run concurrently. I've found that golang is one of those rare language, like Lisp, that actually change the way you thin…

How is that an example an improvement over c++? C++ has had pragmatic omp parallel for for two decades

I'm not familiar with OMP, so please forgive me if I'm incorrect - but OMP from my quick search appears to be thread based, while go routines are much lighter weight than threads. They have their own scheduler - which like most things is both good and bad depending on what you want them for. If you use them as intended, the internal scheduler is a great design decision. This means I can happily spawn 10,000 without concern.

Additionally, combining them with the power of channels makes quick work of many tasks. Channels of course can be implemented in C++ too, but having the compiler take care of it for you with additional tools such as the race detector is very handy.

For a large set of problems, they are very nice to work with.

Re: Why has CPU frequency ceased to grow? (2014)

#123

Post by ta2384428 » Wed, Feb 21, 2018, 1:57 PM UTC

tzahola wrote: post #41

quadcore wrote: post #40

Earlier quoted context omitted.

Wrong. Goroutines are not simply coroutines. GOMAXPROCS defaults to number of cores.

Prove it. Prove it by replacing Sleep in your example with some number crunching, and show how it scales with the number of cores in your CPU.

https://imgur.com/a/DNpw3

Running the following code.

https://play.golang.org/p/k_rRxNAyb0i

I can assure you that go runs across all processors by default.

https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3...

Re: Why has CPU frequency ceased to grow? (2014)

#124

Post by chimprich » Wed, Feb 21, 2018, 1:59 PM UTC

MichaelMoser123 wrote: post #50

krylon wrote: post #22

Earlier quoted context omitted.

Great, now I have to clean all that tea off of my display. ;-P

and i thought that humor is verboten around here... Does HN have an exception in the book that applies to Intel ?

I don't think humour as such is frowned upon on HN - if I was to attempt to write down the unwritten, I would say that posts that are just jokes tend to go down badly, but jokes that make a point or serious posts that are written with some wit are generally accepted.

Re: Why has CPU frequency ceased to grow? (2014)

#125

Post by contravariant » Wed, Feb 21, 2018, 2:00 PM UTC

mjfl wrote: post #8

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm. I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardw…

Haskell kind of qualifies. You can't have race conditions when everything is immutable.

Of course this makes some other things a bit more difficult.

Re: Why has CPU frequency ceased to grow? (2014)

#126

Post by kamaal » Wed, Feb 21, 2018, 2:01 PM UTC

swiley wrote: post #60

mjfl wrote: post #8

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm. I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardw…

There are limits to how much parallelism will improve things, see amdahl's law.

Sure, but let's start moving towards parallelism first. May be a few decades from now we would be close to breaking those limits.

Re: Why has CPU frequency ceased to grow? (2014)

#127

Post by microcolonel » Wed, Feb 21, 2018, 2:02 PM UTC

rbanffy wrote: post #121

BenoitP wrote: post #111

Earlier quoted context omitted.

I'd say exploiting parallelism is not the only way at all. Parallelism is only one way to compute differently. Specialization of hardware to specific workloads will explode in the next years as we can't rely anymore on Moore's law. This will happen on RISC-V, IMHO. We already have these: * Rendering, medium precision mathematics: GPU * Low precision mathematics: TPU * Software Defined Networking: Microsoft is deployi…

Still, if you have only two floats to add and make a decision based on the result, a GPU or a TPU will not help you. Hashing, maybe sorting, blitting and some math could be performed at the on-module DRAM controller level even, without data crossing over the slow DDR4 bus or mangling the CPU caches. I'd love, in fact, to explore such an architecture in a simulator. What would happen to CPU performance if, say, hashes…

> memory be cleared without zeroes

In some specific cases, such as when the Linux kernel maps memory into your process, this is exactly what happens. When you write the page, it faults and clears it on demand; but I don't think there would be a considerable benefit to doing this at a finer granularity.

Re: Why has CPU frequency ceased to grow? (2014)

#128

Post by pjc50 » Wed, Feb 21, 2018, 2:08 PM UTC

jokoon wrote: post #25

mjfl wrote: post #8

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm. I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardw…

I wonder if the future will be massively parallel, when CUDA and opencl came out I thought that future processors would have more and more core, so if you follow Moore's law, CPUs would see their core count double each 6 months. The problem is that GPU don't have error correcting codes, so you cannot really run application code on a GPU. The problem with parallelism is that C-like language don't fit well, only functi…

> The problem is that GPU don't have error correcting codes

.. eh? What are you referring to here? Mainstream CPUs don't have error correction either, unless you're talking about ECC on the higher-end ones.

Re: Why has CPU frequency ceased to grow? (2014)

#129

Post by LaikaF » Wed, Feb 21, 2018, 2:09 PM UTC

kowdermeister wrote: post #86

v_lisivka wrote: post #59

Earlier quoted context omitted.

Crank them up a little more and they will go into deep ultraviolet. Problem solved.

There is no problem here, marketing folks would love it :)

So would gamers.

"Nice LEDs bro" "Nah that's my CPU"

Re: Why has CPU frequency ceased to grow? (2014)

#130

Post by rbanffy » Wed, Feb 21, 2018, 2:09 PM UTC

Androider wrote: post #114

mjfl wrote: post #8

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm. I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardw…

With just Chrome and IntelliJ running on Ubuntu, I have 281 processes and kernel workers running (as reported by ps). Just stitching the apps together for display on your screen requires 3-4 different processes (and a GPU). That's why 4 logical cores is a bare minimum these days for a desktop, even if no individual application takes advantage of more than a single core. On the server side, when we deploy a Node.js we…

Having 281 processes is not a biggie. Having 281 processes that can actually use a piece of the CPUs is quite a different thing.

I think we should, really, start thinking about such things. Maybe prefixing instructions with the execution unit that should handle them (and overflow back to the first one in a circle if we have more EU's in software than the actual hardware provides), separating dependencies within code flow in a more explicit way and, at the same time, not bothering with creating threads.