Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How GPUs Work

cs.virginia.edu

Post reply on HN
Print view View original

21–30 of 54 posts

Re: How GPUs Work

#21

Post by dantillberg »

mej10 wrote: post #16
ak217 wrote: post #7

This overview, while a great start, doesn't really dive into the details of how modern GPUs work. Since 2007, many of the limitations that held GPUs back from being general-purpose computers have been removed (by relentless efforts of NVIDIA and to a lesser extent ATI/AMD, spurred in large part by NVIDIA's traction in the supercomputing space, for example http://en.wikipedia.org/wiki/Titan_%28supercomputer%29 ). My g…

What are you meaning by general-purpose here? Do you not have to use a different programming model anymore?

The parent is referring to how CUDA's introduction enabled developers to write and compile C-ish code to run on a GPU, while previously programmers could take advantage of the power of GPUs for non-render computations by hacking the pixel shaders and such, bending the graphics hardware to do something it was not designed for.

You still have to write your program in a very different way in order to run efficiently on GPUs as opposed to CPUs.

Re: How GPUs Work

#22

Post by com2kid »

yazaddaruvala wrote: post #19

Maybe off topic, but I'm actually really surprised that monitors and GPUs are still different pieces of hardware. I'll admit I only know the basics of GPU architecture, so please forgive/correct me if I'm wrong about something. However, I am just too curious not to share. I'll try to explain. A frame buffer is nothing but a bunch of 1s and 0s in memory, meanwhile a monitor is just a bunch of 1s and 0s in pixels. We c…

A number of problems. A huge one is wiring. Parallel is really complicated electrically, noise drowns out your signal and things run slow. This is why most of our buses have switched over to serial (e.g. USB, PCIe, etc). Sometimes we run those serial busses in parallel, but that still works out to being easier.

Timing is another huge one. Imagine running 2 million wires (for a 1080p display) that have to all be the exact same length to within some tolerance.

The longer those wires gets the harder this gets. This is also another huge reason why the move to serial buses has happened. You can run 4 wires with really tight timings and the bits will fly, but if you try and run 16 wires all together, speed ends up dropping dramatically. Reality is that circuit boards don't have room for a large number of traces running in parallel of all the exact same length!

RAM is a huge exception to this, but extreme measures have been taken to enable this to happen, a good chunk of your Mobo is taken up getting the RAM connected, and memory controllers moved onboard the CPU in part to get RAM closer to the CPU to simplify traces,

Note this is all the perspective of a software guy who has to listen to the hardware team grumble for most of the day. :)

Re: How GPUs Work

#23

Post by _ihaque »

fat0wl wrote: post #14

I'm really sorry I will try to dig up the source myself, but I've read basically the opposite argument in a few technical papers -- that GPU is NOT as fast as claimed for many classic test algorithms (actual speed-up is more like factor of 2 than 10) and that the performance gap between traditional CPUs and GPUs is actually lessening. I'm going to read this article anyway to hear their take & for the learning experie…

tmurray is basically right: most of the really big reported gains are artifacts of unoptimized CPU code. Except for hardware special functions on GPUs, you shouldn't be able to exceed the theoretical perf ratios between GPU and CPU, which are roughly ~30x in FLOPs and ~10x in bandwidth. Depending on whether arithmetic or memory is the algorithmic bottleneck, you'll hit one of those limits.

I wrote a paper [1] on this in one particular domain (computational chemistry) more or less as a rebuttal to a paper that claimed enormous GPU speedups; it was a consequence of slow CPU code, not especially fast GPU code.

[1] http://cs.stanford.edu/people/ihaque/papers/2dtanimoto.pdf

Re: How GPUs Work

#24

Post by mej10 »

dantillberg wrote: post #21
mej10 wrote: post #16

Earlier quoted context omitted.

What are you meaning by general-purpose here? Do you not have to use a different programming model anymore?

The parent is referring to how CUDA's introduction enabled developers to write and compile C-ish code to run on a GPU, while previously programmers could take advantage of the power of GPUs for non-render computations by hacking the pixel shaders and such, bending the graphics hardware to do something it was not designed for. You still have to write your program in a very different way in order to run efficiently on…

The comparison to a Xeon is what confused me. I didn't think such a development had taken place.

Re: How GPUs Work

#25

Post by cscheid »

yazaddaruvala wrote: post #19

Maybe off topic, but I'm actually really surprised that monitors and GPUs are still different pieces of hardware. I'll admit I only know the basics of GPU architecture, so please forgive/correct me if I'm wrong about something. However, I am just too curious not to share. I'll try to explain. A frame buffer is nothing but a bunch of 1s and 0s in memory, meanwhile a monitor is just a bunch of 1s and 0s in pixels. We c…

Myer and Sutherland wrote a classic 1968 paper on what they came to call the Wheel of Reincarnation: simple displays accrue progressively more complexity until someone comes along and throws the whole thing away with a new, clean, design.

Then someone finds that they can add a bit of processing to that display to make it go just a bit faster...

http://cva.stanford.edu/classes/cs99s/papers/myer-sutherland...

Re: How GPUs Work

#27

Post by nemothekid »

The 8800GTX was the first GPU I every bought back in 2007 (obviously I'm not very old). Now 7 years later, its funny how dated the render on "Figure 2." is.

Re: How GPUs Work

#28

Post by sklogic »

_ihaque wrote: post #23
fat0wl wrote: post #14

I'm really sorry I will try to dig up the source myself, but I've read basically the opposite argument in a few technical papers -- that GPU is NOT as fast as claimed for many classic test algorithms (actual speed-up is more like factor of 2 than 10) and that the performance gap between traditional CPUs and GPUs is actually lessening. I'm going to read this article anyway to hear their take & for the learning experie…

tmurray is basically right: most of the really big reported gains are artifacts of unoptimized CPU code. Except for hardware special functions on GPUs, you shouldn't be able to exceed the theoretical perf ratios between GPU and CPU, which are roughly ~30x in FLOPs and ~10x in bandwidth. Depending on whether arithmetic or memory is the algorithmic bottleneck, you'll hit one of those limits. I wrote a paper [1] on this…

One should not forget another important thing: 1GFLOP in CPU is more expensive in terms of power than 1GFLOP in a GPU. So it's not only about chasing GFLOPs, in the mobile and embedded world it's also all about power.

Re: How GPUs Work

#29

Post by JoshTriplett »

yazaddaruvala wrote: post #19

Maybe off topic, but I'm actually really surprised that monitors and GPUs are still different pieces of hardware. I'll admit I only know the basics of GPU architecture, so please forgive/correct me if I'm wrong about something. However, I am just too curious not to share. I'll try to explain. A frame buffer is nothing but a bunch of 1s and 0s in memory, meanwhile a monitor is just a bunch of 1s and 0s in pixels. We c…

Shaders already run heavily in parallel. However, you can't write directly to monitor pixels in parallel like that, because you'd then need milions of individual connections to individual pixels, rather than scanning logic.

Along similar lines, consider that CPUs have billions of transistors but only a little over a thousand pins.

Re: How GPUs Work

#30

Post by brigade »

sklogic wrote: post #28
_ihaque wrote: post #23

Earlier quoted context omitted.

tmurray is basically right: most of the really big reported gains are artifacts of unoptimized CPU code. Except for hardware special functions on GPUs, you shouldn't be able to exceed the theoretical perf ratios between GPU and CPU, which are roughly ~30x in FLOPs and ~10x in bandwidth. Depending on whether arithmetic or memory is the algorithmic bottleneck, you'll hit one of those limits. I wrote a paper [1] on this…

One should not forget another important thing: 1GFLOP in CPU is more expensive in terms of power than 1GFLOP in a GPU. So it's not only about chasing GFLOPs, in the mobile and embedded world it's also all about power.

But normalizing for power reduces the GPU advantage even more! Haswell for instance achieves about 5.3-5.8 GFLOP/W, compared to 24-28 GFLOP/W of Maxwell. That's less than a 5x theoretical computational gain.
Post reply on HN