Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every developer should know about GPU computing

codeconfessions.substack.com

Post reply on HN
Print view View original

61–70 of 186 posts

Re: What every developer should know about GPU computing

#61

Post by smokel »

PTOB wrote: post #53
permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

Try this on: "A non-trivial number of Computer Scientists, Computer Engineers, Electrical Engineers, and hobbyists have ..." Took some philosophy courses for fun in college. I developed a reading skill there that lets me forgive certain statements by improving them instead of dismissing them. My brain now automatically translates over-generalizations and even outright falsehoods into rationally-nearby true statements…

That's a refreshing take.

I tried real hard to understand what some continental philosophers, such as Latour, Deleuze, and Žižek are on about, giving some of their texts quite some benefit of the doubt. After about ten years of doing so, I am more and more returning to my previous opinion that some of these just like to talk, and even though they have lots to say, they say so much, that I still have to do all the actual philosophizing myself.

Re: What every developer should know about GPU computing

#62

Post by touisteur »

choppaface wrote: post #34
mannyv wrote: post #28

Now I understand why ML uses floats for precision. It wasn't a choice, it was because graphics code uses them. Another piece in the "why is ML so inefficient" puzzle! I wonder what that memory copying overhead is IRL. If it's like normal stuff it'll be brutal. I mean, they offload tcp processing into hardware to avoid that. This is way more data, though it is done in bigger chunks.

For a lot of larger modern networks the GPU compute time in computing gradients and doing the backwards pass is so slow that copying float data over the pcie bus is no bottleneck. I.e. copying a minibatch of float images is still plenty fast because the gradients / SGD iteration is so slow and requires so much compute (even with mixed precision). For shallower networks, there can be an advantage to copying just the o…

I would say that some one modern GPU has adopted PCIe 5.0, the NVIDIA H100 - if you want more (FP32) 'compute' (and no HBM...) in one GPU your sourceable choice is the L40 which didn't get PCIe 5.0. This feels like another market/product segmentation (that and L40 didn't get nvlink).

Re: What every developer should know about GPU computing

#63

Post by jokoon »

I wish it was easier to program a GPU...

I've already refrained myself to learn vulkan because it scares me, but similarly, opengl and cuda are a bit mysterious to me, and I don't really know how I could take advantage of it, since most computing tasks cannot be made parallel.

I've read there are data structures that are somehow able to take advantage of a GPU as an alternative to the CPU (for example a database running on a GPU), but it seems a very new domain, and I don't have the skill to explore it.

Re: What every developer should know about GPU computing

#64

Post by nologic01 »

KeplerBoy wrote: post #59
speed_spread wrote: post #58

Earlier quoted context omitted.

Vector Processing Unit would be more appropriate.

Why stop there? Matrix Processing Unit or Tensor Processing Unit if you want to trigger the physicists or are a fan of the google cloud.

Differential Geometric Processing Unit would do it for me.

Re: What every developer should know about GPU computing

#65

Post by robertlagrant »

softfalcon wrote: post #32
cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

Yup, I was coming up under a senior at my work one day and he tried to very incorrectly hand wave how a browser uses a GPU to accelerate draw calls. I had done work in the C/C++ code modifying Chromium directly and went, “that’s not how the browser talks to the OS to use the GPU for acceleration”. They immediately tried to argue I was wrong and made some more nonsense up. This person is a very competent coder, but we…

> and they were not from some rich family that could have paid their way through schooling

Huh? If they'd got private tutoring, that wouldn't make them understand things any less?

Re: What every developer should know about GPU computing

#67

Post by nologic01 »

It is anybody's guess what the future will bring, but on past form GPU programming will remain a niche for specific highly-tuned (HPC) applications and mere mortals can focus on somewhat easier multi-core CPU programming instead.

The main reason GPU gets so much attention these days is that CPU manufacturers (Intel in particular) simply cannot get their act together. Intel had promised significant breakthroughs with Xeon Phi like a decade ago.

In the meantime people have invented more and more applications that need significant computational power. But it will eventually get there. E.g., AMD' latest epyc features 96 cores. Importantly, that computational power is available in principle with simpler / more familiar programming models.

Re: What every developer should know about GPU computing

#68

Post by oivey »

mannyv wrote: post #28

Now I understand why ML uses floats for precision. It wasn't a choice, it was because graphics code uses them. Another piece in the "why is ML so inefficient" puzzle! I wonder what that memory copying overhead is IRL. If it's like normal stuff it'll be brutal. I mean, they offload tcp processing into hardware to avoid that. This is way more data, though it is done in bigger chunks.

I don’t think the choice of using floating point numbers is particularly inefficient. If frameworks were fixed point by default it would be tricky to get the dynamic ranges right all the way through a network. The math in the training assumes the numbers are continuous, too.

Re: What every developer should know about GPU computing

#69

Post by usea »

anonporridge wrote: post #49
permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

Definitely not true about most programmers, but maybe the author meant CS educated engineers. Going through a formal CS program will give you an intimate understanding of CPUs, especially when compared to the very light coverage of GPUs.

> Going through a formal CS program will give you an intimate understanding of CPUs

I did an undergrad in CS, where I did well. I don't feel like I understand CPUs very well. Certainly not anywhere in the realm of "intimate."

Re: What every developer should know about GPU computing

#70

Post by salawat »

smokel wrote: post #61
PTOB wrote: post #53

Earlier quoted context omitted.

Try this on: "A non-trivial number of Computer Scientists, Computer Engineers, Electrical Engineers, and hobbyists have ..." Took some philosophy courses for fun in college. I developed a reading skill there that lets me forgive certain statements by improving them instead of dismissing them. My brain now automatically translates over-generalizations and even outright falsehoods into rationally-nearby true statements…

That's a refreshing take. I tried real hard to understand what some continental philosophers, such as Latour, Deleuze, and Žižek are on about, giving some of their texts quite some benefit of the doubt. After about ten years of doing so, I am more and more returning to my previous opinion that some of these just like to talk, and even though they have lots to say, they say so much, that I still have to do all the act…

The Ur subject of philosophy is not the writing and reading of philosophy, but to exercise the mind in coming to grips with that which we have no conception of yet. Language is it's medium, and the IO is those texts. However the nugget at the center is figuring out that magic blackbox that lets you take all the dead ends in those texts and divine a reasonable guiding principle or recognition of an end to the space of tread discourse.

To that end, you have to approach philosophical reading with a degree of hypothetical detachment that not everyone is immediately comfortable with.

Post reply on HN