Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every developer should know about GPU computing

codeconfessions.substack.com

Post reply on HN
Print view View original

31–40 of 186 posts

Re: What every developer should know about GPU computing

#31

Post by crabbone »

Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

You are interpreting the title literally, while all this article is trying to do is to give an introduction to anyone who wants to get into GPU programming.

Now, about every developer. Ideally, developers should know something, in general, about all fields related to programming. Similar to how in medicine you need to learn about different branches of medicine, even though you'll specialize in one, and so it is in mathematics, physics and so on.

Presently, the demand for quality professionals in programming is very low. There aren't any good testing or certification programs that can tell a good programmer from a bad one. The industry is generally happy with "specialists" who perhaps only know to do one thing, somewhat. So, presently you don't need to know anything about GPU or any other field that's not directly related to your job description.

----

Now, about the article itself. While it gives a lot of valuable factual information, it's missing the forest for the trees. It's very dedicated to how CUDA works or some other particular aspects of NVidia's GPUs. The part that's missing is the part that could make it, potentially, a candidate for the kind of introduction to GPU programming that would make it worth reading to expand your general understanding of how computers work.

If you ever paid attention to how encyclopedic articles are written: the structure of a definition given by encyclopedia often has two components. First puts the object of the definition into the more general category, second explains how the object of the definition is different from other elements of the category. What the GPU article is missing is the first component: putting GPU programming into the more general category. This, in practical terms, means that questions like "is DPU programming anything like GPU programming?" or "can `smart' SSDs (with FPGAs) be treated similar to GPUs?" unanswered.

Re: What every developer should know about GPU computing

#32

Post by softfalcon »

cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

Yup, I was coming up under a senior at my work one day and he tried to very incorrectly hand wave how a browser uses a GPU to accelerate draw calls.

I had done work in the C/C++ code modifying Chromium directly and went, “that’s not how the browser talks to the OS to use the GPU for acceleration”. They immediately tried to argue I was wrong and made some more nonsense up.

This person is a very competent coder, but we got into further discussions and it was made very clear they had lived in a JavaScript landscape for either far too long, or had only barely gone over basic computing concepts. Their understanding of computers largely ended at “and then v8 does some stuff”.

It was weird cause this person had an honours level computer science degree from a very prestigious university and they were not from some rich family that could have paid their way through schooling.

They also had over a decade of experience shipping numerous projects. It really threw me for a loop as to what a software engineer needs to know to succeed at very high levels professionally.

Re: What every developer should know about GPU computing

#33

Post by [deleted] »

wongarsu wrote: post #23
loeg wrote: post #11

Earlier quoted context omitted.

At this point, I think the running shtick / inside joke of "Every Developer Should Know ..." headlines is that of course every developer doesn't need to know the contents of the article that follows.

> of course every developer doesn't need to know the contents of the article that follows. Things every developer should know about English, part 1: why word order matters in negation /s Obviously you meant "not every developer needs to know" instead of "every developer doesn't need to know", but I see this switch so often lately that I'm beginning to wonder if it's a dialect thing (similar to double negative implyin…

[deleted]

Re: What every developer should know about GPU computing

#34

Post by choppaface »

mannyv wrote: post #28

Now I understand why ML uses floats for precision. It wasn't a choice, it was because graphics code uses them. Another piece in the "why is ML so inefficient" puzzle! I wonder what that memory copying overhead is IRL. If it's like normal stuff it'll be brutal. I mean, they offload tcp processing into hardware to avoid that. This is way more data, though it is done in bigger chunks.

For a lot of larger modern networks the GPU compute time in computing gradients and doing the backwards pass is so slow that copying float data over the pcie bus is no bottleneck. I.e. copying a minibatch of float images is still plenty fast because the gradients / SGD iteration is so slow and requires so much compute (even with mixed precision).

For shallower networks, there can be an advantage to copying just the original compressed data to GPU memory and then doing decompression etc there. But modern GPUs haven’t adopted pcie 5 yet because the raw compute is more important.

Lastly tensor cores have had a big impact, depending on the network they can be fast enough to be very under-utilized.

Re: What every developer should know about GPU computing

#35

Post by galangalalgol »

abhi9u wrote: post #17
cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

True. It's becoming a lost art. Although, I think anyone who takes a CS program probably learns enough about CPUs in a Computer Organization or Computer Architecture course. Even if they never have to use that knowledge ever again. :) On the other hand I agree that people who take the non-CS route may not be that intimately familiar with all the details.

What is the non-cs route?

Re: What every developer should know about GPU computing

#36

Post by mackrevinack »

Rizu wrote: post #20
cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

Do you have any recommendations to learn CPUs and Systems well ?, thanks

not sure if this is what you have in mind but Code by Charles Petzold might be worth checking out

Re: What every developer should know about GPU computing

#37

Post by dzign »

cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

Ealier today, a post titled "We have used too many levels of abstraction" was on HN top...

https://news.ycombinator.com/item?id=37965142

Re: What every developer should know about GPU computing

#38

Post by [deleted] »

cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

[deleted]

Re: What every developer should know about GPU computing

#39

Post by yieldcrv »

softfalcon wrote: post #32
cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

Yup, I was coming up under a senior at my work one day and he tried to very incorrectly hand wave how a browser uses a GPU to accelerate draw calls. I had done work in the C/C++ code modifying Chromium directly and went, “that’s not how the browser talks to the OS to use the GPU for acceleration”. They immediately tried to argue I was wrong and made some more nonsense up. This person is a very competent coder, but we…

> I was coming up under a senior at my work one day

> It really threw me for a loop as to what a software engineer needs to know to succeed at very high levels professionally

he is arguably more successful while you have prematurely optimized

do things that make money, since that’s clearly the priority and context here

Re: What every developer should know about GPU computing

#40

Post by [deleted] »

Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

[deleted]
Post reply on HN