Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every developer should know about GPU computing

codeconfessions.substack.com

Post reply on HN
Print view View original

121–130 of 186 posts

Re: What every developer should know about GPU computing

#121

Post by tormeh »

jacquesm wrote: post #105
fancyfredbot wrote: post #97

This article is pretty good but, looking at comments so far, nobody seems to have made the obvious/predictable point that it's terribly Nvidia specific. That would be understandable perhaps a few years ago. But the era where there was no reasonable alternative is over. Sycl is a good language which performs well across multiple hardware vendors. Sapphire rapids is very good hardware. AMD's MI300 looks amazing. Nvidia…

At this point the massive investment in software is what drives this, hardware differences may no longer be dominant but the only way to unseat NV at this point is drop in replacements and/or drop in replacement libraries. And that's getting there. Which is good because GPUs are too expensive and have too little memory, some competition might help move things along rather than to give NV more time to milk their preci…

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

Re: What every developer should know about GPU computing

#122

Post by LouisSayers »

loeg wrote: post #11
Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

At this point, I think the running shtick / inside joke of "Every Developer Should Know ..." headlines is that of course every developer doesn't need to know the contents of the article that follows.

There are at most 10 things every developer should know.

The first being number base systems.

Re: What every developer should know about GPU computing

#123

Post by kdwikzncba »

> We can understand this with the help of Little’s law from queuing theory. It states that the average number of requests in the system (Qd for queue depth) is equal to the average arrival rate of requests (throughput T) multiplied by the average amount of time to serve a request (latency L).

First off, this is obviously false. If you can serve 9req/s and you're getting 10req/s the size of the queue depth is growing at a rate of 1req/s. It's not stationary.

Second, what's the connection between this and gpus? What's the queue? What's the queue depth? What are the requests?

Seems to me that the article focuses more on being smart than actually learning.

Re: What every developer should know about GPU computing

#125

Post by behnamoh »

tormeh wrote: post #121
jacquesm wrote: post #105

Earlier quoted context omitted.

At this point the massive investment in software is what drives this, hardware differences may no longer be dominant but the only way to unseat NV at this point is drop in replacements and/or drop in replacement libraries. And that's getting there. Which is good because GPUs are too expensive and have too little memory, some competition might help move things along rather than to give NV more time to milk their preci…

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

Those ml researchers wanted to get hired…

Re: What every developer should know about GPU computing

#126

Post by minitoar »

kdwikzncba wrote: post #123

> We can understand this with the help of Little’s law from queuing theory. It states that the average number of requests in the system (Qd for queue depth) is equal to the average arrival rate of requests (throughput T) multiplied by the average amount of time to serve a request (latency L). First off, this is obviously false. If you can serve 9req/s and you're getting 10req/s the size of the queue depth is growing…

The scenario is that you’re calculating Qd given a static average latency. Absent that, this formula doesn’t give you a way to compute Qd. What is the average amount of time to service a request in a system where the queue depth is growing without bound?

Re: What every developer should know about GPU computing

#127

Post by ssivark »

kdwikzncba wrote: post #123

> We can understand this with the help of Little’s law from queuing theory. It states that the average number of requests in the system (Qd for queue depth) is equal to the average arrival rate of requests (throughput T) multiplied by the average amount of time to serve a request (latency L). First off, this is obviously false. If you can serve 9req/s and you're getting 10req/s the size of the queue depth is growing…

> First off, this is obviously false. If you can serve 9req/s and you're getting 10req/s the size of the queue depth is growing at a rate of 1req/s. It's not stationary.

I haven’t formally studied any queuing theory, but I think:

1. The rule assumes you have enough processing power to service the average load (otherwise it fails catastrophically like you mentioned)

2. The rule is trying to model the fluctuations in the pending load (which might determine wait time or whatever else).

Re: What every developer should know about GPU computing

#128

Post by squeaky-clean »

abadpoli wrote: post #119
twixfel wrote: post #114

Earlier quoted context omitted.

I don't understand why every other submission on the internet has to have at least one "stopped reading at X" comment relating to it. It adds absolutely nothing.

It provides feedback to the author, and it also has generated quite a significant discussion here about the topic. It could have been worded more constructively sure, but given that the entire point of this website is to have discussions about the material posted, I think it’s added quite a bit.

It hasn't really generated any discussion about the topic at hand though. Just people discussing what they learned or didn't learn about CPUs in school.

Re: What every developer should know about GPU computing

#129

Post by moritzwarhier »

permo-w wrote: post #47

>Most programmers have an intimate understanding of CPUs maybe this article is brilliant, but when the first line is something so blatantly untrue it really makes it hard to take the rest seriously

I think at least 50% of the answers of this would depend on how one defines "intimate understanding"...

I learned basic facts about CPU architectures at university, know in a very basic way the landscape of things and occasionaly stumble upon updates to my limited knowledge... but by no means would I say that, rather like "a basic understanding of how CPUs work / are designed / are to be used" (?)

If I were proficient in assembler, maybe I'd claim to have an "intimate understanding" of how to use CPUs at a low level (still sounds a bit braggy)

It still is not the same though as being an expert in CPU/GPU design.

So yeah I agree.

Article is interesting though, esp. the diagram!

Re: What every developer should know about GPU computing

#130

Post by jacquesm »

tormeh wrote: post #121
jacquesm wrote: post #105

Earlier quoted context omitted.

At this point the massive investment in software is what drives this, hardware differences may no longer be dominant but the only way to unseat NV at this point is drop in replacements and/or drop in replacement libraries. And that's getting there. Which is good because GPUs are too expensive and have too little memory, some competition might help move things along rather than to give NV more time to milk their preci…

I really wish the ML researchers would have stayed with Vulkan or OpenCL instead of standardising on CUDA. Everyone must have known how it would end.

Sometimes people just have a job to do rather than to play politics and in this case doing it with OpenCL or Vulkan would have ceded the market for their application to the competition due the difference in speed and money spent for a given amount of compute. That's starting to change but the competition has reacted much too late. NVidia figured out that software is almost as important as the hardware early enough that betting on that with a sizeable dedicated team gave them a tremendous lead. I've used CUDA and have tried to use OpenCL but the number of hurdles I had to jump through with OpenCL made it a non-starter and that was before factoring in the cost. NV made it easy and made it work right out of the box. Quite often the most visible constraint is developer time and reducing developer friction is a great way to capture a larger slice of the market as well as to actually grow that market.
Post reply on HN