Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every developer should know about GPU computing

codeconfessions.substack.com

Post reply on HN
Print view View original

11–20 of 186 posts

Re: What every developer should know about GPU computing

#11

Post by loeg »

Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

At this point, I think the running shtick / inside joke of "Every Developer Should Know ..." headlines is that of course every developer doesn't need to know the contents of the article that follows.

Re: What every developer should know about GPU computing

#12

Post by cebert »

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU

Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

Re: What every developer should know about GPU computing

#13

Post by HPsquared »

Jeff_Brown wrote: post #7
Jiro wrote: post #4

Earlier quoted context omitted.

Over-specialization is hard on your ability to find another job if you lose your current one.

Completely true. The analogy to hydration holds, I think -- too much and too little can both be problematic. But I see seem to run into much more under-specialization than over-specialization. More than 90% of the wildest successes I know went really deep on something pretty narrow.

Specialization is like putting everything on one number in roulette. It's great if you win (i.e. your specialty is in demand), but you're more likely to lose everything you invested.

Re: What every developer should know about GPU computing

#14

Post by abhi9u »

yodsanklai wrote: post #8
Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

> It can feel like a paintful sacrifice to admit that you'll never (have time to) learn And even if you do have time, you'll probably forget most of it if you don't make use of it in your daily job. Also if you do want to learn about a new topic, it'll take commitment. I remember going through an MIT OS project, it took me something like 50 hours of hours to complete the project. Pretty much impossible when you have…

Thank you :)

Re: What every developer should know about GPU computing

#15

Post by cj »

Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

One thing I find interesting about the software industry is our lack of descriptive job titles.

At big companies you have SWE L1, L2, L3, senior, staff, principal, etc. You also have SRE and maybe some devops or architect roles.

Smaller companies you have lots of people generic “engineer” titles or “full stack engineers”, etc.

Why don’t we encode people’s specialties in their titles if most engineers are working on narrow sections of software?

E.g. “React & Node Developer” instead of “Full stack engineer” … etc

I suppose the easiest rationale is generic titles allow for easier mobility between disciplines.

Re: What every developer should know about GPU computing

#16

Post by dr_kiszonka »

cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

It's not just where you work, in my experience.

Re: What every developer should know about GPU computing

#17

Post by abhi9u »

cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

True. It's becoming a lost art. Although, I think anyone who takes a CS program probably learns enough about CPUs in a Computer Organization or Computer Architecture course. Even if they never have to use that knowledge ever again. :)

On the other hand I agree that people who take the non-CS route may not be that intimately familiar with all the details.

Re: What every developer should know about GPU computing

#18

Post by abhi9u »

nabla9 wrote: post #5
Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

This is just minimum for those don't need to know details. Even general knowledge has depth. Even if you are generalist, or specialist in different area you should deepen your knowledge at every area gradually.

True. This is probably what first few lectures of a first course on GPU computing would cover. The nitty gritty of how to write parallel algorithms for maximum throughput on the GPU is where things become too specific. :)

Re: What every developer should know about GPU computing

#19

Post by abhi9u »

cratermoon wrote: post #6
Jeff_Brown wrote: post #3

Not every developer. I'm not trying to be snarky. I think there's an unhelpful compunction to want to know everything about everything among STEM types like programmers (of which I am one). Specialization is fundamental to the success, not just of whole economies, but of the individuals in them. It can feel like a paintful sacrifice to admit that you'll never (have time to) learn, say, the entire Python language spec…

I dunno. The section "Latency Tolerance, High Throughput and Little’s Law" has many applications in programming. Ever need to scale your cache, or size a connection pool? Little's Law.

Thank you for reading! :)

Re: What every developer should know about GPU computing

#20

Post by Rizu »

cebert wrote: post #12

> Most programmers have an intimate understanding of CPUs and sequential programming because they grow up writing code for the CPU Maybe it’s just where I work, but I feel like even this isn’t true. A lot of the newer/young employees don’t even seem to have a lot of OS/system understanding. A lot of the higher-level languages abstract the immediate need of having any “intimate understanding” of CPUs.

Do you have any recommendations to learn CPUs and Systems well ?, thanks
Post reply on HN