Modern GPU Programming for MLSys

Everything after "Pipelining GEMM with TMA" (inclusive) is specific to NVIDIA. Which is fine but the title (of the guide itself) is clearly misleading.

misleading?

these are model-level frameworks

> Triton

this is a kernel DSL

> [cublas]

this is a BLAS library built atop CUDA

> I know they might be for different purposes, but having some idea what is for what and when to use would be helpful

when people ask this question i always ask: who are you and what is your job? if you're not an ML/DL/AI person then you knowing the specifics is about as useful as me knowing the specifics of react/express/angular/tailwind/django/whatever as an ML person. this is not meant to be condescending, this is meant to allay your anxiety, ie that if you ever find yourself in the position where you have to know these things for your job, it won't be that hard to figure out (just like it isn't that hard to figure out the difference between react and express and django if you're a webdev).

let's swap roles and let's pretend i'm an ML engineer asking you how to enter CRUD. what would you tell me? my strong suspicion (if i caught you in an honest, frank, moment) is you would say to me "why the fuck would you want to do that - it sucks". i have this suspicion because i did actually used to do CRUD and it does suck! but here's your moment of zen: so does ML/DL/AI. it really really does suck. it's basically just as bad as webdev in terms of tedium/boredom/incidental complexity/etc. it's not fun, interesting, exciting, whatever else you're projecting based on an outside-looking-in-perspective.

now i'll acknowledge that there's one big difference: the pay is way better at the far end of the distribution - meaning if you can get to a FAANG ML team then you'll get more money than you're probably getting now (and a ton more stress too) and it's even more than the CRUD devs in FAANG. fine. but ask yourself if it's really worth learning a whole heap of new bullshit just for a chance at more money (no guarantee).

okay now a useful/practical answer: i went back to school for a PhD but i should've just dropped out with the MS. do that. even better do Georgia Tech's online MS.