1–10 of 95 posts
Can anybody give an overview of AI/ML/NPU/TPU/etc chips and pointers to detailed technical papers/books/videos about them? All i am able to find are marketing/sales/general overviews which really don't explain anything.
Am looking for a technical deep dive.
It's a matrix multiplication (https://en.wikipedia.org/wiki/Matrix_multiplication) accelerator chip. Matrix multiplication is important for a few reasons. 1) it's the slow part of a lot of AI algorithms, 2) it's a 'high intensity' algorithm (naively n^3 computation vs. n^2 data), 3) it's easily parallelizable, and 4) it's conceptually and computationally simple.
The situation is kind of analogous to FPUs (floating point math co-processor units) when they were first introduced before they were integrated into computers.
There's more to it than this, but that's the basic idea.
GPUs were optimised to draw, so were able to do dozens of these at a go. So these can be used for AI/ML in both gradient descent and inference (forward passes). Because you can do many at a go, in parallel, they speed things up dramatically. Geoff Hinton experimented with GPUs exploiting their ability to do this, but they aren't actually optimised to do that. It just turned out that it is the best way available to do it at the time, and still currently.
AI chips, are optimised to do either inference or gradient descent. They are not good at drawing like GPUs are. They are optimised for machine learning and joining other AI chips together so you can have massive networks of chips that can parallel compute.
One other class of chips that has not yet shown up are ASICs that mimic the transformers architectures for even more speed - though it changes too much at the moment for it to be useful.
Also because of the mechanics of scale manufacturing: GPUs are currently cheaper per flop of compute as the aggregate of scale is shared with graphical uses. Though with time if there is enough scale AI chips should end up cheaper
Veritasium did a pretty good video on some of them: https://www.youtube.com/watch?v=GVsUOuSjvcg
Veritasium did a pretty good video on some of them: https://www.youtube.com/watch?v=GVsUOuSjvcg
That video is about analog computers
OP asked about chips for AI.
Short story, CPU's can do calculations, they can do them one at a time. Think of something like 1+1, = 2. If you had 1 million equations like these, CPU's will generally do them one at a go, i.e the first one, then the second, etc. GPUs were optimised to draw, so were able to do dozens of these at a go. So these can be used for AI/ML in both gradient descent and inference (forward passes). Because you can do many at…
I read the first book in this list about 10 years ago, and though it’s pretty old the concepts are solid.