Ask HN: What is an A.I. chip and how does it work? • Hacker News Board

Re: Ask HN: What is an A.I. chip and how does it work?

#71

Post by visarga » Sat, May 27, 2023, 4:16 PM UTC

dreamcompiler wrote: post #51

bhu1st wrote: post #40

Earlier quoted context omitted.

> If you are familiar with linear algebra Could you please write what are some common day to day life applications of linear algebra in computing?

Nvidia exists because linear algebra is essential for 3D graphics. They got a second boost from crypto-currency, which isn't strictly about linear algebra but it can put those computation units to good use. Now Nvidia is riding high again because neural nets are all about linear algebra, and LLMs are big neural nets.

I would also add scientific simulations to the list of tasks that GPUs are used for. They parallelise well on many cores.

Re: Ask HN: What is an A.I. chip and how does it work?

#72

Post by rerdavies » Sat, May 27, 2023, 4:23 PM UTC

nologic01 wrote: post #11

It may help your digging and search if you have in mind what those chips really try to do: Accelerate numerical linear algebra calculations. If you are familiar with linear algebra these specialized chips literally etch silicon so as to perform vector (and more general multi-array or tensor) computations faster than a general purpose CPU. They do that by loading and operating a whole set of numbers (a chunk of a vect…

> Accelerate numerical linear algebra calculations.

Technically, for AI, you need to accelerate numerical non-linear algebra calculations, which take the general form of matrix multiplication, but interpose non-linear functions at key points in the calculation.

Re: Ask HN: What is an A.I. chip and how does it work?

#73

Post by visarga » Sat, May 27, 2023, 4:26 PM UTC

Trying to make a list of AI accelerator chip families, anything missing?

- GPU (Graphics Processing Unit)

- TPU (Tensor Processing Unit): ASIC designed for TensorFlow

- IPU (Intelligence Processing Unit): Graphcore

- HPU (Habana Processing Unit): Intel Habana Labs' Gaudi and Goya AI

- NPU (Neural Processing Unit): Huawei, Samsung, Microsoft Brainwave

- VPU (Vision Processing Unit): Intel Movidius

- DPU (Data Processing Unit): NVIDIA data center infrastructure processing unit

- Amazon's Inferentia: Amazon's accelerator chip focused on low cost

- Cerebras Wafer Scale Engine (WSE)

- SambaNova Systems DataScale

- Groq Tensor Streaming Processor (TSP)

Re: Ask HN: What is an A.I. chip and how does it work?

#74

Post by ra1231963 » Sat, May 27, 2023, 4:35 PM UTC

rerdavies wrote: post #72

nologic01 wrote: post #11

It may help your digging and search if you have in mind what those chips really try to do: Accelerate numerical linear algebra calculations. If you are familiar with linear algebra these specialized chips literally etch silicon so as to perform vector (and more general multi-array or tensor) computations faster than a general purpose CPU. They do that by loading and operating a whole set of numbers (a chunk of a vect…

> Accelerate numerical linear algebra calculations. Technically, for AI, you need to accelerate numerical non-linear algebra calculations, which take the general form of matrix multiplication, but interpose non-linear functions at key points in the calculation.

the calculations within a neural network involve both linear and non-linear operations, but the fundamental mathematical framework underlying AI remains rooted in linear algebra

Re: Ask HN: What is an A.I. chip and how does it work?

#75

Post by rramadass » Sat, May 27, 2023, 4:51 PM UTC

visarga wrote: post #73

Trying to make a list of AI accelerator chip families, anything missing? - GPU (Graphics Processing Unit) - TPU (Tensor Processing Unit): ASIC designed for TensorFlow - IPU (Intelligence Processing Unit): Graphcore - HPU (Habana Processing Unit): Intel Habana Labs' Gaudi and Goya AI - NPU (Neural Processing Unit): Huawei, Samsung, Microsoft Brainwave - VPU (Vision Processing Unit): Intel Movidius - DPU (Data Processi…

Nice and very much in line with what i am looking for !

If you don't mind, could you add links to authoritative wiki pages/whitepapers/articles for each of the above? I think it will give us a good starting point to start our study/research from.

Re: Ask HN: What is an A.I. chip and how does it work?

#76

Post by rramadass » Sat, May 27, 2023, 4:56 PM UTC

zoogeny wrote: post #55

There is a YouTube channel TechTechPotato [1] that has a podcast on AI hardware called "The AI Hardware Show". Pretty small and it gives you a view on how niche this market is - but if you want the 10k foot view from young budding tech journalists then I think this fits the bill. Some random examples of video titles from the last 6 months of the channel: * A Deep Dive into IBM's New Machine Learning Chip * Does my PC…

Nice, looks like a good starting point to survey the field.

Re: Ask HN: What is an A.I. chip and how does it work?

#77

Post by sethgoodluck » Sat, May 27, 2023, 4:59 PM UTC

Remember crypto miners (ASICS). Exact same thing but built for the math around AI work instead of Blockchain work.

Re: Ask HN: What is an A.I. chip and how does it work?

#78

Post by m3kw9 » Sat, May 27, 2023, 5:03 PM UTC

AI chip is basically a chip that calculates matrix’s better than general purpose CPUs

Re: Ask HN: What is an A.I. chip and how does it work?

#79

Post by rramadass » Sat, May 27, 2023, 5:08 PM UTC

imakwana wrote: post #58

Relevant: Stanford Online course - CS217 Hardware acceleration for machine learning https://online.stanford.edu/courses/cs217-hardware-accelerat... Course website with lecture notes: https://cs217.stanford.edu/ Reading list: https://cs217.stanford.edu/readings

Excellent!

Re: Ask HN: What is an A.I. chip and how does it work?

#80

Post by rramadass » Sat, May 27, 2023, 5:26 PM UTC

anon291 wrote: post #65

I've worked in this space for the past five years. The chips are essentially highly parallel processors. There's no unifying architecture. You have the graph-based / hpc-simulator chips like Cerebras, Graphcore, etc which are basically a stick-as-many-cores-as-possible situation with a high-speed networking fabric. You have the 'tensor' cores like Groq where the chip operates as a whole and is just well suited for te…

>The chips are essentially highly parallel processors

Right. AFAIK we already were doing SIMD, Vector Processing, VLIW etc. to speed up parallel processing for numerical calculations in AI/ML. What then is the reason for the explosion of these different categories of chips? Are they just glorified ASICs designed for their specific domains or FPGAs programmed accordingly? If so what is their architecture i.e. what are their functional units and how do they work together?