Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New chips for machine intelligence

jameswhanlon.com

Post reply on HN
Print view View original

41–50 of 55 posts

Re: New chips for machine intelligence

#41

Post by trust07007707 »

sabalaba wrote: post #38

Software. Software. Software. Just two companies, Google and NVIDIA, have publicly launched a viable service or software stack. Just two companies have successfully written a "sufficiently advanced compiler". Just two companies actually have a product. And Google refuses to step into the arena and actually compete with NVIDIA. Man, what a time we live in. And no, AMD doesn't count. ROCm is a mess.

I wonder if WebGPU will reduce dependence on CUDA, esp as Tensorflow is being ported to WebGPU. With WebGPU's improved performance and utility and the fact that it runs on top of Vulcan, Metal and D3D with any GPU that has drivers for those, I wonder if DL folks will find it more tempting to use TFJS/WebGPU via Electron or the browser and just be done with CUDA (i.e. break or soften NVIDIA's monopoly)

Re: New chips for machine intelligence

#42

Post by alexhutcheson »

trust07007707 wrote: post #41
sabalaba wrote: post #38

Software. Software. Software. Just two companies, Google and NVIDIA, have publicly launched a viable service or software stack. Just two companies have successfully written a "sufficiently advanced compiler". Just two companies actually have a product. And Google refuses to step into the arena and actually compete with NVIDIA. Man, what a time we live in. And no, AMD doesn't count. ROCm is a mess.

I wonder if WebGPU will reduce dependence on CUDA, esp as Tensorflow is being ported to WebGPU. With WebGPU's improved performance and utility and the fact that it runs on top of Vulcan, Metal and D3D with any GPU that has drivers for those, I wonder if DL folks will find it more tempting to use TFJS/WebGPU via Electron or the browser and just be done with CUDA (i.e. break or soften NVIDIA's monopoly)

I've never heard anyone suggest that WebGPU would be appropriate for ML training workloads. Maybe inference, but not training.

Re: New chips for machine intelligence

#43

Post by nappy-doo »

sabalaba wrote: post #38

Software. Software. Software. Just two companies, Google and NVIDIA, have publicly launched a viable service or software stack. Just two companies have successfully written a "sufficiently advanced compiler". Just two companies actually have a product. And Google refuses to step into the arena and actually compete with NVIDIA. Man, what a time we live in. And no, AMD doesn't count. ROCm is a mess.

There is one more company with a viable compiler: https://ai.facebook.com/tools/glow/

I don't have first hand experience using it, but from people I know, it does work.

Re: New chips for machine intelligence

#46

Post by streetcat1 »

modeless wrote: post #36
brookhaven_dude wrote: post #26

Are deep neural networks really that widely applicable that it's profitable to design custom chips for them? What about other models of AI that involve, say, discrete math or graph search?

Yes. They are far beyond any other AI technique in speech recognition, speech synthesis, translation, OCR, object recognition, playing Go, and many other diverse tasks. And their performance continues to increase with added computing power with no limit that we've seen yet, so custom hardware improves results.

Alas, you do not usually train models from scratch. I think that transfer learning will dominate, and it does not need this power.

Re: New chips for machine intelligence

#47

Post by trust07007707 »

alexhutcheson wrote: post #42
trust07007707 wrote: post #41

Earlier quoted context omitted.

I wonder if WebGPU will reduce dependence on CUDA, esp as Tensorflow is being ported to WebGPU. With WebGPU's improved performance and utility and the fact that it runs on top of Vulcan, Metal and D3D with any GPU that has drivers for those, I wonder if DL folks will find it more tempting to use TFJS/WebGPU via Electron or the browser and just be done with CUDA (i.e. break or soften NVIDIA's monopoly)

I've never heard anyone suggest that WebGPU would be appropriate for ML training workloads. Maybe inference, but not training.

Well, the memory limit of a WebGPU process would be the limiting factor for training. In addition, the bandwidth between the nodes and the parameter server, if doing training in data-parallel fashion, is another limiting factor.

Re: New chips for machine intelligence

#48

Post by lern_too_spel »

Q6T46nT668w6i3m wrote: post #31
zapnuk wrote: post #12

Will there be a customer grade TPU in the near future? Or won't they be able to be as price/performance efficient compared to (nvidia) GPUs?

Apple’s neural engine in their A-series of SoCs.

This article is about hardware for training neural networks, not the inference chips that are in most phones today.

Re: New chips for machine intelligence

#49

Post by lern_too_spel »

JoeDaDude wrote: post #40
zapnuk wrote: post #12

Will there be a customer grade TPU in the near future? Or won't they be able to be as price/performance efficient compared to (nvidia) GPUs?

Do the Google Coral products count as consumer grade? https://coral.withgoogle.com/docs/accelerator/datasheet/

No, those are designed for inference.

Re: New chips for machine intelligence

#50

Post by dgacmu »

zapnuk wrote: post #12

Will there be a customer grade TPU in the near future? Or won't they be able to be as price/performance efficient compared to (nvidia) GPUs?

Consumer turing cards are about the closest you get right now. They're pretty reasonable bang for buck for training. They have tensor cores - not quite as many as Ulta, but the entire chip runs at a higher clock rate and the price/performance is better if you don't mind losing a gig or so of RAM and some memory bandwidth.
Post reply on HN