Will there be a customer grade TPU in the near future? Or won't they be able to be as price/performance efficient compared to (nvidia) GPUs?
Apple’s neural engine in their A-series of SoCs.
New chips for machine intelligence
31–40 of 55 posts
Re: New chips for machine intelligence
#32Some of the numbers in that table do not make any sense and makes me question the quality of the entire article. Where are the numbers for the Cerebras chip coming from?: - How do you have a TDP of 180W for an entire wafer of chips? - Why is there a peak FP32 number when they are clearly working with FP16? Each of these chips is a completely different architecture and it makes no sense to compare them at this level.…
One of the numbers that jumped out at me as being very unusual about the Cerebrus chip was this one: "Speculated clock speed of ~1 GHz and 15 kW power consumption." 15 kW power consumption for 1 chip?!?
Reportedly, it's an insanely large (whole wafer) single chip. But, then, why is it listed with a size that's not commensurate with that?
Re: New chips for machine intelligence
#33Some of the numbers in that table do not make any sense and makes me question the quality of the entire article. Where are the numbers for the Cerebras chip coming from?: - How do you have a TDP of 180W for an entire wafer of chips? - Why is there a peak FP32 number when they are clearly working with FP16? Each of these chips is a completely different architecture and it makes no sense to compare them at this level.…
[deleted]
Re: New chips for machine intelligence
#34Some of the numbers in that table do not make any sense and makes me question the quality of the entire article. Where are the numbers for the Cerebras chip coming from?: - How do you have a TDP of 180W for an entire wafer of chips? - Why is there a peak FP32 number when they are clearly working with FP16? Each of these chips is a completely different architecture and it makes no sense to compare them at this level.…
One of the numbers that jumped out at me as being very unusual about the Cerebrus chip was this one: "Speculated clock speed of ~1 GHz and 15 kW power consumption." 15 kW power consumption for 1 chip?!?
I heard that from an engineer there. He was aghast too.
Re: New chips for machine intelligence
#35Some of the numbers in that table do not make any sense and makes me question the quality of the entire article. Where are the numbers for the Cerebras chip coming from?: - How do you have a TDP of 180W for an entire wafer of chips? - Why is there a peak FP32 number when they are clearly working with FP16? Each of these chips is a completely different architecture and it makes no sense to compare them at this level.…
In the table, the figures are for a single die in the wafer. This is to make a meaningful comparison with the other chips listed (there is a table footnote for this). The 15 KW is the power consumption of the whole wafer (a detail I think was mentioned in the Hot Chips presentation). Why are they clearly working with FP16? Are there any public details on this?
Re: New chips for machine intelligence
#36Are deep neural networks really that widely applicable that it's profitable to design custom chips for them? What about other models of AI that involve, say, discrete math or graph search?
Yes. They are far beyond any other AI technique in speech recognition, speech synthesis, translation, OCR, object recognition, playing Go, and many other diverse tasks. And their performance continues to increase with added computing power with no limit that we've seen yet, so custom hardware improves results.
Re: New chips for machine intelligence
#37Re: New chips for machine intelligence
#38Software. Software. Software. Just two companies, Google and NVIDIA, have publicly launched a viable service or software stack. Just two companies have successfully written a "sufficiently advanced compiler". Just two companies actually have a product. And Google refuses to step into the arena and actually compete with NVIDIA. Man, what a time we live in.
And no, AMD doesn't count. ROCm is a mess.
Re: New chips for machine intelligence
#39I am surprised Amazon has not jumped in the game, renting out an accelerator like Google does with the TPUs
AWS gpu compute is extremely expensive. If this is due to datacenter licensing costs, I hope they come out with their own hardware soon to reduce these costs. If on the otherhand, it's because their value-add is not in renting out the hardware but burst scalability, then I'm less optimistic that they'd cannibalize their own cloud product. Currently, it only takes about 1 month to break even if you buy a consumer gpu…
Exactly. AWS is simply too expensive. You can buy a Lambda Quad GPU workstation and pay it back in a couple of months. If you want to save more you can just build it yourself.
https://lambdalabs.com/blog/8-v100-server-on-prem-vs-p3-inst...
Re: New chips for machine intelligence
#40Will there be a customer grade TPU in the near future? Or won't they be able to be as price/performance efficient compared to (nvidia) GPUs?
Do the Google Coral products count as consumer grade?