Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google supercharges machine learning tasks with TPU custom chip

cloudplatform.googleblog.com

Post reply on HN
Print view View original

191–200 of 283 posts

Re: Google supercharges machine learning tasks with TPU custom chip

#191

Post by DigitalJack »

chlestakoff wrote: post #171
luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

Now that you are free to talk about it, could you explain how ASICs stack up against GPUs, in practical terms?

ASIC stands for "application specific integrated circuit" and the name conveys a special purpose design of an integrated circuit, like for example bit coin mining.

But that's just terminology/convention. One could argue a GPU is an ASIC, and a CPU is an ASIC. The only thing to argue is how specific does an application have to be to call it an ASIC instead of some other made up name.

The prime differentiators are the process, which is the term used for the many steps of fabrication of an IC (processes are often referred to as nodes, distinguished by the smallest feature size of a transistor they create), and the ability of the design engineers to create an optimal design, in terms of boolean logic and semiconductor physics.

Given the right team of engineers, and a top notch foundry, and a great deal of experience in the problem domain (machine learning in this case), a custom IC could very likely trounce a GPU.

GPUs were designed for the domain of graphics processing, which happens to have some commonality with the processing in machine learning. But, at least until recently, GPUs weren't focused on machine learning. Just graphics.

Now the GPU vendors are trying to leverage their knowledge of graphics processing and building of graphics processors to create machine learning processors, but the thing they are leveraging could also be what handcuffs them. Which gives opportunity for a company like Google to do a fresh take on the problem domain without the baggage of the knowledge of graphics processing.

Re: Google supercharges machine learning tasks with TPU custom chip

#192

Post by eggy »

Pretty quick implementation.

On the energy savings and space savings front, this type of implementation coupled with the space-saving, energy-saving claims of going to unums vs. float should get it to the next order of magnitude. Come on, Google, make unums happen!

Re: Google supercharges machine learning tasks with TPU custom chip

#193

Post by pritambarhate »

vgt wrote: post #157
jbooth wrote: post #115

Earlier quoted context omitted.

If I recall correctly, it took Google a while to actually offer the boring stuff. For a while, you could get a Google Compute Engine but you couldn't just get a dang VM image, because Google knows better than you and you should do things their way. They've fixed it now, but lost a lot of potential market share for that conceit.

If I recall correctly myself (getting old!), Google has had the ability to build your own custom image since GCE went GA 2.5 years ago. Now, admittadly, it took a while to get IAM and VPC going, but we done did it now! I'd love to hear what other boring stuff has been a showstopper for you, in case we missed something dumb :)

Not having Managed Postgres and Managed Redis are 2 main showstoppers for me.

Re: Google supercharges machine learning tasks with TPU custom chip

#194

Post by hyperopt »

breatheoften wrote: post #143

A podcast I listen to posted an interview with an expert last week saying that he perceived that much of the interest in custom hardware for machine learning tasks died when people realized how effective GPUs were at the (still-evolving-set-of) tasks. http://www.thetalkingmachines.com/blog/2016/5/5/sparse-codin... I wonder how general the gains from these ASIC's are and whether the performance/power efficiency wins w…

I listen to the Talking Machines as well. Great podcast. Another question would be are the gains worth the cost of an ML-specific ASIC. GPUs have the entire, massive gaming industry driving the cost down. I suppose that as adoption of gradient-descent-based neural networks increases, it may be worth the cost in a similar way that GPUs are worth the cost. Then again, I have never implemented SGD on a GPU so I'm not aware if there are any bottlenecks that could be solved with an ML-specific ASIC. Can anyone else shed some light on this?

Re: Google supercharges machine learning tasks with TPU custom chip

#195

Post by Veratyr »

blue11 wrote: post #113
luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

OK, so what is it? The announcement neither says what a TPU actually is nor what it can do. It's a magic black box. No specs. No price.

Since nobody has pointed this out, although he may be able to tell people "I was working on that thing", he likely can't say much more. Many large companies don't allow employees to disclose details that haven't been released publicly.

Re: Google supercharges machine learning tasks with TPU custom chip

#196

Post by zhemao »

hristov wrote: post #140

It is interesting that they would make this into an ASIC, provided how notoriously high the development costs for ASICs are. Are those costs coming down? If so life will get very hard for the FPGA makers of the world soon. It would be interesting to see what the economics of this project are. I.e., what are the development costs and costs per chip. Of course it is very doubtful I will ever get to see the economics of…

It's mainly a question of volume. ASICs have a big economy of scale, so the cost-per-chip goes down considerably once you go over a certain number of chips. Plus, there's all the NRE costs of an ASIC design over an FPGA design. Google probably figured they could use enough chips to make the cost of manufacturing ASICs worthwhile.

I don't think FPGAs are going to be beat out by ASICs for low volume applications anytime soon.

Re: Google supercharges machine learning tasks with TPU custom chip

#198

Post by sandGorgon »

luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

What was your design flow? Did you use the standard cadence, Synopsys toolset to build this? How did you achieve basic design and JIT(or whatever it is you call the machine code part of the Python VM ) together?

Rrally interested to know how this was managed...

Re: Google supercharges machine learning tasks with TPU custom chip

#199

Post by zhemao »

amelius wrote: post #105
zymhan wrote: post #96

Earlier quoted context omitted.

Wouldn't you typically prototype the design on an FPGA and then manufacture an ASIC once you'd worked out the kinks?

I'm suspecting that the design contains a lot of repetitive units, each of which can easily be tested in software. So perhaps testing on an FPGA isn't even needed.

I think zymhan meant prototyping on an FPGA to see if you could actually see an improvement by mapping parts of your algorithm to custom hardware. This would be a sensible first step before committing to a full ASIC run. I'm sure Google did do this initially.

As for testing for correctness, ASIC designs are almost always tested in software simulation instead of on FPGAs. Mainly because RTL written for FPGAs can look quite different from RTL written for ASIC synthesis. There's also a lot incidental engineering work needed to get something working on an FPGA.

Post reply on HN