Google supercharges machine learning tasks with TPU custom chip • Hacker News Board

Re: Google supercharges machine learning tasks with TPU custom chip

#161

Post by LogicFailsMe » Wed, May 18, 2016, 11:51 PM UTC

dgacmu wrote: post #149

LogicFailsMe wrote: post #73

Perf/W, the official metric of slow but efficient processors. How many times must we go down this road? Let's see this sucker train AlexNet...

Wearing my CMU hat for a moment (but keeping in mind Google's paying me this year): Google's always been cautious about the balance of speed and efficiency, out of concerns about programmer productivity, parallelization, and generality. See, for example, Urs's article in response to my and a few other people's crazy-academic research on using "Wimpy" nodes: http://static.googleusercontent.com/media/research.google.co…

Because there's an absolute sippy straw of bandwidth to the thing if that's a 1x pci-e connection.

For if it were delivering performance on par with a $1000 Maxwell class GPU, why wouldn't you guys crow about it? That would be a really big deal wouldn't it? TitanX for 20W? That'd be awesome.

And having suffered through multiple pitches for us to buy various FPGA and boutique processors, I have yet to see someone who produced perf per watt numbers first, subsequently produce an impressive performance number. In fact, it took nearly yelling at one vendor for them to finally admit perf was abysmal.

Finally, training does not equal inference. Training requires strong scaling, but inference need only weak scale. So I suspect that Urs had to bite his tongue and buy a bunch of gpus for training networks.

Am I missing something?

Re: Google supercharges machine learning tasks with TPU custom chip

#162

Post by hedgehog » Wed, May 18, 2016, 11:53 PM UTC

manav wrote: post #150

Interesting. Plenty of work has been done with FPGAs, and a few have developed ASICs like DaDianNao in China [1]. Google though actually has the resources to deploy them in their datacenters. Microsoft explored something similar to accelerate search with FPGAs [2]. The results show that the Arria 10 (20nm latest from Altera) had about 1/4th the processing ability at 10% of the power usage of the Nvidia Tesla K40 (25w…

You can build an ASIC with fast external memory, it adds to the cost but then you can handle larger models similar to a GPU. Software support is an issue but for deep learning applications there's no reason in principle you couldn't add support to TensorFlow etc for new hardware to make it simple for application developers to adopt. Movidius has announced that they're doing this and it's likely that other ML chip vendors will do the same.

Re: Google supercharges machine learning tasks with TPU custom chip

#163

Post by habitue » Thu, May 19, 2016, 12:00 AM UTC

nickysielicki wrote: post #42

honkhonkpants wrote: post #22

Earlier quoted context omitted.

Apple designs their own CPUs. I think they'd be able to field a massively parallel FMAC chip if they thought that was a good idea. Where Apple really looks weak is in datacenters, networking, and cloud services.

What does the iPhone of 2021 look like? I get the feeling from today's announcements that Google sees the 2021 version of Google Now as the selling point for their 2021 Nexus line. I don't think Apple is preparing to compete on that.

I would say they're already not competing on the assistant side. Siri is considerably worse than Google Now, even though it came out first

Re: Google supercharges machine learning tasks with TPU custom chip

#164

Post by hedgehog » Thu, May 19, 2016, 12:02 AM UTC

mrpippy wrote: post #120

Bah, SGI made a Tensor Processing Unit XIO card 15 years ago. evidence suggests they were mostly for defense customers: http://forums.nekochan.net/viewtopic.php?t=16728751 http://manx.classiccmp.org/mirror/techpubs.sgi.com/library/m...

The term is pretty generic, Nervana also calls their chip a "tensor processing unit".

Re: Google supercharges machine learning tasks with TPU custom chip

#165

Post by nxzero » Thu, May 19, 2016, 12:05 AM UTC

Is there anyway to detect what hardware to being used by the cloud service if you're using the cloud service? (yes, realize this question is a bit of a paradox, but figured I'd ask.)

Re: Google supercharges machine learning tasks with TPU custom chip

#166

Post by nxzero » Thu, May 19, 2016, 12:05 AM UTC

Is there anyway to detect what hardware to being used by the cloud service if you're using the cloud service? (yes, realize this question is a bit of a paradox, but figured I'd ask.)

Re: Google supercharges machine learning tasks with TPU custom chip

#167

Post by Keyframe » Thu, May 19, 2016, 12:08 AM UTC

mrpippy wrote: post #120

Bah, SGI made a Tensor Processing Unit XIO card 15 years ago. evidence suggests they were mostly for defense customers: http://forums.nekochan.net/viewtopic.php?t=16728751 http://manx.classiccmp.org/mirror/techpubs.sgi.com/library/m...

Damn, what a riveting read would be to find out what IT toys defense has now! Even an informed speculation would be nice to read.

Re: Google supercharges machine learning tasks with TPU custom chip

#168

Post by dekhn » Thu, May 19, 2016, 12:34 AM UTC

rcpt wrote: post #136

luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

> it's impossible to deploy GPUs at scale, let alone ASICs A bit old but cf. DE Shaw and Anton https://en.m.wikipedia.org/wiki/Anton_(computer)

I think the jury is truly still out on whether the anton designs are worth it (compared to commodity clusters/GPUs, etc).

and Anton wasn't really "scale" in this sense. It was a vertically scaled single (or several) machines.

Re: Google supercharges machine learning tasks with TPU custom chip

#169

Post by Razengan » Thu, May 19, 2016, 12:36 AM UTC

nickysielicki wrote: post #42

honkhonkpants wrote: post #22

Earlier quoted context omitted.

Apple designs their own CPUs. I think they'd be able to field a massively parallel FMAC chip if they thought that was a good idea. Where Apple really looks weak is in datacenters, networking, and cloud services.

What does the iPhone of 2021 look like? I get the feeling from today's announcements that Google sees the 2021 version of Google Now as the selling point for their 2021 Nexus line. I don't think Apple is preparing to compete on that.

Apple's strength is in consumer (and to a lesser extent, developer) ecosystems; the cozy comfortable bubble you get when you're surrounded by everything Apple. Getting access to your stuff across multiple devices is virtually effortless and continually seamless, with almost no configuration required.

Whether that's good or not may be arguable, but it's certainly a selling point for many and I don't see Google or any other company's offerings approaching the same experience, and I suspect that's by design; they have to be more open and support all devices but that kinda dilutes everything. Apple will only get stronger in that aspect IMO.

Re: Google supercharges machine learning tasks with TPU custom chip

#170

Post by bravo22 » Thu, May 19, 2016, 12:55 AM UTC

Given the insane mask costs for lower geometries, the ASIC is most likely an Xilinx EasyPath or Altera Hardcopy. Otherwise the amortization of the mask and dev costs -- even for a structured cell ASIC -- over 1K unit wouldn't make much sense versus the extra cooling/power costs for a GPU.