Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google supercharges machine learning tasks with TPU custom chip

cloudplatform.googleblog.com

Post reply on HN
Print view View original

121–130 of 283 posts

Re: Google supercharges machine learning tasks with TPU custom chip

#121

Post by panic »

luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

How different are TPUs from GPUs? From the article, it sounds like TPUs use lower-precision arithmetic: are there any other differences?

Re: Google supercharges machine learning tasks with TPU custom chip

#122

Post by Symmetry »

dnautics wrote: post #25
reitzensteinm wrote: post #11

Earlier quoted context omitted.

But isn't 3 generations ahead just 8x? Which doesn't sound at all unreasonable for a custom hardware.

This is about right! 64-bit IEEE fp -> 16-bit IEEE-style fp[0] is a 4x bit size reduction, and multiplication is O(n^2) is silicon transistor count. [0] If google is smart, they'd ditch +/- infinity and if they were ballsy, they'd ditch zero in their FP implementation.

Generally speaking GPUs are already very good at running with float32s, usually much better than they are at using float64s in fact. The big advantages of using an ASIC are mostly on the storage side but they also allow you to get away with non IEEE floating point numbers that don't necessarily implement subnormals, NaN, etc.

Re: Google supercharges machine learning tasks with TPU custom chip

#123

Post by asimuvPR »

protomok wrote: post #116
asimuvPR wrote: post #104

Earlier quoted context omitted.

Very interesting fact. But the average programmer is not Rob Pike. How do you see this panning out for the average programmer? Will people need to learn a bit about chips to build more efficient CRUD apps?

I think there will always be an API. In the case of Deep Learning you already see Caffe, TensorFlow, etc. readily available for developers. I don't think the average developer will need to understand chip design but I do think _many_ developers will need to know how to use deep learning frameworks.

It had definitely started as an API side detail. But it might become an industry thing. It allows very easy vendor lock-in for all the *AAS products (PAAS, SAAS, etc). You could be required an add-on for your server or machine to be able to use their product.

Oh, and imagine a Facebook chip. :)

Re: Google supercharges machine learning tasks with TPU custom chip

#124

Post by randyrand »

rdtsc wrote: post #108
luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

"We have an FPGA, oh but need to go faster, ok, build an ASIC then" is a natural thing to come up with. That's kind of what bitcoin farms did. Obviously details and plans how it was done is where all the good stuff is, so that being hidden is understandable.

I'm not even a hardware or AI person, and even I could have told you ASICs would make way more sense than GPUs or FPGAs for machine learning. It's all about data locality. Fetching memory is the most costly thing a GPU does, and for ML (DNNs) there no big need for global access memory 99% of the time.

Anyone the casually follows AI knows that people have been talking about making DNN ASICS for some time. It was all a matter of time and $$$$$$

There is no doubt FB is working on them too. Which is why Google is finally publicaly saying that "we did it first ;)"

Re: Google supercharges machine learning tasks with TPU custom chip

#125

Post by dharma1 »

read_only wrote: post #99
skaevola wrote: post #82

Earlier quoted context omitted.

What are the advantages of analog computing for this application?

If you have an application that can tolerate error (like classification), then analog computing can give enormous gains in terms of speed _and_ power efficiency. Essentially, the savings come from using physics to perform the math (see Kirchhoff's current law) vs. using discrete time steps vs. fully-unrolling the logic. Google may not be using analog processing for this version, but I read an analog neural network re…

What do you think about http://optalysys.com/ or http://lighton.io?

Re: Google supercharges machine learning tasks with TPU custom chip

#126

Post by sitkack »

panic wrote: post #121
luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

How different are TPUs from GPUs? From the article, it sounds like TPUs use lower-precision arithmetic: are there any other differences?

Not just lower precision but probably less error correction in the lower bits. -Or- ... they convert the floating point values to analog values and do all the math in the analog domain and convert it back, just like old analog computers. But by going fully analog, I think the gains would be over 10x better, so probably not.

See 'low-power (inexact|approximate) computing'

Re: Google supercharges machine learning tasks with TPU custom chip

#127

Post by retox »

omarforgotpwd wrote: post #119
retox wrote: post #88

Earlier quoted context omitted.

GPUs also have this nice side effect of being great at playing games on. Purely as a guess I'd think that the gaming market is bigger than the AI researcher market.

In a future where AI is everywhere, Nvidia hopes it can sell GPUs by the hundreds and thousands to large data centers. You can make a lot more money a lot faster selling your hardware this way, and Nvidia is very interested in it judging from how much they talked about it at their recent conference.

I would be surprised if they weren't working on their own specialised chips then, though Google have the advantage of already having the software specs to build for.

Re: Google supercharges machine learning tasks with TPU custom chip

#128

Post by dsl »

arcanus wrote: post #94
honkhonkpants wrote: post #78

Earlier quoted context omitted.

Training and inference are not the same thing.

Sure... I think you are missing the point. In Nov., the Nature paper which contains the AlphaGo algorithm, the hardware detailed was exclusively CPU+GPUs. Between that time and the Lee Sedol match, the hardware running AlphaGo was switched, to these TPU. From the paper: "The final version of AlphaGo used 40 search threads, 48 CPUs, and 8 GPUs. We also implemented a distributed version of AlphaGo that exploited multip…

Or they consider the TPU to be something similar to a math co-processor, it just offloads work from the CPU. They also didn't discuss the custom built power modules, or the custom built network switches used for interconnects.

Re: Google supercharges machine learning tasks with TPU custom chip

#129

Post by dekhn »

asimuvPR wrote: post #104
dekhn wrote: post #89

Earlier quoted context omitted.

At some point in the past, Rob Pike mentioned that when we was working on Voyager (that spaceship that almost 40 years after launch, has left the solar system and continues to send back valuable science data), he had a relatively good understanding of the system from the quantum level (transistors are based on quantum theory) to the solar system. He wasn't kidding, either.

Very interesting fact. But the average programmer is not Rob Pike. How do you see this panning out for the average programmer? Will people need to learn a bit about chips to build more efficient CRUD apps?

i'm an average programmer but reading "High Performance Computing" http://shop.oreilly.com/product/9781565923126.do made a huge difference for me even when writing more efficient CRUD apps. A big issue is understanding the cost of the operations in the stack you are using and the overheads caused by your stack.

Re: Google supercharges machine learning tasks with TPU custom chip

#130

Post by Symmetry »

fiatmoney wrote: post #27
azinman2 wrote: post #21

Earlier quoted context omitted.

Ndvidia has to be general purpose. This is not and thus can be better optimized.

"General purpose" isn't that general, if you look at the actual operations they support and their threading model. It's already fairly optimized for these sorts of operations, and this amount of claimed headroom makes me suspicious.

Google has a lot of potential options that NVidia doesn't have. They can size their cache heirarchy to the task at hand. They can partition their memory space. They can drop scatter/gather. They can gang ALUs into dataflows that they know are the majority of machine learning workloads. They can partition their register file at the ISA level or maybe even drop it entirely. They can drop the parts of the IEEE754 floating point spec they don't need and they can size their numbers to the precision they need.
Post reply on HN