Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google supercharges machine learning tasks with TPU custom chip

cloudplatform.googleblog.com

Post reply on HN
Print view View original

231–240 of 283 posts

Re: Google supercharges machine learning tasks with TPU custom chip

#231

Post by Coding_Cat »

I wonder if we will be seeing more of this in the (near) future. I expect so, and from more people then just Google. Why? Look at the problems the fab labs have had with the latest generation of chips and as they grow smaller the problems will probably rise. We are already close to the physical limit of transistor size. So, it is fair to assume that Moore's law will (hopefully) not outlive me.

So what then? I certainly hope the tech sector will not just leave it at that. If you want to continue to improve performance (per-watt) there is only one way you can go then: improve the design at an ASIC level. ASIC design will probably stay relatively hard, although there will probably be some technological solutions to make it easier with time, but if fabrication stalls at a certain nm level, production costs will probably start to drop with time as well.

I've been thinking about this quite a bit recently because I hope to start my PhD in ~1 year, and I'm torn between HPC or Computer Architecture. This seems to be quite a pro for Comp. Arch ;).

Re: Google supercharges machine learning tasks with TPU custom chip

#232

Post by amelius »

blue11 wrote: post #113
luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

OK, so what is it? The announcement neither says what a TPU actually is nor what it can do. It's a magic black box. No specs. No price.

It seems it is a GPU-like device but tuned towards neural net applications, and with a more convenient HDD-like formfactor.

Re: Google supercharges machine learning tasks with TPU custom chip

#233

Post by nathan_f77 »

I'm thinking that this has the potential to change the context of many debates about the "technological singularity", or AI taking over the world. Because it all seems to be based on FUD.

While reading this article, one of my first reactions was "holy shit, Google might actually build a general AI with these, and they've probably already been working on it for years".

But really, nothing about these chips is unknown or scary. They use algorithms that are carefully engineered and understood. They can be scaled up horizontally to crunch numbers, and they have a very specific purpose. They improve search results and maps.

What I'm trying to say is that general artificial intelligence is such a lofty goal, that we're going to have to understand every single piece of the puzzle before we get anywhere close. Including building custom ASICs, and writing all of the software by hand. We're not going to accidentally leave any loopholes open where AI secretly becomes conscious and decided to take over the world.

Re: Google supercharges machine learning tasks with TPU custom chip

#234

Post by Houshalter »

nathan_f77 wrote: post #233

I'm thinking that this has the potential to change the context of many debates about the "technological singularity", or AI taking over the world. Because it all seems to be based on FUD. While reading this article, one of my first reactions was "holy shit, Google might actually build a general AI with these, and they've probably already been working on it for years". But really, nothing about these chips is unknown…

Just because you understand a machine doesn't mean it can't be dangerous. I could completely understand every aspect of a nuclear bomb, and I could still make a mistake and cause quite a bit of damage with a real one. Complex systems are notorious for having all sorts of unexpected problems, and mistakes happen all the time. How much complex software is entirely bug free?

The danger of AI is more than just a random bug though. It's that an intelligent AI is inherently not good. If you give it a goal, like to make as many paperclips as possible, it will do everything in it's power to convert the world to paperclips. If you give it the goal of self preservation, it try to destroy anything that has a 0.0001% chance of hurting it, and make as many redundant copies as possible. Etc.

Very, very few goals actually result in an AI that wants to do exactly what you want it to do. And if the AI is incredibly powerful, that will be a very bad outcome for humanity.

Re: Google supercharges machine learning tasks with TPU custom chip

#235

Post by auvi »

luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

any idea which foundry google have used for the fabrication?

Re: Google supercharges machine learning tasks with TPU custom chip

#236

Post by cm3 »

kuschku wrote: post #95
nickysielicki wrote: post #8

Earlier quoted context omitted.

There's no way Google lets this leave their datacenters. Chip fabrication is a race to the bottom at this point. [1] Google is doubling down on hosting as a source of future revenue, and they're doing that by building an ecosystem around Tensorflow. What I think is interesting is how weak Apple looks. Amazon has the talent and money to be able to compete with Google on this playing field. Microsoft is late, but they…

> There's no way Google lets this leave their datacenters. Chip fabrication is a race to the bottom at this point. [1] I’d hope someone somewhere steals the blueprints and posts all of them publicly online. The whole point of patents was that companies would publish everything, but get 20 years of protection. But by now, especially companies like Google don’t do so anymore – and everyone loses out. EDIT: I’ll add the…

Same misuse happens with copyright. Both were invented to foster publishing and not creating life-long monopolies. The life times of copyright and patents must be way shorter as well. Everyone builds on something that came before. It's impossible to build a better bike if you have to test drive on a street with patent mines.

Re EDIT: Downvotes must be comment-mandatory or not allowed otherwise.

Re: Google supercharges machine learning tasks with TPU custom chip

#237

Post by redcalx »

sitkack wrote: post #126
panic wrote: post #121

Earlier quoted context omitted.

How different are TPUs from GPUs? From the article, it sounds like TPUs use lower-precision arithmetic: are there any other differences?

Not just lower precision but probably less error correction in the lower bits. -Or- ... they convert the floating point values to analog values and do all the math in the analog domain and convert it back, just like old analog computers. But by going fully analog, I think the gains would be over 10x better, so probably not. See 'low-power (inexact|approximate) computing'

They likely use half precision (i.e. 16 bit) floats in some of the computing stages. They've also discussed using even fewer bits of precision in some of their research output. When training and following an error gradient you kind of need lots of precision, but when running a learned model you often don't need much at all, 8 or even 4 bit numbers are sufficient in some calcs where rounding error propagation isn't an issue.

Re: Google supercharges machine learning tasks with TPU custom chip

#238

Post by dzdt »

tptacek wrote: post #155
bd wrote: post #37

So now open sourcing of "crown jewels" AI software makes sense. Competitive advantage is protected by custom hardware (and huge proprietary datasets). Everything else can be shared. In fact it is now advantageous to share as much as you can, the bottleneck is a number of people who know how to use new tech.

http://www.joelonsoftware.com/articles/StrategyLetterV.html

Joel Spoelsky : "commoditize your complements." Steve Ballmer : "developers! developers! developers!"

Developers are a complement to hardware, software, and data based businesses. Google probably spends more on employees than on hardware or data. They really want to commoditize developers that can work on their stuff.

Google's business is based on having more and better data than their competitors. In many cases they have been happy to commoditize hardware, making open their datacenter designs for instance. That helps to cheapen their costs for building datcenters.

In other cases they open source software, like tensorflow and go language. These choices are made to commoditize developers. Google wants there to be a big pool of people who know how to use the technologies that Google uses. More developers means less costs for Google to hire and train employees. Which is their biggest expense: win!

With the TPU, as long as no one else is doing that kind of hardware for machine learning, it is a proprietary advantage to keep it secret. But at some point the logic flips: when others start to do similar things Google would rather commoditize their version of the tech. Because the lesson of the last 40 years is any widely used hardware WILL become commoditized. The inertia is with software codebases and developer knowledge.

Developers developers developers developer developers!

Re: Google supercharges machine learning tasks with TPU custom chip

#239

Post by vidarh »

manav wrote: post #152
Krakulf wrote: post #111

Earlier quoted context omitted.

It really boosts Google Cloud Platform with image and prestige, they're coming out with what is pretty much tensorflow as a service with their machine learning product. Coming out saying they can give you a service no one else can right down to a custom chip may sway a few buyers in the market. https://cloud.google.com/ml/

Well this also shows their commitment to Tensorflow if they willing to pay to fab custom chips for it. Then again, in the Google scheme of things that's probably not a huge cost. They've probably carefully modeled the energy savings over GPU/FPGA and found it to be substantial enough, even taking into account costs of design changes.

If you need enough chips it doesn't take particularly careful modelling to say that you'll beat an FPGA quite substantially in most cases.

Re: Google supercharges machine learning tasks with TPU custom chip

#240

Post by vidarh »

kuschku wrote: post #95
nickysielicki wrote: post #8

Earlier quoted context omitted.

There's no way Google lets this leave their datacenters. Chip fabrication is a race to the bottom at this point. [1] Google is doubling down on hosting as a source of future revenue, and they're doing that by building an ecosystem around Tensorflow. What I think is interesting is how weak Apple looks. Amazon has the talent and money to be able to compete with Google on this playing field. Microsoft is late, but they…

> There's no way Google lets this leave their datacenters. Chip fabrication is a race to the bottom at this point. [1] I’d hope someone somewhere steals the blueprints and posts all of them publicly online. The whole point of patents was that companies would publish everything, but get 20 years of protection. But by now, especially companies like Google don’t do so anymore – and everyone loses out. EDIT: I’ll add the…

There's little need for anyone to steal the blueprints. It's unlikely there's anything particularly "special" there other than identifying operations in Tensorflow that take long enough and are carried out often enough and are simple enough to be worth turning into an ASIC. If there's a market for it, there will be other people designing chips for it too.
Post reply on HN