Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google supercharges machine learning tasks with TPU custom chip

cloudplatform.googleblog.com

Post reply on HN
Print view View original

181–190 of 283 posts

Re: Google supercharges machine learning tasks with TPU custom chip

#181

Post by lightcatcher »

Florin_Andrei wrote: post #148
blue11 wrote: post #113

Earlier quoted context omitted.

OK, so what is it? The announcement neither says what a TPU actually is nor what it can do. It's a magic black box. No specs. No price.

Well, it's a first announcement on a blog. They say it accelerates TensorFlow by 10x. They say it fits in an HDD slot. And the whole announcement must stay within a page or two. It's a "more details to follow" type of thing. Pretty standard actually.

> They say it accelerates TensorFlow by 10x

They say 10x performance / watt, nothing about performance per unit time.

Re: Google supercharges machine learning tasks with TPU custom chip

#182

Post by p1esk »

aaronsnoswell wrote: post #151

I'm curious to know; is this announcement something that an expert in these sorts of areas could have (or did?) predict months or years ago, given Google's recent jumps forwards in Machine learning products? Can someone with more knowledge about this comment?

Yes. Back in 1989, Intel built one of the first custom chips for ML (ETANN). Since then, there have been hundreds of such designs, and dozens of working chips. So, people have been building them long before "deep learning" arrived in all its glory. Now that it did, it was expected that efforts in custom hardware should intensify.

However, such initiatives still face the same problems as 30 years ago: custom hardware is expensive, inflexible, hard to program, and quickly becomes obsolete. It's still worth it if there's no other way to speed things up, but Moore's law is still alive and kicking, as evidenced by 15B transistor GP100 chip form Nvidia, so we can still just wait a little bit for the next gen GPUs.

Google is certainly in a good position, having developed a very popular ML framework, and having enough resources to develop good hardware (the blog post was written by Norm Jouppi - one of the best computer architects in history). It remains to be seen, however, how well these TPUs are supported in TensorFlow. What kind of models will get the advertised speed up?

Re: Google supercharges machine learning tasks with TPU custom chip

#183

Post by wyldfire »

fiatmoney wrote: post #33

I'm guessing that the performance / watt claims are heavily predicated on relatively low throughput, kind of similar to ARM vs Intel CPUs - particularly because they're only powering it & supplying bandwidth via what looks like a 1X PCIE slot. IOW, taking their claims at face value, a Nvidia card or Xeon Phi would be expected to smoke one of these, although you might be able to run N of these in the same power envelo…

> IOW, taking their claims at face value, a Nvidia card or Xeon Phi would be expected to smoke one of these, although you might be able to run N of these in the same power envelope.

That seems unlikely, right? GPGPU software beating an ASIC? I guess it depends on just how abstract and adaptable the TPU IC is. Sounds like they use a custom number representation -- if they can squeeze it into less than half precision (IEEE FP16) then it would be super hard for a Phi or GPU to beat it.

But ultimately it all comes down to specific applications' bottlenecks. If they don't have to go off-card ever and their workset fits into the on-die memory, then they'd have no real advantage by using a PCIe GPU with GDDRx on it.

Re: Google supercharges machine learning tasks with TPU custom chip

#184

Post by hellameta »

aaronsnoswell wrote: post #151

I'm curious to know; is this announcement something that an expert in these sorts of areas could have (or did?) predict months or years ago, given Google's recent jumps forwards in Machine learning products? Can someone with more knowledge about this comment?

Sure. I don't have anything to link on the spot but this was/is/has been foreseeable for some time. Although it's all very cool and shiny - most business applications of machine learning remain squarely in the territory of classic algos like GLM & forests (random, boosted trees etc. etc.). As a fun note, advances like these highlight that data scientists etc. will not be beaten by more complex automated methods, but simply by speed. Much like the filing system that 'runs' whatever you're using to see these words (https://www.youtube.com/watch?v=EKWGGDXe5MA).

Edit: to elaborate... single model training runs are possible to do quite fast now, but knowing how to tune hyper parameters remains the 'voodoo' of the field. But the best hyper params are also possible to discover through brute force: try every combination you can! Today, you can use various heuristics to improve this process, but either way, being able to train whatever X times faster just means we can search hyper parameter space that much faster. The robots are coming :)

Re: Google supercharges machine learning tasks with TPU custom chip

#185

Post by tcarey83 »

euyyn wrote: post #75
BooneJS wrote: post #12

Earlier quoted context omitted.

From the article: "TPU is tailored to machine learning applications, allowing the chip to be more tolerant of reduced computational precision, which means it requires fewer transistors per operation."

Do you reckon that means it's using small floats?

When I first read the blog post earlier in the day, it actually said they were using 8-bits. I remember it because it seem quite small to me.

Re: Google supercharges machine learning tasks with TPU custom chip

#186

Post by p1esk »

read_only wrote: post #99
skaevola wrote: post #82

Earlier quoted context omitted.

What are the advantages of analog computing for this application?

If you have an application that can tolerate error (like classification), then analog computing can give enormous gains in terms of speed _and_ power efficiency. Essentially, the savings come from using physics to perform the math (see Kirchhoff's current law) vs. using discrete time steps vs. fully-unrolling the logic. Google may not be using analog processing for this version, but I read an analog neural network re…

True. ML applications are well-suited to analog computation not only because they can tolerate errors, they also have an ability to adapt to errors, provided training algorithms are ran in hardware.

Re: Google supercharges machine learning tasks with TPU custom chip

#187

Post by jmoiron »

incepted wrote: post #180
kristianp wrote: post #176

Earlier quoted context omitted.

Cool how he foreshadows the end of Sun (takeover by Oracle in 2010) in that article from 2002: "Sun's two strategies are (a) make software a commodity by promoting and developing free software (Star Office, Linux, Apache, Gnome, etc), and (b) make hardware a commodity by promoting Java, with its bytecode architecture and WORA. OK, Sun, pop quiz: when the music stops, where are you going to sit down? Without proprieta…

Predicting that a company will fail without giving a date is not foreshadowing, it's stating the obvious.

What? He also predicted how and why it would fail. Sun was a big enough player then that it could have survived plenty of other ways. Apple of today looks nothing like the company in 2000, but Sun got caught out more or less exactly as described and never adapted.

Re: Google supercharges machine learning tasks with TPU custom chip

#188

Post by hyperopt »

The Cloud Machine Learning service is one that I'm highly anticipating. Setting up arbitrary cloud machines for training models is a mess right now. I think if Google sets it up correctly, it could be a game changer for ML research for the rest of us. Especially if they can undercut AWS's GPU instances on cost per unit of performance through specialized hardware. I don't think the coinciding releases/announcements of TensorFlow, Cloud ML, and now this are an accident. There is something brewing and I think it's going to be big.

Re: Google supercharges machine learning tasks with TPU custom chip

#189

Post by dgacmu »

LogicFailsMe wrote: post #161
dgacmu wrote: post #149

Earlier quoted context omitted.

Wearing my CMU hat for a moment (but keeping in mind Google's paying me this year): Google's always been cautious about the balance of speed and efficiency, out of concerns about programmer productivity, parallelization, and generality. See, for example, Urs's article in response to my and a few other people's crazy-academic research on using "Wimpy" nodes: http://static.googleusercontent.com/media/research.google.co…

Because there's an absolute sippy straw of bandwidth to the thing if that's a 1x pci-e connection. For if it were delivering performance on par with a $1000 Maxwell class GPU, why wouldn't you guys crow about it? That would be a really big deal wouldn't it? TitanX for 20W? That'd be awesome. And having suffered through multiple pitches for us to buy various FPGA and boutique processors, I have yet to see someone who…

That wasn't really my point - I'm simply noting that Urs is one of the last people I'd think to hop on the wimpy crazy train. His published articles suggest that he's got a very good grasp of the tradeoffs involved in "real" TCO -- i.e., taking a fairly global view of both the human, capital, and operating expenses involved in a technology decision such as using wimpies (no) or fabricating a custom ASIC for machine learning (yes).

That doesn't mean a TPU is faster or slower than anything in particular, it just means that quite likely that it's good for some machine learning tasks that Google cares enough about to spend the whatever dollars it cost to make the thing.

The WSJ article has a few more quotes from Norm Jouppi, btw.: http://www.wsj.com/articles/google-isnt-playing-games-with-n... (Sorry if that gets paywalled. Googling "wall street journal google tensor processing unit" got me there.)

Re: Google supercharges machine learning tasks with TPU custom chip

#190

Post by DigitalJack »

lightcatcher wrote: post #181
Florin_Andrei wrote: post #148

Earlier quoted context omitted.

Well, it's a first announcement on a blog. They say it accelerates TensorFlow by 10x. They say it fits in an HDD slot. And the whole announcement must stay within a page or two. It's a "more details to follow" type of thing. Pretty standard actually.

> They say it accelerates TensorFlow by 10x They say 10x performance / watt, nothing about performance per unit time.

You can make some assumptions though. If the power consumption was equal, the performance is 10x.

The speed at which an ASIC will run is constrained by temperature (power dissipation) and and logic timing, which itself has a dependency on temperature.

So we could call that vertical scaling, to some power ceiling which may not take us all the way to 10x, but it's not impossible.

Then there is horizontal, which I assume is applicable to these problems... running more in parallel.

In both cases, I think it's safe to assume they are getting a performance increase in the instantaneous sense.

Post reply on HN