Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google supercharges machine learning tasks with TPU custom chip

cloudplatform.googleblog.com

Post reply on HN
Print view View original

171–180 of 283 posts

Re: Google supercharges machine learning tasks with TPU custom chip

#171

Post by chlestakoff »

luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

Now that you are free to talk about it, could you explain how ASICs stack up against GPUs, in practical terms?

Re: Google supercharges machine learning tasks with TPU custom chip

#172

Post by boulos »

jbooth wrote: post #115
vgt wrote: post #87

Earlier quoted context omitted.

Your point is valid, but I think what the OP was saying is that Google is offering all this stuff IN ADDITION to the boring stuff. Google does boring stuff very well too.. and one can argue much better than AWS as well.. take a look at Quizlet's story: https://quizlet.com/blog/whats-the-best-cloud-probably-gcp (shamelessly biased Googler)

If I recall correctly, it took Google a while to actually offer the boring stuff. For a while, you could get a Google Compute Engine but you couldn't just get a dang VM image, because Google knows better than you and you should do things their way. They've fixed it now, but lost a lot of potential market share for that conceit.

"So"?

If you're evaluating something today, how does it change your decision that we were late to market with Compute Engine (and in this specific case "bring-your-own-kernel")?

If it's about future boring stuff, I think the list of boring stuff isn't too long ;).

Disclosure: I work on Compute Engine.

Re: Google supercharges machine learning tasks with TPU custom chip

#173

Post by CydeWeys »

Florin_Andrei wrote: post #148
blue11 wrote: post #113

Earlier quoted context omitted.

OK, so what is it? The announcement neither says what a TPU actually is nor what it can do. It's a magic black box. No specs. No price.

Well, it's a first announcement on a blog. They say it accelerates TensorFlow by 10x. They say it fits in an HDD slot. And the whole announcement must stay within a page or two. It's a "more details to follow" type of thing. Pretty standard actually.

The cool thing here is that it's already widely deployed in production, so we know it actually works and that it's not just vaporware. The "more details to follow" is thus a lot more convincing than for a lot of similar-seeming announcements of things that don't actually exist yet (and often don't ever).

Re: Google supercharges machine learning tasks with TPU custom chip

#174

Post by CydeWeys »

retox wrote: post #88
babo wrote: post #79

Earlier quoted context omitted.

Sure, but that's the deal. I'll buy the latest nVidia 1080 card as soon as I can but renting these custom chips per minute would be a way better option for me.

GPUs also have this nice side effect of being great at playing games on. Purely as a guess I'd think that the gaming market is bigger than the AI researcher market.

> Purely as a guess I'd think that the gaming market is bigger than the AI researcher market.

Machine learning isn't just targeting the AI researcher market though -- it's widely used by a huge number of companies, and of course, by many of Google's most important products. I would argue that those markets combined are larger than gaming.

Re: Google supercharges machine learning tasks with TPU custom chip

#176

Post by kristianp »

tptacek wrote: post #155
bd wrote: post #37

So now open sourcing of "crown jewels" AI software makes sense. Competitive advantage is protected by custom hardware (and huge proprietary datasets). Everything else can be shared. In fact it is now advantageous to share as much as you can, the bottleneck is a number of people who know how to use new tech.

http://www.joelonsoftware.com/articles/StrategyLetterV.html

Cool how he foreshadows the end of Sun (takeover by Oracle in 2010) in that article from 2002:

"Sun's two strategies are (a) make software a commodity by promoting and developing free software (Star Office, Linux, Apache, Gnome, etc), and (b) make hardware a commodity by promoting Java, with its bytecode architecture and WORA. OK, Sun, pop quiz: when the music stops, where are you going to sit down? Without proprietary advantages in hardware or software, you're going to have to take the commodity price, which barely covers the cost of cheap factories in Guadalajara, not your cushy offices in Silicon Valley."

Re: Google supercharges machine learning tasks with TPU custom chip

#177

Post by hueving »

vgt wrote: post #87
gcr wrote: post #80

Earlier quoted context omitted.

Vanilla? Boring? I read "Vanilla" and "Boring" as "Horray, I don't have to spend time rewriting all this complicated code I already have!" If I'm just dipping my toes into (say) Caffe or Theano, I don't have to rewrite it from scratch. That is a huge advantage---not a disadvantage!---of AWS over google.

Your point is valid, but I think what the OP was saying is that Google is offering all this stuff IN ADDITION to the boring stuff. Google does boring stuff very well too.. and one can argue much better than AWS as well.. take a look at Quizlet's story: https://quizlet.com/blog/whats-the-best-cloud-probably-gcp (shamelessly biased Googler)

Minus the massive downtime that just occurred recently. https://status.cloud.google.com/incident/compute/16007

Not implying that AWS hasn't had them. It's just that adopting GCE this early makes you a bit of a guinea pig because GCE isn't used internally at Google.

Re: Google supercharges machine learning tasks with TPU custom chip

#178

Post by emcq »

hedgehog wrote: post #162
manav wrote: post #150

Interesting. Plenty of work has been done with FPGAs, and a few have developed ASICs like DaDianNao in China [1]. Google though actually has the resources to deploy them in their datacenters. Microsoft explored something similar to accelerate search with FPGAs [2]. The results show that the Arria 10 (20nm latest from Altera) had about 1/4th the processing ability at 10% of the power usage of the Nvidia Tesla K40 (25w…

You can build an ASIC with fast external memory, it adds to the cost but then you can handle larger models similar to a GPU. Software support is an issue but for deep learning applications there's no reason in principle you couldn't add support to TensorFlow etc for new hardware to make it simple for application developers to adopt. Movidius has announced that they're doing this and it's likely that other ML chip ven…

External memory kills your power budget. Efficient processors, whether they be GPUs, cpus, or asics all gain efficiencies by keeping data local to computation. Having your chip focused on off chip data is difficult to optimize power for if you don't have compute bound problems.

Re: Google supercharges machine learning tasks with TPU custom chip

#179

Post by emcq »

fiatmoney wrote: post #27
azinman2 wrote: post #21

Earlier quoted context omitted.

Ndvidia has to be general purpose. This is not and thus can be better optimized.

"General purpose" isn't that general, if you look at the actual operations they support and their threading model. It's already fairly optimized for these sorts of operations, and this amount of claimed headroom makes me suspicious.

If they balance compute to memory better than GPUs, you could definitely see a 10x. GPUs have large off chip memory and small caches (like 256kb). Cost to going to off chip memory can be 1-2 orders of magnitude more than on chip memory. You can certainly fit 4+MB on modern processors, but they likely bought designs from a company like Samsung because designing high performance, low power memory cells is tricky. I'm surprised they were able to keep things a secret.

Re: Google supercharges machine learning tasks with TPU custom chip

#180

Post by incepted »

kristianp wrote: post #176
tptacek wrote: post #155

Earlier quoted context omitted.

http://www.joelonsoftware.com/articles/StrategyLetterV.html

Cool how he foreshadows the end of Sun (takeover by Oracle in 2010) in that article from 2002: "Sun's two strategies are (a) make software a commodity by promoting and developing free software (Star Office, Linux, Apache, Gnome, etc), and (b) make hardware a commodity by promoting Java, with its bytecode architecture and WORA. OK, Sun, pop quiz: when the music stops, where are you going to sit down? Without proprieta…

Predicting that a company will fail without giving a date is not foreshadowing, it's stating the obvious.
Post reply on HN