Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google supercharges machine learning tasks with TPU custom chip

cloudplatform.googleblog.com

Post reply on HN
Print view View original

261–270 of 283 posts

Re: Google supercharges machine learning tasks with TPU custom chip

#261

Post by Florin_Andrei »

lightcatcher wrote: post #181
Florin_Andrei wrote: post #148

Earlier quoted context omitted.

Well, it's a first announcement on a blog. They say it accelerates TensorFlow by 10x. They say it fits in an HDD slot. And the whole announcement must stay within a page or two. It's a "more details to follow" type of thing. Pretty standard actually.

> They say it accelerates TensorFlow by 10x They say 10x performance / watt, nothing about performance per unit time.

> They say 10x performance / watt

Well, that's the kind of metric you'd expect from a cloud provider. That's what's important to them.

If you're a tinkerer dabbling in TPU acceleration on your gaming/coding PC alongside with GPU acceleration, then the metric that would be interesting for you is speed increase per unit.

Re: Google supercharges machine learning tasks with TPU custom chip

#262

Post by hyperopt »

visarga wrote: post #224
hyperopt wrote: post #188

The Cloud Machine Learning service is one that I'm highly anticipating. Setting up arbitrary cloud machines for training models is a mess right now. I think if Google sets it up correctly, it could be a game changer for ML research for the rest of us. Especially if they can undercut AWS's GPU instances on cost per unit of performance through specialized hardware. I don't think the coinciding releases/announcements of…

I'd love to have a place to experiment cheaply but still not be required to invest 2-3K$ in it.

It doesn't just need to be a place to experiment cheaply. Many companies building software around ML techniques still rent time on EC2. Unless you are training models 24/7 and have your machines located in a very cost efficient location in terms of power/cooling, It's probably better for your training to be done in the cloud. It think very few use cases fall into the latter category.

Re: Google supercharges machine learning tasks with TPU custom chip

#263

Post by tlarkworthy »

engizeer wrote: post #201
boulos wrote: post #172

Earlier quoted context omitted.

"So"? If you're evaluating something today, how does it change your decision that we were late to market with Compute Engine (and in this specific case "bring-your-own-kernel")? If it's about future boring stuff, I think the list of boring stuff isn't too long ;). Disclosure: I work on Compute Engine.

All given, the fact that Google itself doesn't extensively use GC is kind of a red flag(I know quite a few Googlers from search infrastructure and none of them said their teams used GCE internally). A solid guarantee with AWS is if AWS goes down, then a multitude of Amazon's services also will go down(ex Amazonian myself), so it gives me a belief that AWS's uptime is more important to Amazon itself that it is for ext…

Google Firebase uses GCP including GCE extensively.

(Firebase Engineer here)

Re: Google supercharges machine learning tasks with TPU custom chip

#264

Post by nickpsecurity »

bravo22 wrote: post #258
nickpsecurity wrote: post #257

Earlier quoted context omitted.

Cool stuff. Previously, I was looking at eASIC or Triad if I needed this cuz I thought FPGA people cancelled S-ASIC's. Good to know there's a high-end one from Xilinx. Here's the others in case you didn't know about them: http://www.easic.com/products/28-nm-easic-nextreme-3/ https://www.triadsemi.com/reconfigurable-full-custom-asic/ eASIC has a maskless capability where they straight-up print your silicon for prototy…

That's awesome. I had heard about them before but never used. Crazy how low that price is.

Yeah, Triad is still in startup mode and picky. eASIC has been around quite a while. They also have ezCopy or something to produce ASIC's from their S-ASIC's. A side benefit is there's lots of pre-tested IP, including Gaisler OSS CPU.

So, worth considering. I need to get numbers on Xilinx, though, in terms of pricing and royalties. Esp if they have something for 28nm, 45nm, or 65nm that will be significantly cheaper than other one.

Re: Google supercharges machine learning tasks with TPU custom chip

#265

Post by hyperopt »

lightcatcher wrote: post #181
Florin_Andrei wrote: post #148

Earlier quoted context omitted.

Well, it's a first announcement on a blog. They say it accelerates TensorFlow by 10x. They say it fits in an HDD slot. And the whole announcement must stay within a page or two. It's a "more details to follow" type of thing. Pretty standard actually.

> They say it accelerates TensorFlow by 10x They say 10x performance / watt, nothing about performance per unit time.

In March at the GCP NEXT keynote [1], Jeff Dean demos Cloud ML on the GCP. He casually mentions passing in the argument "replicas=20" to get "20 way parallelism in optimizing this particular model". GCE does not currently offer GPU instances. I've never heard the term replicas in the GPU ML discourse. These devices may enable a type of parallelism that we have not seen before. Furthermore, his experiments are apparently using the Criteo dataset, which is a 10GB dataset. Now, I haven't looked into the complexity of the model or to what extent they train it to, but right now that sounds really impressive to me.

1: https://youtu.be/HgWHeT_OwHc?t=2h13m6s

Re: Google supercharges machine learning tasks with TPU custom chip

#266

Post by hyperopt »

nkw wrote: post #4

I guess this explains why Google Cloud Compute hasn't offered GPU instances.

That's what I'm thinking. I was anticipating the release of GPU instances, but now I'm thinking that they will simply leapfrog over GPU instances straight to this.

Re: Google supercharges machine learning tasks with TPU custom chip

#267

Post by PeterisP »

blue11 wrote: post #113
luu wrote: post #63

I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…

OK, so what is it? The announcement neither says what a TPU actually is nor what it can do. It's a magic black box. No specs. No price.

The article doesn't actually imply (though people might wish so) that the described TPU is intended to ever be a product available to others.

It's a post saying "that's how we do it" that serves a bunch of political and PR goals, but it's quite likely that this will stay an internal technology; maybe available indirectly as a cloud computing offering - in which case they'll give the specs and price of the whole solution, not of a particular model of TPU chip.

Re: Google supercharges machine learning tasks with TPU custom chip

#268

Post by PeterisP »

Bromskloss wrote: post #13

What is the capabilities that a piece of hardware like this needs to have to be suitable for machine learning (and not just one specific machine learning problem)?

Sacrifice generality, accuracy and ability to randomly access a lot of memory so that you can implement fast and power-efficient matrix operations with a single, low accuracy datatype thus requiring less memory, bandwidth and transistors.

Re: Google supercharges machine learning tasks with TPU custom chip

#269

Post by PeterisP »

simunaga wrote: post #209

In what sense in this a great news? Yes, it's a progress, so what? After all, you - programmers - earn money for your jobs and pretty soon you might not have one. Because of these kinds of great news -- "Whayyy, this is really interesting, AI, maching learning. Aaaaa!". "I'll get fired, won't have money for living and AI will take my place, but the world will be better! Yes! Progress!" Who will benefit from this? Sur…

Who will benefit from this? All of us.

The jobs can can be lost - those will be lost and should be lost. Prolonging the process doesn't help anyone as well. The transition can be hard and painful, though, so speeding it up is all the better.

Post reply on HN