I wonder if opening this up as a cloud offering is a way to get a whole bunch of excess capacity (if it needs it for something big?) but have it paid for.
Google supercharges machine learning tasks with TPU custom chip
61–70 of 283 posts
Re: Google supercharges machine learning tasks with TPU custom chip
#62Earlier quoted context omitted.
No - tensorflow is open source and you can run it on many platforms. TPUs are about efficiency. You might not be able to do image recognition as efficiently without one, but you can still perform exactly the same tasks. (I work on TF this year.)
I would be shocked if tensorflow optimizations where useful 1:1 for stock Intel chips or GPU's. So, there is still plenty of lock-in even if your process runs. GPU vendors love to play this game by helping optimize games.
It's possible, but I think that the majority of ML optimization as seen by a programmer using tensorflow is more about optimizing the balance of accuracy, training & inference speed, and memory use, and a lot of the solutions in this space are pretty hardware independent. There's an entire other type of optimization about, e.g., making conv2d insanely fast, but that's not something that a typical data scientist-type user deals with.
(To elaborate -- it's questions like "how deep should I make this convolution? Should I use tf.relu or tf.sigmoid? How many fully-connected layers should I put here, and how big should I make them?". These are really knotty deep learning design questions, but they're often h/w independent. Not always - we certainly have some ops on TF that we only support in CPUs and not on GPUs, for example - but often.)
Re: Google supercharges machine learning tasks with TPU custom chip
#63I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-).
I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished that they'd change, but it was the prevailing attitude. Currently, if you look at what's being hyped up at a couple of large companies that could conceivably build a competing chip, it's all FPGAs all the time, so announcing that we built an ASIC could change what other companies do, which is exactly what Google was trying to avoid back when I was there.
If this signals that Google is going to be less secretive about infrastructure, that's great news.
When I joined Microsoft, I tried to gently bring up the possibility of doing either GPUs or ASICs and was told, very confidentially by multiple people, that it's impossible to deploy GPUs at scale, let alone ASICs. Since I couldn't point to actual work I'd done elsewhere, it seemed impossible to convince folks, and my job was in another area, I gave up on it, but I imagine someone is having that discussion again right now.
Just as an aside, I'm being fast and loose with language when I use the word impossible. It's more than my feeling is that you have a limited number of influence points and I was spending mine on things like convincing my team to use version control instead of mailing zip files around.
Re: Google supercharges machine learning tasks with TPU custom chip
#64Earlier quoted context omitted.
I don't see any mention of offering these chips for sale. You can rent them it seems via cloud offerings & that's it.
Yup, I assume they're gonna keep them in house as a competitive advantage for a time. I doubt they'll do it forever; the most valuable part of NVIDIA's CUDA is the ecosystem, and I think Google knows that.
I would assume that the API to use these is tensorflow.
So... just use Google's machine learning cloud thingy.
The software can build the community, where the supercharging is only available when you run it on Google cloud.
(although GPU performance isn't bad either, so you don't have to, thus community)
Re: Google supercharges machine learning tasks with TPU custom chip
#653 generations ahead of moore law??? I really wonder how they are accomplishing this beyond implementing the kernels in hardware. I suspect they are using specialized memory and an extremely wide architecture. Sounds they also used this for AlphaGo. I wonder how badly we were off on AlphaGo's power estimates. Seems everyone assumed they were using GPU's, sounds like they were not. At least partially. I would really LO…
The article confirms that "AlphaGo was powered by TPUs in the matches against Go world champion, Lee Sedol, enabling it to "think" much faster and look farther ahead between moves."
Re: Google supercharges machine learning tasks with TPU custom chip
#66Now this is really interesting. I've been asking myself why this hadn't happened before. Its been all software, software, software for the last decade or so. But now I get it. We are at a point in time where it makes sense to adjust the hardware to the software. Funny how things work. It used to be the other way around.
This is known as the Wheel of Reincarnation. Functionality moves to special-purpose hardware then back to software, and the cycle repeats. (The computer term is from 1968 so this has been happening for a long time.)
http://www.catb.org/jargon/html/W/wheel-of-reincarnation.htm...
Re: Google supercharges machine learning tasks with TPU custom chip
#67I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…
Google has always been strategic about announcing what it was doing, even since the early days (I used to work there too). Think about the impact the first MapReduce and GFS/BigTable papers had.
My guess as to why they're announcing the TPU is that they are feeling the pressure from Facebook and other AI labs, and want to reinforce their reputation as being the best place to do AI research. By revealing that AlphaGo was based on this hardware, they indicate to researchers around the world that if you want to build the most advanced ML models you need to be at Google. Same reason they talked about MapReduce/GFS back in the day.
Re: Google supercharges machine learning tasks with TPU custom chip
#68I'm happy to hear that this is finally public so I can actually talk about the work I did when I was at Google :-). I'm a bit surprised they announced this, though. When I was there, there was this pervasive attitude that if "we" had some kind of advantage over the outside world, we shouldn't talk about it lest other people get the same idea. To be clear, I think that's pretty bad for the world and I really wished th…
My vague impression is that there had started to be more and more research on use of ASICs for ML (though I can't say I follow it that closely). Enough of a changing tide that the competitive advantage was going?
Re: Google supercharges machine learning tasks with TPU custom chip
#69Earlier quoted context omitted.
But isn't 3 generations ahead just 8x? Which doesn't sound at all unreasonable for a custom hardware.
This is about right! 64-bit IEEE fp -> 16-bit IEEE-style fp[0] is a 4x bit size reduction, and multiplication is O(n^2) is silicon transistor count. [0] If google is smart, they'd ditch +/- infinity and if they were ballsy, they'd ditch zero in their FP implementation.
Actually for many neural network apps 8, 4 even 1 bit is sufficient for representing weights.
Re: Google supercharges machine learning tasks with TPU custom chip
#70I think this shows a fundamental difference between Amazon (AWS) and Google Cloud. AWSs offerings seem fairly vanilla and boring. Google are offering more and more really useful stuff: - cloud machine learning - custom hardware - live migration of hosts without downtime - Cold storage with access in seconds - bigquery - dataflow
to start a REAL business you SHOULD act boring. for everything else there is google.
...are you saying Google isn't a real business?