I wonder if this architecture is the same Lanai architecture that was recently introduced by Google on LLVM. http://lists.llvm.org/pipermail/llvm-dev/2016-February/09511...
No, this is an ASIC. It's not general-purpose.
Google supercharges machine learning tasks with TPU custom chip
211–220 of 283 posts
Re: Google supercharges machine learning tasks with TPU custom chip
#212Earlier quoted context omitted.
> They say it accelerates TensorFlow by 10x They say 10x performance / watt, nothing about performance per unit time.
You can make some assumptions though. If the power consumption was equal, the performance is 10x. The speed at which an ASIC will run is constrained by temperature (power dissipation) and and logic timing, which itself has a dependency on temperature. So we could call that vertical scaling, to some power ceiling which may not take us all the way to 10x, but it's not impossible. Then there is horizontal, which I assum…
> You can make some assumptions though. If the power consumption was equal, the performance is 10x.
While I agree some performance per unit increase is likely, how does a direct 10x increased based on power savings follow? Less power usage does not mean that the chip can run through more flops in the same amount of time, right?
Re: Google supercharges machine learning tasks with TPU custom chip
#213One question: what has this got to do with tensors?
That they're used a lot in machine learning? If you're processing video for instance, you might have a 5-dimensional tensor: x, y, color channel, time index and batch index.
Re: Google supercharges machine learning tasks with TPU custom chip
#214Earlier quoted context omitted.
There were no references to this TPU in the TensorFlow source code?
No but if you look at the code it's clearly designed to support pluggable hardware.
Very interesting observation -- this seems so much more obvious in retrospect given the meticulous mapping of graph to devices outlined in the paper.
Re: Google supercharges machine learning tasks with TPU custom chip
#215Earlier quoted context omitted.
If I recall correctly myself (getting old!), Google has had the ability to build your own custom image since GCE went GA 2.5 years ago. Now, admittadly, it took a while to get IAM and VPC going, but we done did it now! I'd love to hear what other boring stuff has been a showstopper for you, in case we missed something dumb :)
Not having Managed Postgres and Managed Redis are 2 main showstoppers for me.
True but There are 3rd party managed services from both...
Re: Google supercharges machine learning tasks with TPU custom chip
#216Earlier quoted context omitted.
It's possible, but I think that the majority of ML optimization as seen by a programmer using tensorflow is more about optimizing the balance of accuracy, training & inference speed, and memory use, and a lot of the solutions in this space are pretty hardware independent. There's an entire other type of optimization about, e.g., making conv2d insanely fast, but that's not something that a typical data scientist-type…
I am more thinking in terms of: 1. Best price/performance is tensorflow right now. So, the best software choice is platform X. 2. Then in 2 years.. Well we are using Platform X so tensorflow is clearly the best option. In other words once you pick conv2d, you tend to also stick with whatever conv2d is optimized for. Which also means HW vendors love to help optimize popular platforms.
Others can make TF hardware too. Now that Google has opened the way, they will compete.
Re: Google supercharges machine learning tasks with TPU custom chip
#217For the curious, that's a plaque on the side if the rack showing the Go board at the end of AlphaGo vs Lee Sedol Game 3, at the moment Lee Sedol resigned and AlphaGo won the tournament (of five games).
By the way, more merit for Lee Sedol now that we know what he played against.
Re: Google supercharges machine learning tasks with TPU custom chip
#218I'm guessing that the performance / watt claims are heavily predicated on relatively low throughput, kind of similar to ARM vs Intel CPUs - particularly because they're only powering it & supplying bandwidth via what looks like a 1X PCIE slot. IOW, taking their claims at face value, a Nvidia card or Xeon Phi would be expected to smoke one of these, although you might be able to run N of these in the same power envelo…
> I'm guessing that the performance / watt claims are heavily predicated on relatively low throughput, kind of similar to ARM vs Intel CPUs - particularly because they're only powering it & supplying bandwidth via what looks like a 1X PCIE slot. Agreed. Also tells you that they don't need to communicate with the CPU much, given that it only has a PCIE. Reminds me of Knights Ferry, in this respect. > a Nvidia card or…
I am so happy that Nvidia got a real competitor in the ML hardware market. Maybe now they will be two times more creative.
Re: Google supercharges machine learning tasks with TPU custom chip
#219In what sense in this a great news? Yes, it's a progress, so what? After all, you - programmers - earn money for your jobs and pretty soon you might not have one. Because of these kinds of great news -- "Whayyy, this is really interesting, AI, maching learning. Aaaaa!". "I'll get fired, won't have money for living and AI will take my place, but the world will be better! Yes! Progress!" Who will benefit from this? Sur…
I thought long and hard about the future when 95% of the jobs as they exist today will be taken by robots. Initially, it would seem that people would be reduced to beggars, as they depend on BHI or other forms of welfare to subsist. But then it struck me:
You don't need jobs as long as you have land, renewable energy sources and robots (and 3d printers). You can live in a community that is self sufficient. You will be employed by your land, as it always was up until 100 years ago. We will also have robots, maybe not the latest generation, but we don't need to go back to the 19th century agriculture.
It is you who will benefit in the end, if you can use AI to improve your life. As long as AI doesn't remain locked in the hands of one entity and we all share into the benefits, it will work out ok. In the short run we need some sort of social welfare though, and to invest in renewables and self-sufficiency technologies.
How much self-sufficient a country, city, village or small farm could be? There is a lot of potential to migrate back to small community agrarian economy with robotics and 3d printing and solar panels.
People could trade using a different currency than that used for robotic produced goods. This currency will have to enforce differentiation of economic agents (diversity) and integration (low barriers of entry). A currency that will automatically disable the accumulation of power in a few hands and work for humans. We have to build an economy that functions more like the brain. In the brain there is no master neuron. They all share in the activity. So should be an enlightened human society.
Re: Google supercharges machine learning tasks with TPU custom chip
#220Earlier quoted context omitted.
You can make some assumptions though. If the power consumption was equal, the performance is 10x. The speed at which an ASIC will run is constrained by temperature (power dissipation) and and logic timing, which itself has a dependency on temperature. So we could call that vertical scaling, to some power ceiling which may not take us all the way to 10x, but it's not impossible. Then there is horizontal, which I assum…
> You can make some assumptions though. If the power consumption was equal, the performance is 10x. While I agree some performance per unit increase is likely, how does a direct 10x increased based on power savings follow? Less power usage does not mean that the chip can run through more flops in the same amount of time, right?
It does if power was the limiting factor in clock speed.