Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
blog.riseml.com
Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
1–10 of 132 posts
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#2Hi, author here. The motivation for this article came out of the HN discussion on a previous post (https://news.ycombinator.com/item?id=16447096). There was a lot of valuable feedback - thanks for that.
Happy to answer questions!
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#3Was this running the AWS Deep Learning AMI or did you build your own.
Because Intel was involved in its development and made a number of tweaks to improve performance.
Be curious if it actually was significant or not.
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#4Thanks for sharing and very insightful. Guess the TPUs are the real deal. About 1/2 the cost for similar performance.
Would assume Google is able to do that because of the less power required.
I am actually more curious to get a paper on the new speech NN Google is using. Suppose to be 16k samples a second through a NN is hard to imagine how they did that and was able to roll it out as you would think the cost would be prohibitive.
You are ultimately competing with a much less compute heavy solution.
https://cloudplatform.googleblog.com/2018/03/introducing-Clo...
Suspect this was only possible because of the TPUs.
Can't think of anything else where controlling the entire stack including the silicon would be more important than AI applications.
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#5Was this running the AWS Deep Learning AMI or did you build your own. Because Intel was involved in its development and made a number of tweaks to improve performance. Be curious if it actually was significant or not.
On AWS this was using nvidia-docker with the TensorFlow Docker images. Probably, the AWS AMI Deep Learning gives very similar performance (with same versions of CUDA, TensorFlow etc.). There's only so much you can tweak if the GPU itself is the bottleneck...
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#6Excellent! Thanks for these numbers, I wanted to see exactly this kind of benchmarks! Do you plan to try different benchmarks with the same setup for different problems, like semantic segmentation, DenseNet, LSTM training performance etc. as well?
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#7How much detail do we know about the TPUs' design? Does Google disclose a block-diagram level? ISA details? Do they release a toolchain for low-level programming or only higher-level functions like TensorFlow?
EDIT: I found [1] which describes "tensor cores", "vector/matrix units" and HBM interfaces. The design sounds similar in concept to GPUs. Maybe they don't have or need interpolation hw or other GPU features?
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#8Hi, author here. The motivation for this article came out of the HN discussion on a previous post ( https://news.ycombinator.com/item?id=16447096 ). There was a lot of valuable feedback - thanks for that. Happy to answer questions!
AMD - Where does their hardware stand in the race for ML? What changes would AMD need to make to be competitive?
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#9Thanks for sharing and very insightful. Guess the TPUs are the real deal. About 1/2 the cost for similar performance. Would assume Google is able to do that because of the less power required. I am actually more curious to get a paper on the new speech NN Google is using. Suppose to be 16k samples a second through a NN is hard to imagine how they did that and was able to roll it out as you would think the cost would…
Half the cost? Where are you reading that? Yeah on demand rental in AWS is expensive, but both long term and buying V100 yourself is significantly cheaper. Cloud companies have pretty fat margins on on demand rentals.
You can’t buy a TPU, it’s a cloud only thing. They also show it’s not a huge difference in both perf and time to converge (albeit only one architecture)
I would say kudos to V100 and this benchmark that breaks the TPU hype.
Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50
#10How much detail do we know about the TPUs' design? Does Google disclose a block-diagram level? ISA details? Do they release a toolchain for low-level programming or only higher-level functions like TensorFlow? EDIT: I found [1] which describes "tensor cores", "vector/matrix units" and HBM interfaces. The design sounds similar in concept to GPUs. Maybe they don't have or need interpolation hw or other GPU features? [1…
So far only very few details are disclosed. Here are two presentations:
https://supercomputersfordl2017.github.io/Presentations/Imag... http://learningsys.org/nips17/assets/slides/dean-nips17.pdf
For the last version of the TPU, Google provided more detail, e.g., in this paper:
https://arxiv.org/pdf/1704.04760.pdf
Hopefully, Google will publish something similar for TPUv2, but I have no knowledge whether or when that might happen.