Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

Post by ndesaulniers » Thu, Apr 26, 2018, 4:49 PM UTC

eanzenberg wrote: post #50

jacksmith21006 wrote: post #4

Thanks for sharing and very insightful. Guess the TPUs are the real deal. About 1/2 the cost for similar performance. Would assume Google is able to do that because of the less power required. I am actually more curious to get a paper on the new speech NN Google is using. Suppose to be 16k samples a second through a NN is hard to imagine how they did that and was able to roll it out as you would think the cost would…

The impression I got was opposite: TPU is not the hot shit that Google claims it is. Pricing is kind of irrelevant since they can subsidize this to create that story.

Post by elmarhaussmann » Thu, Apr 26, 2018, 5:03 PM UTC

antognini wrote: post #59

sdenton4 wrote: post #56

Earlier quoted context omitted.

"As shown above, the top-1 accuracy after 90 epochs for the TPU implementation is 0.7% better. This may seem minor, but making improvements at this already very high level is extremely difficult and, depending on the application, such small improvements may make a big difference in the end." Any idea of how much variation in accuracy you get on different training runs of the same model on the same hardware? My unders…

I've been running a lot of these resnet-50 experiments lately and the run-to-run variation is very small, on the order of 0.1%. It's actually pretty amazing how consistent training is given that the initialization is always different and the data is sampled differently on each run. (As an aside, it took us about three weeks to track down a bug that was causing the model to consistently reach an accuracy 1% lower than…

Post by ndesaulniers » Thu, Apr 26, 2018, 5:06 PM UTC

shaklee3 wrote: post #45

jacksmith21006 wrote: post #32

Earlier quoted context omitted.

"The cost performance ratio reflects this." But the TPUs are half the cost per this article? Plus Google does the entire stack and can better optimize the hardware versus Nvidia. So it seem Google can improve faster I would think. If there ever was a huge advantage doing the entire stack it is with neural networks. A perfect example is Google new speech doing 16k samples a second with a NN. https://cloudplatform.goog…

I'm not sure what you mean by google does the entire stack. Nvidia writes all of the major CUDA libraries used behind the scenes in the NN libraries, such as cuDNN, cuBLAS, etc. Nvidia can likely improve their hardware significantly faster/more efficiently than Google can because their entire business depends on it. Google has incentive for improving their TPU for internal use, but they don't make any money by sellin…

Post by ndesaulniers » Thu, Apr 26, 2018, 5:12 PM UTC

eanzenberg wrote: post #64

ndesaulniers wrote: post #61

Earlier quoted context omitted.

> Pricing is kind of irrelevant since they can subsidize this to create that story. Depends on how much you plan to use the hardware. If it's running near continuously, total cost of ownership is very important. Power costs can quickly dominate TCO.

At the pricing extreme, Google could make their TPUs free to use and charge elsewhere in their cloud. This shows that literal pricing is pretty irrelevant.

Post by ndesaulniers » Thu, Apr 26, 2018, 5:21 PM UTC

shaklee3 wrote: post #55

jacksmith21006 wrote: post #46

Earlier quoted context omitted.

Yes in the cloud as you would expect in 2018. Available to the general public.

They announced in 2016 they had TPUs. So no, I would not expect that 2 full years later they're just now being available in the public cloud. These are not new products to them; they likely just don't want to deal with supporting them in different configurations.