Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

blog.riseml.com

Post reply on HN
Print view View original

11–20 of 132 posts

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#11

Post by jacksmith21006 »

nojvek wrote: post #9
jacksmith21006 wrote: post #4

Thanks for sharing and very insightful. Guess the TPUs are the real deal. About 1/2 the cost for similar performance. Would assume Google is able to do that because of the less power required. I am actually more curious to get a paper on the new speech NN Google is using. Suppose to be 16k samples a second through a NN is hard to imagine how they did that and was able to roll it out as you would think the cost would…

Half the cost? Where are you reading that? Yeah on demand rental in AWS is expensive, but both long term and buying V100 yourself is significantly cheaper. Cloud companies have pretty fat margins on on demand rentals. You can’t buy a TPU, it’s a cloud only thing. They also show it’s not a huge difference in both perf and time to converge (albeit only one architecture) I would say kudos to V100 and this benchmark that…

The chart has 6.7 per hour for 3186 images Google and 12.2 per hour for 3128 AWS.

Or maybe reading it wrong?

That is close to half has much to use Google is it not?

BTW, The TPUs are also about twice as fast also.

Sounds like Google is pretty far ahead of Nvidia. Which really just makes sense as Google does the entire stack and just going to have the data to optimize the silicon.

About half the cost is hype?

I want in the cloud and not have to deal with updating, etc. Would think most are the same for anything of any scale. Could not imagine any longer building up rigs and dealing with all the issues. Plus much harder to scale.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#12

Post by my123 »

TheLoneAdmin wrote: post #8
elmarhaussmann wrote: post #2

Hi, author here. The motivation for this article came out of the HN discussion on a previous post ( https://news.ycombinator.com/item?id=16447096 ). There was a lot of valuable feedback - thanks for that. Happy to answer questions!

AMD - Where does their hardware stand in the race for ML? What changes would AMD need to make to be competitive?

Their hardware is fine. Their software is starting to get good too now. They're finishing MIOpen, a set of CUDA compatible libraries with which you can use Tensorflow (TF uses the builtin CUDA libs too, not only CUDA itself, as does CNTK). ROCm provides a CUDA implementation for AMD systems.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#13

Post by elmarhaussmann »

bitL wrote: post #6

Excellent! Thanks for these numbers, I wanted to see exactly this kind of benchmarks! Do you plan to try different benchmarks with the same setup for different problems, like semantic segmentation, DenseNet, LSTM training performance etc. as well?

Happy to hear the benchmark is useful to you! We'd love to try different setups and further models/networks. On the other hand, such benchmarks are a LOT of effort (which we underestimated it initially), so we'll have to see.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#14

Post by nightski »

jacksmith21006 wrote: post #11
nojvek wrote: post #9

Earlier quoted context omitted.

Half the cost? Where are you reading that? Yeah on demand rental in AWS is expensive, but both long term and buying V100 yourself is significantly cheaper. Cloud companies have pretty fat margins on on demand rentals. You can’t buy a TPU, it’s a cloud only thing. They also show it’s not a huge difference in both perf and time to converge (albeit only one architecture) I would say kudos to V100 and this benchmark that…

The chart has 6.7 per hour for 3186 images Google and 12.2 per hour for 3128 AWS. Or maybe reading it wrong? That is close to half has much to use Google is it not? BTW, The TPUs are also about twice as fast also. Sounds like Google is pretty far ahead of Nvidia. Which really just makes sense as Google does the entire stack and just going to have the data to optimize the silicon. About half the cost is hype? I want i…

It's more a comparison of AWS vs. Google Cloud pricing than Nvidia vs. TPUv2.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#15

Post by jacksmith21006 »

nightski wrote: post #14
jacksmith21006 wrote: post #11

Earlier quoted context omitted.

The chart has 6.7 per hour for 3186 images Google and 12.2 per hour for 3128 AWS. Or maybe reading it wrong? That is close to half has much to use Google is it not? BTW, The TPUs are also about twice as fast also. Sounds like Google is pretty far ahead of Nvidia. Which really just makes sense as Google does the entire stack and just going to have the data to optimize the silicon. About half the cost is hype? I want i…

It's more a comparison of AWS vs. Google Cloud pricing than Nvidia vs. TPUv2.

Strongly disagree. If Google is able to offer at about 1/2 the cost using their own silicon versus AWS using Nvidia that is all about the silicon difference.

But we also have the V1 TPU paper and can see the TPUs are able to use less joules per inference compared to an older Nvidia architecture. Was not that close. Just makes sense Google V2 TPUs would do the same.

Hope Google does a V3 TPU and then will share a V2 TPU paper like they did on V1 of the TPUs.

What is far more impressive of the TPUs is

https://cloudplatform.googleblog.com/2018/03/introducing-Clo...

If really doing 16k a second through a NN and at a price you can offer generally now that is incredible. I want this paper even more so.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#16

Post by jacksmith21006 »

wyldfire wrote: post #7

How much detail do we know about the TPUs' design? Does Google disclose a block-diagram level? ISA details? Do they release a toolchain for low-level programming or only higher-level functions like TensorFlow? EDIT: I found [1] which describes "tensor cores", "vector/matrix units" and HBM interfaces. The design sounds similar in concept to GPUs. Maybe they don't have or need interpolation hw or other GPU features? [1…

Great paper on the Generation 1 TPU. But Google has not shared much details on gen 2 and in some ways kind of hid information.

Suspect we will need a gen 3 to get a paper on the gen 2.

Here is the gen 1 paper and highly recommend. Pretty interesting using 65536 very simple cores.

https://arxiv.org/ftp/arxiv/papers/1704/1704.04760.pdf

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#17

Post by MrBuddyCasino »

elmarhaussmann wrote: post #2

Hi, author here. The motivation for this article came out of the HN discussion on a previous post ( https://news.ycombinator.com/item?id=16447096 ). There was a lot of valuable feedback - thanks for that. Happy to answer questions!

I found it interesting that they are so close together in performance - I mean what are the odds that they end up within 2% of each other?

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#19

Post by elmarhaussmann »

MrBuddyCasino wrote: post #17
elmarhaussmann wrote: post #2

Hi, author here. The motivation for this article came out of the HN discussion on a previous post ( https://news.ycombinator.com/item?id=16447096 ). There was a lot of valuable feedback - thanks for that. Happy to answer questions!

I found it interesting that they are so close together in performance - I mean what are the odds that they end up within 2% of each other?

Yeah, pretty big coincidence. However, this may change with the next TensorFlow versions, which supposedly has further speed improvements for the TPUv2.

Note also, that the ~2% performance difference is only on one model (ResNet-50) and cannot be generalized to all workloads/all of deep learning (at least not without further proof).

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#20

Post by shaklee3 »

elmarhaussmann wrote: post #2

Hi, author here. The motivation for this article came out of the HN discussion on a previous post ( https://news.ycombinator.com/item?id=16447096 ). There was a lot of valuable feedback - thanks for that. Happy to answer questions!

Nice work. I've only seen anecdotal stories about how TPU is faster, but never something as detailed as this.
Post reply on HN