Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

blog.riseml.com

Post reply on HN
Print view View original

91–100 of 132 posts

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#91

Post by shaklee3 »

ndesaulniers wrote: post #69
shaklee3 wrote: post #55

Earlier quoted context omitted.

They announced in 2016 they had TPUs. So no, I would not expect that 2 full years later they're just now being available in the public cloud. These are not new products to them; they likely just don't want to deal with supporting them in different configurations.

> they likely just don't want to deal with supporting them in different configurations. It is a lot of work. But mainly that TPUv1 only did inference while TPUv2 does training+inference.

Exactly my point. It's a lot of work. That's the reason why Nvidia has such a large team doing it, and also why they spent 3 billion dollars to build the V100 ASIC.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#92

Post by dimitry12 »

bitL wrote: post #89
dimitry12 wrote: post #88

An important hidden cost here is coding a model which can take advantage of mixed-precision training. It is not trivial: you have to empirically discover scaling factors for loss functions, at the very least. It's great that there is now wider choice of (pre-trained?) models formulated for mixed-precision training. When I was comparing Titan V (~V100) and 1080ti 5 months ago, I was only able to get 90% increase in fo…

How did you get your hands on Titan V 5 months ago? I still can't find it anywhere in retail in EU...

It was in stock on and off and I was able to order it directly from Nvidia US.

After 59 days of playing with it, I sent it back (initiated return on 30th day, after I already figured out it doesn't live up to the hype, then had another 30 days to actually send it back).

With $3,000 I can buy 4 1080ti's, while only two are necessary to beat Titan V (in Titan V's best game). I only bought one though. NowInStock.net helped with buying 1080ti directly from Nvidia.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#93

Post by verall »

dgacmu wrote: post #84
verall wrote: post #73

Earlier quoted context omitted.

The new "datacenter" restriction only applies to GeForce branded cards. The Titan V is now called the "NVIDIA Titan V" and with no GeForce branding to be found anywhere. So the restriction applies to the 1080ti but _not_ the titan V. I completely agree the restriction is total bullshit but it's important to get the facts straight.

Not according to the statement from NVidia quoted in this article: https://www.cnbc.com/2017/12/27/nvidia-limits-data-center-us... It applies to both GeForce and Titan.

You're right - it seems like they have added "Titan" to the agreement since it was first posted on HN:

http://www.nvidia.com/content/DriverDownload-March2009/licen...

Thanks for the tip!

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#94

Post by jacksmith21006 »

shaklee3 wrote: post #91
ndesaulniers wrote: post #69

Earlier quoted context omitted.

> they likely just don't want to deal with supporting them in different configurations. It is a lot of work. But mainly that TPUv1 only did inference while TPUv2 does training+inference.

Exactly my point. It's a lot of work. That's the reason why Nvidia has such a large team doing it, and also why they spent 3 billion dollars to build the V100 ASIC.

Big difference is Google does the entire stack and also has scale to conceptually be able to create a better solution. But that is theoretical.

Here we can see the results where the Google TPU gets almost twice result per dollar over Nvidia. But then Google should be able to iterate more quickly.

Take the move from using CNN to using capsule networks. The idea for Capsules came from Hinton and Google is going to be there first to optimize in hardware. This is the benefit of playing in all layers of the stack.

Or the using NN for text to speech and offering at scale. Google just has inherent advantages over Nvidia and now we get to see a little more concrete results. But hope we get a lot more similar and see if the Google advantage holds up.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#95

Post by jacksmith21006 »

shaklee3 wrote: post #90
ndesaulniers wrote: post #65

Earlier quoted context omitted.

> I'm not sure what you mean by google does the entire stack. Consider that Google has some of the best machine learning researchers, compiler engineers, hardware engineers, and infrastructure in the business working on this.

Huh? Machine learning and infrastructure Engineers, yes. Compiler and Hardware engineers? No. What gives you reason to believe they have a lead in either of those departments other than they have a lot of money? They're forced to use the same foundry as Nvidia, and their Hardware team is likely significantly smaller.

Google been buying up AI resources well before anyone else and has the strongest and deepest team at this point.

It is why so many of the break throughs have come from Google. Great example is winning at Go almost a decade earlier than anyone thought possible.

They probably two of the strongest teams with one the Brain team and then the Deepmind team. But all the other engineers and infrastructure is first rate at Google.

Really at this point do not think the $100B cash is as important as Google already built the team and now experinced resources are far more difficult to get.

The other advantage for Google is their ability to attract the top engineers in addition.

Google just got started a lot earlier on all of this.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#96

Post by shaklee3 »

jacksmith21006 wrote: post #95
shaklee3 wrote: post #90

Earlier quoted context omitted.

Huh? Machine learning and infrastructure Engineers, yes. Compiler and Hardware engineers? No. What gives you reason to believe they have a lead in either of those departments other than they have a lot of money? They're forced to use the same foundry as Nvidia, and their Hardware team is likely significantly smaller.

Google been buying up AI resources well before anyone else and has the strongest and deepest team at this point. It is why so many of the break throughs have come from Google. Great example is winning at Go almost a decade earlier than anyone thought possible. They probably two of the strongest teams with one the Brain team and then the Deepmind team. But all the other engineers and infrastructure is first rate at Go…

Google got started a lot earlier on this? Did you read what you are saying? Nvidia has been making hardware longer than Google has been a company. No, Google does not have a better hardware team. Google has the luxury of making a device that is used for a single purpose that they control. Nvidia made a device that can be used for far more and works on commodity hardware. By the way, deepmind/alphago uses Nvidia GPUs, so that was an extremely bad example.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#97

Post by elmarhaussmann »

scottlegrand2 wrote: post #75

What they're not saying is that one can't use all nvlink bandwidth for gradient reduction on a DGX-1V with only 4 GPUs because nvlink is composed of 2 8-node rings. And given the data parallel nature of this benchmark, I'm very interested in where time was spent on each architecture. That said, they fixed this on NVSwitch so it's just another HW hiccup like int8 was on Pascal.

For this benchmark, NVLink and gradient reduction isn't the bottleneck. The performance scales almost perfectly linearly from one GPU to four.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#98

Post by elmarhaussmann »

dimitry12 wrote: post #88

An important hidden cost here is coding a model which can take advantage of mixed-precision training. It is not trivial: you have to empirically discover scaling factors for loss functions, at the very least. It's great that there is now wider choice of (pre-trained?) models formulated for mixed-precision training. When I was comparing Titan V (~V100) and 1080ti 5 months ago, I was only able to get 90% increase in fo…

Author here.

Yes, thanks for mentioning that! That's what the article is alluding to at the end. There's also something like a "cost-to-model" and that's influenced by how easy it is to make efficient use of the performance and how much tweaking it needs. It's also influenced by the framework you use... However, that's difficult to compare and almost impossible to measure.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#99

Post by jacksmith21006 »

shaklee3 wrote: post #96
jacksmith21006 wrote: post #95

Earlier quoted context omitted.

Google been buying up AI resources well before anyone else and has the strongest and deepest team at this point. It is why so many of the break throughs have come from Google. Great example is winning at Go almost a decade earlier than anyone thought possible. They probably two of the strongest teams with one the Brain team and then the Deepmind team. But all the other engineers and infrastructure is first rate at Go…

Google got started a lot earlier on this? Did you read what you are saying? Nvidia has been making hardware longer than Google has been a company. No, Google does not have a better hardware team. Google has the luxury of making a device that is used for a single purpose that they control. Nvidia made a device that can be used for far more and works on commodity hardware. By the way, deepmind/alphago uses Nvidia GPUs,…

Hardware optimize for NN. Nvidia dominate focus had been graphics. Big difference which we can see the results in this article.

Plus benefits not having the baggage that Nvidia would have.

But never going to be able to use a TPU for graphics.

In the end it is about results.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#100

Post by jacksmith21006 »

shaklee3 wrote: post #96
jacksmith21006 wrote: post #95

Earlier quoted context omitted.

Google been buying up AI resources well before anyone else and has the strongest and deepest team at this point. It is why so many of the break throughs have come from Google. Great example is winning at Go almost a decade earlier than anyone thought possible. They probably two of the strongest teams with one the Brain team and then the Deepmind team. But all the other engineers and infrastructure is first rate at Go…

Google got started a lot earlier on this? Did you read what you are saying? Nvidia has been making hardware longer than Google has been a company. No, Google does not have a better hardware team. Google has the luxury of making a device that is used for a single purpose that they control. Nvidia made a device that can be used for far more and works on commodity hardware. By the way, deepmind/alphago uses Nvidia GPUs,…

BTW,. Deepmind now uses TPUs both for training and inference and with the results we can see why.

https://www.theverge.com/circuitbreaker/2016/5/19/11716818/g... Google reveals the mysterious custom hardware that powers AlphaGo

Post reply on HN