Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

blog.riseml.com

Post reply on HN
Print view View original

31–40 of 132 posts

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#31

Post by dgacmu »

samfisher83 wrote: post #23

>For GPUs, there are further interesting options to consider next to buying. For example, Cirrascale offers monthly rentals of a server with four V100 GPUs for around $7.5k (~$10.3 per hour). However, further benchmarks are required to allow a direct comparison since the hardware differs from that on AWS (type of CPU, memory, NVLink support etc.). Can't you just buy some 1080s for cheaper than this. I understand ther…

Yes, you can. The problem starts when "you" are a large company -- NVidia restricts "datacenter" use of consumer GPUs (see previous HN discussion of that one: https://news.ycombinator.com/item?id=15983587 ). A single Titan V is somewhere in the 90% range of a V100 at less than 1/3 the cost, and a 1080ti, if you can find one, likely offers a slightly better price/performance spot. 4-GPU training may suffer due the lack of NVlink, but not enough for it to matter too much. As you scale, though, the lack of NVlink will hurt more. And, of course, all of these things come with a capex vs opex tradeoff, and a sysadmin vs cloud tradeoff, that will appeal differently to different situations.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#32

Post by jacksmith21006 »

Nokinside wrote: post #21

Nvidia is currently in cashing out phase. They have monopoly and money flows in effortlessly. The cost performance ratio reflects this. AMD will enter the game soon once they get their software working, Intel will follow. I suspect that Nvidia will respond with its own specialized machine learning and inference chips to match the cost/performance ratio. As long as Nvidia can maintain high manufacturing volumes and sm…

"The cost performance ratio reflects this."

But the TPUs are half the cost per this article?

Plus Google does the entire stack and can better optimize the hardware versus Nvidia. So it seem Google can improve faster I would think.

If there ever was a huge advantage doing the entire stack it is with neural networks.

A perfect example is Google new speech doing 16k samples a second with a NN.

https://cloudplatform.googleblog.com/2018/03/introducing-Clo...

Do not think Google could offer this service as a competitive cost without the TPUs.

This new method is replacing the method that was far less compute intensive so to offer at a competitive price requires lowering compute cost which suspect is only possible with the TPUs.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#33

Post by elmarhaussmann »

Tenoke wrote: post #29

>For the V100 experiments, we used a p3.8xlarge instance ( Xeon E5–2686@2.30GHz 16 cores, 244 GB memory , Ubuntu 16.04) on AWS with four V100 GPUs (16 GB of memory each). For the TPU experiments, we used a small n1-standard-4 instance as host ( Xeon@2.3GHz two cores, 15 GB memory , Debian 9) for which we provisioned a Cloud TPU (v2–8) consisting of four TPUv2 chips (16 GB of memory each). A bit odd that the TPUs are…

All of the computation, including pre-processing, is offloaded to the TPU. The weak machine is really just idling. A bigger one will only cost money and have no measurable effect on the performance.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#34

Post by elmarhaussmann »

drej wrote: post #18

Thanks for this, just a minor thing: You have price per hour and performance per second. Thus that ratio is not performance per image per $, you need to scale that. Also, the metric is not "images per second per $", but just "images per $".

Thanks for catching this!

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#35

Post by [deleted] »

Tenoke wrote: post #29

>For the V100 experiments, we used a p3.8xlarge instance ( Xeon E5–2686@2.30GHz 16 cores, 244 GB memory , Ubuntu 16.04) on AWS with four V100 GPUs (16 GB of memory each). For the TPU experiments, we used a small n1-standard-4 instance as host ( Xeon@2.3GHz two cores, 15 GB memory , Debian 9) for which we provisioned a Cloud TPU (v2–8) consisting of four TPUv2 chips (16 GB of memory each). A bit odd that the TPUs are…

[deleted]

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#36

Post by mtgx »

nightski wrote: post #14
jacksmith21006 wrote: post #11

Earlier quoted context omitted.

The chart has 6.7 per hour for 3186 images Google and 12.2 per hour for 3128 AWS. Or maybe reading it wrong? That is close to half has much to use Google is it not? BTW, The TPUs are also about twice as fast also. Sounds like Google is pretty far ahead of Nvidia. Which really just makes sense as Google does the entire stack and just going to have the data to optimize the silicon. About half the cost is hype? I want i…

It's more a comparison of AWS vs. Google Cloud pricing than Nvidia vs. TPUv2.

If anything, the pricing likely benefits Google. As in Google may be more profitable with the TPU usage, even at 1/2 the cost of Amazon's V100 usage.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#37

Post by nightski »

jacksmith21006 wrote: post #15
nightski wrote: post #14

Earlier quoted context omitted.

It's more a comparison of AWS vs. Google Cloud pricing than Nvidia vs. TPUv2.

Strongly disagree. If Google is able to offer at about 1/2 the cost using their own silicon versus AWS using Nvidia that is all about the silicon difference. But we also have the V1 TPU paper and can see the TPUs are able to use less joules per inference compared to an older Nvidia architecture. Was not that close. Just makes sense Google V2 TPUs would do the same. Hope Google does a V3 TPU and then will share a V2 T…

What makes you so sure it is all the silicon difference and not just AWS pricing their product at a more profitable price point?

These costs also ignore transferring and storing massive data sets in the cloud. In general the cloud is a huge pain and I'd avoid it like the plague unless I was caught and really, really needed the scalability. But even then that only works if you have a scalable implementation of the algorithm you are working on.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#38

Post by deepnotderp »

jacksmith21006 wrote: post #11
nojvek wrote: post #9

Earlier quoted context omitted.

Half the cost? Where are you reading that? Yeah on demand rental in AWS is expensive, but both long term and buying V100 yourself is significantly cheaper. Cloud companies have pretty fat margins on on demand rentals. You can’t buy a TPU, it’s a cloud only thing. They also show it’s not a huge difference in both perf and time to converge (albeit only one architecture) I would say kudos to V100 and this benchmark that…

The chart has 6.7 per hour for 3186 images Google and 12.2 per hour for 3128 AWS. Or maybe reading it wrong? That is close to half has much to use Google is it not? BTW, The TPUs are also about twice as fast also. Sounds like Google is pretty far ahead of Nvidia. Which really just makes sense as Google does the entire stack and just going to have the data to optimize the silicon. About half the cost is hype? I want i…

fwiw, the "TPU instance " has more than one tpu chip on it.

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#39

Post by Jabbles »

elmarhaussmann wrote: post #2

Hi, author here. The motivation for this article came out of the HN discussion on a previous post ( https://news.ycombinator.com/item?id=16447096 ). There was a lot of valuable feedback - thanks for that. Happy to answer questions!

Do you have more information about this bit?

the TPU implementation applies very compute-intensive image pre-processing steps and actually sacrifices raw throughput

Thanks

Re: Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

#40

Post by my123 »

dgacmu wrote: post #31
samfisher83 wrote: post #23

>For GPUs, there are further interesting options to consider next to buying. For example, Cirrascale offers monthly rentals of a server with four V100 GPUs for around $7.5k (~$10.3 per hour). However, further benchmarks are required to allow a direct comparison since the hardware differs from that on AWS (type of CPU, memory, NVLink support etc.). Can't you just buy some 1080s for cheaper than this. I understand ther…

Yes, you can. The problem starts when "you" are a large company -- NVidia restricts "datacenter" use of consumer GPUs (see previous HN discussion of that one: https://news.ycombinator.com/item?id=15983587 ). A single Titan V is somewhere in the 90% range of a V100 at less than 1/3 the cost, and a 1080ti, if you can find one, likely offers a slightly better price/performance spot. 4-GPU training may suffer due the lac…

With a mining exception for some reason, and their drivers blocking themselves when running in a virtualized environment unless you do some hacks.
Post reply on HN