Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud TPUs in Beta

cloudplatform.googleblog.com

Post reply on HN
Print view View original

21–30 of 138 posts

Re: Cloud TPUs in Beta

#21

Post by minimaxir »

twtw wrote: post #15
minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

Why does it make any sense to compare the price/hour for a single TPU (4 ASICs) to the price/hour for p3.16xlarge, which has 8x V100? Also, that benchmark cost of $358 is for Resnet-152, not Resnet-50.

Whoops, I misread, added edit.

Re: Cloud TPUs in Beta

#22

Post by ebikelaw »

bloudermilk wrote: post #3
Talyen42 wrote: post #2

How does this compare to Nvidia GPUs on AWS price/perf-wise? The article makes it sound like this is a new thing...

Google claims[0] the TPU is many times faster for the workloads they've designed it for. > On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs. As far as I know this will be the first opportunity for the public to prove those claims, as until now they've not been available on GCP. I don't mean to sound skeptical–I'm quite confident they're…

Perf per watt matters to Google but not you. You should only think of it on a perf/$ basis, right?

Re: Cloud TPUs in Beta

#23

Post by Cyph0n »

bicubic wrote: post #16
perfmode wrote: post #7

Earlier quoted context omitted.

It is. TPUs perform calculations on weights using low-precision floating point and integer types. This saves a ton of computation, but doesn't matter much for training models.

But GPU is also able to use lower resolution types. There must be more to the TPU advantage.

GPUs are much more complex (general-purpose) and therefore cannot be optimized beyond a certain point due to timing requirements and PVT (process, temperature, voltage) variations. In other words, the more stuff you have on an ASIC, the more careful you have to be ensure a margin of tolerance for variations.

Re: Cloud TPUs in Beta

#25

Post by dharma1 »

Cyph0n wrote: post #20
minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

A better comparison would be the f1.16xlarge[1] instance @ ~$4/hr. It comes with 8 FPGAs (12 Gbps link) and 64 vCPUs. [1]: https://aws.amazon.com/ec2/instance-types/f1/ Edit: I'm genuinely curious about why this comment is getting downvotes.

Do much deep learning training with the FPGAs?

Re: Cloud TPUs in Beta

#26

Post by twtw »

Some things:

A "single TPU" is 4 ASICs. It is not clear if it makes sense to compare a "single TPU" to a "single GPU."

As a point of reference, NVIDIA's numbers are 6 hours for Resnet-50 on Imagenet when training with 8xV100. From a naive extrapolation, 4xV100 would probably take ~12 hours and 1xV100 about two days.

Google has previously only compared TPUs to K80, so it will be interesting to see some benchmarks that compare TPUs to more recent GPUs. K80 was released in 2014, and the Kepler architecture was introduced in 2012.

Re: Cloud TPUs in Beta

#27

Post by yazr »

What are the chances of TensorFlow code gradually optimizing for TPUs over GPUs?!

(Yes TF is OSS, but realistically Google is putting much more resources into it)

Re: Cloud TPUs in Beta

#28

Post by Cyph0n »

dharma1 wrote: post #25
Cyph0n wrote: post #20

Earlier quoted context omitted.

A better comparison would be the f1.16xlarge[1] instance @ ~$4/hr. It comes with 8 FPGAs (12 Gbps link) and 64 vCPUs. [1]: https://aws.amazon.com/ec2/instance-types/f1/ Edit: I'm genuinely curious about why this comment is getting downvotes.

Do much deep learning training with the FPGAs?

Nope, but I'm sure others do.

Re: Cloud TPUs in Beta

#29

Post by saguro »

Cyph0n wrote: post #23
bicubic wrote: post #16

Earlier quoted context omitted.

But GPU is also able to use lower resolution types. There must be more to the TPU advantage.

GPUs are much more complex (general-purpose) and therefore cannot be optimized beyond a certain point due to timing requirements and PVT (process, temperature, voltage) variations. In other words, the more stuff you have on an ASIC, the more careful you have to be ensure a margin of tolerance for variations.

So the only advantage of the TPU is it's a simpler and more specialized asic? Google didn't break any new ground in terms of training perf?

Re: Cloud TPUs in Beta

#30

Post by gcp »

Interestingly, GCP now appears to be available to individuals in Europe. It wasn't like that before, no idea when that policy got changed. Before, GCP wasn't even a consideration compared to AWS (which always handled that).
Post reply on HN