Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud TPUs in Beta

cloudplatform.googleblog.com

Post reply on HN
Print view View original

11–20 of 138 posts

Re: Cloud TPUs in Beta

#11

Post by indescions_2018 »

bloudermilk wrote: post #3
Talyen42 wrote: post #2

How does this compare to Nvidia GPUs on AWS price/perf-wise? The article makes it sound like this is a new thing...

Google claims[0] the TPU is many times faster for the workloads they've designed it for. > On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs. As far as I know this will be the first opportunity for the public to prove those claims, as until now they've not been available on GCP. I don't mean to sound skeptical–I'm quite confident they're…

The reserve TPU button has been available on the dashboard for the last few months. But I assume instances have been prioritized for large customers such as Two Sigma.

From the paper:

"Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU."

In-Datacenter Performance Analysis of a Tensor Processing Unit

https://arxiv.org/abs/1704.04760

Price is about 5x cloud nvidia gpu instance on an hourly basis.

Re: Cloud TPUs in Beta

#12

Post by twtw »

bloudermilk wrote: post #3
Talyen42 wrote: post #2

How does this compare to Nvidia GPUs on AWS price/perf-wise? The article makes it sound like this is a new thing...

Google claims[0] the TPU is many times faster for the workloads they've designed it for. > On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs. As far as I know this will be the first opportunity for the public to prove those claims, as until now they've not been available on GCP. I don't mean to sound skeptical–I'm quite confident they're…

It will be interesting to see some benchmarks that compare TPUs to V100, since all previously published comparisons from Google compare TPU to K80 (3 GPU architectures ago).

Re: Cloud TPUs in Beta

#13

Post by [deleted] »

phamilton wrote: post #8
bloudermilk wrote: post #3

Earlier quoted context omitted.

Google claims[0] the TPU is many times faster for the workloads they've designed it for. > On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs. As far as I know this will be the first opportunity for the public to prove those claims, as until now they've not been available on GCP. I don't mean to sound skeptical–I'm quite confident they're…

I wonder how these would compare with Amazon's FPGA instances with a comparable core running.

[deleted]

Re: Cloud TPUs in Beta

#14

Post by xiphias »

bloudermilk wrote: post #3
Talyen42 wrote: post #2

How does this compare to Nvidia GPUs on AWS price/perf-wise? The article makes it sound like this is a new thing...

Google claims[0] the TPU is many times faster for the workloads they've designed it for. > On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs. As far as I know this will be the first opportunity for the public to prove those claims, as until now they've not been available on GCP. I don't mean to sound skeptical–I'm quite confident they're…

One thing to keep in mind that Google was using Tensorflow for comparision, which is heavily optimized for TPU, and GPU is just a second class citizen. Of course it was a great strategy on Google's side, and TPUs perform better than GPUs, but this is a little bit of cheating in the benchmarks.

Re: Cloud TPUs in Beta

#15

Post by twtw »

minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

Why does it make any sense to compare the price/hour for a single TPU (4 ASICs) to the price/hour for p3.16xlarge, which has 8x V100?

Also, that benchmark cost of $358 is for Resnet-152, not Resnet-50.

Re: Cloud TPUs in Beta

#16

Post by bicubic »

perfmode wrote: post #7
Talyen42 wrote: post #2

How does this compare to Nvidia GPUs on AWS price/perf-wise? The article makes it sound like this is a new thing...

It is. TPUs perform calculations on weights using low-precision floating point and integer types. This saves a ton of computation, but doesn't matter much for training models.

But GPU is also able to use lower resolution types. There must be more to the TPU advantage.

Re: Cloud TPUs in Beta

#17

Post by briffle »

Talyen42 wrote: post #2

How does this compare to Nvidia GPUs on AWS price/perf-wise? The article makes it sound like this is a new thing...

This is a new thing. Google also has Nvidia GPUs. these are new custom designed ASICs google has designed for certain ML tasks.

Re: Cloud TPUs in Beta

#18

Post by douglasfshearer »

minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

P3.16x benchmark is ResNet-152, TPU cost of $200 was for ResNet-50.

Tensorflow benchmarks show ResNet-152 resulting in 2.4x lower throughput than ResNet-50. [0]

[0] https://www.tensorflow.org/performance/benchmarks

Re: Cloud TPUs in Beta

#19

Post by vomjom »

bloudermilk wrote: post #3
Talyen42 wrote: post #2

How does this compare to Nvidia GPUs on AWS price/perf-wise? The article makes it sound like this is a new thing...

Google claims[0] the TPU is many times faster for the workloads they've designed it for. > On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs. As far as I know this will be the first opportunity for the public to prove those claims, as until now they've not been available on GCP. I don't mean to sound skeptical–I'm quite confident they're…

Keep in mind that what you linked refers to TPUv1, which is built for quantized 8-bit inference. The TPUv2, which was announced in this blog post, is for general purpose training and uses 32-bit weights, activations, and gradients.

It will have very different performance characteristics.

Re: Cloud TPUs in Beta

#20

Post by Cyph0n »

minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

A better comparison would be the f1.16xlarge[1] instance @ ~$4/hr. It comes with 8 FPGAs (12 Gbps link) and 64 vCPUs.

[1]: https://aws.amazon.com/ec2/instance-types/f1/

Edit: I'm genuinely curious about why this comment is getting downvotes.

Post reply on HN