Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud TPUs in Beta

cloudplatform.googleblog.com

Post reply on HN
Print view View original

71–80 of 138 posts

Re: Cloud TPUs in Beta

#71

Post by deepnotderp »

boulos wrote: post #58
borramakot wrote: post #38

Earlier quoted context omitted.

Back of the envelope, a TPU costs a little more than 2x as much as a Volta on AWS P3, and delivers a little less than 2x the performance (180 TOPs for the TPU, 100 for Volta). On a raw performance/$ metric, I'm not sure the TPU is that interesting. It might be worth it if I were willing to pay a huge amount to get back results from an experiment faster, by using lots of TPUs- distributed learning on GPUs doesn't seem…

Disclosure: I work on Google Cloud. Peak ops/second isn’t the only thing that matters though. You have to be able to feed the units. The V100 does lots of finer-grained matrix multiplies which can make it harder to keep up. Don’t get me wrong, the V100 is a great chip. And we’re all looking forward to more (preferably third-party) benchmark results, to tease out when one is the better choice for a workload. But don’t…

This makes no sense, the V100 has more memory bandwidth than both the TPU and TPUv2

Re: Cloud TPUs in Beta

#72

Post by dgacmu »

yazr wrote: post #27

What are the chances of TensorFlow code gradually optimizing for TPUs over GPUs?! (Yes TF is OSS, but realistically Google is putting much more resources into it)

Very low. A lot of the performance on GPUs comes from Nvidia's optimizations in CuDNN -- it's mostly a matter of making sure TensorFlow feeds the right formats/etc. to CuDNN for core NN ops. TF should run well on CPUs, GPUs, TPUs, and likely future embedded accelerators (via tensorflow lite, which already supports the Android Neural Networks API).

(I'm part time on Brain, but, of course, this isn't some kind of Official Statement(tm)).

Re: Cloud TPUs in Beta

#73

Post by deepnotderp »

jlebar wrote: post #57
borramakot wrote: post #38

Earlier quoted context omitted.

Back of the envelope, a TPU costs a little more than 2x as much as a Volta on AWS P3, and delivers a little less than 2x the performance (180 TOPs for the TPU, 100 for Volta). On a raw performance/$ metric, I'm not sure the TPU is that interesting. It might be worth it if I were willing to pay a huge amount to get back results from an experiment faster, by using lots of TPUs- distributed learning on GPUs doesn't seem…

I know people don't know what to expect from tpu performance, but does anyone actually get 100tops out of Volta? I thought you'd have to spin the tensorcores and never touch memory, which is...not realistic. I know you hedged by saying "back of the envelope", but I'd much rather compare on real benchmarks than based on cited peak performance numbers, which are kind of meaningless.

This is true of the TPU as well, check out their paper's utilization numbers. If you ignore one outlier at ~90% utilization, their utilization plummets. I'm glad people are finally looking past the b.s. "peak" numbers for once though.

Re: Cloud TPUs in Beta

#74

Post by staticfloat »

minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

Just a minor nitpick that ResNet is the model you are referring to, ImageNet is the dataset that ResNet is trained on.

Re: Cloud TPUs in Beta

#75

Post by whataretensors »

puzzle wrote: post #68
whataretensors wrote: post #62

I don't like it. Google is mixing too many things. No way to buy a TPU. No competition from other cloud providers. Proprietary hardware and vendor lock-in.

This is really Tensorflow as a service. You get an IP address and a port you send gRPC requests to: https://github.com/tensorflow/tpu/blob/master/tools/diagnost... Presumably, there's a whole server behind that address that has all the right drivers and libraries: details you don't need to care about. The only partial lock-in is that not all ops are supported and you need to figure if there are any parts of the graph…

That's my point. Competitors are largely moated out by high costs of TPU production and proprietary drivers.

Re: Cloud TPUs in Beta

#76

Post by boulos »

minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

Disclosure: I work on Google Cloud.

Note that the post says “less than $200” not $200. There are lots of values between 0 and 200. What we’d love is for third-party folks like yourself to do the comparison (which I know you can, Max!)

Re: Cloud TPUs in Beta

#78

Post by DannyBee »

yazr wrote: post #27

What are the chances of TensorFlow code gradually optimizing for TPUs over GPUs?! (Yes TF is OSS, but realistically Google is putting much more resources into it)

TF funds one of my teams explicitly just to optimize CPUs and GPUs. Every discussion i've had with them tells me they care about making customers succeed, period.

So i'm going to with "pretty low".

Re: Cloud TPUs in Beta

#80

Post by jamesblonde »

deepnotderp wrote: post #71
boulos wrote: post #58

Earlier quoted context omitted.

Disclosure: I work on Google Cloud. Peak ops/second isn’t the only thing that matters though. You have to be able to feed the units. The V100 does lots of finer-grained matrix multiplies which can make it harder to keep up. Don’t get me wrong, the V100 is a great chip. And we’re all looking forward to more (preferably third-party) benchmark results, to tease out when one is the better choice for a workload. But don’t…

This makes no sense, the V100 has more memory bandwidth than both the TPU and TPUv2

Yes, when training DNNs memory bandwidth is the only figure you need to look at. That's why the 1080Ti is by far and away the best bang for buck right now (ignore the EULA nonsense). It has about 55% of the memory b/w of the V100 for 10% of the price.
Post reply on HN