When is off the shelf edition coming?
Cloud TPUs in Beta
41–50 of 138 posts
Re: Cloud TPUs in Beta
#42Reading the TOS it seems like this is a really great deal for Google:
"When you upload, submit, store, send or receive content to or through our Services, you give Google (and those we work with) a worldwide license to use, host, store, reproduce, modify, create derivative works (such as those resulting from translations, adaptations or other changes we make so that your content works better with our Services), communicate, publish, publicly perform, publicly display and distribute such content. The rights you grant in this license are for the limited purpose of operating, promoting, and improving our Services, and to develop new ones."
All your training data are belong to us.
We can use your models to improve ours.
The terms will prevent me from using it. I can't grant Google permission to redistribute HIPAA PHI.
Re: Cloud TPUs in Beta
#43This is exciting. There are lots of specific reasons to choose Google Cloud over AWS (and vice versa), but proprietary hardware is surely an advantage that is going to be hard to replicate / compete with. If TPUs hold up to the hype, GCloud may become the de facto for ML/AI startups.
>If TPUs hold up to the hype, GCloud may become the de facto for ML/AI startups.
Don't startups want to win a big exit though? Google won't need to buy the startup for billions, because the TOS already grants them permission to use all the models and training data for free. Seems like a Faustian bargain to me.
Re: Cloud TPUs in Beta
#44I'm puzzled by the phrase "differentiated performance per dollar." Is it more performant, or less? If it's less performant, why mention it at all? If it's more performant, why not simply say "better performance per dollar"?
It is both more performant overall as well as per dollar.
Re: Cloud TPUs in Beta
#45Some things: A "single TPU" is 4 ASICs. It is not clear if it makes sense to compare a "single TPU" to a "single GPU." As a point of reference, NVIDIA's numbers are 6 hours for Resnet-50 on Imagenet when training with 8xV100. From a naive extrapolation, 4xV100 would probably take ~12 hours and 1xV100 about two days. Google has previously only compared TPUs to K80, so it will be interesting to see some benchmarks that…
The comparison was the first generation TPUs not the second generation which is what these are.
But ultimately it comes down to the cost to complete some amount of work. Google also offers Nvidia GPUs in their cloud for training and should be able to compare the cost of using one over the other as both are supported by TF.
That is the ultimate guide on how good or not good the TPUs really are.
Re: Cloud TPUs in Beta
#46Earlier quoted context omitted.
Google claims[0] the TPU is many times faster for the workloads they've designed it for. > On our production AI workloads that utilize neural network inference, the TPU is 15x to 30x faster than contemporary GPUs and CPUs. As far as I know this will be the first opportunity for the public to prove those claims, as until now they've not been available on GCP. I don't mean to sound skeptical–I'm quite confident they're…
Perf per watt matters to Google but not you. You should only think of it on a perf/$ basis, right?
They're closely related though, since if the perf per watt is lower then Google can charge you less doller per perf. The price they charge you is ultimately tied to the operating cost.
Re: Cloud TPUs in Beta
#47That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…
Back of the envelope, a TPU costs a little more than 2x as much as a Volta on AWS P3, and delivers a little less than 2x the performance (180 TOPs for the TPU, 100 for Volta). On a raw performance/$ metric, I'm not sure the TPU is that interesting. It might be worth it if I were willing to pay a huge amount to get back results from an experiment faster, by using lots of TPUs- distributed learning on GPUs doesn't seem…
It'll make a lot more sense when the TPU pods they alluded to come out.
Re: Cloud TPUs in Beta
#48Some things: A "single TPU" is 4 ASICs. It is not clear if it makes sense to compare a "single TPU" to a "single GPU." As a point of reference, NVIDIA's numbers are 6 hours for Resnet-50 on Imagenet when training with 8xV100. From a naive extrapolation, 4xV100 would probably take ~12 hours and 1xV100 about two days. Google has previously only compared TPUs to K80, so it will be interesting to see some benchmarks that…
On 4x1080Ti it takes 2 days to train ResNet-50. 4xASICs doing it in a day is not that impressive.
Re: Cloud TPUs in Beta
#49I'm puzzled by the phrase "differentiated performance per dollar." Is it more performant, or less? If it's less performant, why mention it at all? If it's more performant, why not simply say "better performance per dollar"?
[deleted]
Re: Cloud TPUs in Beta
#50I assume Azure and AWS have some buddying up with Intel/Nervana and Nvidia counterstroke to Google TPUs. I can’t quite imagine what it will be though.
Amazon announced today they are working on their own TPU type chips.