When is off the shelf edition coming?
if there ever was a chance for a hardware start-up to become the next big thing it's entering this space. unfortunately Nervana sold out to intel.
Cloud TPUs in Beta
101–110 of 138 posts
Re: Cloud TPUs in Beta
#102Earlier quoted context omitted.
We mostly focus on the “whole board” numbers. So it’s not only units “local” HBM, but NVLINK versus TPU to TPU. Sorry for the confusion. Edit for this part of the thread: the best public numbers are in the linked presentation [1]. [1] https://supercomputersfordl2017.github.io/Presentations/Imag...
That's... a skewed ... comparison, NVLINK is a board to board connection whereas you're talking about TPU to TPU on board communication if I understand correctly?
That's sort of the point though! We're actually selling these as the "board". So the right way to compare things is sort of DGX-1 style "deep learning rig" versus a board of four TPU units (or several connected). The on-chip network is a big part of its overall efficiency.
I don't recall what (if anything) we've said about how we link up the boards across racks, but the folks at Next Platform looked pretty carefully at the pictures: https://www.nextplatform.com/2017/05/22/hood-googles-tpu2-ma...
Re: Cloud TPUs in Beta
#103Earlier quoted context omitted.
Has Google published data on the memory bandwidth of TPU v2 (aka "cloud TPU")? I'm having trouble finding it. In any case I agree, we shouldn't be looking at the stated peak compute of either of the chips. (Disclaimer: I work at Google on XLA, and have in the past worked on TPUs.)
From the blog post is the link to the fairly recent NIPS presentation: https://supercomputersfordl2017.github.io/Presentations/Imag... which claims 2400 GB/s for the board and 600 GB/s per “chip”.
This is in comparison to 900gb/s for V100.
Re: Cloud TPUs in Beta
#104This seems a bit pricey compared to other offerings. Wouldn't an ASIC make things more economical?
Seems like in terms of cost per performance, both AWS P3 spot instances and Paperspace v100 offerings are more economical.
Are these prices expected to become more competitive once it is out of beta?
Re: Cloud TPUs in Beta
#105This seems a bit pricey compared to other offerings. Wouldn't an ASIC make things more economical? Seems like in terms of cost per performance, both AWS P3 spot instances and Paperspace v100 offerings are more economical. Are these prices expected to become more competitive once it is out of beta?
isn't the tpu kind of a deep learning asic?
Re: Cloud TPUs in Beta
#106Disclosure: I work on Google Cloud. I want to highlight this paragraph from the post: > Here at Google Cloud, we want to provide customers with the best cloud for every ML workload and will offer a variety of high-performance CPUs (including Intel Skylake) and GPUs (including NVIDIA’s Tesla V100) alongside Cloud TPUs. We fundamentally want Google Cloud to be the best place to do computing. That includes AI/ML and so…
Any plans to support AMD GPUs and the Radeon Open Compute project? The AI/ML community really needs viable alternatives to NVIDIA, otherwise they will continue to flex pricing power. Google, via TensorFlow, is in a phenomenal position to promote open source alternatives to the proprietary Deep Learning software ecosystem that we see today with CUDA/CuDNN.
Re: Cloud TPUs in Beta
#107Disclosure: I work on Google Cloud. I want to highlight this paragraph from the post: > Here at Google Cloud, we want to provide customers with the best cloud for every ML workload and will offer a variety of high-performance CPUs (including Intel Skylake) and GPUs (including NVIDIA’s Tesla V100) alongside Cloud TPUs. We fundamentally want Google Cloud to be the best place to do computing. That includes AI/ML and so…
Did you guys ever reveal the internal math model of TPU 2?
We know V100 is FP16/FP32 on their tensor cores, when will you follow suit?
Edit: sort of, from https://www.theregister.co.uk/2017/12/14/google_tpu2_specs_i...
"32-bit floating-point precision math units for scalars and vectors, and 32-bit floating-point-precision matrix multiplication units with reduced precision for multipliers."
So what does "reduced" mean exactly?
Re: Cloud TPUs in Beta
#108Earlier quoted context omitted.
My guess: Never
I hope that's not true, for the sake of progress. Todays clouds wouldn't have happened if AMD and Intel had restricted cloud use of their processors.
Among other things, it would be expensive (in a ton of ways), a digression, require providing direct end user support in a way they aren't good at.
It also would have significant export restrictions: Neural network related asics are very tightly export controlled:
https://www.bis.doc.gov/index.php/forms-documents/pdfs/1245-...
(search for neural network)
My 2c: It would be an expensive waste of time for Google :)
Though certainly, not gonna disagree it would be cool for the sake of progress.
Re: Cloud TPUs in Beta
#109Disclosure: I work on Google Cloud. I want to highlight this paragraph from the post: > Here at Google Cloud, we want to provide customers with the best cloud for every ML workload and will offer a variety of high-performance CPUs (including Intel Skylake) and GPUs (including NVIDIA’s Tesla V100) alongside Cloud TPUs. We fundamentally want Google Cloud to be the best place to do computing. That includes AI/ML and so…
Did you guys ever reveal the internal math model of TPU 2? We know V100 is FP16/FP32 on their tensor cores, when will you follow suit? Edit: sort of, from https://www.theregister.co.uk/2017/12/14/google_tpu2_specs_i... "32-bit floating-point precision math units for scalars and vectors, and 32-bit floating-point-precision matrix multiplication units with reduced precision for multipliers." So what does "reduced" mean…
We still don’t document it exactly, but [1] shows that bfloat16 is supported on lots of ops.
Re: Cloud TPUs in Beta
#110Earlier quoted context omitted.
Microsoft uses FPGAs for Deep Learning. Source: https://techcrunch.com/2017/08/22/microsoft-brainwave-aims-t...
... for inference. I don't know of anyone who takes training on FPGAs seriously. They tend to get crushed by GPU/TPU/other ASIC in throughput, perf/watt, and perf/$.
Training on FPGAs starts to make a lot of sense when you consider low bit precision computation (e.g. DoReFa-Net).