51–60 of 95 posts
Google claim 29x better performance-per-Watt with TPUs than contemporary GPUs[0]. Interesting to contrast that to the images-per-$ figure in this post, which is more like 2x. I assume there's a high capital cost for this new hardware, but when they scale it up I wonder if the ratio of cost TPU to GPU will trend towards the ratio of power-per-Watt between the platforms? Seems like a natural limit, even if it never qui…
TPUv2 is specially optimized for deep learning.
Nvidia's Volta microarchitecture is graphics processor with additional tensor units. It's a General-purpose (GPGPU) chip designed with graphics and other scientific computing tasks in mind. Nvidia has enjoyed monopoly power in the market and single microarchitecture has been enough in every high performance category.
Next logical step for Nvidia is to develop specialized deep learning TPU to compete with TPUv2 and others.
Disclosure: I work on Google Cloud. While not perfect, I want to commend the RiseML folks for doing not only an “just out of the box” run in both regular and fp16 mode (for V100), but also adding their own LSTM experiment to the mix. We need third-party benchmarks whenever new hardware or software are being sold by vendors (reminder: I benefit from you buying Google Cloud!). I hope the authors are able to collect som…
Thanks for your feedback and your suggestions (and from everybody else)! We'll make sure to gather all of the valuable feedback and run additional experiments. Different batch sizes and a comparison against >1 GPUs is already planned (and partly executed).
Specialization brings speedups. TPUv2 is specially optimized for deep learning. Nvidia's Volta microarchitecture is graphics processor with additional tensor units. It's a General-purpose (GPGPU) chip designed with graphics and other scientific computing tasks in mind. Nvidia has enjoyed monopoly power in the market and single microarchitecture has been enough in every high performance category. Next logical step for…
Earlier quoted context omitted.
So this is a chip that no one outside of Google is going to be able to get a physical copy of ever? It makes any benchmarks become Google-cloud benchmarks, right? Edit: I am complaining a bit about the lack of availability but there's also a real point here. If there's no source for TPUs outside of Google, Google Cloud competes only with other cloud providers and with owning physical GPUs - long term, it has no incen…
It's going to be a very exciting multi-company arms race -- at minimum, Google, Intel, Nvidia. Microsoft has their FPGAs, Amazon has their rumors. And there are several startups trying to enter the space. I don't think we're looking at stagnating; very much the opposite. It's going to be fantastic for the field. (I'm saying this with my CMU hat, not my Google hat.)
The era of the mainframe, with each provider competing with a custom chip, wasn't necessarily beneficial for individual buying computer power.
The bar graph seems a little whacky. It groups the TPU (which can only do FP16) with the FP32 results from the GPUs, then puts the FP16 GPU results off to the side even though that's much closer to what the TPU is doing. Impressive results regardless though; quite a bit faster than V100 than the paper specs would suggest.
Good point, I agree that the FP16 GPU results should be closer or grouped with the TPU results. We'll try to update accordingly.
[Edited] The top line results focus on comparing four TPUs in a rack node (which marketing cleverly named “one cloud TPU”), running ~16 bit mixed precision, to one GPU (out of 8 in a rack node), also capable of 16 bit or mixed precision, but handicapped to 32 bit IEEE 754. That is a misleading comparison. Images/$ are obviously more directly comparable, but again the emphasized comparisons are at different precision.…
agreed, it's almost purposefully very misleading. He's not even using the same version of tensorflow, or the current version of cuda (9.1).
It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…
I wouldn't be surprised if public TPUs are to some degree a way to print money: at least for a while, Google can probably just rent out its unused capacity that it had already planned and paid for. :-)
Earlier quoted context omitted.
I thought they were going to provide to other cloud providers? I’m also guessing if you’re willing to purchase a lot of them then they’re willing to talk...
I'd be interested if anyone has details. It may be that the other cloud providers would then sell them to those individuals. Indeed, the job of entities called "distributors" to buy big lots from manufacturers and break them up. And of course, I don't know what the point of (apparently) keeping them out of the average person's hand would be.
[Edited] The top line results focus on comparing four TPUs in a rack node (which marketing cleverly named “one cloud TPU”), running ~16 bit mixed precision, to one GPU (out of 8 in a rack node), also capable of 16 bit or mixed precision, but handicapped to 32 bit IEEE 754. That is a misleading comparison. Images/$ are obviously more directly comparable, but again the emphasized comparisons are at different precision.…
Point well taken, we'll make sure to add a comparison to 4 and 8 GPUs. For now, a "Cloud TPU" (containing 8 cores) seems to be the smallest unit to allocate. The question of what exactly makes up a single device and how many to compare against each other is not easy to answer.