Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking Google’s new TPUv2

blog.riseml.com

Post reply on HN
Print view View original

11–20 of 95 posts

Re: Benchmarking Google’s new TPUv2

#11

Post by jlebar »

bhouston wrote: post #4

It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…

> I am surprised the TPU doesn't kick the ass of the NVIDIA chips.

30% cheaper e2e price for the company's first public offering, compared to the market leader's top-of-the-line chip sounds...pretty good to me?

Re: Benchmarking Google’s new TPUv2

#13

Post by Veelox »

bhouston wrote: post #4

It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…

I agree with you that the cost of TPU development probably out ways the number of dollars that Google will earn renting TPUs. The thing is, no one else has a TPU but Google. That doesn't look like it will change any time soon. That means that if you want to run the fastest machine learning models, you have to use Google Cloud. Now, Google doesn't just benefit from the TPUs, they can now sell more customers to come to their cloud. After that starts happening, all of the best machine learning people will have Google Cloud experience. Then when they start something new, they will use what they know: Google Cloud. Also, they will create the tooling that only works with TPUs and gives an advantage you cannot use outside of Google Cloud. So, it will be a net win for Google even if it is more expensive to run a TPU than what they are renting them for.

tl;dr TPU helps Google Clouds' network effect.

Re: Benchmarking Google’s new TPUv2

#14

Post by twtw »

IIRC, TPUv2 uses 16 bit floating point in some format with higher dynamic range and lower precision than standard fp16. Can someone confirm?

If that is right, is the "Tensorflow-optimized" Resnet-50 using 16bit floats when running on TPUv2?

Re: Benchmarking Google’s new TPUv2

#15

Post by chapill »

NelsonMinar wrote: post #5
chapill wrote: post #3

I wonder if Chinese companies will use (or be allowed to use) TPUs. It seems like a pretty obvious way to have the NSA scoop up any Chinese AI advancements China may want to keep secret.

I wonder which Chinese companies are developing their own processors like TPUs.

Well, they do have the fastest supercomputer in the world currently and it's made with homegrown chips. No Intel ME backdoors there. Smaller chinese companies could, for a little more money, get similar performance buying 8x V100 machines from NVidia. I don't think they want to share their advancements in AI fighter pilots with USA. They have a big lead.

Re: Benchmarking Google’s new TPUv2

#16

Post by chapill »

bhouston wrote: post #4

It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…

>I am surprised the TPU doesn't kick the ass of the NVIDIA chips.

Yeah, I'm a bit disappointed myself. When announced initially, it seemed Google had a huge lead. But they dragged their feet for two years getting it to market, and now NVidia is nipping at their heels already.

I suspect they are using the TPUs internally for competitive advantage, and these are the leftovers they are done with. They're probably using v4 or v5 internally already.

Re: Benchmarking Google’s new TPUv2

#17

Post by olfactory »

chapill wrote: post #15
NelsonMinar wrote: post #5

Earlier quoted context omitted.

I wonder which Chinese companies are developing their own processors like TPUs.

Well, they do have the fastest supercomputer in the world currently and it's made with homegrown chips. No Intel ME backdoors there. Smaller chinese companies could, for a little more money, get similar performance buying 8x V100 machines from NVidia. I don't think they want to share their advancements in AI fighter pilots with USA. They have a big lead.

What is the hardest thing to accomplish with something like a TPU? Is it the IP or the fabrication?

How does the TPU design offer improved performance? By leveraging IP or fabrication improvements?

Re: Benchmarking Google’s new TPUv2

#18

Post by deepnotderp »

twtw wrote: post #14

IIRC, TPUv2 uses 16 bit floating point in some format with higher dynamic range and lower precision than standard fp16. Can someone confirm? If that is right, is the "Tensorflow-optimized" Resnet-50 using 16bit floats when running on TPUv2?

Re: fp16 dynamic range: yes.

Re: Benchmarking Google’s new TPUv2

#19

Post by voxadam »

bhouston wrote: post #4

It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…

Since TPUs are used at Google to process data for its own service offerings (e.g. image classification, voice recognition, language translation, NLP, route planning, etc.) wouldn't it be fair to say that they will also be able to recoup the sunk costs (R&D) by purchasing fewer GPUs?

Re: Benchmarking Google’s new TPUv2

#20

Post by bitL »

bhouston wrote: post #4

It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…

> TPU doesn't kick the ass of the NVIDIA chips

It used to until Volta came out with basically TPUs embedded on the board. We will see if AMD will join them as Vega in theory should be around Volta as well, just tooling is not there.

Post reply on HN