Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking Google’s new TPUv2

blog.riseml.com

Post reply on HN
Print view View original

61–70 of 95 posts

Re: Benchmarking Google’s new TPUv2

#61

Post by puzzle »

boulos wrote: post #23
bhouston wrote: post #4

It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…

Disclosure: I work on Google Cloud. By the logic above, you would conclude that TPUv1 (the inference-only chip) might have been a mistake, but we’ve been very public about how it “saved us from building lots of datacenters”. That wasn’t ever sold as part of Cloud, so the benefit there is all from the second bit you mentioned: cheaper and more efficient than GPUs at the time . The paper also goes into more detail, but…

Is there going to be an updated paper on performance per Watt, now that TPUv2 is public and V100 has been preannounced on the Google blog?

Re: Benchmarking Google’s new TPUv2

#62

Post by twtw »

BLanen wrote: post #48
jrk wrote: post #25

[Edited] The top line results focus on comparing four TPUs in a rack node (which marketing cleverly named “one cloud TPU”), running ~16 bit mixed precision, to one GPU (out of 8 in a rack node), also capable of 16 bit or mixed precision, but handicapped to 32 bit IEEE 754. That is a misleading comparison. Images/$ are obviously more directly comparable, but again the emphasized comparisons are at different precision.…

The amount of devices is what is completely irrelevant. It's all about performance per dollar.

Sure, for a customer. But from a technological point of view, performance per dollar doesn't tell us everything. A company could subsidize their compute service and get astounding perf/dollar with a not particularly impressive chip.

I'd like to know perf/watt, for instance, even if it doesn't matter to the customer.

Re: Benchmarking Google’s new TPUv2

#63

Post by twtw »

Nokinside wrote: post #52

Specialization brings speedups. TPUv2 is specially optimized for deep learning. Nvidia's Volta microarchitecture is graphics processor with additional tensor units. It's a General-purpose (GPGPU) chip designed with graphics and other scientific computing tasks in mind. Nvidia has enjoyed monopoly power in the market and single microarchitecture has been enough in every high performance category. Next logical step for…

> Next logical step for Nvidia is to develop specialized deep learning TPU to compete with TPUv2 and others

I don't know, this benchmark seems to show V100 doing pretty well against a specialized ASIC. It may well be that all NVIDIA has to do is cut costs on V100 to make a two V100s about as expensive as the cloud TPUv2. With increased batch size, it looks like two V100s would have performance comparable to TPUv2.

Re: Benchmarking Google’s new TPUv2

#64

Post by twtw »

bitL wrote: post #20
bhouston wrote: post #4

It is hard for Google to make money on these TPUs as the whole engineering cost has to be made back from its pricing on Google Cloud, where as with NVIDIA it can pay back its engineering costs via multiple mature channels (games, super computers, and multiple cloud providers.) I wonder which is higher, the cost for creating the TPUs in terms of engineering and manufacturing or the cost differential in terms of usage…

> TPU doesn't kick the ass of the NVIDIA chips It used to until Volta came out with basically TPUs embedded on the board. We will see if AMD will join them as Vega in theory should be around Volta as well, just tooling is not there.

How long has Google had the TPUv2 for internal use? I was under the impression that V100 and TPUv2 where developed around same time. They were certainly announced around the same time at least. Just seems weird to say "it used to," when V100 has been shipping since mid-summer 2017.

Re: Benchmarking Google’s new TPUv2

#65

Post by ucaetano »

joe_the_user wrote: post #55
dgacmu wrote: post #49

Earlier quoted context omitted.

It's going to be a very exciting multi-company arms race -- at minimum, Google, Intel, Nvidia. Microsoft has their FPGAs, Amazon has their rumors. And there are several startups trying to enter the space. I don't think we're looking at stagnating; very much the opposite. It's going to be fantastic for the field. (I'm saying this with my CMU hat, not my Google hat.)

I would not see things stagnating but it seems like there's a potential for the individual to get cut out of this excitement if each of these entities is keeping it's chips close to it's chest. The era of the mainframe, with each provider competing with a custom chip, wasn't necessarily beneficial for individual buying computer power.

Industries go through cycles of innovation and concentration. During innovation cycles, many new non-standard products appear with innovative solutions, the entire pie grows really fast. As growth eventually stabilizes, standards become more relevant and consolidation happens, eventually leading to a stagnation that makes the industry ripe for disruption and change again.

If you look at processors, you see that with the early custom processors, followed by some standardization and copy around the IBM S/360, followed by more proprietary innovation around the PC era, resulting finally in the x86, eventually disrupted by the mobile chips, which then consolidated around ARM and so on.

Re: Benchmarking Google’s new TPUv2

#67

Post by boulos »

BLanen wrote: post #48
jrk wrote: post #25

[Edited] The top line results focus on comparing four TPUs in a rack node (which marketing cleverly named “one cloud TPU”), running ~16 bit mixed precision, to one GPU (out of 8 in a rack node), also capable of 16 bit or mixed precision, but handicapped to 32 bit IEEE 754. That is a misleading comparison. Images/$ are obviously more directly comparable, but again the emphasized comparisons are at different precision.…

The amount of devices is what is completely irrelevant. It's all about performance per dollar.

Disclosure: I work on Google Cloud.

Not necessarily. The DGX-1, for example, has pretty poor perf/$$ but reduces the time a data scientist spends waiting. For some organizations, their people time is so valuable that what matters is “what gets me my answers back faster”, because that employee is easily $100/hr+.

That’s actually why the 8xV100 with NVLINK is so attractive (and why the TPUs also have board to board networking, not just chip to chip).

Re: Benchmarking Google’s new TPUv2

#68

Post by chapill »

KirinDave wrote: post #45
chapill wrote: post #36

Earlier quoted context omitted.

Oh, I totally agree with you there. It's just I consider Google a government actor too.

Does this mean you consider Google a government unto itself, or part of an existing government?

Google is the same as NSA, but exists as a dance around 4th amendment. Google can do the sort of spying the US government can't constitutionally do, then hand that over to the government, constitutionally, under gag order if necessary. It's all stagecraft. Same for Apple, Amazon, Intel, etc. Eric Schmidt runs HRCs campaign. Al Gore is on Apple's board.

They are all set up to spy on us. Deep state. They hunt sys admins. If you're here, you're a target.

Re: Benchmarking Google’s new TPUv2

#69

Post by Nokinside »

deepnotderp wrote: post #54
Nokinside wrote: post #52

Specialization brings speedups. TPUv2 is specially optimized for deep learning. Nvidia's Volta microarchitecture is graphics processor with additional tensor units. It's a General-purpose (GPGPU) chip designed with graphics and other scientific computing tasks in mind. Nvidia has enjoyed monopoly power in the market and single microarchitecture has been enough in every high performance category. Next logical step for…

Volta V100 already has "tensor cores" which are basically little matrix multiplication ASICs.

That's what I said.

The microarchctiecture has many unnecessary things and it's not optimized as a whole for deep learning.

Re: Benchmarking Google’s new TPUv2

#70

Post by bitL »

twtw wrote: post #64
bitL wrote: post #20

Earlier quoted context omitted.

> TPU doesn't kick the ass of the NVIDIA chips It used to until Volta came out with basically TPUs embedded on the board. We will see if AMD will join them as Vega in theory should be around Volta as well, just tooling is not there.

How long has Google had the TPUv2 for internal use? I was under the impression that V100 and TPUv2 where developed around same time. They were certainly announced around the same time at least. Just seems weird to say "it used to," when V100 has been shipping since mid-summer 2017.

I think at least for inference TPUv1 was beating all previously available GPUs by a wide margin. TPUv2 did that for training as well, with the exception of Volta.
Post reply on HN