Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud TPUs in Beta

cloudplatform.googleblog.com

Post reply on HN
Print view View original

111–120 of 138 posts

Re: Cloud TPUs in Beta

#111

Post by IanCal »

danjoc wrote: post #95
shafyy wrote: post #56

Earlier quoted context omitted.

Regardless of the TOS saying that or not (I haven't read them), I can think of at least two reasons why your statement doesn't hold: 1) AI startups usually don't have a lot of value to potential acquirers based on their data, but based on other things (e.g., talent, customers, business model, platform, brand). That's like saying you shouldn't use AWS because Amazon can just steal and commercialize all your data. 2) T…

>They can surely look at it and maybe do stuff with them internally, but I am pretty sure that they can't use them commercially. How would you ever know if they did? People who worked at Google have been accused, by Google, of stealing the entire self driving car program and taking it to a competitor.

That is just not true, the suit was about LIDAR.

It's also vastly different. Of course someone working at at google on a project has access to that project. It doesn't mean they have access to your stuff.

Re: Cloud TPUs in Beta

#112

Post by jacksmith21006 »

danjoc wrote: post #43
tejasmanohar wrote: post #9

This is exciting. There are lots of specific reasons to choose Google Cloud over AWS (and vice versa), but proprietary hardware is surely an advantage that is going to be hard to replicate / compete with. If TPUs hold up to the hype, GCloud may become the de facto for ML/AI startups.

>If TPUs hold up to the hype, GCloud may become the de facto for ML/AI startups. Don't startups want to win a big exit though? Google won't need to buy the startup for billions, because the TOS already grants them permission to use all the models and training data for free. Seems like a Faustian bargain to me.

Ha! No Google does not get the models and data.

Re: Cloud TPUs in Beta

#113

Post by jacksmith21006 »

danjoc wrote: post #95
shafyy wrote: post #56

Earlier quoted context omitted.

Regardless of the TOS saying that or not (I haven't read them), I can think of at least two reasons why your statement doesn't hold: 1) AI startups usually don't have a lot of value to potential acquirers based on their data, but based on other things (e.g., talent, customers, business model, platform, brand). That's like saying you shouldn't use AWS because Amazon can just steal and commercialize all your data. 2) T…

>They can surely look at it and maybe do stuff with them internally, but I am pretty sure that they can't use them commercially. How would you ever know if they did? People who worked at Google have been accused, by Google, of stealing the entire self driving car program and taking it to a competitor.

Because you do not stay in business if you operate in such a manner. Plus it is not good from an employee standpoint in retaining. Most people prefer to conduct themselves in an ethical manner.

Hard to get employees to not steal from you if you are stealing from your customers.

Re: Cloud TPUs in Beta

#114

Post by cstejerean »

riku_iki wrote: post #89
brd wrote: post #66

Earlier quoted context omitted.

I don't know what qualifies as novel for you but some use cases I've seen: On the retail side: Using computer vision to deliver alerts about shelf condition. For farming: Using computer vision + ML to devise and track health monitoring for crops. For manufacturing: Predictive maintenance of equipment has been a very popular area of focus. There have been countless use cases on the finance side of things. For instance…

I am curious if DL really can deliver good results in such spaces. We all see success stories for very refined and well defined problems with huge amount of training data, with models created by 1% top engineers, but for average business such conditions may not be achievable, to train model to recognize various shelf conditions in different situations, buildings, etc. you need nontrivial set of training data, and wil…

Most businesses will probably not develop and train their own systems, but rather implemented solutions developed by the folks with the expertise and training data.

Re: Cloud TPUs in Beta

#116

Post by jlebar »

deepnotderp wrote: post #71
boulos wrote: post #58

Earlier quoted context omitted.

Disclosure: I work on Google Cloud. Peak ops/second isn’t the only thing that matters though. You have to be able to feed the units. The V100 does lots of finer-grained matrix multiplies which can make it harder to keep up. Don’t get me wrong, the V100 is a great chip. And we’re all looking forward to more (preferably third-party) benchmark results, to tease out when one is the better choice for a workload. But don’t…

This makes no sense, the V100 has more memory bandwidth than both the TPU and TPUv2

V100 has 900gb/s memory bandwidth [0].

TPUv2 has 600gb/s per chip x 4 chips, so 2400gb/s [1].

As we've discussed elsewhere [2], comparing TPUv2 to V100 on a per chip basis doesn't make much sense. Who cares how many chips are on the board? If Google announced tomorrow that TPUv3 is coming out, which is identical to TPUv2 but the four chips are glued together, nobody would care.

The questions that we should instead be asking are, how fast can I train my model and how much does it cost?

Per elsewhere in thread [3], on Volta you have 900gb/s per 100Tops/s = 0.9 bytes/s per op/s, whereas on TPUv2 you have 2400gb/s memory bandwidth over 180Tops/s = 1.33 bytes/s per op/s. This means that TPUv2's memory-bandwidth-to-compute ratio is 1.33/9 = 1.5x higher than Volta's.

We can do a similar comparison for memory available. V100 has 16gb per 100Tops, TPUv2 has 64gb per 180Tops. So the memory-to-compute ratio for Volta is 16g/100T = .16 milli while for TPUv2 it's 64g/180T = .36 milli, for a ratio of .36/.16 = 2.25x higher on TPUv2.

Does any of this matter? Does it translate into faster and/or cheaper training? Do models actually need and benefit from this additional memory and memory bandwidth? My guess from working on GPUs is yes, at least insofar as bandwidth is concerned, but it's just a guess. I'm excited to find out for real.

(Disclaimer: I work at Google on XLA, and used to work on TPUs.)

[0] https://images.nvidia.com/content/technologies/volta/pdf/437... [1] https://supercomputersfordl2017.github.io/Presentations/Imag... [2] https://news.ycombinator.com/item?id=16360212 [3] https://news.ycombinator.com/item?id=16359531

Re: Cloud TPUs in Beta

#117

Post by londons_explore »

sabalaba wrote: post #106
boulos wrote: post #67

Disclosure: I work on Google Cloud. I want to highlight this paragraph from the post: > Here at Google Cloud, we want to provide customers with the best cloud for every ML workload and will offer a variety of high-performance CPUs (including Intel Skylake) and GPUs (including NVIDIA’s Tesla V100) alongside Cloud TPUs. We fundamentally want Google Cloud to be the best place to do computing. That includes AI/ML and so…

Any plans to support AMD GPUs and the Radeon Open Compute project? The AI/ML community really needs viable alternatives to NVIDIA, otherwise they will continue to flex pricing power. Google, via TensorFlow, is in a phenomenal position to promote open source alternatives to the proprietary Deep Learning software ecosystem that we see today with CUDA/CuDNN.

Google would happily accept patches to enable support for it.

AMD hopefully has a team writing such patches now. It makes business sense for them to do so.

Google is getting even more price gouging from Nvidia than the general public, and has even more incentive to level the playing field.

Re: Cloud TPUs in Beta

#118

Post by londons_explore »

boulos wrote: post #109
oneshot908 wrote: post #107

Earlier quoted context omitted.

Did you guys ever reveal the internal math model of TPU 2? We know V100 is FP16/FP32 on their tensor cores, when will you follow suit? Edit: sort of, from https://www.theregister.co.uk/2017/12/14/google_tpu2_specs_i... "32-bit floating-point precision math units for scalars and vectors, and 32-bit floating-point-precision matrix multiplication units with reduced precision for multipliers." So what does "reduced" mean…

We still don’t document it exactly, but [1] shows that bfloat16 is supported on lots of ops. [1] https://cloud.google.com/tpu/docs/tensorflow-ops

It is documented here: https://github.com/tensorflow/tensorflow/blob/master/tensorf...

Re: Cloud TPUs in Beta

#119

Post by deepnotderp »

boulos wrote: post #102
deepnotderp wrote: post #97

Earlier quoted context omitted.

That's... a skewed ... comparison, NVLINK is a board to board connection whereas you're talking about TPU to TPU on board communication if I understand correctly?

That's sort of the point though! We're actually selling these as the "board". So the right way to compare things is sort of DGX-1 style "deep learning rig" versus a board of four TPU units (or several connected). The on-chip network is a big part of its overall efficiency. I don't recall what (if anything) we've said about how we link up the boards across racks, but the folks at Next Platform looked pretty carefully…

It's not the point though. You're comparing whole board tpu FLOPs (4x 45) but then comparing tpu single chip chip2chip communication with nVidia board2board communication.

You can't have your cake and eat it too.

Re: Cloud TPUs in Beta

#120

Post by boulos »

londons_explore wrote: post #118
boulos wrote: post #109

Earlier quoted context omitted.

We still don’t document it exactly, but [1] shows that bfloat16 is supported on lots of ops. [1] https://cloud.google.com/tpu/docs/tensorflow-ops

It is documented here: https://github.com/tensorflow/tensorflow/blob/master/tensorf...

That doesn’t prove that the chip operates at 16 bits. For example, we could do 18-bit multipliers (or anything >= 16) and still use 16-bit floats.
Post reply on HN