Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud TPUs in Beta

cloudplatform.googleblog.com

Post reply on HN
Print view View original

81–90 of 138 posts

Re: Cloud TPUs in Beta

#81

Post by obmelvin »

52-6F-62 wrote: post #65
karpodiem wrote: post #54

Earlier quoted context omitted.

Can you describe a line of business function that makes novel use of ML?

From a media standpoint: frontline comment moderation. It would take a lot of the legwork out of filtering for advertisements, uncivil discussion, attacks, off topic posts, and trolling. I believe NYT does this already, but using minimal oversight to prevent any edge case misses or false positives. Presently there’s not much in the way of suitable options for large media that build their modules in house. At the same…

For those who want to read more: https://www.nytimes.com/2017/06/13/insider/have-a-comment-le... [not particularly techincal, but given the GP seemed to be skeptical about real world use I think this is still appropriate]

Re: Cloud TPUs in Beta

#82

Post by dgacmu »

saguro wrote: post #29
Cyph0n wrote: post #23

Earlier quoted context omitted.

GPUs are much more complex (general-purpose) and therefore cannot be optimized beyond a certain point due to timing requirements and PVT (process, temperature, voltage) variations. In other words, the more stuff you have on an ASIC, the more careful you have to be ensure a margin of tolerance for variations.

So the only advantage of the TPU is it's a simpler and more specialized asic? Google didn't break any new ground in terms of training perf?

So, a way to think of this is: The speed (and therefore, cost) of training a machine learning model depends on (a) the ML techniques (how rapidly the model converges and to what accuracy); and (b) how quickly the processor executes the operations involved in the ML techniques.

The TPU is only an improvement in (b). It's not going to result in a big-O style speedup, because the same training algorithms and architectures will run on it that we run on CPUs & GPUs today.

I'm not sure what counts as "breaking new ground" - is that 10%? 100%? 1000? :-) The things to watch out for in benchmarks will be:

(a) Perf/$. This is actually a big deal - one of my students recently blew through $5000 of Google Cloud credits running Imagenet experiments, in a week. And we didn't finish them! As this cost really drops, it enables things like Neural Architecture Search, which uses tons of compute capability to explore architectural variants automatically.

(b) Absolute perf.

(c) Performance scaling. To what degree will the fast, 2D torroidal mesh allow a full pod of Cloud TPUs to scale nearly-linearly? Absolute training times matter from a user productivity standpoint. Waiting 30 minutes for a result is very different from waiting 12 hours (you can do one of these while you sneak out to go running! :-).

The NIPS'17 slides have more technical context for some of this: https://supercomputersfordl2017.github.io/Presentations/Imag...

Re: Cloud TPUs in Beta

#83

Post by puzzle »

whataretensors wrote: post #75
puzzle wrote: post #68

Earlier quoted context omitted.

This is really Tensorflow as a service. You get an IP address and a port you send gRPC requests to: https://github.com/tensorflow/tpu/blob/master/tools/diagnost... Presumably, there's a whole server behind that address that has all the right drivers and libraries: details you don't need to care about. The only partial lock-in is that not all ops are supported and you need to figure if there are any parts of the graph…

That's my point. Competitors are largely moated out by high costs of TPU production and proprietary drivers.

Why are proprietary drivers a blocker? As long as you expose the same gRPC interface, your customers don't need to know what happens behind the scenes. You could have an FPGA or a Beowulf cluster of Raspberry Pis hiding.

Re: Cloud TPUs in Beta

#85

Post by jlebar »

deepnotderp wrote: post #73
jlebar wrote: post #57

Earlier quoted context omitted.

I know people don't know what to expect from tpu performance, but does anyone actually get 100tops out of Volta? I thought you'd have to spin the tensorcores and never touch memory, which is...not realistic. I know you hedged by saying "back of the envelope", but I'd much rather compare on real benchmarks than based on cited peak performance numbers, which are kind of meaningless.

This is true of the TPU as well, check out their paper's utilization numbers. If you ignore one outlier at ~90% utilization, their utilization plummets. I'm glad people are finally looking past the b.s. "peak" numbers for once though.

Has Google published data on the memory bandwidth of TPU v2 (aka "cloud TPU")? I'm having trouble finding it.

In any case I agree, we shouldn't be looking at the stated peak compute of either of the chips.

(Disclaimer: I work at Google on XLA, and have in the past worked on TPUs.)

Re: Cloud TPUs in Beta

#87

Post by riku_iki »

boulos wrote: post #67

Disclosure: I work on Google Cloud. I want to highlight this paragraph from the post: > Here at Google Cloud, we want to provide customers with the best cloud for every ML workload and will offer a variety of high-performance CPUs (including Intel Skylake) and GPUs (including NVIDIA’s Tesla V100) alongside Cloud TPUs. We fundamentally want Google Cloud to be the best place to do computing. That includes AI/ML and so…

> high-performance CPUs (including Intel Skylake)

Any plans for ryzen?

Re: Cloud TPUs in Beta

#88

Post by boulos »

Cyph0n wrote: post #20
minimaxir wrote: post #6

That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU ( https://aws.amazon.com/ec2/pricing/on-demand/ , the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr . Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper ( https://cloud.google.com/compute/pricing , n1-highmem-64, +$3.78/hr t…

A better comparison would be the f1.16xlarge[1] instance @ ~$4/hr. It comes with 8 FPGAs (12 Gbps link) and 64 vCPUs. [1]: https://aws.amazon.com/ec2/instance-types/f1/ Edit: I'm genuinely curious about why this comment is getting downvotes.

Disclosure: I work on Google Cloud.

I didn’t downvote you, but presumably people disagree with “Here’s an FPGA” as comparable to being given a working piece of hardware. That is, would you have said that the best comparison to a V100 is this same FPGA box?

I (and others) get what you were trying to say: TPUs are ASICs that aren’t general purpose at all, so an FPGA is a better comparison than a more general purpose GPU. As an end user, that just isn’t true though. If someone hands you an f1.16xlarge, you have to build your own psuedo-chip for machine learning. While with this offering, TensorFlow handles the acceleration / offload for you.

Re: Cloud TPUs in Beta

#89

Post by riku_iki »

brd wrote: post #66
karpodiem wrote: post #54

Earlier quoted context omitted.

Can you describe a line of business function that makes novel use of ML?

I don't know what qualifies as novel for you but some use cases I've seen: On the retail side: Using computer vision to deliver alerts about shelf condition. For farming: Using computer vision + ML to devise and track health monitoring for crops. For manufacturing: Predictive maintenance of equipment has been a very popular area of focus. There have been countless use cases on the finance side of things. For instance…

I am curious if DL really can deliver good results in such spaces.

We all see success stories for very refined and well defined problems with huge amount of training data, with models created by 1% top engineers, but for average business such conditions may not be achievable, to train model to recognize various shelf conditions in different situations, buildings, etc. you need nontrivial set of training data, and will have unclear expectations about model performance.

Re: Cloud TPUs in Beta

#90

Post by danjoc »

barrus wrote: post #55
danjoc wrote: post #42

Reading the TOS it seems like this is a really great deal for Google: "When you upload, submit, store, send or receive content to or through our Services, you give Google (and those we work with) a worldwide license to use, host, store, reproduce, modify, create derivative works (such as those resulting from translations, adaptations or other changes we make so that your content works better with our Services), commu…

Cloud TPU product manager here. The TOS you are quoting only refers to the information you provide in the survey. Here are the Google Cloud TOS: https://cloud.google.com/terms/ if you're interested in what Cloud does with customers data. 5.2 Use of Customer Data. Google will not access or use Customer Data, except as necessary to provide the Services to Customer. Your training data and models are secure.

This URL isn't on the TPU beta signup page. The Google TOS is. Perhaps you can see the confusion? I would be reluctant to trust random 37 karma guy on Hacker News message board on this particularly important consideration.
Post reply on HN