91–100 of 138 posts
Earlier quoted context omitted.
That's my point. Competitors are largely moated out by high costs of TPU production and proprietary drivers.
Why are proprietary drivers a blocker? As long as you expose the same gRPC interface, your customers don't need to know what happens behind the scenes. You could have an FPGA or a Beowulf cluster of Raspberry Pis hiding.
Disclosure: I work on Google Cloud. I want to highlight this paragraph from the post: > Here at Google Cloud, we want to provide customers with the best cloud for every ML workload and will offer a variety of high-performance CPUs (including Intel Skylake) and GPUs (including NVIDIA’s Tesla V100) alongside Cloud TPUs. We fundamentally want Google Cloud to be the best place to do computing. That includes AI/ML and so…
> high-performance CPUs (including Intel Skylake) Any plans for ryzen?
But I’d like to note that even if we were to use parts internally at Google (or not!), that for Cloud what matters is market demand. If there really was enormous customer demand for say ARM64, then we would look into it, even if the rest of Google wasn’t interested.
Earlier quoted context omitted.
This is really Tensorflow as a service. You get an IP address and a port you send gRPC requests to: https://github.com/tensorflow/tpu/blob/master/tools/diagnost... Presumably, there's a whole server behind that address that has all the right drivers and libraries: details you don't need to care about. The only partial lock-in is that not all ops are supported and you need to figure if there are any parts of the graph…
That's my point. Competitors are largely moated out by high costs of TPU production and proprietary drivers.
The hardest part in DNNs is the model and data. That's basically platform-independent. My students mix and match TensorFlow and Caffe, for example, on several different models.
The next part is getting the model implemented in a framework (TensorFlow? Caffe? MXNet? PyTorch?). That's work to change, particularly if you're in a production environment. But it's not the same amount of work as collecting data and building a model.
The final part is running training - CPUs, GPUs, TPUs, etc. This is really fungible. The platform-specific optimizations are relatively small here.
Looking at it from a customer perspective:
- Can a trained model be exported (weights included) for use on another platform? (yes)
- Can the code written for training be used on the customer's own hardware? (yes, absent any small tweaks needed for TPU, but they're *small*).
- Might the customer not want to leave because of ease-of-use, particularly at scale, or performance, or total cost of ownership? (yes, and I think that's what the sales pitch is).
Earlier quoted context omitted.
Disclosure: I work on Google Cloud. Peak ops/second isn’t the only thing that matters though. You have to be able to feed the units. The V100 does lots of finer-grained matrix multiplies which can make it harder to keep up. Don’t get me wrong, the V100 is a great chip. And we’re all looking forward to more (preferably third-party) benchmark results, to tease out when one is the better choice for a workload. But don’t…
This makes no sense, the V100 has more memory bandwidth than both the TPU and TPUv2
Edit for this part of the thread: the best public numbers are in the linked presentation [1].
[1] https://supercomputersfordl2017.github.io/Presentations/Imag...
Earlier quoted context omitted.
>If TPUs hold up to the hype, GCloud may become the de facto for ML/AI startups. Don't startups want to win a big exit though? Google won't need to buy the startup for billions, because the TOS already grants them permission to use all the models and training data for free. Seems like a Faustian bargain to me.
Regardless of the TOS saying that or not (I haven't read them), I can think of at least two reasons why your statement doesn't hold: 1) AI startups usually don't have a lot of value to potential acquirers based on their data, but based on other things (e.g., talent, customers, business model, platform, brand). That's like saying you shouldn't use AWS because Amazon can just steal and commercialize all your data. 2) T…
How would you ever know if they did? People who worked at Google have been accused, by Google, of stealing the entire self driving car program and taking it to a competitor.
Earlier quoted context omitted.
This is true of the TPU as well, check out their paper's utilization numbers. If you ignore one outlier at ~90% utilization, their utilization plummets. I'm glad people are finally looking past the b.s. "peak" numbers for once though.
Has Google published data on the memory bandwidth of TPU v2 (aka "cloud TPU")? I'm having trouble finding it. In any case I agree, we shouldn't be looking at the stated peak compute of either of the chips. (Disclaimer: I work at Google on XLA, and have in the past worked on TPUs.)
which claims 2400 GB/s for the board and 600 GB/s per “chip”.
Earlier quoted context omitted.
This makes no sense, the V100 has more memory bandwidth than both the TPU and TPUv2
We mostly focus on the “whole board” numbers. So it’s not only units “local” HBM, but NVLINK versus TPU to TPU. Sorry for the confusion. Edit for this part of the thread: the best public numbers are in the linked presentation [1]. [1] https://supercomputersfordl2017.github.io/Presentations/Imag...
Disclosure: I work on Google Cloud. I want to highlight this paragraph from the post: > Here at Google Cloud, we want to provide customers with the best cloud for every ML workload and will offer a variety of high-performance CPUs (including Intel Skylake) and GPUs (including NVIDIA’s Tesla V100) alongside Cloud TPUs. We fundamentally want Google Cloud to be the best place to do computing. That includes AI/ML and so…
Earlier quoted context omitted.
From a media standpoint: frontline comment moderation. It would take a lot of the legwork out of filtering for advertisements, uncivil discussion, attacks, off topic posts, and trolling. I believe NYT does this already, but using minimal oversight to prevent any edge case misses or false positives. Presently there’s not much in the way of suitable options for large media that build their modules in house. At the same…
For those who want to read more: https://www.nytimes.com/2017/06/13/insider/have-a-comment-le... [not particularly techincal, but given the GP seemed to be skeptical about real world use I think this is still appropriate]
This is exciting. There are lots of specific reasons to choose Google Cloud over AWS (and vice versa), but proprietary hardware is surely an advantage that is going to be hard to replicate / compete with. If TPUs hold up to the hype, GCloud may become the de facto for ML/AI startups.