81–90 of 95 posts
Earlier quoted context omitted.
What is the hardest thing to accomplish with something like a TPU? Is it the IP or the fabrication? How does the TPU design offer improved performance? By leveraging IP or fabrication improvements?
Neither, it's a matrix multiply systolic array ASIC, that's been done decades ago. There are host of Chinese companies developing similar processors.
Earlier quoted context omitted.
Well, they do have the fastest supercomputer in the world currently and it's made with homegrown chips. No Intel ME backdoors there. Smaller chinese companies could, for a little more money, get similar performance buying 8x V100 machines from NVidia. I don't think they want to share their advancements in AI fighter pilots with USA. They have a big lead.
What is the hardest thing to accomplish with something like a TPU? Is it the IP or the fabrication? How does the TPU design offer improved performance? By leveraging IP or fabrication improvements?
Becomes a chicken-and-egg problem.
Google was able to solve it by designing TPUs for internal usage first, therefore reaching minimum scale, and then making changes and offering it publicly.
So you'd need to be:
1) Chip designer
2) Massive internal user
3) Cloud service provider
I don't see any other company that has all 3. Amazon has (3), and maybe (1) if they hired the right people. Microsoft could have enough (2) to justify it, but they're going the way of FPGAs.
I'm also not sure how we can take googles word for the numbers, since they might as well be eating a less-than-ideal power cost to promote their platform. Any upfront cost will probably offset by locked-in customers later on.
I might just be a bit cynical though.
Earlier quoted context omitted.
That's why you scroll down the page to the cost comparison, which places it on a more even keel. They do also compare float16 on Volta. Physical packaging is irrelevant -- what matters is dollars to convergence and time to convergence. (I'm obviously biased - I helped with parts of the cloud-side of cloud TPU - but I presume this comment stands on its own. :-)
To be clear, I had read the whole post, I was just being terse since the emphasis seemed to be so heavily on an apples to bananas comparison (I believe 100% of the results cited in the prose, many in bold, are with mismatched precision and batch size), with minimal articulation of the many axes of nuance here. Precision isn't defined at all in the LSTM case, and could easily be the cause of the failure of the TPU run…
(2) Unless Google releases the specs to the h/w, I'd argue that cost is our best proxy. But if you assume that both Google and Amazon want to make a profit on their cloud rentals, it at least gives us a way to get to something we can normalize to (the V100's list price is public, though who knows how much Amazon pays). And, given that you can't buy a Cloud TPU, the price Google charges really is the meaningful answer. It doesn't tell us about fundamentals, but it's the right answer from a consumer standpoint.
I think it's a fair bigger-picture question to ask how we fairly and informatively benchmark cloud-only services in ways that we can not only get consumer-oriented price comparisons, but also learn from the underlying technical choices. The longer-term answer is that we beg Google to write a paper about TPUv2, as they (surprisingly!) did about TPUv1 -- because without that, we just get black box numbers combined with informed speculation based upon glossy board and heatsink photos.
btw - the best current source of specs about TPUv2 was Jeff's NIPS talks: http://learningsys.org/nips17/assets/slides/dean-nips17.pdf
Which mentions a few details like 16GB HBM per chip with 600GB/s memory bandwidth.
(3) I agree completely with you that the comparisons are hard. I'm very glad the authors of the blog post are listening to the feedback they're getting here -- on the LSTM, on batch size comparisons, and about precision and being clear about which things they're measuring.
(Reminder disclosure: It's awkward talking about Google in the third person since they pay me part time, but I'm trying to take this discussion with my academic hat also. This nested series of disclaimers is an amusing commentary about how small the machine learning + systems community is.)
Earlier quoted context omitted.
Does this mean you consider Google a government unto itself, or part of an existing government?
Google is the same as NSA, but exists as a dance around 4th amendment. Google can do the sort of spying the US government can't constitutionally do, then hand that over to the government, constitutionally, under gag order if necessary. It's all stagecraft. Same for Apple, Amazon, Intel, etc. Eric Schmidt runs HRCs campaign. Al Gore is on Apple's board. They are all set up to spy on us. Deep state. They hunt sys admin…
You are a conspiracy theorist.
> Google can do the sort of spying the US government can't constitutionally do, then hand that over to the government, constitutionally
This is an agenda-driven redefinition of the word "spy" that I find disingenuous in the extreme.
> It's all stagecraft. Same for Apple, Amazon, Intel, etc. Eric Schmidt runs HRCs campaign. Al Gore is on Apple's board.
Firstly: Eric Schmidt founded a company designed to legally channel lots of money via analytics expertise into his favorite candidate, like every rich person does under the current set of laws. Neither of us has to like it, but he did not "run her campaign" and if they actually did? Wow, not a great job there.
As for Al Gore?
I've worked with Al Gore. He did some advisory work for my financial data startup, as part of our first round's venture firm. Our data was some of the most valuable data about consumers that can possibly exist, and had incredible applications for both surveillance and law enforcement.
To the best of my knowledge, we were never pressured to hand over a byte to anyone. Quite the opposite. I specifically remember a conversation where he mentioned user data privacy was the single most important priority he felt we could have.
So between my lying eyes, ears, and email history and your wild gesticulations about the evil overlordship, I'm gonna have to lean towards my own personal experience.
Unless, of course, you got some actual evidence and not a room full of old coffee cups and red string up on the walls.
Just to clarify, is this benchmark leveraging mixed-precision mode on the Volta V100? The major innovation of the Volta generation is mixed-precision which NVIDIA claims is a huge performance increase over the Pascal generation (P100 in the case of your benchmark). Link to NVIDIA documentation on mixed-precision TensorCores: https://devblogs.nvidia.com/inside-volta/
Where specified "fp16", the V100 benchmarks use the code from https://github.com/tensorflow/benchmarks/tree/master/scripts... with the flag --use_fp16=true which enables fp16 for some but not all Tensors.
Disclosure: I work on Google Cloud. While not perfect, I want to commend the RiseML folks for doing not only an “just out of the box” run in both regular and fp16 mode (for V100), but also adding their own LSTM experiment to the mix. We need third-party benchmarks whenever new hardware or software are being sold by vendors (reminder: I benefit from you buying Google Cloud!). I hope the authors are able to collect som…
So this is a chip that no one outside of Google is going to be able to get a physical copy of ever? It makes any benchmarks become Google-cloud benchmarks, right? Edit: I am complaining a bit about the lack of availability but there's also a real point here. If there's no source for TPUs outside of Google, Google Cloud competes only with other cloud providers and with owning physical GPUs - long term, it has no incen…
For a while, it's very likely that Google will be the main user of these, so there's still plenty of incentive for it to increase efficiency and reduce costs.
Earlier quoted context omitted.
Google is the same as NSA, but exists as a dance around 4th amendment. Google can do the sort of spying the US government can't constitutionally do, then hand that over to the government, constitutionally, under gag order if necessary. It's all stagecraft. Same for Apple, Amazon, Intel, etc. Eric Schmidt runs HRCs campaign. Al Gore is on Apple's board. They are all set up to spy on us. Deep state. They hunt sys admin…
> Google is the same as NSA, but exists as a dance around 4th amendment You are a conspiracy theorist. > Google can do the sort of spying the US government can't constitutionally do, then hand that over to the government, constitutionally This is an agenda-driven redefinition of the word "spy" that I find disingenuous in the extreme. > It's all stagecraft. Same for Apple, Amazon, Intel, etc. Eric Schmidt runs HRCs ca…
Earlier quoted context omitted.
Where specified "fp16", the V100 benchmarks use the code from https://github.com/tensorflow/benchmarks/tree/master/scripts... with the flag --use_fp16=true which enables fp16 for some but not all Tensors.
It's my understanding that fp16 (available on the previous generation P100) and mixed-precision (major innovation of V100) are different things and the speedup of TensorCores is entirely missing from this benchmark. Unlike the general purpose P100, the TPU is a heavily optimized chip built for Deep Learning, hence it's performance increase. However, the V100 is also heavily optimized for Deep Learning (arguably the f…