The Looming Battle Over AI Chips

Never overestimate the intelligence of the decision makers at big bureaucratic tech companies. Also, it is not in the best interest of any of them to be reliant on NVDA or any other single vendor for any critical workload whatsoever. Doubly not so for NVDA's mostly closed source and haphazardly optimized libraries.

All that said, Bill Daly rocks, and NVDA is a hardened target. But the DL frameworks have enormous performance holes once one stops running Resnet-152 and other popular benchmark graphs in the same way that 3DMark performance is not necessarily representative of actual gaming performance unless NVDA took it upon themselves to make it so.

And since DL is such a dynamic field (just like game engines), I expect this situation to persist for a very, very long time.

> Never overestimate the intelligence of the decision makers at big bureaucratic tech companies.

See Google and anything chat related after 2010

And Intel knew that mobile chips will one day become very popular, too - two decades ago. Much good did that knowledge do to the company.

It's not about them being ignorant about it. It's about them making decisions in spite of that knowledge - decisions that Make Sense™ for the advancement and increased profitability of the incumbent cash cow, but they are often contradictory or have a negative impact on new tech investment.

Here's one main reason why Nvidia will not go "full TPU" with its chips - it wants "scalability". That means it wants an architecture that can be "flexible and server different markets".

The companies that specialize in AI chips will likely beat them in performance because they only care about winning one market at a time (and the AI market is a pretty big one).

Intel's AI strategy is even more of a mess, because it has no clue what it can use to beat Nvdia, so its investments and developer ecosystems are all over the place.

Thank you for this tip. Out of curiosity why does this trick work?

The new Google Speech solution is the perfect example on why Google had to do their own silicon.

Doing speech with 16k samples a second through a NN and keep at a reasonable cost is really, really difficult.

The old way was far more power efficient and if you are going to use this new technique which gets you a far better result and do it at a reasonable cost you have to go all the way down into the silicon.

Here listen to the results.

https://cloudplatform.googleblog.com/2018/03/introducing-Clo...

Now I am curious on the cost difference Google as able to achieve. It is still going to be more then the old way but how close did Google come?

But my favorite new thing with these chips is the Jeff Dean paper.

https://www.arxiv-vanity.com/papers/1712.01208v1/

Can't wait to see the cost difference using Google TPUs and this technique versus traditional approaches.

Plus this approach support multi-core inherently. How would you ever do a tree search with multiple cores?

Ultimately to get the new applications we need Google and others doing the silicon. We are getting to extremes where the entire stack has to be tuned together.

I think Google vision for Lens is going to be a similar situation.