Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

191–200 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#191

Post by ForHackernews »

timmg wrote: post #20
sbarre wrote: post #2

A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.

They will, I'm sure. The big difference is that Google is both the chip designer *and* the AI company. So they get both sets of profits. Both Google and Nvidia contract TSMC for chips. Then Nvidia sells them at a huge profit. Then OpenAI (for example) buys them at that inflated rate and them puts them into production. So while Nvidia is "selling shovels", Google is making their own shovels and has their own mines.

Selling shovels may still turn out to be the right move: Nvidia got rich off the cryptocurrency bubble, now they're getting even richer off the AI bubble.

Having your own mines only pays off if you actually do strike gold. So far AI undercuts Google's profitable search ads, and loses money for OpenAI.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#192

Post by decimalenough »

bastawhiz wrote: post #186

I don't think what the article writes about matters all that much. Gemini 3 Pro is arguably not even the best model anymore, and it's _weeks_ old, and Google has far more resources than Anthropic does. If the hardware actually was the secret sauce, Google would be wiping the floor with little everyone else. But they're not. There's a few confounding problems: 1. Actually using that hardware effectively isn't easy. It…

_Weeks_ old! What a fossil!

Slightly more seriously: what you say makes sense if and only if you're projecting Sam Altman and assuming that a) real legit superhuman AGI is just around the corner, and b) all the spoils will accrue to the first company that finds it, which means you need to be 100% in on building the next model that will finally unlock AGI.

But if this is not the case -- and it's increasingly looking like it's not -- it's going to continue to be a race of competing AIs, and that race will be won by the company that can deliver AI at scale the most cheaply. And the article is arguing that company will be Google.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#193

Post by decimalenough »

lukeschlather wrote: post #130

This feels a lot like the RISC/CISC debate. More academic than it seems. Nvidia is designing their GPUs primarily to do exactly the same tasks TPUs are doing right now. Even within Google it's probably hard to tell whether or not it matters on a 5-year timeframe. It certainly gives Google an edge on some things, but in the fullness of time "GPUs" like the H100 are primarily used for running tensor models and they're…

Google does not sell them, but you can rent them:

https://cloud.google.com/tpu

As you note, they'll set the margins to benefit themselves, but you can still eke out some benefit.

Also, you can buy Edge TPUs, but as the name says these are for edge AI inference and useless for any heavy lifting workloads like training or LLMs.

https://www.amazon.com/Google-Coral-Accelerator-coprocessor-...

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#194

Post by zzzoom »

jauntywundrkind wrote: post #136
sbarre wrote: post #2

A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.

Nvidia doesn't have the software stack to do a TPU. They could make a systolic array TPU and software, perhaps. But it would mean abandoning 18 years of CUDA. The top post right now is talking about TPU's colossal advantage in scaling & throughput. Ironwood is massively bigger & faster than what Nvidia is shooting for, already. And that's a huge advantage. But imo that is a replicateable win. Throw gobs more at netwo…

> They could make a systolic array TPU and software, perhaps. But it would mean abandoning 18 years of CUDA.

Tensor cores are specialized and have CUDA support.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#195

Post by decimalenough »

siliconc0w wrote: post #15

Google has always had great tech - their problem is the product or the perseverance, conviction, and taste needed to make things people want.

This is a bizarre argument to make for AI, since Google started working on TPUs in 2013 (12 years ago) and Sundar started publicly banging on about being an AI-first company in 2016. They missed the first boat on LLMs, but Google has been invested in AI for way longer than any of the competition.

https://aibusiness.com/companies/google-ceo-sundar-pichai-we...

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#196

Post by Scene_Cast2 »

qeternity wrote: post #181
svara wrote: post #153

Earlier quoted context omitted.

... but not in deep learning or am I missing something important here?

Yes, absolutely in deep learning. Custom fused CUDA kernels everywhere.

Yep. MoE, FlashAttention, or sparse retrieval architectures for example.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#197

Post by thisisauserid »

bastawhiz wrote: post #186

I don't think what the article writes about matters all that much. Gemini 3 Pro is arguably not even the best model anymore, and it's _weeks_ old, and Google has far more resources than Anthropic does. If the hardware actually was the secret sauce, Google would be wiping the floor with little everyone else. But they're not. There's a few confounding problems: 1. Actually using that hardware effectively isn't easy. It…

They are using that hardware to wipe the floor with everyone if you look at the price per million tokens.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#198

Post by cyanydeez »

bastawhiz wrote: post #186

I don't think what the article writes about matters all that much. Gemini 3 Pro is arguably not even the best model anymore, and it's _weeks_ old, and Google has far more resources than Anthropic does. If the hardware actually was the secret sauce, Google would be wiping the floor with little everyone else. But they're not. There's a few confounding problems: 1. Actually using that hardware effectively isn't easy. It…

Fairly certain google is aiming for "realtime" model training which would definitely require a new arcjitscture

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#199

Post by cyanydeez »

jeffbee wrote: post #172
Traster wrote: post #132

Earlier quoted context omitted.

Training is taking an enormous problem and trying to break it into lots of pieces and managing the data dependency between those pieces. It's solving 1 really hard problem. Inference is the opposite, it's lots of small independent problems. All of this "we have X many widgets connected to Y many high bandwidth optical telescopes" is all a training problem that they need to solve. Inference is "I have 20 tokens and I…

I can't think of any case where inference doesn't care about latency.

I cant thinl of any reason training isnt going to become real time with a significant cpu budget.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#200

Post by torginus »

sbarre wrote: post #2

A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.

My personal guess would be what drives the cost and size of these chips is the memory bandwidth and the transcievers required to support it. Since transcievers/memory controllers are on the edge of the chip, you get a certain minimum circumference for a given bandwidth, which determines your min surface area.

It might be even 'free' to fill it with more complicated logic (especially one that allows you write clever algorithms that let you save on bandwidth).

Post reply on HN