Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

71–80 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#71

Post by HarHarVeryFunny »

zenoprax wrote: post #36

I have read in the past that ASICs for LLMs are not as simple a solution compared to cryptocurrency. In order to design and build the ASIC you need to commit to a specific architecture: a hashing algorithm for a cryptocurrency is fixed but the LLMs are always changing. Am I misunderstanding "TPU" in the context of the article?

Regardless of architecture (which is anyways basically the same for all LLMs), the computational needs of modern neural networks are pretty generic, centered around things like matrix multiply, which is what the TPU provides. There is even TPU support for some operations built into PyTorch - it is not just a proprietary interface that Google use themselves.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#72

Post by NaomiLehman »

1980phipsi wrote: post #60

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prim…

inference is often a static, bounded problem solvable by generic compilers. training requires the mature ecosystem and numerical stability of cuda to handle mixed-precision operations. unless you rewrite the software from the ground up like Google but for most companies it's cheaper and faster to buy NVIDIA hardware

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#73

Post by baby_souffle »

1980phipsi wrote: post #60

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prim…

That quote left me with the same question. Something about decent amount of ram on one board perhaps? That’s advantageous for training but less so for inference?

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#74

Post by mupuff1234 »

villgax wrote: post #16

https://killedbygoogle.com

That's actually one of the reasons why Google might win.

Nvidia is tied down to support previous and existing customers while Google can still easily shift things around without needing to worry too much about external dependencies.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#75

Post by mrbungie »

lvl155 wrote: post #52
OccamsMirror wrote: post #47

Earlier quoted context omitted.

To be fair, they weren't increasing Ads revenue.

They literally gave away their secret sauce to OpenAI and pretended like it wasn’t a big opportunity.

Just as expected from a big firm with slower organizational speed. They can afford to make those mistakes.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#76

Post by Workaccount2 »

riku_iki wrote: post #51
bgwalter wrote: post #40

Earlier quoted context omitted.

Google Hangouts wasn't small. Google+ was big and supposedly "the future" and is the canonical example of a huge misallocation of resources. Google will have no problem discontinuing Google "AI" if they finally notice that people want a computer to shut up rather than talk at them.

> Google+ was big how you define big? My understanding they failed to compete with facebook, and decided to redirect resources somewhere else.

Google completely fumbled Google+ by doing a slow invite only launch.

The hype when it was first coming to market was intense. But then nobody could get access because they heavily restricted sign ups.

By the time it was in "open beta" (IIRC like 6-7 mos later), the hype had long died and nobody cared about it anymore.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#77

Post by HarHarVeryFunny »

thesz wrote: post #58

5 days ago: https://news.ycombinator.com/item?id=45926371 Sparse models have same quality of results but have less coefficients to process, in case described in the link above sixteen (16) times as less. This means that these models need 8 times less data to store, can be 16 and more times faster and use 16+ times less energy. TPUs are not all that good in the case of sparse matrices. They can be used to train dense…

TPUs do include dedicated hardware, SparseCores, for sparse operations.

https://docs.cloud.google.com/tpu/docs/system-architecture-t...

https://openxla.org/xla/sparsecore

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#78

Post by llm_nerd »

1980phipsi wrote: post #60

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prim…

It's just more common as a legacy artifact from when nvidia was basically the only option available. Many shops are designing models and functions, and then training and iterating on nvidia hardware, but once you have a trained model it's largely fungible. See how Anthropic moved their models from nvidia hardware to Inferentia to XLA on Google TPUs.

Further it's worth noting that the Ironwood, Google's v7 TPU, supports only up to BF16 (a 16-bit floating point that has the range of FP32 minus the precision. Many training processes rely upon larger types, quantizing later, so this breaks a lot of assumptions. Yet Google surprised and actually training Gemini 3 with just that type, so I think a lot of people are reconsidering assumptions.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#79

Post by the_af »

gostsamo wrote: post #64
qwertox wrote: post #53

Earlier quoted context omitted.

What if China becomes self-sufficient enough to no longer rely on Taiwanese Fabs, and hence having no issues with those Fabs getting destroyed. That would put China as the leader once and for all.

First, the US has advanced fab capabilities and in case of a need can develop them further. On the other side, China will suffer a Russia style blockback while caught up in a nasty war with Taiwan. Totally possible, but the second order effects are much more complex than "leader once for all". The path for victory for China is not war despite the west, but a war when the west would not care.

The best path for victory for China is probably no war at all. War is wasteful and risky.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#80

Post by jimbohn »

subroutine wrote: post #59
jimbohn wrote: post #42

Given the importance of scale for this particular product, any company placing itself on "just" one layer of the whole story is at a heavy disadvantage, I guess. I'd rather have a winning google than openai or meta anyway.

> I'd rather have a winning google than openai or meta anyway. Why? To me, it seems better for the market, if the best models and the best hardware were not controlled by the same company.

I agree, it would be the best of bad cases, in a sense. I have low trust in OpenAI due to its leadership, and in Meta, because, well, Meta has history, let's say.
Post reply on HN