Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

141–150 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#141

Post by pzo »

timmg wrote: post #20
sbarre wrote: post #2

A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.

They will, I'm sure. The big difference is that Google is both the chip designer *and* the AI company. So they get both sets of profits. Both Google and Nvidia contract TSMC for chips. Then Nvidia sells them at a huge profit. Then OpenAI (for example) buys them at that inflated rate and them puts them into production. So while Nvidia is "selling shovels", Google is making their own shovels and has their own mines.

on top of that Google is also cloud infrastructure provider - contrary to OpenAI that need to have someone like Azure plug those GPUs and host servers.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#142

Post by qeternity »

sharpy wrote: post #133
morkalork wrote: post #118

Earlier quoted context omitted.

Anti-moat like commoditizing the compliment?

If they get things like PyTorch to work well without carinng what hardware it is running on, it erodes Nvidia's CUDA moat. Nvidia's chips are excellent, without doubt, but their real moat is the ecosystem around CUDA.

PyTorch is only part of it. There is still a huge amount of CUDA that isn’t just wrapped by PyTorch and isn’t easily portable.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#143

Post by pzo »

IncreasePosts wrote: post #110
villgax wrote: post #66

Earlier quoted context omitted.

Exactly, ChatGPT pretty much ate away ad volume & retention if th already garbage search results weren't enough. Don't even get me started on Android & Android TV as an ecosystem.

That's not the story that GOOGs quarterly earning reports tell(ad revenue up 12% YoY)

most likely because they got more aggressive with campaign against adblock in chrome and more ads in youtube.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#144

Post by ithkuil »

iamtheworstdev wrote: post #120
acoustics wrote: post #112

Earlier quoted context omitted.

I think people are confusing the bubble popping with AI being over. When the dot-com bubble popped, it's not like internet infrastructure immediately became useless and worthless.

that's actually not all that true... a lot of fiber that had been laid went dark, or was never lit, and was hoarded by telecoms in an intentional supply constrained market in order to drive up the usage cost of what was lit.

Are you saying that the internet business didn't grow a lot after the bubble popped?

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#145

Post by PunchyHamster »

m4r1k wrote: post #89
villgax wrote: post #61

Earlier quoted context omitted.

100 times more chips for equivalent memory, sure.

Check the specs again. Per chip, TPU 7x has 192GB of HBM3e, whereas the NVIDIA B200 has 186GB. While the B200 wins on raw FP8 throughput (~9000 vs 4614 TFLOPs), that makes sense given NVIDIA has optimized for the single-chip game for over 20 years. But the bottleneck here isn't the chip—it's the domain size. NVIDIA's top-tier NVL72 tops out at an NVLink domain of 72 Blackwell GPUs. Meanwhile, Google is connecting 921…

Yet everyone uses NVIDIA and Google is at catchup position.

Ecosystem is MASSIVE factor and will be a massive factor for all but the biggest models

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#146

Post by rbanffy »

1980phipsi wrote: post #60

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prim…

This is a very important point - the market for training chips might be a bubble, but the market for inference is much, much larger. At some point we might have good enough models and the need for new frontier models will cool down. The big power-hungry datacenters we are seeing are mostly geared towards training, while inference-only systems are much simpler and power efficient.

A real shame, BTW, all that silicon doesn't do FP32 (very well). After training ceases to be that needed, we could use all that number crunching for climate models and weather prediction.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#147

Post by coredog64 »

sbarre wrote: post #7
blibble wrote: post #5

Earlier quoted context omitted.

the entire organisation has been built over the last 25 years to produce GPUs turning a giant lumbering ship around is not easy

For sure, I did not mean to imply they could do it quickly or easily, but I have to assume that internally at Nvidia there's already work happening to figure out "can we make chips that are better for AI and cheaper/easier to make than GPUs?"

Isn't that a bit like Kodak knowing that digital cameras were a thing but not wanting to jeopardize their film business?

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#148

Post by bryanlarsen »

iamtheworstdev wrote: post #120
acoustics wrote: post #112

Earlier quoted context omitted.

I think people are confusing the bubble popping with AI being over. When the dot-com bubble popped, it's not like internet infrastructure immediately became useless and worthless.

that's actually not all that true... a lot of fiber that had been laid went dark, or was never lit, and was hoarded by telecoms in an intentional supply constrained market in order to drive up the usage cost of what was lit.

And then they sold it to Google who lit it up.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#149

Post by epolanski »

PunchyHamster wrote: post #145
m4r1k wrote: post #89

Earlier quoted context omitted.

Check the specs again. Per chip, TPU 7x has 192GB of HBM3e, whereas the NVIDIA B200 has 186GB. While the B200 wins on raw FP8 throughput (~9000 vs 4614 TFLOPs), that makes sense given NVIDIA has optimized for the single-chip game for over 20 years. But the bottleneck here isn't the chip—it's the domain size. NVIDIA's top-tier NVL72 tops out at an NVLink domain of 72 Blackwell GPUs. Meanwhile, Google is connecting 921…

Yet everyone uses NVIDIA and Google is at catchup position. Ecosystem is MASSIVE factor and will be a massive factor for all but the biggest models

Catch-up in what exactly? Google isn't building hardware to sell, they aren't in the same market.

Also I feel you completely misunderstand that the problem isn't how fast is ONE gpu vs ONE tpu, what matters is the costs for the same output. If I can fill a datacenter at half the cost for the same output, does it matters I've used twice the TPUs and that a single Nvidia Blackwell was faster? No...

And hardware cost isn't even the biggest problem, operational costs, mostly power and cooling are another huge one.

So if you design a solution that fits your stack (designed for it) and optimize for your operational costs you're light years ahead of your competition using the more powerful solution, that costs 5 times more in hardware and twice in operational costs.

All I say is more or less true for inference economics, have no clue about training.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#150

Post by qeternity »

llm_nerd wrote: post #78
1980phipsi wrote: post #60

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prim…

It's just more common as a legacy artifact from when nvidia was basically the only option available. Many shops are designing models and functions, and then training and iterating on nvidia hardware, but once you have a trained model it's largely fungible. See how Anthropic moved their models from nvidia hardware to Inferentia to XLA on Google TPUs. Further it's worth noting that the Ironwood, Google's v7 TPU, suppor…

This is not the case for LLMs. FP16/BF16 training precision is standard, with FP8 inference very common. But labs are moving to FP8 training and even FP4.
Post reply on HN