Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

321–328 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#321

Post by Dylan16807 »

oivey wrote: post #252
ZiiS wrote: post #247

Earlier quoted context omitted.

They are suggesting it is easier for others to buy buy more NVidia chips and feed them more power. Whilst operating costs are covered by investors. If they move on to competing on having to do inference the cheepest then the TPUs will shine.

The original post made no comments about inference or training or even cost in any way. It said you could hook up more TPUs together with more memory and higher average bandwidth than you could with a datacenter of Nvidia GPUs. From an architectural point of view, it isn’t clear (and is not explained) what that enables. It clearly hasn’t led to a business outcome for Google where they are the clear market leader. See…

> My point: cool benchmark, what does it matter?

And then people explained why the effects are smoothed over right now but will matter eventually and you rejected them as if they didn't understand your question. They answered it, take the answer.

> It didn’t merely say Google’s TPUs were better. It said that Nvidia can’t compete.

Can't compete at clusters of a certain size. The argument is that anyone on nVidia simply isn't building clusters that big.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#322

Post by pu_pe »

bastawhiz wrote: post #297
pu_pe wrote: post #263

Earlier quoted context omitted.

I think you are addressing the issue from a developer's perspective. I don't think TPUs are going to be sold to individual users anytime soon. What the article is pointing out is that Google is now able to squeeze significantly more performance per dollar than their peer competitors in the LLM space. For example, OpenAI has announced trillion-dollar investments in data centers to continue scaling. They need to go thr…

> Google is now able to squeeze significantly more performance per dollar than their peer competitors in the LLM space Performance per dollar doesn't "win" anything though. Performance (as in speed) hardly cracks the top five concerns that most folks have when choosing a model provider, because fast, good models already exist at price points that are acceptable. That might mean slightly better margins for Google, but…

It's not slightly better margins, we are talking about huge cost reductions on the main expense which is compute. In a context where companies are making trillion dollar investments, it matters a lot.

Also, performance and user choice are definitely impacted by compute. If they ever find a way to replace a job with LLMs, those who can throw more compute at it for a lower price point will win.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#323

Post by saagarjha »

overfeed wrote: post #318
saagarjha wrote: post #315

Earlier quoted context omitted.

Link? I only know of Rain and they raised <<$1 billion IIRC

Note that I said "attempted". https://www.wsj.com/tech/ai/sam-altman-seeks-trillions-of-do...

I think this was to build datacenters

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#324

Post by neilmovva »

bjourne wrote: post #45
fooker wrote: post #10

Earlier quoted context omitted.

That's exactly what Nvidia is doing with tensor cores.

Except the native width of Tensor Cores are about 8-32 (depending on scalar type), whereas the width of TPUs is up to 256. The difference in scale is massive.

I think Hopper's native matmul tile is 64x64, and Blackwell is 128x128.

see this blog for a reference on Blackwell:

https://hazyresearch.stanford.edu/blog/2025-03-15-tk-blackwe...

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#325

Post by foota »

bigyabai wrote: post #159
mrbungie wrote: post #88

Earlier quoted context omitted.

It's fun when then you read last Nvidia tweet [1] suggesting that still their tech is better, based on pure vibes as anything in the (Gen)AI-era. [1] https://x.com/nvidianewsroom/status/1993364210948936055

> based on pure vibes The tweet gives their justification; CUDA isn't ASIC. Nvidia GPUs were popular for crypto mining, protein folding, and now AI inference too. TPUs are tensor ASICs. FWIW I'm inclined to agree with Nvidia here. Scaling up a systolic array is impressive but nothing new.

Sure, but their company's 4.3 trillion valuation isn't based on how good their GPUs are for general purpose computing, it's based on how good they are at AI.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#326

Post by veunes »

breppp wrote: post #276
veunes wrote: post #275

The funniest thing about this story is that NVIDIA has essentially become a TPU company. Look at the Hopper and Blackwell architectures: Tensor Cores are taking up more space, the Transformer Engine has appeared, and NVLink has started to look like a supercomputer interconnect. Jensen Huang isn't stupid. He saw the threat of specialized ASICs and just built the ASIC inside the GPU. Now we have a GPU that is 80% matri…

There's an issue with building a swiss knife chip that supports everything back to the 80s, it works great until it doesn't (Intel)

Timing is everything here. The swiss army knife approach only loses when tasks stop changing. Intel suffered when workloads like web and mobile stabilized

In AI, we're still in the explosion phase. If you build the perfect ASIC for Transformers today, and tomorrow a paper drops with a new architecture, your chip becomes a brick. NVIDIA pays the "legacy tax" and keeps CUDA specifically as insurance against algorithm churn. As long as the industry moves this fast, flexibility beats raw efficiency

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#327

Post by musicale »

pezezin wrote: post #313
musicale wrote: post #308

Earlier quoted context omitted.

I guess that's why Nintendo stopped making game consoles and Sony's PlayStation never went anywhere.

Consoles still exist, but they have evolved to use PC hardware.

Switch is more like smartphone hardware, though with differentiation and innovation in the form factor, hybrid docking design, and joy-con controllers; and it's clearly still a game console, running a custom OS and software and supporting game cartridges.

The influence goes both ways however - both the Steam Deck and Steam Machine are attempts to build a PC that resembles a game console (the former being Switch-like and the latter as something of a "GabeCube.") Steam Deck's Game Mode UI and Steam's Big Picture mode try to provide a console-like experience as well.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#328

Post by coolsunglasses »

CamperBob2 wrote: post #305
coolsunglasses wrote: post #209

Earlier quoted context omitted.

I'd love for someone to give me an alternative to CUDA but I don't primarily use GPUs for inference, I do 64-bit unsigned integer workloads and the only people who seem to care even a little about this currently are NVidia, if imperfectly. I _really_ want an alternative but the architecture churn imposed by targeting ROCm for say an MI350X is brutal. The way their wavefronts and everything work is significantly diffe…

That's new to me -- what sorts of workloads are centered on 64-bit uints?

cryptography
Post reply on HN