Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

201–210 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#201

Post by torginus »

lukeschlather wrote: post #130

This feels a lot like the RISC/CISC debate. More academic than it seems. Nvidia is designing their GPUs primarily to do exactly the same tasks TPUs are doing right now. Even within Google it's probably hard to tell whether or not it matters on a 5-year timeframe. It certainly gives Google an edge on some things, but in the fullness of time "GPUs" like the H100 are primarily used for running tensor models and they're…

I feel like this is more like the console/PC debate in the 90s. Consoles like the SNES had dedicated fixed function graphics hardware with weaker general specs, but with the special HW they could perform as well as a much more expensive PC - but as devs made more and more varied and clever games, that fixed function hardware couldn't support it and the PC became the superior choice.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#202

Post by throwawayffffas »

pzo wrote: post #141
timmg wrote: post #20

Earlier quoted context omitted.

They will, I'm sure. The big difference is that Google is both the chip designer *and* the AI company. So they get both sets of profits. Both Google and Nvidia contract TSMC for chips. Then Nvidia sells them at a huge profit. Then OpenAI (for example) buys them at that inflated rate and them puts them into production. So while Nvidia is "selling shovels", Google is making their own shovels and has their own mines.

on top of that Google is also cloud infrastructure provider - contrary to OpenAI that need to have someone like Azure plug those GPUs and host servers.

I am pretty sure OpenAI has data centers of its own.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#203

Post by torginus »

qwertox wrote: post #17

How high are the chances that as soon as China produces their own competitive TPU/GPU, they'll invade Taiwan in order to starve the West in regards to processing power, while at the same time getting an exclusive grip on the Taiwanese Fabs?

Imo having the best logic process nodes is not necessary to win at AI - having the most memory bandwidth is - and China has SOTA HBMs.

I'd guess most of their handicap comes from their hardware and software not being as refined as the US's

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#204

Post by saagarjha »

oivey wrote: post #189
pests wrote: post #166

Earlier quoted context omitted.

You say that like I d a bad thing. Nvidia architectures keep changing and getting more advanced as well, with specialized tensor operations, different accumulators and caches, etc. I see no issue with progress.

That’s missing the point. Things like tensor cores were added in parallel with improvements to existing computer and CUDA kernels from 10 years ago generally run without modification. Hardware architecture may change, but Nvidia has largely avoided changing how you interact with it.

Modern CUDA programs that hit roofline look absolutely nothing like those from 10 or even 5 years ago. Or even 2 if you’re on Blackwell.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#205

Post by saagarjha »

wood_spirit wrote: post #123
Workaccount2 wrote: post #68

Earlier quoted context omitted.

Deepmind gets to work directly with the TPU team to make custom modifications and designs specifically for deepmind projects. They get to make pickaxes that are made exactly for the mine they are working. Everyone using Nvidia hardware has a lot of overlap in requirements, but they also all have enough architectural differences that they won't be able to match Google. OpenAI announced they will be designing their own…

Isn’t there a suspicion that OpenAI buying custom chips from another Sam Altman venture is just graft? Wasn’t that one of the things that came up when the board tried to out him?

The chips are being done in-house.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#206

Post by saagarjha »

bjourne wrote: post #45
fooker wrote: post #10

Earlier quoted context omitted.

That's exactly what Nvidia is doing with tensor cores.

Except the native width of Tensor Cores are about 8-32 (depending on scalar type), whereas the width of TPUs is up to 256. The difference in scale is massive.

Nobody cares about width; they care about TFLOPs.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#209

Post by coolsunglasses »

sharpy wrote: post #133
morkalork wrote: post #118

Earlier quoted context omitted.

Anti-moat like commoditizing the compliment?

If they get things like PyTorch to work well without carinng what hardware it is running on, it erodes Nvidia's CUDA moat. Nvidia's chips are excellent, without doubt, but their real moat is the ecosystem around CUDA.

I'd love for someone to give me an alternative to CUDA but I don't primarily use GPUs for inference, I do 64-bit unsigned integer workloads and the only people who seem to care even a little about this currently are NVidia, if imperfectly.

I _really_ want an alternative but the architecture churn imposed by targeting ROCm for say an MI350X is brutal. The way their wavefronts and everything work is significantly different enough that if you're trying to get last-mile perf (which for GPUs unfortunately yawns back into the 2-5x stretch) you're eating a lot of pain to get the same cost-efficiency out of AMD hardware.

FPGAs aren't really any more cost effective unless the $/kwh goes into the stratosphere which is a hypothetical I don't care to contemplate.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#210

Post by onlyrealcuzzo »

siliconc0w wrote: post #15

Google has always had great tech - their problem is the product or the perseverance, conviction, and taste needed to make things people want.

Odd way to describe the most used product in the history of the world.
Post reply on HN