Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

151–160 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#151

Post by augment_me »

1980phipsi wrote: post #60

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prim…

NVIDIA chips are more versatile. During training, you might need to schedule things to the SFU(Special Function unit that does sin, cos, 1/sqrt(x), etc), you might need to run epilogues, save intermediary computations, save gradients, etc. When you train, you might need to collect data from various GPUs, so you need to support interconnects, remote SMEM writing, etc.

Once you have trained, you have frozen weights/feed-forward networks that consist out of frozen weights that you can just program in and run data over. These weights can be duplicated across any amount of devices and just sit there and run inference with new data.

If this turns out to be the future use-case for NNs(it is today), then Google are better set.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#152

Post by DonHopkins »

Will Google sell TPUs that can be plugged into stock hardware, or custom hardware with lots of TPUs? Our customers want all their video processing to happen on site, and don't want their video or other data to touch the cloud, so they're not happy about renting cloud TPUs or GPUs. Also it would be nice to have smart cameras with built-in TPUs.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#153

Post by svara »

qeternity wrote: post #142
sharpy wrote: post #133

Earlier quoted context omitted.

If they get things like PyTorch to work well without carinng what hardware it is running on, it erodes Nvidia's CUDA moat. Nvidia's chips are excellent, without doubt, but their real moat is the ecosystem around CUDA.

PyTorch is only part of it. There is still a huge amount of CUDA that isn’t just wrapped by PyTorch and isn’t easily portable.

... but not in deep learning or am I missing something important here?

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#154

Post by pksebben »

iamtheworstdev wrote: post #120
acoustics wrote: post #112

Earlier quoted context omitted.

I think people are confusing the bubble popping with AI being over. When the dot-com bubble popped, it's not like internet infrastructure immediately became useless and worthless.

that's actually not all that true... a lot of fiber that had been laid went dark, or was never lit, and was hoarded by telecoms in an intentional supply constrained market in order to drive up the usage cost of what was lit.

If it was hoarded by anyone, then by definition not useless OR worthless. Also, you are currently on the internet if you're reading this, so the point kinda stands.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#156

Post by butvacuum »

epolanski wrote: post #149
PunchyHamster wrote: post #145

Earlier quoted context omitted.

Yet everyone uses NVIDIA and Google is at catchup position. Ecosystem is MASSIVE factor and will be a massive factor for all but the biggest models

Catch-up in what exactly? Google isn't building hardware to sell, they aren't in the same market. Also I feel you completely misunderstand that the problem isn't how fast is ONE gpu vs ONE tpu, what matters is the costs for the same output. If I can fill a datacenter at half the cost for the same output, does it matters I've used twice the TPUs and that a single Nvidia Blackwell was faster? No... And hardware cost is…

Also, isn't memory a bit moot? At scale I thought that the ASICs frequently sat idle waiting for memory.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#157

Post by nutjob2 »

m4rtink wrote: post #103
timmg wrote: post #20

Earlier quoted context omitted.

They will, I'm sure. The big difference is that Google is both the chip designer *and* the AI company. So they get both sets of profits. Both Google and Nvidia contract TSMC for chips. Then Nvidia sells them at a huge profit. Then OpenAI (for example) buys them at that inflated rate and them puts them into production. So while Nvidia is "selling shovels", Google is making their own shovels and has their own mines.

So when the bubble pops the companies making the shovels (TSMC, NVIDIA) might still have the money they got for their products and some of the ex-AI companies might least be able to sell standard compliant GPUs on the wider market. And Google will end up with lots of useless super specialized custom hardware.

How could Google's custom hardware become useless? They've used it for their business for years now and will do so for years into the future. It's not like their hardware is LLM specific. Google cannot lose with their vast infrastructure.

Meanwhile OpenAI et al dumping GPUs while everyone else is doing the same will get pennies on the dollar. It's exactly the opposite to what you describe.

I hope that comes to pass, because I'll be ready to scoop up cheap GPUs and servers.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#159

Post by bigyabai »

mrbungie wrote: post #88
m4r1k wrote: post #41

Google's real moat isn't the TPU silicon itself—it's not about cooling, individual performance, or hyper-specialization—but rather the massive parallel scale enabled by their OCS interconnects. To quote The Next Platform: "An Ironwood cluster linked with Google’s absolutely unique optical circuit switch interconnect can bring to bear 9,216 Ironwood TPUs with a combined 1.77 PB of HBM memory... This makes a rackscale…

It's fun when then you read last Nvidia tweet [1] suggesting that still their tech is better, based on pure vibes as anything in the (Gen)AI-era. [1] https://x.com/nvidianewsroom/status/1993364210948936055

> based on pure vibes

The tweet gives their justification; CUDA isn't ASIC. Nvidia GPUs were popular for crypto mining, protein folding, and now AI inference too. TPUs are tensor ASICs.

FWIW I'm inclined to agree with Nvidia here. Scaling up a systolic array is impressive but nothing new.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#160

Post by grandmczeb »

augment_me wrote: post #151
1980phipsi wrote: post #60

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prim…

NVIDIA chips are more versatile. During training, you might need to schedule things to the SFU(Special Function unit that does sin, cos, 1/sqrt(x), etc), you might need to run epilogues, save intermediary computations, save gradients, etc. When you train, you might need to collect data from various GPUs, so you need to support interconnects, remote SMEM writing, etc. Once you have trained, you have frozen weights/fee…

All of those are things you can do with TPUs
Post reply on HN