Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

261–270 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#261

Post by 01100011 »

eikenberry wrote: post #185
augment_me wrote: post #151

Earlier quoted context omitted.

NVIDIA chips are more versatile. During training, you might need to schedule things to the SFU(Special Function unit that does sin, cos, 1/sqrt(x), etc), you might need to run epilogues, save intermediary computations, save gradients, etc. When you train, you might need to collect data from various GPUs, so you need to support interconnects, remote SMEM writing, etc. Once you have trained, you have frozen weights/fee…

Won't the need to train increase as the need for specialized, smaller models increases and we need to train their many variations? Also what about models that continuously learn/(re)train? Seems to me the need for training will only go up in the future.

That's the thing - nobody knows. LLM architecture is constantly evolving and people are trying all kinds of things.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#262

Post by 01100011 »

Workaccount2 wrote: post #68
sbarre wrote: post #2

A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.

Deepmind gets to work directly with the TPU team to make custom modifications and designs specifically for deepmind projects. They get to make pickaxes that are made exactly for the mine they are working. Everyone using Nvidia hardware has a lot of overlap in requirements, but they also all have enough architectural differences that they won't be able to match Google. OpenAI announced they will be designing their own…

> Deepmind gets to work directly with the TPU team to make custom modifications

You don't think Nvidia has field-service engineers and applications engineers with their big customers? Come on man. There is quite a bit of dialogue between the big players and the chipmaker.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#263

Post by pu_pe »

bastawhiz wrote: post #186

I don't think what the article writes about matters all that much. Gemini 3 Pro is arguably not even the best model anymore, and it's _weeks_ old, and Google has far more resources than Anthropic does. If the hardware actually was the secret sauce, Google would be wiping the floor with little everyone else. But they're not. There's a few confounding problems: 1. Actually using that hardware effectively isn't easy. It…

I think you are addressing the issue from a developer's perspective. I don't think TPUs are going to be sold to individual users anytime soon. What the article is pointing out is that Google is now able to squeeze significantly more performance per dollar than their peer competitors in the LLM space.

For example, OpenAI has announced trillion-dollar investments in data centers to continue scaling. They need to go through a middle-man (Nvidia), while Google does not, and will be able to use their investment much more efficiently to train and serve their own future models.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#264

Post by Voultapher »

benreesman wrote: post #235
m4r1k wrote: post #41

Google's real moat isn't the TPU silicon itself—it's not about cooling, individual performance, or hyper-specialization—but rather the massive parallel scale enabled by their OCS interconnects. To quote The Next Platform: "An Ironwood cluster linked with Google’s absolutely unique optical circuit switch interconnect can bring to bear 9,216 Ironwood TPUs with a combined 1.77 PB of HBM memory... This makes a rackscale…

NVFP4 is the thing no one saw coming. I wasn't watching the MX process really, so I cast no judgements, but it's exactly what it sounds like, a serious compromise in resource constrained settings. And it's in the silicon pipeline. NVFP4 is to put it mildly a masterpiece, the UTF-8 of its domain and in strikingly similar ways it is 1. general 2. robust to gross misuse 3. not optional if success and cost both matter. I…

This reads like a badly done, sponsored hype video on YouTube.

So if we look at what NVIDIA has to say about NVFP4 it sure sounds impressive [1]. But look closely that initial graph never compares fp8 and fp4 on the same hardware. They jump from H100 to B200 while implying a 5x jump of going with fp4 which it isn't. Accompanied with scary words like if you use MXFP4 "Risk of noticeable accuracy drop compared to FP8" .

Contrast that with what AMD has to say on the open MXFP4 approach which is quite similar to NVFP4 [2]. Ohh the horrors of getting 79.6 instead of 79.9 on GPQA Diamond when using MXFP4 instead of FP8.

[1] https://developer.nvidia.com/blog/introducing-nvfp4-for-effi...

[2] https://rocm.blogs.amd.com/software-tools-optimization/mxfp4...

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#265

Post by wangii »

It's all about the right mind set at the very top level. At the beginning of the PC era, nobody would bet IBM to lose. Same in the dawn of internet, all money was on MS. so it happened to Nokia and Ericsson.

Google is a giant without a direction. The ads money is so good that it just doesn't have the gut to leave it on the table.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#266

Post by ZiiS »

oivey wrote: post #252
ZiiS wrote: post #247

Earlier quoted context omitted.

They are suggesting it is easier for others to buy buy more NVidia chips and feed them more power. Whilst operating costs are covered by investors. If they move on to competing on having to do inference the cheepest then the TPUs will shine.

The original post made no comments about inference or training or even cost in any way. It said you could hook up more TPUs together with more memory and higher average bandwidth than you could with a datacenter of Nvidia GPUs. From an architectural point of view, it isn’t clear (and is not explained) what that enables. It clearly hasn’t led to a business outcome for Google where they are the clear market leader. See…

The fact that NVidia are currently winning is undisputed.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#267

Post by a96 »

DonHopkins wrote: post #221
nish__ wrote: post #214

Earlier quoted context omitted.

Why don't your customers trust Google Cloud?

It's not Google Cloud per se, it's any cloud. There are a million reasons not to trust (or spend money on) any cloud. They want all their video and data on premises and completely under their control.

Well, there's also an ever increasing list of reasons not to trust anything from Google. They're even pretty similar reasons.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#268

Post by a96 »

DonHopkins wrote: post #222
riku_iki wrote: post #51

Earlier quoted context omitted.

> Google+ was big how you define big? My understanding they failed to compete with facebook, and decided to redirect resources somewhere else.

Orkut was HUGE in Brazil.

In Finnish, it pretty much means orgasms, so it wasn't. Though that's probably not the sole reason, of course.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#269

Post by qcnguy »

kllrnohj wrote: post #228
oivey wrote: post #189

Earlier quoted context omitted.

That’s missing the point. Things like tensor cores were added in parallel with improvements to existing computer and CUDA kernels from 10 years ago generally run without modification. Hardware architecture may change, but Nvidia has largely avoided changing how you interact with it.

And yet current versions of Whisper GPU will not run on my not-quite-10-year old Pascal GPU anymore because the hardware CUDA version is too old. Just because it's still called CUDA doesn't mean it's portable over a not-that-long of a timeframe.

Portable doesn't normally mean that it runs on arbitrarily old hardware. CUDA was never portable, it only runs on Nvidia hardware. The question is whether old versions of Whisper GPU run on newer hardware, that'd be backwards compatibility.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#270

Post by qcnguy »

saagarjha wrote: post #204
oivey wrote: post #189

Earlier quoted context omitted.

That’s missing the point. Things like tensor cores were added in parallel with improvements to existing computer and CUDA kernels from 10 years ago generally run without modification. Hardware architecture may change, but Nvidia has largely avoided changing how you interact with it.

Modern CUDA programs that hit roofline look absolutely nothing like those from 10 or even 5 years ago. Or even 2 if you’re on Blackwell.

But for research you often don't have to max out the hardware right away.

And the question is what do programs that max out Ironwood look like vs TPU programs written 5 years ago?

Post reply on HN