Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

301–310 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#301

Post by jauntywundrkind »

morkalork wrote: post #118
jauntywundrkind wrote: post #105

Earlier quoted context omitted.

Google's work on Jax, pytorch, tensorflow, and the more general XLA underneath are exactly the kind of anti-moat everyone has been clamoring for.

Anti-moat like commoditizing the compliment?

Yes!

Pytorch, Jax, tensorflow are all examples to me of very capable products, that compete very well in ML space.

But more broadly work like XLA and IREE are very interesting toolkits for mapping a huge variety of computation onto many types of hardware. While Pytorch et al are fine example applications, are things you can do, XLA is the Big Tent idea, the toolkit to erode not just specific CUDA use cases, but to allow hardware in general to be more broadly useful.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#302

Post by karmakaze »

Voultapher wrote: post #264
benreesman wrote: post #235

Earlier quoted context omitted.

NVFP4 is the thing no one saw coming. I wasn't watching the MX process really, so I cast no judgements, but it's exactly what it sounds like, a serious compromise in resource constrained settings. And it's in the silicon pipeline. NVFP4 is to put it mildly a masterpiece, the UTF-8 of its domain and in strikingly similar ways it is 1. general 2. robust to gross misuse 3. not optional if success and cost both matter. I…

This reads like a badly done, sponsored hype video on YouTube. So if we look at what NVIDIA has to say about NVFP4 it sure sounds impressive [1]. But look closely that initial graph never compares fp8 and fp4 on the same hardware. They jump from H100 to B200 while implying a 5x jump of going with fp4 which it isn't. Accompanied with scary words like if you use MXFP4 "Risk of noticeable accuracy drop compared to FP8"…

Looking into NVFP4/Nvidia vs MXFP4/AMD the summation was that seem to be pretty close when including the MI355X which leads in VRAM and throughput but trails in accuracy (slightly)--and for that mixing in MXFP6 makes up for it.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#303

Post by epistasis »

bloppe wrote: post #296
epistasis wrote: post #217

Earlier quoted context omitted.

The LLM provider I trust the most right now is AWS. Anybody else seems to have very conflicted purposes when it comes to sending them my data and interactions.

Amazon makes an LLM?

They make Amazon Q, but they don't make a general purpose LLM, which is precisely why I fid them more trustworthy. They run others' LLMs [1] , but don't have an interest in using customer data to improve the next generation of their LLM, or move into adjacent businesses to commercialize LLMs. At least not yet.

[1] https://aws.amazon.com/bedrock/model-choice/

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#304

Post by 01100011 »

Workaccount2 wrote: post #293
01100011 wrote: post #262

Earlier quoted context omitted.

> Deepmind gets to work directly with the TPU team to make custom modifications You don't think Nvidia has field-service engineers and applications engineers with their big customers? Come on man. There is quite a bit of dialogue between the big players and the chipmaker.

They do, but they need to appease a dozen different teams from a dozen different labs, forcing nvidia to take general approaches and/or dictating approaches and pigeonholing labs into using those methods. Deepmind can do whatever they want, and get the exact hardware to match it. It's a massive advantage when you can discover a bespoke way of running a filter, and you can get a hardware implementation of it without h…

This ignores the way it often works: Customer comes to NVDA with a problem and NVDA comes up with a solution. This solution now adds value for every customer.

In your example, if OpenAI makes a massive new find they aren't taking it to NVDA.

Nvidia has the advantage of a broad base of customers that gives it a lot of information on what needs work and it tries to quickly respond to those deficiencies.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#305

Post by CamperBob2 »

coolsunglasses wrote: post #209
sharpy wrote: post #133

Earlier quoted context omitted.

If they get things like PyTorch to work well without carinng what hardware it is running on, it erodes Nvidia's CUDA moat. Nvidia's chips are excellent, without doubt, but their real moat is the ecosystem around CUDA.

I'd love for someone to give me an alternative to CUDA but I don't primarily use GPUs for inference, I do 64-bit unsigned integer workloads and the only people who seem to care even a little about this currently are NVidia, if imperfectly. I _really_ want an alternative but the architecture churn imposed by targeting ROCm for say an MI350X is brutal. The way their wavefronts and everything work is significantly diffe…

That's new to me -- what sorts of workloads are centered on 64-bit uints?

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#306

Post by tarsinge »

zmmmmm wrote: post #215

I always enjoy being wrong and I was very wrong in my predictions about Google : I thought they should theoretically win, but I was also very confident they couldn't possibly turn their execution ship around to actually pull together a coherent competitor to OpenAI. But they do seem to have done that and it's very impressive. If they do continue to execute, I can't see anybody stopping them dominating and I would be…

I don’t know if Google will win but the message they sent is that Nvidia will not: if Google can release the best model without using their own chips instead of GPUs, and save money in the process, then others (Microsoft, Amazon, etc.) will do it too sooner or later.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#307

Post by blinding-streak »

thelastgallon wrote: post #70

With its AI offerings, can Google suck the oxygen out of AWS? AWS grew big because of compute. The AI spend will be far larger than compute. Can Google launch AI/Cloud offerings with free compute bundled? Use our AI, and we'll throw in compute for free.

Interesting thought. If that strategy worked too well, I could see the government going after them for monopolistic "bundling"

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#308

Post by musicale »

torginus wrote: post #201
lukeschlather wrote: post #130

This feels a lot like the RISC/CISC debate. More academic than it seems. Nvidia is designing their GPUs primarily to do exactly the same tasks TPUs are doing right now. Even within Google it's probably hard to tell whether or not it matters on a 5-year timeframe. It certainly gives Google an edge on some things, but in the fullness of time "GPUs" like the H100 are primarily used for running tensor models and they're…

I feel like this is more like the console/PC debate in the 90s. Consoles like the SNES had dedicated fixed function graphics hardware with weaker general specs, but with the special HW they could perform as well as a much more expensive PC - but as devs made more and more varied and clever games, that fixed function hardware couldn't support it and the PC became the superior choice.

I guess that's why Nintendo stopped making game consoles and Sony's PlayStation never went anywhere.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#309

Post by charleshn »

qcnguy wrote: post #163
mrbungie wrote: post #88

Earlier quoted context omitted.

It's fun when then you read last Nvidia tweet [1] suggesting that still their tech is better, based on pure vibes as anything in the (Gen)AI-era. [1] https://x.com/nvidianewsroom/status/1993364210948936055

Not vibes. TPUs have fallen behind or had to be redesigned from scratch many times as neural architectures and workloads evolved, whereas the more general purpose GPUs kept on trucking and building on their prior investments. There's a good reason so much research is done on Nvidia clusters and not TPU clusters. TPU has often turned out to be over-specialized and Nvidia are pointing that out.

> There's a good reason so much research is done on Nvidia clusters and not TPU clusters.

You are aware that Gemini was trained on TPU, and that most research at Deepmind is done on TPU?

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#310

Post by Workaccount2 »

01100011 wrote: post #304
Workaccount2 wrote: post #293

Earlier quoted context omitted.

They do, but they need to appease a dozen different teams from a dozen different labs, forcing nvidia to take general approaches and/or dictating approaches and pigeonholing labs into using those methods. Deepmind can do whatever they want, and get the exact hardware to match it. It's a massive advantage when you can discover a bespoke way of running a filter, and you can get a hardware implementation of it without h…

This ignores the way it often works: Customer comes to NVDA with a problem and NVDA comes up with a solution. This solution now adds value for every customer. In your example, if OpenAI makes a massive new find they aren't taking it to NVDA. Nvidia has the advantage of a broad base of customers that gives it a lot of information on what needs work and it tries to quickly respond to those deficiencies.

>In your example, if OpenAI makes a massive new find they aren't taking it to NVDA.

Right, and therefore they are stuck doing it in software, while google can do it in hardware.

Post reply on HN