Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

121–130 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#121

Post by immibis »

zenoprax wrote: post #36

I have read in the past that ASICs for LLMs are not as simple a solution compared to cryptocurrency. In order to design and build the ASIC you need to commit to a specific architecture: a hashing algorithm for a cryptocurrency is fixed but the LLMs are always changing. Am I misunderstanding "TPU" in the context of the article?

Cryptocurrency architectures also change - Bitcoin is just about the lone holdout that never evolves. The hashing algorithm for Monero is designed so that a Monero hashing ASIC is literally just a CPU, and it doesn't even matter what the instruction set is.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#122

Post by almostgotcaught »

mrbungie wrote: post #88
m4r1k wrote: post #41

Google's real moat isn't the TPU silicon itself—it's not about cooling, individual performance, or hyper-specialization—but rather the massive parallel scale enabled by their OCS interconnects. To quote The Next Platform: "An Ironwood cluster linked with Google’s absolutely unique optical circuit switch interconnect can bring to bear 9,216 Ironwood TPUs with a combined 1.77 PB of HBM memory... This makes a rackscale…

It's fun when then you read last Nvidia tweet [1] suggesting that still their tech is better, based on pure vibes as anything in the (Gen)AI-era. [1] https://x.com/nvidianewsroom/status/1993364210948936055

> NVIDIA is a generation ahead of the industry

a generation is 6 months

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#123

Post by wood_spirit »

Workaccount2 wrote: post #68
sbarre wrote: post #2

A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.

Deepmind gets to work directly with the TPU team to make custom modifications and designs specifically for deepmind projects. They get to make pickaxes that are made exactly for the mine they are working. Everyone using Nvidia hardware has a lot of overlap in requirements, but they also all have enough architectural differences that they won't be able to match Google. OpenAI announced they will be designing their own…

Isn’t there a suspicion that OpenAI buying custom chips from another Sam Altman venture is just graft? Wasn’t that one of the things that came up when the board tried to out him?

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#124

Post by kittikitti »

You can't really buy a TPU, you have to buy the entire data center that includes the TPU plus the services and support. In Google Colab, I often don't prefer the TPU either because the documentation for the AI isn't made for it. While this could all change in the long term, I also don't see these changes in Google's long term strategy. There's also the problem with Google's graveyard which isn't mentioned in the long term of the original article. Combined with these factors, I'm still skeptical about Google's lead on AI.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#125

Post by wmf »

almostgotcaught wrote: post #122
mrbungie wrote: post #88

Earlier quoted context omitted.

It's fun when then you read last Nvidia tweet [1] suggesting that still their tech is better, based on pure vibes as anything in the (Gen)AI-era. [1] https://x.com/nvidianewsroom/status/1993364210948936055

> NVIDIA is a generation ahead of the industry a generation is 6 months

For GPUs a generation is 1-2 years.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#126

Post by kcb »

zenoprax wrote: post #36

I have read in the past that ASICs for LLMs are not as simple a solution compared to cryptocurrency. In order to design and build the ASIC you need to commit to a specific architecture: a hashing algorithm for a cryptocurrency is fixed but the LLMs are always changing. Am I misunderstanding "TPU" in the context of the article?

LLMs require memory and interconnect bandwidth so needs a whole package that is capable of feeding data to the compute. Crypto is 100% compute bound. Crypto is a trivially parallelized application that runs the same calculation over N inputs.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#127

Post by LogicFailsMe »

lvl155 wrote: post #57
LogicFailsMe wrote: post #22

Earlier quoted context omitted.

That's what it is on paper. But in practice you trade one set of hardware idiosyncrasies for another and unless you have the right people to deal with that, it's a hassle.

On top, when you get locked into Google Cloud, you’re effectively at the mercy of their engineers to optimize and troubleshoot. Do you think Google will help their potential competitors before they help themselves? Highly unlikely considering their actions in the past decade plus.

Given my Fitbit's inability to play nice with my pixel phone, I have zero faith in Google engineers.

What else would one expect when their core value is hiring generalists over specialists* and their lousy retention record?

*Pay no attention to the specialists they acquihire and pay top dollar... And even they don't stick around.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#128

Post by sidibe »

nostrademons wrote: post #84
thomascgalvin wrote: post #39

Earlier quoted context omitted.

Their incentive structure doesn't lead to longevity. Nobody gets promoted for keeping a product alive, they get promoted for shipping something new. That's why we're on version 37 of whatever their chat client is called now. I think we can be reasonably sure that search, Gmail, and some flavor of AI will live on, but other than that, Google apps are basically end-of-life at launch.

It's telling that basically all of Google's successful projects were either acquisitions or were sponsored directly by the founders (or sometimes, were acquisitions that were directly sponsored by the founders). Those are the only situations where you are immune from the performance review & promotion process.

They've actually had many very successful projects that make the few products and acquisitions you are thinking of work. It's true most of their end products don't work or get abandoned but it stretches their infrastructure in ways that works out well in the long run

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#129

Post by tim333 »

bryanlarsen wrote: post #26
bhouston wrote: post #14

In my 20+ years of following NVIDIA, I have learned to never bet against them long-term. I actually do not know exactly why they continually win, but they do. The main issue they have a 3-4 year gap between wanting a new design pivot and realizing it (silicon has a long "pipeline"), it can seem that they may be missing a new trend or swerve in the demands of the market, it is often simply because there is this delay.

You could have said the same thing about Intel for ~50 years.

Depends on the top management though. I imagine Nvidia will keep doing well while Jensen Huang is running things.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#130

Post by lukeschlather »

This feels a lot like the RISC/CISC debate. More academic than it seems. Nvidia is designing their GPUs primarily to do exactly the same tasks TPUs are doing right now. Even within Google it's probably hard to tell whether or not it matters on a 5-year timeframe. It certainly gives Google an edge on some things, but in the fullness of time "GPUs" like the H100 are primarily used for running tensor models and they're going to have hardware that is ruthlessly optimized for that purpose.

And outside of Google this is a very academic debate. Any efficiency gains over GPUs will primarily turn into profit for Google rather than benefit for me as a developer or user of AI systems. Since Google doesn't sell TPUs, they are extremely well-positioned to ensure no one else can profit from any advantages created by TPUs.

Post reply on HN