Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

Post reply on HN
Print view View original

51–60 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#51

Post by riku_iki »

bgwalter wrote: post #40
riku_iki wrote: post #19

Earlier quoted context omitted.

It's all small products which didn't receive traction.

Google Hangouts wasn't small. Google+ was big and supposedly "the future" and is the canonical example of a huge misallocation of resources. Google will have no problem discontinuing Google "AI" if they finally notice that people want a computer to shut up rather than talk at them.

> Google+ was big

how you define big? My understanding they failed to compete with facebook, and decided to redirect resources somewhere else.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#52

Post by lvl155 »

OccamsMirror wrote: post #47
lvl155 wrote: post #43

Earlier quoted context omitted.

And they forgot to pay those people so most of them left.

To be fair, they weren't increasing Ads revenue.

They literally gave away their secret sauce to OpenAI and pretended like it wasn’t a big opportunity.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#53

Post by qwertox »

gostsamo wrote: post #21
qwertox wrote: post #17

How high are the chances that as soon as China produces their own competitive TPU/GPU, they'll invade Taiwan in order to starve the West in regards to processing power, while at the same time getting an exclusive grip on the Taiwanese Fabs?

Not very. Those fabs are vulnerable things, shame if something happens to them. If China attacks, it would be for various other reasons and processors are only one of many considerations, no matter how improbable it might sound to an HN-er.

What if China becomes self-sufficient enough to no longer rely on Taiwanese Fabs, and hence having no issues with those Fabs getting destroyed. That would put China as the leader once and for all.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#54

Post by p-e-w »

zenoprax wrote: post #36

I have read in the past that ASICs for LLMs are not as simple a solution compared to cryptocurrency. In order to design and build the ASIC you need to commit to a specific architecture: a hashing algorithm for a cryptocurrency is fixed but the LLMs are always changing. Am I misunderstanding "TPU" in the context of the article?

It’s true that architectures change, but they are built from common components. The most important of those is matrix multiplication, using a relatively small set of floating point data types. A device that accelerates those operations is, effectively, an ASIC for LLMs.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#55

Post by villgax »

riku_iki wrote: post #19
villgax wrote: post #16

https://killedbygoogle.com

It's all small products which didn't receive traction.

Wait until Apple's ChromeBook competitor shows up to eat their lunch just like switching to another proprietary stack with no dev ecosystem will die out. Sure they'll go after big ticket accounts, also take a guess at what else gets sanctioned next.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#56

Post by bfrog »

p-e-w wrote: post #54
zenoprax wrote: post #36

I have read in the past that ASICs for LLMs are not as simple a solution compared to cryptocurrency. In order to design and build the ASIC you need to commit to a specific architecture: a hashing algorithm for a cryptocurrency is fixed but the LLMs are always changing. Am I misunderstanding "TPU" in the context of the article?

It’s true that architectures change, but they are built from common components. The most important of those is matrix multiplication, using a relatively small set of floating point data types. A device that accelerates those operations is, effectively, an ASIC for LLMs.

We used to call these things DSPs

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#57

Post by lvl155 »

LogicFailsMe wrote: post #22
svantana wrote: post #13

Earlier quoted context omitted.

That's mentioned in the article, but is the lock-in really that big? In some cases, it's as easy as changing the backend of your high-level ML library.

That's what it is on paper. But in practice you trade one set of hardware idiosyncrasies for another and unless you have the right people to deal with that, it's a hassle.

On top, when you get locked into Google Cloud, you’re effectively at the mercy of their engineers to optimize and troubleshoot. Do you think Google will help their potential competitors before they help themselves? Highly unlikely considering their actions in the past decade plus.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#58

Post by thesz »

5 days ago: https://news.ycombinator.com/item?id=45926371

Sparse models have same quality of results but have less coefficients to process, in case described in the link above sixteen (16) times as less.

This means that these models need 8 times less data to store, can be 16 and more times faster and use 16+ times less energy.

TPUs are not all that good in the case of sparse matrices. They can be used to train dense versions, but inference efficiency with sparse matrices may be not all that great.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#59

Post by subroutine »

jimbohn wrote: post #42

Given the importance of scale for this particular product, any company placing itself on "just" one layer of the whole story is at a heavy disadvantage, I guess. I'd rather have a winning google than openai or meta anyway.

> I'd rather have a winning google than openai or meta anyway.

Why? To me, it seems better for the market, if the best models and the best hardware were not controlled by the same company.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#60

Post by 1980phipsi »

> It is also important to note that, until recently, the GenAI industry’s focus has largely been on training workloads. In training workloads, CUDA is very important, but when it comes to inference, even reasoning inference, CUDA is not that important, so the chances of expanding the TPU footprint in inference are much higher than those in training (although TPUs do really well in training as well – Gemini 3 the prime example).

Does anyone have a sense of why CUDA is more important for training than inference?

Post reply on HN