Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

cnbc.com

Post reply on HN
Print view View original

51–60 of 137 posts

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#51

Post by rjzzleep »

Moto7451 wrote: post #44
rjzzleep wrote: post #42

Earlier quoted context omitted.

The first revisions were stuff made by qualcomm right? I don't think we have much data on how much customizations they make and where they their IP from, but given how much of the Tensor cores comes from Samsung I think it's safe to say to assume that there is a decent amount coming from some of the big vendors.

To be fair that’s a pretty good approach if you look at Apple’s progression from assembled IPs in the first iPhone CPU to the A and M series.

Yeah but at least when it comes to mobile CPUs Apple seemed vastly more competent in how they approached it.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#52

Post by floxy »

On a slightly different tangent, is anyone working on analog machine learning ASICs? Sub-threshold CMOS or something? I mean even at the research level? Using a handful of transistor for an analog multiplier. And get all of the crazy fascinating translinear stuff of Barrie Gilbert fame.

https://www.electronicdesign.com/technologies/analog/article...

https://www.analog.com/en/resources/analog-dialogue/articles...

http://madvlsi.olin.edu/bminch/talks/090402_atact.pdf

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#53

Post by fennecbutt »

fidotron wrote: post #21
ambicapter wrote: post #18

Earlier quoted context omitted.

Microsoft, famously resource-poor.

They have so much money it is harmful to their ability to execute. Just look at the implosion of the XBox business.

Granted, if everyone had done what the highly paid executives had told them to do, xbox would never have existed.

And I'm guessing that the decline is due to executive meddling.

What is it that executives do again? Beyond collecting many millions of dollars a year, that is.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#54

Post by fidotron »

fennecbutt wrote: post #53
fidotron wrote: post #21

Earlier quoted context omitted.

They have so much money it is harmful to their ability to execute. Just look at the implosion of the XBox business.

Granted, if everyone had done what the highly paid executives had told them to do, xbox would never have existed. And I'm guessing that the decline is due to executive meddling. What is it that executives do again? Beyond collecting many millions of dollars a year, that is.

They sit around fantasizing about buying Nintendo because that would be the top thing they could achieve in their careers.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#55

Post by kcb »

philipwhiuk wrote: post #47
alephnerd wrote: post #37

I've mentioned this before on HN [0][1]. The name of the game has been custom SoCs and ASICs for a couple years now, because inference and model training is an "embarrassingly parallel" problem, and models that are optimized for older hardware can provide similar gains to models that are run on unoptimized but more performant hardware. Same reason H100s remain a mainstay in the industry today, as their performance pr…

> The name of the game has been custom SoCs and ASICs for a couple years now, because inference and model training is an "embarrassingly parallel" problem, and models that are optimized for older hardware can provide similar gains to models that are run on unoptimized but more performant hardware. Is anyone else getting crypto flashbacks?

The difference is crypto wasn't memory and throughput dependent. That's why a small asic on a USB stick could outperform a GPU.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#57

Post by imtringued »

Der_Einzige wrote: post #50
harrall wrote: post #8

Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.

I'm 99.999% sure that the claim of "all deep learning for Gemini on their own silicon" is not true. Maybe if you restrict it similarly to the Deepseek paper to "Gemini uses TPU for the final successful training run and for scaled inference" you might be correct, but there's no way that GPUs aren't involved for at minimum comparability and more rapid iteration reasons during the extremely buggy and error prone point o…

Google is the creator of JAX and XLA. Maybe the developer laptops have Nvidia GPUs and they do some testing there, but for Google there is literally no point in bothering with CUDA, pytorch or any other ecosystem strongly focused on Nvidia GPUs.

In my experience JAX is way more flexible than pytorch the moment you want to do things that aren't training ML models. E.g. you want to build an optimizer that uses the derivative of your model with respect to the input.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#58

Post by kenjackson »

alephnerd wrote: post #40
bee_rider wrote: post #14

The most important note is: > The software titan is rather late to the custom silicon party. While Amazon and Google have been building custom CPUs and AI accelerators for years, Microsoft only revealed its Maia AI accelerators in late 2023. They are too late for now, they realistically hardware takes a couple generations to become a serious contender and by the time Microsoft has a chance to learn from their hardwar…

> hardware takes a couple generations to become a serious contender Not really and for the same reason Chinese players like Biren are leapfrogging - much of the workload profile in AI/ML is "embarrassingly parallel", thus reducing the need for individual ASICs to be bleeding edge performant. If you are able to negotiate competitive fabrication and energy supply deals, you can mass produce your way into providing "goo…

And current LLM architectures affinitize differently to HW than DNNs even just a decade ago. If you have the money and technical expertise (both of which I assume MS has access to) then a late start might actually be beneficial.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#59

Post by latchkey »

amelius wrote: post #30
latchkey wrote: post #23

It always falls back on the software. AMD is behind, not because the hardware is bad, but because their software historically has played second fiddle to their hardware. The CUDA moat is real. So, unless they also solve that issue with their own hardware, then it will be like the TPU, which is limited to usage primarily at Google, or within very specific use cases. There are only so many super talented software engin…

> The CUDA moat is real. I don't know. The transformer architecture uses only a limited number of primitives. Once you have ported those to your new architecture, you're good to go. Also, Google has been using TPUs for a long time now, and __they__ never hit a brick wall for a lack of CUDA.

It is beyond porting, it is mentality of developers. Change is expensive and I'm not just talking about $ value.

> Also, Google has been using TPUs for a long time now, and __they__ never hit a brick wall for a lack of CUDA.

That's exactly what I'm saying. __they__ is the keyword.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#60

Post by bongodongobob »

appleaday1 wrote: post #48

The current M$ sure is doing a great job at making people move to alternatives.

People, sure, but that's not their target demographic. It's businesses and they aren't moving away from MS anytime soon.
Post reply on HN