Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

cnbc.com

Post reply on HN
Print view View original

71–80 of 137 posts

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#71

Post by latchkey »

amelius wrote: post #63
latchkey wrote: post #59

Earlier quoted context omitted.

It is beyond porting, it is mentality of developers. Change is expensive and I'm not just talking about $ value. > Also, Google has been using TPUs for a long time now, and __they__ never hit a brick wall for a lack of CUDA. That's exactly what I'm saying. __they__ is the keyword.

Not sure what you mean. Google is a big company. Their TPUs have many users internally.

Very few developers outside of Google have ever written code for a TPU. In a similar way, far fewer have written code for AMD, compared to NVIDIA.

If you're going to design a custom chip and deploy it in your data centers, you're also committing to hiring and training developers to build for it.

That's a kind of moat, but with private chips. While you solve one problem (getting the compute you want), you create another: supporting and maintaining that ecosystem long term.

NVIDIA was successful because they got their hardware into developers hands, which created a feedback loop, developers asked for fixes/features, NVIDIA built them, the software stack improved, and the hardware evolved alongside it. That developer flywheel is what made CUDA dominant and is extremely hard to replicate because the shortage of talented developers is real.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#72

Post by xadhominemx »

floxy wrote: post #52

On a slightly different tangent, is anyone working on analog machine learning ASICs? Sub-threshold CMOS or something? I mean even at the research level? Using a handful of transistor for an analog multiplier. And get all of the crazy fascinating translinear stuff of Barrie Gilbert fame. https://www.electronicdesign.com/technologies/analog/article... https://www.analog.com/en/resources/analog-dialogue/articles... http…

For large models, the bottlenecks are memory bandwidth, network, and power consumption by the DAC/ADC arrays

It’s never come even close to penciling out in practice.

For small models there are people working on this implemented in flash memory eg Mythic.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#74

Post by imtringued »

latchkey wrote: post #23

It always falls back on the software. AMD is behind, not because the hardware is bad, but because their software historically has played second fiddle to their hardware. The CUDA moat is real. So, unless they also solve that issue with their own hardware, then it will be like the TPU, which is limited to usage primarily at Google, or within very specific use cases. There are only so many super talented software engin…

Internal ASICs are a completely different market. You know your workloads and there is a finite number of them. It's as if you had to build a web browser, normally an impossible task, except it only needs to work with your company website, which only uses 1% of all of the features a browser offers.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#75

Post by kccqzy »

rjzzleep wrote: post #42
harrall wrote: post #8

Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.

The first revisions were stuff made by qualcomm right? I don't think we have much data on how much customizations they make and where they their IP from, but given how much of the Tensor cores comes from Samsung I think it's safe to say to assume that there is a decent amount coming from some of the big vendors.

It's "made by" TSMC as usual. Their customization comes from identifying which compute operations that want optimized in hardware and do it themselves. And then they buy non-compute IP like HBM from Broadcom. And Broadcom also does things like physical design.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#76

Post by latchkey »

imtringued wrote: post #74
latchkey wrote: post #23

It always falls back on the software. AMD is behind, not because the hardware is bad, but because their software historically has played second fiddle to their hardware. The CUDA moat is real. So, unless they also solve that issue with their own hardware, then it will be like the TPU, which is limited to usage primarily at Google, or within very specific use cases. There are only so many super talented software engin…

Internal ASICs are a completely different market. You know your workloads and there is a finite number of them. It's as if you had to build a web browser, normally an impossible task, except it only needs to work with your company website, which only uses 1% of all of the features a browser offers.

Very true indeed. I'm not arguing against that at all.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#77

Post by kccqzy »

Der_Einzige wrote: post #50
harrall wrote: post #8

Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.

I'm 99.999% sure that the claim of "all deep learning for Gemini on their own silicon" is not true. Maybe if you restrict it similarly to the Deepseek paper to "Gemini uses TPU for the final successful training run and for scaled inference" you might be correct, but there's no way that GPUs aren't involved for at minimum comparability and more rapid iteration reasons during the extremely buggy and error prone point o…

You seem to think GPUs are better than TPUs for rapid iteration. Why is that? There's no inherent reason why one is more suited to rapid iteration than another; it's entirely a matter of developer tooling and infrastructure. And Google famously has excellent tooling. And furthermore, the tooling Google exposes to the outside world is usually poorer than the tooling used internally by Googlers.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#78

Post by pjmlp »

Den_VR wrote: post #38
cjbgkagh wrote: post #15

I guess Microsoft’s investment into Graphcore didn’t pay off. Not sure what they’re planning but more of that isn’t going to cut it. At the time (late 2019) I was arguing for either a GPU approach or specialized architecture targeting transformers. There was a split at MS where the ‘Next Gen’ bayesian was being done in the US and the frequentist work was being shipped off to China. Chris Bishop was promoted to head o…

Microsoft lacks the credibility and track record for this to be anything but talk. Hardware doesn’t simply go from zero to gigawatts of infrastructure on talk. Even Apple is better positioned for such a thing.

Microsoft has plenty of home grown hardware on Azure, some of which even has firmware written in Rust nowadays.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#79

Post by migueldeicaza »

cjbgkagh wrote: post #15

I guess Microsoft’s investment into Graphcore didn’t pay off. Not sure what they’re planning but more of that isn’t going to cut it. At the time (late 2019) I was arguing for either a GPU approach or specialized architecture targeting transformers. There was a split at MS where the ‘Next Gen’ bayesian was being done in the US and the frequentist work was being shipped off to China. Chris Bishop was promoted to head o…

They do have such a dedicated chip, the MAIA 100 chip which is an in-house chip, and it is a chip that was designed in the era of transformers, and this is what is being discussed in the interview.
Post reply on HN