Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

cnbc.com

Post reply on HN
Print view View original

91–100 of 137 posts

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#91

Post by latchkey »

amelius wrote: post #86
latchkey wrote: post #71

Earlier quoted context omitted.

Very few developers outside of Google have ever written code for a TPU. In a similar way, far fewer have written code for AMD, compared to NVIDIA. If you're going to design a custom chip and deploy it in your data centers, you're also committing to hiring and training developers to build for it. That's a kind of moat, but with private chips. While you solve one problem (getting the compute you want), you create anoth…

I mean it's all true to some extent. But that doesn't mean implementing the few primitives to get transformers running requires CUDA, or that it's an impossible task. Remember, we're talking about >$1B companies here who can easily assemble teams of 10s-100s of developers. You can compare CUDA to the first PC OS, DOS 1.0. Sure, DOS was viewed as a moat at the time, but it didn't keep others from kicking its ass.

> You can compare CUDA to the first PC OS, DOS 1.0.

Sorry, I don't understand this comparison at all. CUDA isn't some first version of an OS, not even close. It's been developed for almost 20 years now. Bucketloads of documentation, software and utility have been created around it. It won't have its ass kicked by any stretch of imagination.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#92

Post by cjbgkagh »

migueldeicaza wrote: post #79
cjbgkagh wrote: post #15

I guess Microsoft’s investment into Graphcore didn’t pay off. Not sure what they’re planning but more of that isn’t going to cut it. At the time (late 2019) I was arguing for either a GPU approach or specialized architecture targeting transformers. There was a split at MS where the ‘Next Gen’ bayesian was being done in the US and the frequentist work was being shipped off to China. Chris Bishop was promoted to head o…

They do have such a dedicated chip, the MAIA 100 chip which is an in-house chip, and it is a chip that was designed in the era of transformers, and this is what is being discussed in the interview.

I missed that, it’s been a few years since I’ve paid attention to MS hardware and it is very possible that my thoughts are out of date. I left MS with a rather bad taste in my mouth. I’m checking out the info on that chip and what I am seeing is a little light on details. Just TPUs and fast interconnects.

What I’ve found; MIAI 200 the next version is having issues due to brain drain, and MIAI 300 is to be an entirely new architecture so the status for that is rather uncertain.

I think a big reason MS invested so heavily into OpenAI was to have a marquee customer push cultural change through the org, which was a necessary decision. If that eventually yields in a useful chip I will be impressed, I hope it does.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#93

Post by amelius »

latchkey wrote: post #91
amelius wrote: post #86

Earlier quoted context omitted.

I mean it's all true to some extent. But that doesn't mean implementing the few primitives to get transformers running requires CUDA, or that it's an impossible task. Remember, we're talking about >$1B companies here who can easily assemble teams of 10s-100s of developers. You can compare CUDA to the first PC OS, DOS 1.0. Sure, DOS was viewed as a moat at the time, but it didn't keep others from kicking its ass.

> You can compare CUDA to the first PC OS, DOS 1.0. Sorry, I don't understand this comparison at all. CUDA isn't some first version of an OS, not even close. It's been developed for almost 20 years now. Bucketloads of documentation, software and utility have been created around it. It won't have its ass kicked by any stretch of imagination.

Yes, CUDA has a history. And it shows. CUDA has very bad integration with the OS for example. It's time some other company (Microsoft sounds like a good contender) showed them how you do this the right way.

Anyway, this all distracts from the fact that you don't need an entire "OS" just to run some arithmetic primitives to get transformers running.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#95

Post by interestpiqued »

buildbot wrote: post #20
harrall wrote: post #8

Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.

Not that it matters, but Microsoft has been doing AI accelerators for a bit too - project Brainwave has been around since 2018 - https://blogs.microsoft.com/ai/build-2018-project-brainwave/

Yeah I worked in the hardware org around this time. We got moved from under Xbox org to azure and our main work became AI related accelerators

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#96

Post by kingstnap »

imtringued wrote: post #57
Der_Einzige wrote: post #50

Earlier quoted context omitted.

I'm 99.999% sure that the claim of "all deep learning for Gemini on their own silicon" is not true. Maybe if you restrict it similarly to the Deepseek paper to "Gemini uses TPU for the final successful training run and for scaled inference" you might be correct, but there's no way that GPUs aren't involved for at minimum comparability and more rapid iteration reasons during the extremely buggy and error prone point o…

Google is the creator of JAX and XLA. Maybe the developer laptops have Nvidia GPUs and they do some testing there, but for Google there is literally no point in bothering with CUDA, pytorch or any other ecosystem strongly focused on Nvidia GPUs. In my experience JAX is way more flexible than pytorch the moment you want to do things that aren't training ML models. E.g. you want to build an optimizer that uses the deri…

Honestly Pytorch is weird imo, I'm surprised people love it so much.

Loss.backward? Tensor.grad? Optimizer.zero grad()? With torch.no_grad()?

What is with all these objects holding pointers to stuff? An ndarray is a pointer to memory and a shape my dudes. A gradient is the change in a scalar function w.r.t to some inputs.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#97

Post by jtfrench »

"Microsoft Silicon", coming up.

Is it practical for them to buy an existing chip maker? Or would they just go home-grown?

- As of today, Nvidia's market cap is a whopping 4.51 trillion USD compared to Microsoft's 3.85 trillion USD, so that might not work.

- AMD's market cap is 266.49 billion USD, which is more in reach.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#98

Post by interestpiqued »

bee_rider wrote: post #14

The most important note is: > The software titan is rather late to the custom silicon party. While Amazon and Google have been building custom CPUs and AI accelerators for years, Microsoft only revealed its Maia AI accelerators in late 2023. They are too late for now, they realistically hardware takes a couple generations to become a serious contender and by the time Microsoft has a chance to learn from their hardwar…

Most of the big players started working on hardware for this stuff in 2018/2019. I worked at MSFT silicon org during this time. Meta was also hiring my coworkers for similar projects. I left a few years ago and don’t know current state but they already have some generations under their belt

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#99

Post by latchkey »

amelius wrote: post #93
latchkey wrote: post #91

Earlier quoted context omitted.

> You can compare CUDA to the first PC OS, DOS 1.0. Sorry, I don't understand this comparison at all. CUDA isn't some first version of an OS, not even close. It's been developed for almost 20 years now. Bucketloads of documentation, software and utility have been created around it. It won't have its ass kicked by any stretch of imagination.

Yes, CUDA has a history. And it shows. CUDA has very bad integration with the OS for example. It's time some other company (Microsoft sounds like a good contender) showed them how you do this the right way. Anyway, this all distracts from the fact that you don't need an entire "OS" just to run some arithmetic primitives to get transformers running.

> CUDA has very bad integration with the OS for example.

If you want to cherry pick anything, you can. But in my eyes, you're just solidifying my point. Software is critical. Minimizing the surface is obviously a good thing (tinygrad for example), but you're still going to need people who are willing and able to write the code.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#100

Post by accrual »

nickpsecurity wrote: post #66
floxy wrote: post #52

On a slightly different tangent, is anyone working on analog machine learning ASICs? Sub-threshold CMOS or something? I mean even at the research level? Using a handful of transistor for an analog multiplier. And get all of the crazy fascinating translinear stuff of Barrie Gilbert fame. https://www.electronicdesign.com/technologies/analog/article... https://www.analog.com/en/resources/analog-dialogue/articles... http…

A bunch of people. Just type these terms into DuckDuckGo: analog neural network hardware physical neural network hardware Put "this paper" after each one to get academic research. Try it with and without that phrase. Also, add "survey" to the next iteration. The papers that pop up will have the internal jargon the researchers use to describe their work. You can further search with it. The "this paper," "survey," and…

Thanks for these helpful search terms!
Post reply on HN