Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

cnbc.com

Post reply on HN
Print view View original

131–137 of 137 posts

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#131

Post by john01dav »

tyre wrote: post #127
jlarocco wrote: post #125

Earlier quoted context omitted.

But there's a real risk that in the long term all of the "serious" AI hardware research will get done inside a few big companies, essentially shutting out smaller players with a hardware moat. Unless they start selling the hardware, but in the current AI market nobody would do that because it's their special sauce. On the other hand, maybe it's no different than any other hardware, and other makers will catch up even…

And some of those people will leave to start their own companies or join competitor chipmakers. Innovation will continue.

Independent chip companies, some of which were made that way, often get acquired and turned into in house only for big tech.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#132

Post by lostlogin »

Herring wrote: post #107
alecco wrote: post #94

For many years, every few months Microsoft and Meta say they are going to do AI hardware. But nothing tangible is delivered.

Yeah they could just be negotiating a better deal w nvidia.

It’s probably a helpful thing to say in the current political environment too.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#133

Post by musicale »

interestpiqued wrote: post #95
buildbot wrote: post #20

Earlier quoted context omitted.

Not that it matters, but Microsoft has been doing AI accelerators for a bit too - project Brainwave has been around since 2018 - https://blogs.microsoft.com/ai/build-2018-project-brainwave/

Yeah I worked in the hardware org around this time. We got moved from under Xbox org to azure and our main work became AI related accelerators

> We got moved from under Xbox org to azure and our main work became AI related accelerators

Since I prefer games to AI, this makes me rather sad.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#134

Post by IshKebab »

cjbgkagh wrote: post #15

I guess Microsoft’s investment into Graphcore didn’t pay off. Not sure what they’re planning but more of that isn’t going to cut it. At the time (late 2019) I was arguing for either a GPU approach or specialized architecture targeting transformers. There was a split at MS where the ‘Next Gen’ bayesian was being done in the US and the frequentist work was being shipped off to China. Chris Bishop was promoted to head o…

I used to work for Graphcore. Microsoft gave up on Graphcore fairly early on actually and I think it was a wise decision. There were a number of issues with Graphcore's GC1 and GC2 chips:

* a bet on storing the entire model (and code) in 900MB of SRAM. Hell of a lot of SRAM but it only really works for small models and the world wants enormous models.

* Blew it's weirdness budget by a lot. Everything is quite different so it's a significant effort to port software to it. Often you did get a decent speedup (like 2-10x) but I doubt many people thought that was worth the software pain.

* The price was POA so normal people couldn't buy one. (And it would have been too expensive for individuals anyway.) So there was little grass roots community support and research. Nvidia gets that because it's very easy to buy a consumer 4090 or whatever and run AI on it.

* Nvidia were killing it with Grace Hopper.

GC3 was way more ambitious and supports a mountain of DRAM with crazy memory bandwidth so if they ever finish it maybe they'll make a comeback.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#135

Post by tyleo »

musicale wrote: post #133
interestpiqued wrote: post #95

Earlier quoted context omitted.

Yeah I worked in the hardware org around this time. We got moved from under Xbox org to azure and our main work became AI related accelerators

> We got moved from under Xbox org to azure and our main work became AI related accelerators Since I prefer games to AI, this makes me rather sad.

You know, I prefer games too but I’ve suspected that the AI boom improved hardware for games and AI rather than just AI in isolation.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#136

Post by edude03 »

imtringued wrote: post #68
edude03 wrote: post #17

Earlier quoted context omitted.

> I wonder if it will spur nvidia to work on an inference only accelerator. Arguably that's a GPU? Other than (currently) exotic ways to run LLMs like photonics or giant SRAM tiles there isn't a device that's better at inference than GPUs and they have the benefit that they can be used for training as well. You need the same amount of memory and the same ability to do math as fast as possible whether its inference or…

The AMD NPU has more than 2x the performance per watt versus basically any Nvidia GPU. Nvidia isn't leading because they are power efficient. And no, the NPU isn't a GPU.

Maybe a better way to make my point - the GPU is nvidias golden goose egg and it's good enough that they may go down with the ship. For example (illustrative numbers) - if it costs nvidia $100 to make a GPU they can sell to gamers for $2000, researchers for $5000 and enterprise for $15,000, would it make sense for them to start from scratch and invest billions to make something that's today an unknown amount better and that would only be interesting to the $15,000 market they've already cornered? (Yes, I'm assuming there are more gamers than people who want to run a local LLM)
Post reply on HN