Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

cnbc.com

Post reply on HN
Print view View original

31–40 of 137 posts

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#31

Post by dekhn »

chrismustcode wrote: post #12
harrall wrote: post #8

Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.

I thought they use GPU for learning and TPU for inference, I’m open to been corrected.

no. for internal training most work is done on TPUs, which have been explicitly designed for high performance training.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#33

Post by sgerenser »

outside1234 wrote: post #27

For GPUs at least this is pretty obvious. For CPUs it is less clear to me that they can do it more efficiently.

When Microsoft talks about “making their own CPUs,” they just mean putting together a large number of off-the-shelf Arm Neoverse cores into their own SoC, not designing a fully custom CPU. This is the same thing that Google and Amazon are doing as well.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#34

Post by startupsfail »

hkt wrote: post #6

Even just saying this applies downward pressure on pricing: NVIDIA has an enormous amount of market power (~"excess" profit) right now and there aren't enough near competitors to drive that down. The only thing that will work is their biggest _consumers_ investing, or threatening to invest, if their prices are too high. Long term, I wonder if we're exiting the "platform compute" era, for want of a better term. By tha…

In that case, it's great that Microsoft is building their silicon. Keeps NVIDIA in check, otherwise these profits would evaporate into nonsense and NVIDIA would lose the AI industry to competition from China. Which, depending if AGI/ASI is possible or not, may or may not be a great move.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#35

Post by georgeburdell »

CharlesW wrote: post #19
giancarlostoro wrote: post #4

So similar to Apple Silicon. If this means they'll be on par with Apple Silicon I'm okay with this, I'm surprised they didn't do this sooner for their Surface devices. Oh right, for their data centers. I could see this being useful there too, brings costs down lower.

> So similar to Apple Silicon. Yes, in the sense that this is at least partially inspired by Apple's vertical integration playbook, which has now been extended to their own data centers based on custom Apple Silicon¹ and a built-for-purpose, hardened edition of Darwin². ¹ https://security.apple.com/blog/private-cloud-compute/ ² https://en.wikipedia.org/wiki/Darwin_(operating_system)

Vertical integration only works if your internal teams can stay in the race at each level well enough to keep the stack competitive as a whole. Microsoft can’t attract the same level of talent as Apple because their pay is close to the industry median

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#36

Post by edude03 »

CharlesW wrote: post #28
edude03 wrote: post #17

Earlier quoted context omitted.

> I wonder if it will spur nvidia to work on an inference only accelerator. Arguably that's a GPU? Other than (currently) exotic ways to run LLMs like photonics or giant SRAM tiles there isn't a device that's better at inference than GPUs and they have the benefit that they can be used for training as well. You need the same amount of memory and the same ability to do math as fast as possible whether its inference or…

> Arguably that's a GPU? Yes, and to @quadrature 's point, NVIDIA is creating GPUs explicitly focused on inference, like the Rubin CPX: https://www.tomshardware.com/pc-components/gpus/nvidias-new-... "…the company announced its approach to solving that problem with its Rubin CPX— Content Phase aXcelerator — that will sit next to Rubin GPUs and Vera CPUs to accelerate specific workloads."

Yeah, I'm probably splitting hairs here but as far as I understand (and honestly maybe I don't understand) - Rubin CPX is "just" a normal GPU with GDDR instead of HBM.

In fact - I'd say we're looking at this backwards - GPUs used to be the thing that did math fast and put the result into a buffer where something else could draw it to a screen. Now a "GPU" is still a thing that does math fast, but now sometimes, you don't include the hardware to put the pixels on a screen.

So maybe - CPX is "just" a GPU but with more generic naming that aligns with its use cases.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#37

Post by alephnerd »

I've mentioned this before on HN [0][1].

The name of the game has been custom SoCs and ASICs for a couple years now, because inference and model training is an "embarrassingly parallel" problem, and models that are optimized for older hardware can provide similar gains to models that are run on unoptimized but more performant hardware.

Same reason H100s remain a mainstay in the industry today, as their performance profile is well understood now.

[0] - https://news.ycombinator.com/item?id=45275413

[1] - https://news.ycombinator.com/item?id=43383418

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#38

Post by Den_VR »

cjbgkagh wrote: post #15

I guess Microsoft’s investment into Graphcore didn’t pay off. Not sure what they’re planning but more of that isn’t going to cut it. At the time (late 2019) I was arguing for either a GPU approach or specialized architecture targeting transformers. There was a split at MS where the ‘Next Gen’ bayesian was being done in the US and the frequentist work was being shipped off to China. Chris Bishop was promoted to head o…

Microsoft lacks the credibility and track record for this to be anything but talk. Hardware doesn’t simply go from zero to gigawatts of infrastructure on talk. Even Apple is better positioned for such a thing.

Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

#40

Post by alephnerd »

bee_rider wrote: post #14

The most important note is: > The software titan is rather late to the custom silicon party. While Amazon and Google have been building custom CPUs and AI accelerators for years, Microsoft only revealed its Maia AI accelerators in late 2023. They are too late for now, they realistically hardware takes a couple generations to become a serious contender and by the time Microsoft has a chance to learn from their hardwar…

> hardware takes a couple generations to become a serious contender

Not really and for the same reason Chinese players like Biren are leapfrogging - much of the workload profile in AI/ML is "embarrassingly parallel", thus reducing the need for individual ASICs to be bleeding edge performant.

If you are able to negotiate competitive fabrication and energy supply deals, you can mass produce your way into providing "good enough" performance.

Finally, the persona who cares about hardware performance in training isn't in the market for cloud offered services.

Post reply on HN