[flagged]
Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
11–20 of 137 posts
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#12Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.
I thought they use GPU for learning and TPU for inference, I’m open to been corrected.
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#13Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.
I thought they use GPU for learning and TPU for inference, I’m open to been corrected.
I've heard its a mixture because they can't source enough in-house compute
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#14The most important note is:
> The software titan is rather late to the custom silicon party. While Amazon and Google have been building custom CPUs and AI accelerators for years, Microsoft only revealed its Maia AI accelerators in late 2023.
They are too late for now, they realistically hardware takes a couple generations to become a serious contender and by the time Microsoft has a chance to learn from their hardware mistakes the “AI” bubble will have popped.
But, there will probably be some little LLM tools that do end up having practical value; maybe there will be a happy line-crossing point for MS and they’ll have cheap in-house compute when the models actually need to be able to turn a profit.
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#15I guess Microsoft’s investment into Graphcore didn’t pay off. Not sure what they’re planning but more of that isn’t going to cut it. At the time (late 2019) I was arguing for either a GPU approach or specialized architecture targeting transformers.
There was a split at MS where the ‘Next Gen’ bayesian was being done in the US and the frequentist work was being shipped off to China. Chris Bishop was promoted to head of MSR Cambridge which didn’t help.
Microsoft really is an institutionally stupid organization so I have no idea on which direction they actually go. My best guess is that it’s all talk.
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#16Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.
I thought they use GPU for learning and TPU for inference, I’m open to been corrected.
Some details here: https://news.ycombinator.com/item?id=42392310
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#17Not suprising that the hyperscalers will make this decision for inference and maybe even a large chunk of training. I wonder if it will spur nvidia to work on an inference only accelerator.
> I wonder if it will spur nvidia to work on an inference only accelerator.
Arguably that's a GPU? Other than (currently) exotic ways to run LLMs like photonics or giant SRAM tiles there isn't a device that's better at inference than GPUs and they have the benefit that they can be used for training as well. You need the same amount of memory and the same ability to do math as fast as possible whether its inference or training.
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#18Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#19So similar to Apple Silicon. If this means they'll be on par with Apple Silicon I'm okay with this, I'm surprised they didn't do this sooner for their Surface devices. Oh right, for their data centers. I could see this being useful there too, brings costs down lower.
> So similar to Apple Silicon.
Yes, in the sense that this is at least partially inspired by Apple's vertical integration playbook, which has now been extended to their own data centers based on custom Apple Silicon¹ and a built-for-purpose, hardened edition of Darwin².
¹ https://security.apple.com/blog/private-cloud-compute/ ² https://en.wikipedia.org/wiki/Darwin_(operating_system)
Re: Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips
#20Google has been using its own TPU silicon for machine learning since 2015. I think they do all deep learning for Gemini on ther own silicon. But they also invented AI as we know it when they introduced transformer architecture and they’ve been more invested in machine learning than most companies for a very long time.
Not that it matters, but Microsoft has been doing AI accelerators for a bit too - project Brainwave has been around since 2018 - https://blogs.microsoft.com/ai/build-2018-project-brainwave/