Earlier quoted context omitted.
They'll sound better after we hear Microsoft's names.
Microsoft Azure Inference for Cloud Apps 365 Pro Live Series X
Viva
The first two custom silicon chips designed by Microsoft for its cloud
151–160 of 282 posts
Re: The first two custom silicon chips designed by Microsoft for its cloud
#152Not a lot of information about the chips yet. About 100B transistors in the AI chip. For comparison, an RTX 4090 has 76B, and an H100 has about 80B. So the Maia chip is pretty massive.
This is like performance review based on written line of code.
In a way, it is the opposite. Code is what you execute. The transistors are the engines that do the execution. They are going to expect a chip they designed with 105B transistors to perform (speed/efficiency/whatever) in the same ballpark as a high=end GPU for their AI workloads.
It is like knowing the kind of engine a car has. Not all V8 gas engines produce the same power, but knowing that it is a V8 instead of an inline three cylinder does give you an idea of the expected performance characteristics.
Re: The first two custom silicon chips designed by Microsoft for its cloud
#153It was only a matter of time. Google announced theirs years ago, Amazon announced theirs last year. Right now NVIDIA has the lead because they have the better software, but they can't make the chips fast enough. Will be interesting to see if their better software continues to keep them in the lead or if people are more interested in getting the capacity in any form.
nVidia still has the monopoly on training. Everyone else is just making chips for inference.
Shouldn't inference be 99.999% of the compute cost over time?
Especially for MS.
Look how many Copilots they are cramming into their products
Re: The first two custom silicon chips designed by Microsoft for its cloud
#154Earlier quoted context omitted.
If they “can’t make the chips fast enough” being TSMC’s second highest volume customer behind Apple and probably second in priority, what chance does Microsoft have getting enough of TSMCs capacity?
It blows my mind that not only are Apple, Nvidia, and all other enormous "chip makers" fabless... but there's ONE single company actually making the majority of all their chips. Why aren't any other companies entering this space? TSMC's growth and profits are immense, it's not like the market couldn't bear more competitors. Also, the current situation is geopolitical insanity. It feels like China and the U.S. are on…
Building and operating a fab is an extremely difficult and expensive task that will take at least 5-10 years to bear fruit. That makes it a high-risk venture which is hard to sell to the board and investors.
More importantly, the new chip production lines are dependent on hyper-specialized EUV lithography systems that only company is able to manufacture (ASML). ASML has their own limits to production too.
Re: The first two custom silicon chips designed by Microsoft for its cloud
#155> Microsoft said it does not plan to sell the chips Add it to the list of things you can't buy at any price, and can only rent. That list is getting pretty long, especially if you count "any electronic device you can't fully control or modify".
It's the new HaaS business model: Hardware as a Service.
I’m curious to know what everyone thinks of this trend. Do you view it as a good thing, bringing efficiency and economy of scale, competition and so forth? Or do you consider it a bad thing, another salvo in the War on General Purpose Computing [1] so vividly described by Cory Doctorow?
I, personally, am interested in retrocomputing, amateur/hobbyist electronics, and hobbyist computing (including semiconductors [2]). While these techniquess and devices may be light years away from anything resembling a computer that can compete with SotA commercial offerings, they do offer the promise of “keeping the candle lit” as it were. I will note that if you follow Sam Zeloof’s chronicles, he progressed through the earliest phases of semiconductor development far faster than the industry did back when it was pioneering the technology. Of course, he had the benefits of knowing it was already possible and access to the written knowledge of the experts who went before him.
Re: The first two custom silicon chips designed by Microsoft for its cloud
#156Earlier quoted context omitted.
If they “can’t make the chips fast enough” being TSMC’s second highest volume customer behind Apple and probably second in priority, what chance does Microsoft have getting enough of TSMCs capacity?
It blows my mind that not only are Apple, Nvidia, and all other enormous "chip makers" fabless... but there's ONE single company actually making the majority of all their chips. Why aren't any other companies entering this space? TSMC's growth and profits are immense, it's not like the market couldn't bear more competitors. Also, the current situation is geopolitical insanity. It feels like China and the U.S. are on…
[deleted]
Re: The first two custom silicon chips designed by Microsoft for its cloud
#157I am a little confused on all this. I thought conventional wisdom was that creating "super good/ top performant " computer processors was hard, and unless you could produce them at scale prohibitable expensive. Is this done as a bridge until/if Nvidia is able to deliver their processors fast enough? I would think that competitors to Nvidea would have serious competitors on the market already if competing can be done…
It remains to be seen how well these chips will compete with Nvidia.
Note the huge difference in the kind of hardware required to do training and to do interference.
Re: The first two custom silicon chips designed by Microsoft for its cloud
#158Earlier quoted context omitted.
Awesome! Someone who knows about Brainwave!
Yeah! Did/do you work on it? The original publications were good timing; I was working as a consultant on an FPGA-based ML accelerator at the time the original stuff was talked about, and I really enjoyed reading everything I could about Brainwave! Really neat project from both a system design perspective (e.g. heterogeneous compiler) to the choice of using and interconnecting FPGAs and integrating the network/softwa…
Sorta! Tangentially on the data science side - Brainwave pre-dates me being at MS though. It's a really cool project and I'm glad someone else also thinks so :) I was enamored with FPGAs from undergrad, one of the professors I did research with was a consultant on the original Catapult project, I thought it was the coolest idea. It was pretty awesome to be able to come full circle a fews years later.
Re: The first two custom silicon chips designed by Microsoft for its cloud
#159Earlier quoted context omitted.
If they “can’t make the chips fast enough” being TSMC’s second highest volume customer behind Apple and probably second in priority, what chance does Microsoft have getting enough of TSMCs capacity?
It blows my mind that not only are Apple, Nvidia, and all other enormous "chip makers" fabless... but there's ONE single company actually making the majority of all their chips. Why aren't any other companies entering this space? TSMC's growth and profits are immense, it's not like the market couldn't bear more competitors. Also, the current situation is geopolitical insanity. It feels like China and the U.S. are on…
TSMC is solving some of the hardest physics, electrical engineering, mechanical engineering, and other kinds of engineering I don't even know about. It's unlikely anyone catches up to them in terms of lithography ability for a long, long time given how well they've executed 45nm and lower nodes.
Re: The first two custom silicon chips designed by Microsoft for its cloud
#160Earlier quoted context omitted.
If they “can’t make the chips fast enough” being TSMC’s second highest volume customer behind Apple and probably second in priority, what chance does Microsoft have getting enough of TSMCs capacity?
> what chance does Microsoft have getting enough of TSMCs capacity? Looking at their balance sheet: plenty. TSMC is all about pay to play. Apple is first in line because they’re willing to pay to be first in line. I have no doubt Microsoft can justify spending the money they save not buying nvidia into getting some priority access from TSMC. Also keep in mind Apple is now on 3nm so there’s likely spare 5nm.
My guess is that Microsoft booked capacity years ago. Either by strategic decision, with the belief they could yield the capacity to someone else, or through their other hardware divisions (see: Xbox).
It’s very likely they knew something like this was coming, as they’ve been doing FPGAs for more than a decade now.