Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first two custom silicon chips designed by Microsoft for its cloud

theverge.com

Post reply on HN
Print view View original

121–130 of 282 posts

Re: The first two custom silicon chips designed by Microsoft for its cloud

#121

Post by buildbot »

aseipp wrote: post #69
paulpan wrote: post #53

Earlier quoted context omitted.

A bit surprised to see Jensen's stage appearance since clearly Microsoft's success with its own AI chips means less business for Nvidia's chips. Because different than the ARM chip also announced in the same Ignite event, Microsoft doesn't exactly "need" nor can fully utilize an AI chip. Google trains its foundational models (e.g. Gemini) on its own TPU hardware but Microsoft's is heavily reliant on OpenAI for its ge…

Microsoft absolutely runs their own models on their own hardware, at scale, and they have done so for years just like every other hyperscaler -- Project Brainwave was first publicly talked about as far back as 2018. The generative LLM craze is a recent phenomenon in comparison. They are absolutely going to go all in on putting AI functionality in Bing, in Excel, in Windows, etc etc. To do that, you need hardware. Non…

Awesome! Someone who knows about Brainwave!

Re: The first two custom silicon chips designed by Microsoft for its cloud

#122

Post by brucethemoose2 »

aleph_minus_one wrote: post #112
brucethemoose2 wrote: post #86

Earlier quoted context omitted.

The capital costs are enormous, not even counting the CUDA moat. It takes years to start producing a big AI processor. Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. Nvidia is massively supply constrained. AI customers would be buying up MI250s, CS-2s, IPUs, Tenstorrent accelerators, Gaudi 2s and so on en masse if they wanted to..…

> Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. As I already hinted in my post: I see a huge problem in the fact that in my opinion it still is not completely clear to this day which capabilities an AI accelerator really needs - too much is in my opinion still in a state of flux.

The answer is kinda "whatever Nvidia implements." Research papers literally build around their hardware capabilities.

A good example of this is Intel canceling, and AMD sidelining, their unified memory CPU/GPU chips for AI. They are super useful!.. In theory. But actually, they totally useless because no one is programming frameworks with unified memory SoCs in mind, as Nvidia does not make something like that.

Re: The first two custom silicon chips designed by Microsoft for its cloud

#126

Post by hulitu »

wargames wrote: post #14
monlockandkey wrote: post #10

Arm in inevitable for the server. It's interesting how now days, efficiency/power consumption is a consideration over pure raw performance.

I genuinely don't see how x86 architecture will continue to survive the next 10 years. It will of course take longer to change home desktop users to new architectures; they will be the last segment to switch, but it seems all but inevitable. BTW, I'm not even speaking to whether x86 can compete at the same power per watt... I think it just won't make sense financially to be out of sync with the industry.

> I genuinely don't see how x86 architecture will continue to survive the next 10 years.

ARM is ok only for reasonable performance at low power (if we forget about VIA).

Re: The first two custom silicon chips designed by Microsoft for its cloud

#127

Post by kmeisthax »

echelon wrote: post #57
paulpan wrote: post #53

Earlier quoted context omitted.

A bit surprised to see Jensen's stage appearance since clearly Microsoft's success with its own AI chips means less business for Nvidia's chips. Because different than the ARM chip also announced in the same Ignite event, Microsoft doesn't exactly "need" nor can fully utilize an AI chip. Google trains its foundational models (e.g. Gemini) on its own TPU hardware but Microsoft's is heavily reliant on OpenAI for its ge…

> Unless Microsoft is planning to acquire OpenAI fully They're going to play a modified version of the old Rareware trick. It's also a pretty great game to buy up OpenAI equity, which ultimately gets spent on Microsoft compute. Two birds, one stone.

...they're going to overbid on a studio that was actively falling apart, after being rebuffed from buying one of the biggest giants in the business[0], all as part of an ill-advised attempt to muscle into a game business they didn't understand?

[0] Microsoft tried to buy Nintendo very early on

Re: The first two custom silicon chips designed by Microsoft for its cloud

#128

Post by er4hn »

gary_0 wrote: post #16

> Microsoft said it does not plan to sell the chips Add it to the list of things you can't buy at any price, and can only rent. That list is getting pretty long, especially if you count "any electronic device you can't fully control or modify".

This is a custom chip that they are making. I don't think that they should be required to sell it, but if others find it valuable you could expect to see hardware startups making their own RISC-V AI chips as well that you could buy.

Re: The first two custom silicon chips designed by Microsoft for its cloud

#129

Post by aseipp »

buildbot wrote: post #121
aseipp wrote: post #69

Earlier quoted context omitted.

Microsoft absolutely runs their own models on their own hardware, at scale, and they have done so for years just like every other hyperscaler -- Project Brainwave was first publicly talked about as far back as 2018. The generative LLM craze is a recent phenomenon in comparison. They are absolutely going to go all in on putting AI functionality in Bing, in Excel, in Windows, etc etc. To do that, you need hardware. Non…

Awesome! Someone who knows about Brainwave!

Yeah! Did/do you work on it? The original publications were good timing; I was working as a consultant on an FPGA-based ML accelerator at the time the original stuff was talked about, and I really enjoyed reading everything I could about Brainwave! Really neat project from both a system design perspective (e.g. heterogeneous compiler) to the choice of using and interconnecting FPGAs and integrating the network/software/ML stack (IIRC, there was a good paper on the overlay network they used to make those custom functions available on the global network fabric.)

I'm guessing at this point the ASICs make a lot more economic sense, though. :)

Re: The first two custom silicon chips designed by Microsoft for its cloud

#130

Post by Slartie »

hulitu wrote: post #80
pseudosavant wrote: post #5

Not a lot of information about the chips yet. About 100B transistors in the AI chip. For comparison, an RTX 4090 has 76B, and an H100 has about 80B. So the Maia chip is pretty massive.

This is like performance review based on written line of code.

GPUs (and AI chips) are highly parallel, containing thousands upon thousands of the same compute units. The performance of these chips is very much dependent on having a sheer number of transistors to form into as many compute units as possible.

If we assume that Microsoft is roughly able to architect compute units of a similar performance-to-number-of-transistors ratio as nVidia is, then having twice the number of transistors should roughly result in twice the performance.

That is very different than it is with typical software. If you give a programmer who needs to write 100 lines of code to solve a given problem 100 more lines to fill, he won't simply be able to copy-paste his 100 lines another time and by that action be twice as fast at solving whatever problem you tasked him with. With GPU compute units, such copy-pasting of compute units is exactly what's being done (at least until you hit the limits of other resources such as management units, memory bandwidth etc.).

Post reply on HN