Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD's Instinct MI455X: Aiming for the Sun

chipsandcheese.com

Post reply on HN
Print view View original

21–30 of 38 posts

Re: AMD's Instinct MI455X: Aiming for the Sun

#21

Post by ACCount37 »

stevefan1999 wrote: post #17
dist-epoch wrote: post #11

Earlier quoted context omitted.

Co-founder and Chief Compute Officer Anthropic: > I think the thing that we were thinking about originally was whenever we're bringing up a new hardware platform, it's a big effort. It's like a huge thing. And so as we were thinking about this, we started doing our own evaluation of MI 355. You guys generously got us a rack to start working. And we expected this to be kind of a big process. > Our actual experience wa…

That only works if you have all the unit tests and making sure the behaviors between different GPUs are expected, and can tolerate difference in implementation details such as numerical stability and complexity for some functions. AMD and Nvidia are doing vastly different computation for different implementation to the same algorithm for their APIs (can't tell due to NDA, but one algorithm Nvidia did it using broadca…

That, or, you know. Ask Claude Code to write all of those unit tests.

Variance in numerical instability across hardware is basically a given, because all the different accelerators implement their "fast paths" in slightly different ways. Modern AI just takes it on the chin.

Anthropic, for example, already has to support both Nvidia CUDA and Google/Broadcom TPUs - AMD aside. And I expect those two to have more architectural and software differences than AMD ROCm and Nvidia CUDA accelerators would.

And yes. If you're wondering how they solved the problem of variance between those two: they didn't! Claude instances that run in the TPU land produce slightly different outputs than those in Nvidia racks! Just not different enough for, you know. Anyone to give a shit. No statistically significant effect on model performance.

Re: AMD's Instinct MI455X: Aiming for the Sun

#22

Post by foobiekr »

InTheArena wrote: post #20
paoliniluis wrote: post #12

Earlier quoted context omitted.

Using AI slop in software that runs ML models is like buying a Ferrari with a motorbike engine. Might be good for demos or as a proof of concept, but that can’t be used for serving LLMs at scale

At the risk of noting that this is a PR statement from someone who needs something from AMD - you do realize that you just shat upon the CTO of the largest possible consumer of a technology talking about that technology? It's not 2022 anymore. Just pointing out, there is a reason the large LLMs keep working to try to make it so their own tools can't attack their own moats, by preventing the exact behavior that you di…

He has engaged in a ton of bullshit marketing for years.

Re: AMD's Instinct MI455X: Aiming for the Sun

#23

Post by ux266478 »

ACCount37 wrote: post #16
fancyfredbot wrote: post #7

It's mind blowing seeing these multi exaflop single rack systems. The world's first exaflop supercomputer was Frontier. It was launched only 4 years ago in 2022. It's not a fair comparison of course. FP4 in Helios barely qualifies as floating point. Frontier was proper fp64, 16 times the bit width and probably 256x as many transistors. All the same just wow. Much compute.

Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision. Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a…

> the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.

It still is. Just like how "mainframe" used to be a very general word, and over time gained a very unintuitive definition referring to a very specific type of computer with a specific purpose, "supercomputer" almost invariably means it's a highly bespoke cluster dealing with FP64 workloads. I don't see anyone referring to these AI clusters as supercomputers, for the same reason they aren't referring to the racks as mainframes.

I wonder if there's a term for this kind of semantic narrowing?

Re: AMD's Instinct MI455X: Aiming for the Sun

#24

Post by ktm5j »

stevefan1999 wrote: post #19
nolist_policy wrote: post #15

Earlier quoted context omitted.

Spir-v is already happening: https://rocm.blogs.amd.com/software-tools-optimization/spir-...

> PYTORCH_ROCM_ARCH="gfx900;gfx906;gfx908;gfx90a;gfx942;gfx950;\ gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;\ gfx1150;gfx1151;gfx1200;gfx1201" That kicked off my PTSD to setup ROCm on my 6600XT and 9070XT, not even with different containers. I have to build an image for EACH architecture. That's duping 15GB for each arch on my already terribly small SSD.

Well, that appears to be the exact problem that they're trying to solve with spir-v.

> The shift SPIR-V enables is from ahead-of-time compilation per target to compile once, just-in-time specialize on device.

Re: AMD's Instinct MI455X: Aiming for the Sun

#25

Post by fancyfredbot »

ux266478 wrote: post #23
ACCount37 wrote: post #16

Earlier quoted context omitted.

Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision. Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a…

> the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision. It still is. Just like how "mainframe" used to be a very general word, and over time gained a very unintuitive definition referring to a very specific type of computer with a specific purpose, "supercomputer" almost invariably means it's a highly bespoke cluster dealing with FP64 w…

I think the term in this case is marketing.

NVIDIA used to use the term supercomputer more often five or six years ago. It's now fully committed to the term AI factory. I think this is supposed to make you think of their expensive kit as a productive asset building your business rather than an expensive tool for your boffins. Your average business executive is instinctively going to question whether they need a supercomputer but don't yet have the same preconceptions about AI factories.

I don't think that Nvidia single handedly caused the move away from the term supercomputer. But I think they had a hand helping push firmly in that direction.

Re: AMD's Instinct MI455X: Aiming for the Sun

#28

Post by Goronmon »

nl wrote: post #5
dist-epoch wrote: post #4

432 GB of RAM per chip. Tens of terabytes per rack. Clients queuing up to buy them. RAM prices are not coming down any time soon.

Did anyone think RAM prices were coming down soon?

There are plenty of laymen that think "AI bubble pop" means that "AI" usage will go to 0 and then all of a sudden all these hardware and datacenters related companies are buying will become totally unused and flood the market.

Re: AMD's Instinct MI455X: Aiming for the Sun

#29

Post by storus »

ACCount37 wrote: post #16
fancyfredbot wrote: post #7

It's mind blowing seeing these multi exaflop single rack systems. The world's first exaflop supercomputer was Frontier. It was launched only 4 years ago in 2022. It's not a fair comparison of course. FP4 in Helios barely qualifies as floating point. Frontier was proper fp64, 16 times the bit width and probably 256x as many transistors. All the same just wow. Much compute.

Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision. Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a…

FP64 is not that precise; proper simulations usually need much higher precision. Even ancient Intel could do 80-bit FP.

Re: AMD's Instinct MI455X: Aiming for the Sun

#30

Post by addaon »

storus wrote: post #29
ACCount37 wrote: post #16

Earlier quoted context omitted.

Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision. Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a…

FP64 is not that precise; proper simulations usually need much higher precision. Even ancient Intel could do 80-bit FP.

This very much depends on the simulation. CFD historically used 64-bit, with extended precision used internal to an extremely small number of poorly conditioned operations, or with very stiff equations; today it's not at all unusual to use single precision or (strategically) mixed precision, depending on the problem. I think the strongest statement you could make about simulation workloads, rather than them "needing" a specific precision, is that they are usually properly analyzed to determine the needed precision, and then run at that precision.
Post reply on HN