An apt title. To get to the sun, you need to slow down from orbital speed to zero. And then you fall in. Do they run CUDA yet or is this yet more impressive but unusable hardware?
AMD never misses a chance to miss a chance.
AMD's Instinct MI455X: Aiming for the Sun
31–38 of 38 posts
Re: AMD's Instinct MI455X: Aiming for the Sun
#32Earlier quoted context omitted.
FP64 is not that precise; proper simulations usually need much higher precision. Even ancient Intel could do 80-bit FP.
This very much depends on the simulation. CFD historically used 64-bit, with extended precision used internal to an extremely small number of poorly conditioned operations, or with very stiff equations; today it's not at all unusual to use single precision or (strategically) mixed precision, depending on the problem. I think the strongest statement you could make about simulation workloads, rather than them "needing"…
One could argue that the interesting parts of simulation require higher precision. When you are in stable conditions, that's where your intuition might be sufficient already; once you hit those ugly parts then you start requiring as high precision as you can get to have any correlation with reality. Even a simple solar system simulation through gravity interactions has significant drifts in FP64.
Re: AMD's Instinct MI455X: Aiming for the Sun
#33Earlier quoted context omitted.
Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision. Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a…
FP64 is not that precise; proper simulations usually need much higher precision. Even ancient Intel could do 80-bit FP.
ancient intel did 80 bit fp until your intermediate result was written from registers to memory. Since you typically have no control over when that happens, even in very low level languages like C, the 80 bit precision mostly had the effect of making your code produce different results on different compilers. If you needed 80 bit precision and deterministic results you had to write the algorithm in assembly.
Re: AMD's Instinct MI455X: Aiming for the Sun
#34Earlier quoted context omitted.
This very much depends on the simulation. CFD historically used 64-bit, with extended precision used internal to an extremely small number of poorly conditioned operations, or with very stiff equations; today it's not at all unusual to use single precision or (strategically) mixed precision, depending on the problem. I think the strongest statement you could make about simulation workloads, rather than them "needing"…
One could argue that the interesting parts of simulation require higher precision. When you are in stable conditions, that's where your intuition might be sufficient already; once you hit those ugly parts then you start requiring as high precision as you can get to have any correlation with reality. Even a simple solar system simulation through gravity interactions has significant drifts in FP64.
Datatype requirements vary in time and space for a given application. We're just beginning to optimize for this, so there is yet much to discover.
Re: AMD's Instinct MI455X: Aiming for the Sun
#35Earlier quoted context omitted.
> the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision. It still is. Just like how "mainframe" used to be a very general word, and over time gained a very unintuitive definition referring to a very specific type of computer with a specific purpose, "supercomputer" almost invariably means it's a highly bespoke cluster dealing with FP64 w…
I think the term in this case is marketing. NVIDIA used to use the term supercomputer more often five or six years ago. It's now fully committed to the term AI factory. I think this is supposed to make you think of their expensive kit as a productive asset building your business rather than an expensive tool for your boffins. Your average business executive is instinctively going to question whether they need a super…
It's definitely older than that. Originally, way back in the mechanical days when super-computing was hyphenated, it was about how many millions of instructions per second could be handled. That changed in the 1960s with a series of CDC mainframes that put all of their focus on floating point pipelines, moving the category to refer to floating-point crunchers used for scientific simulation (which is what the frontier of computing was used for). Eventually parallelism would be added in at some point, and the term has fossilized into a "monolithic cluster with massive FLOPS intended specifically for scientific simulation". Because for the longest time, that's who needed all the compute.
The PIM/m of the FGCS defied this a little, sometimes being called a "symbolic supercomputer", but never without the "symbolic" qualifier. It was the first time in history where the frontier of computational throughput wasn't about scientific simulations. It wasn't even about arithmetic!
Right now is also interesting, because the world's most powerful FLOPS monster monoliths aren't being used for scientific simulation. They fail at being supercomputers for that reason, but a computer scientist from the 60s would probably raise his eyebrow at that.
Re: AMD's Instinct MI455X: Aiming for the Sun
#36I don't have that many kidneys
Re: AMD's Instinct MI455X: Aiming for the Sun
#37Earlier quoted context omitted.
Co-founder and Chief Compute Officer Anthropic: > I think the thing that we were thinking about originally was whenever we're bringing up a new hardware platform, it's a big effort. It's like a huge thing. And so as we were thinking about this, we started doing our own evaluation of MI 355. You guys generously got us a rack to start working. And we expected this to be kind of a big process. > Our actual experience wa…
Using AI slop in software that runs ML models is like buying a Ferrari with a motorbike engine. Might be good for demos or as a proof of concept, but that can’t be used for serving LLMs at scale
This is untrue.
Read the "Chipping Away at the CUDA Moat: Using Agents to Enable Day 0 Support" section of [1] where they list a number of agent-created PRs that have been integrated into production vLLM/SGLang code and as well as AMD kernel fixes.
[1] https://newsletter.semianalysis.com/i/208362548/chipping-awa...
Re: AMD's Instinct MI455X: Aiming for the Sun
#38432 GB of RAM per chip. Tens of terabytes per rack. Clients queuing up to buy them. RAM prices are not coming down any time soon.
but you have to rewrite all your software to ROCm. And ROCm, to this day, still sucks. ZLUDA basically crash on high memory demand, and only accounted for ~70% of CUDA API coverage (and it is still buggy). But hey, at least it does run on Rust-CUDA. I'm one of the few who ported it and fixed a few bugs on ZLUDA. I used it to run a simple SHA256 kernel and it ran sure, but I gave it up because of those fundamental pro…
That depends on the workload. When buying for on-prem inference, MI355x is quite capable.
It still requires a ton more ops time, not every dc handles it cooling-wise, and you have to be careful with the model selection, but when it fits, it fits well