Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
rocm.blogs.amd.com
Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
1–10 of 12 posts
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#2So when can I buy MI350s for my homelab?
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#3As much as I enjoy these articles and for AMD to write more light technical articles, it really feels constrained, even strained, to be unable to cite the equivalent terms from the precursor here (NVIDIA). Another batch of jargon for very similar architectures and programming models... HIP and ROCm have actually made amazing strides in making CUDA developers' porting work easy, and I know playing catchup to a (monopolist) moving target you have no power over is bad... but I feel this is part of the thousand paper cuts.
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#4So when can I buy MI350s for my homelab?
Hopefully the MI350P is available soon, at last a standard PCIe SKU, if a bit too-much previous-generation and gimped compared to the MI350X
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#5Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#6Earlier quoted context omitted.
Hopefully the MI350 P is available soon, at last a standard PCIe SKU, if a bit too-much previous-generation and gimped compared to the MI350X
MI350 happens to not fit DeepSeek V4 Flash (144GB vs 180+ needed) so it will be dead on arrival.
For production loads? No one will run DS without expert parallelism so it’s not important for model to fit on one gpu.
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#7Earlier quoted context omitted.
MI350 happens to not fit DeepSeek V4 Flash (144GB vs 180+ needed) so it will be dead on arrival.
For production loads? No one will run DS without expert parallelism so it’s not important for model to fit on one gpu.
Do you mean people will run multiple GPUs, or that streaming a small % of experts from disk won't completely kill performance?
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#8Earlier quoted context omitted.
Hopefully the MI350 P is available soon, at last a standard PCIe SKU, if a bit too-much previous-generation and gimped compared to the MI350X
MI350 happens to not fit DeepSeek V4 Flash (144GB vs 180+ needed) so it will be dead on arrival.
This thing has 50% more memory than GTX PRO 6000, which fetch 18k-25k+ each right now on newegg, and generally sold out. If nvdia can still sell out 96GB PCI cards (on a 2 year old architecture) at those prices, how is an AMD equivalent with better specs dead on arrival?
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#9Earlier quoted context omitted.
MI350 happens to not fit DeepSeek V4 Flash (144GB vs 180+ needed) so it will be dead on arrival.
This thing has 50% more memory than GTX PRO 6000, which fetch 18k-25k+ each right now on newegg, and generally sold out. If nvdia can still sell out 96GB PCI cards (on a 2 year old architecture) at those prices, how is an AMD equivalent with better specs dead on arrival?
Market takes time to catch up. When released Qwen 3.6 had a lot of sense and it did fit, but frontier since went too far ahead. Now you'd want DS V4 Flash from yesterday, or you are too far behind.
Unless MI350P will be much cheaper than 6000 Pro, which is unlikely given its VRAM, it will lose to it, because you'd need 2 of either.
Re: Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
#10Earlier quoted context omitted.
For production loads? No one will run DS without expert parallelism so it’s not important for model to fit on one gpu.
Do you mean people will run multiple GPUs, or that streaming a small % of experts from disk won't completely kill performance?
Time for AMD to improve their directstorage story I guess. Is there a good writeup on how one would use it there ?
Same for gpudirect (more useful for scale-out or training).
Everyone is focused on AI but these are two interesting techs trickling down from the NVIDIA tree, relatively "easy" to use there, which maybe exist in AMD world but I somehow missed the docs and APIs and demos on how to use them...