AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
1–10 of 24 posts
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#2Is ROCm actually usable in this years machine learning ecosystem? Can I just drop in any PyTorch model that was developed on CUDA and expect it to work?
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#3Honestly, the likes of OpenAI and Mosaic need to consider Nvidia a huge threat long term.
Nvidia has shown time and time again, they will royally fuck over anyone they have to in order to drive profit.
Then, if they dare talk negatively about them, they will just discontinue their access to hardware.
Not saying AMD is a savior, but having only ONE option will lead to long term issues.
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#4Somewhat related: starting a new Ubuntu-pytorch-cuda project? That’ll be 10-15 gigabytes, please. Is there some way to strip down the individual deps? I imagine it’s the way that it is because the drivers and the pytorch source are probably machine generated before compilation, to some extent. Is there any hope for triton becoming a lightweight interstitial frame in the codebase stack that can just allow me to do the hw codegen after pulling, without having to rebuild completely everything, and also simultaneously allowing more accelerators?
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#5Honestly, the likes of OpenAI and Mosaic need to consider Nvidia a huge threat long term. Nvidia has shown time and time again, they will royally fuck over anyone they have to in order to drive profit. Then, if they dare talk negatively about them, they will just discontinue their access to hardware. Not saying AMD is a savior, but having only ONE option will lead to long term issues.
This is a very real take. Large institutions tend to have teams dedicated to future proofing and mitigating these risks, i'd hope and imagine it is on their radar.
I do hope that AMD gets their shit in order because we need the competition to keep this space energized.
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#6Is ROCm actually usable in this years machine learning ecosystem? Can I just drop in any PyTorch model that was developed on CUDA and expect it to work?
Is ROCm actually usable in this years machine learning ecosystem?
I don't known, as I'm only just now building out my first AMD based ML machine to run ROCm. All I can really say is that AMD really seem to be making a genuine effort to get ROCm to that level. See the two links I submitted yesterday[1][2] for more details.
The two things in particular that stand out to me from all this are:
1. They are at least publicly declaring their intention to make ROCm a player in AI/ML. Previously there was at least a perception (and quite possibly a reality) that ROCm was more focused on other HPC workloads and not really AI / ML. AMD seems committed to changing that.
2. It seems that they are finally serious about getting ROCm working on their consumer Radeon cards. Even though 5.6 didn't include the long hoped-for announcement of such support, the blog post they put out did at least officially declare their intent to do so in a release this fall. And maybe more to the point, the batch of changes in 5.6 did actually include some fixes for problems encountered running on Radeon cards, even though they aren't yet officially listed as supported.
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#7Somewhat related: starting a new Ubuntu-pytorch-cuda project? That’ll be 10-15 gigabytes, please. Is there some way to strip down the individual deps? I imagine it’s the way that it is because the drivers and the pytorch source are probably machine generated before compilation, to some extent. Is there any hope for triton becoming a lightweight interstitial frame in the codebase stack that can just allow me to do the…
Here is what really annoys me:
The dependencies are such a mess that even if you try to install only pytorch-cpu, at some point some random package will cause pytorch-cuda and those 10GBs to be installed.
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#8I've been waiting years and will wait more for PyTorch etc to work under Mesa rather than install RocM
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#9Is ROCm actually usable in this years machine learning ecosystem? Can I just drop in any PyTorch model that was developed on CUDA and expect it to work?
obviously not there yet, otherwise AMD stock will be doubled.
Re: AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton
#10Is ROCm actually usable in this years machine learning ecosystem? Can I just drop in any PyTorch model that was developed on CUDA and expect it to work?
On my projects it kind of works. Given the usual driver installing work. Depending on the card you'd have to use some env flags like HSA_OVERRIDE_GFX_VERSION to make it not crash.