Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why CUDA translation wont unlock AMD

eliovp.com

Post reply on HN
Print view View original

61–70 of 85 posts

Re: Why CUDA translation wont unlock AMD

#61

Post by djsjajah »

manjose2018 wrote: post #33

https://geohot.github.io//blog/jekyll/update/2025/03/08/AMD-... https://tinygrad.org/ is the only viable alternative to CUDA that I have seen popup in the past few years.

I can't tell if you are making a joke or not.

They are not even remotely equivalent. tinygrad is a toy.

If you are serious, I would be interested to hear how you see tinygrad replacing CUDA. I could see a tiny grad zealot arguing that it is gong to replace torch, but CUDA??

Have you looked into AMD support in torch? I would wager that like for like, a torch/amd implementation of a models is going to run rings around a tinygrad/amd implementation.

Re: Why CUDA translation wont unlock AMD

#62

Post by bigyabai »

measurablefunc wrote: post #60
bigyabai wrote: post #55

Earlier quoted context omitted.

It's hard to catch-on to a deliberately dishonest pretense. You could clone 10,000 John Carmacks to do the job for you, Nvidia would still be a $5 trillion business next time you wake up.

[flagged]

I'm not talking to them. I am responding to you - your sardonic piss-take is against HN guidelines and written in bad-faith.

Re: Why CUDA translation wont unlock AMD

#65

Post by suprjami »

latchkey wrote: post #58
mandevil wrote: post #18

Yeah, ROCm focused code will always beat generic code compiled down. But this is a really difficult game to win. For example, Deepseek R-1 released optimized for running on Nvidia HW, and needed some adaption to run as well on ROCm. This was for the exact same reasons that ROCm code will beat generic code compiled into ROCm, in the same way. Basically the Deepseek team, for their own purposes, created R-1 to fit Nvid…

Kimi is the latest model that isn't running correctly on AMD. Apparently close to Deepseek in design, but different enough that it just doesn't work. It isn't just the model, it is the engine to run it. From what I understand this model works with sglang, but not with vLLM.

This is normal. An inference engine needs support for a model's particular implementation of the transformer architecture. This has been true for almost every model release since we got local weights.

Really good model providers send a launch-day patch to llama.cpp and vllm to make sure people can run their model instantly.

Re: Why CUDA translation wont unlock AMD

#66

Post by latchkey »

suprjami wrote: post #65
latchkey wrote: post #58

Earlier quoted context omitted.

Kimi is the latest model that isn't running correctly on AMD. Apparently close to Deepseek in design, but different enough that it just doesn't work. It isn't just the model, it is the engine to run it. From what I understand this model works with sglang, but not with vLLM.

This is normal. An inference engine needs support for a model's particular implementation of the transformer architecture. This has been true for almost every model release since we got local weights. Really good model providers send a launch-day patch to llama.cpp and vllm to make sure people can run their model instantly.

It isn't about normal or not. It is that those patches are done for Nvidia, but not AMD. It is that it takes time and energy to vet them and merge them into those projects. Kimi has been out for 3 months now and it still doesn't run out of the box on vLLM on AMD, but it works just fine with Nvidia.

Re: Why CUDA translation wont unlock AMD

#67

Post by pjmlp »

fulafel wrote: post #52

Vulkan Compute is catching up with HIP (or whatever the compatibility stuff is called now), which seems like a welcome break from CUDA - in this benchmark it beats CUDA in some benchmarks on AMD: https://www.phoronix.com/review/rocm-71-llama-cpp-vulkan

For most devs using GLSL instead of C++20, or Python GPU JIT, is a downgrade in developer experience.

Re: Why CUDA translation wont unlock AMD

#68

Post by bigyabai »

measurablefunc wrote: post #63
bigyabai wrote: post #62

Earlier quoted context omitted.

I'm not talking to them. I am responding to you - your sardonic piss-take is against HN guidelines and written in bad-faith.

[flagged]

Sure, and thieves probably recommend that the cops move on & refrain from following where they're headed.

Be honest and you won't have to fend-off accusations of bad-faith. I'm inclined to agree with your overall point of AI being overhyped, but you've gutted your own logic so hard in the process that your stance is unrecognizable. You've developed a meaningfully ambiguous stance to an elaborate and deeply incorrect series of arguments.

Re: Why CUDA translation wont unlock AMD

#69

Post by measurablefunc »

bigyabai wrote: post #68
measurablefunc wrote: post #63

Earlier quoted context omitted.

[flagged]

Sure, and thieves probably recommend that the cops move on & refrain from following where they're headed. Be honest and you won't have to fend-off accusations of bad-faith. I'm inclined to agree with your overall point of AI being overhyped, but you've gutted your own logic so hard in the process that your stance is unrecognizable. You've developed a meaningfully ambiguous stance to an elaborate and deeply incorrect…

[flagged]

Re: Why CUDA translation wont unlock AMD

#70

Post by fulafel »

pjmlp wrote: post #67
fulafel wrote: post #52

Vulkan Compute is catching up with HIP (or whatever the compatibility stuff is called now), which seems like a welcome break from CUDA - in this benchmark it beats CUDA in some benchmarks on AMD: https://www.phoronix.com/review/rocm-71-llama-cpp-vulkan

For most devs using GLSL instead of C++20, or Python GPU JIT, is a downgrade in developer experience.

For Python: PyTorch has Vulkan support according to https://docs.pytorch.org/executorch/stable/backends/vulkan/v... - wonder how performance is there.
Post reply on HN