Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

VUDA: A Vulkan Implementation of CUDA

github.com

Post reply on HN
Print view View original

61–70 of 83 posts

Re: VUDA: A Vulkan Implementation of CUDA

#61

Post by wmf »

causality0 wrote: post #59
paulmd wrote: post #56

Earlier quoted context omitted.

“Just works” in this context means executing the compiled CUDA or the PTX bytecode without recompiling. Nobody is ever going to utilize ROCm if it requires distributing as source and recompiling. To make it even more insulting, even simply installing ROCm itself is a massive burden, even on an ostensibly-supported (as geohot discovered) and even just “it works out of the box if you distribute and compile it locally”…

Don't forget AMD doesn't seem to even care about ROCm themselves. Six months in and RDNA3 cards still don't support it. Can you imagine if Nvidia launched RTX40- cards with no DLSS even though 30- cards already had it, and six months started boasting about how DLSS support was "coming this fall"?

ROCm is for CDNA not RDNA. It has limited, best-effort RDNA support for a few cards.

Re: VUDA: A Vulkan Implementation of CUDA

#62

Post by i80and »

causality0 wrote: post #59
paulmd wrote: post #56

Earlier quoted context omitted.

“Just works” in this context means executing the compiled CUDA or the PTX bytecode without recompiling. Nobody is ever going to utilize ROCm if it requires distributing as source and recompiling. To make it even more insulting, even simply installing ROCm itself is a massive burden, even on an ostensibly-supported (as geohot discovered) and even just “it works out of the box if you distribute and compile it locally”…

Don't forget AMD doesn't seem to even care about ROCm themselves. Six months in and RDNA3 cards still don't support it. Can you imagine if Nvidia launched RTX40- cards with no DLSS even though 30- cards already had it, and six months started boasting about how DLSS support was "coming this fall"?

I've been running PyTorch on my Radeon 7900 XT using ROCm. Is that not supposed to work?

Re: VUDA: A Vulkan Implementation of CUDA

#63

Post by HarHarVeryFunny »

netheril96 wrote: post #30

Just in case other people who have AMD GPU and run Windows have the same needs as I have, that is, to train or run machine learning models, please checkout torch-directml and tensorflow-directml.

I'm not sure this really makes any more sense than AMD chasing CUDA compatibility with ROCm/MiOpen/HIP. CUDA and DirectX seem too low level to be used as a compatibility API over widely divergent hardware (AMD vs NVidia) without giving up a lot of performance.

cuDNN being higher level offers more opportunity for compatibility without losing performance (i.e different implementations of kernels fine-tuned for optimal performance on AMD vs NVidia hardware), but the trouble is that so much of what frameworks like PyTorch do is based on custom kernels, not just cuDNN.

It seems the best bet for AMD would be a rock solid low level API (not a moving target) and support of high level optimizing ML compilers to reduce the level of effort for the framework (PyTorch, TensorFlow, JAX ...) vendors to provide framework-level support on top of that. Ultimately they'd need to work very closely with the framework vendors to provide this support, since they are the ones who would be benefiting from it.

It's odd how much of an afterthought ML support has seemed to be for AMD over the years... maybe the relative size of the consumer ML market vs graphics/gaming market didn't seem to make it worth their effort, but as NVidia has shown this is a path to gaining much more lucrative data center wins.

Re: VUDA: A Vulkan Implementation of CUDA

#64

Post by graphe »

i80and wrote: post #62
causality0 wrote: post #59

Earlier quoted context omitted.

Don't forget AMD doesn't seem to even care about ROCm themselves. Six months in and RDNA3 cards still don't support it. Can you imagine if Nvidia launched RTX40- cards with no DLSS even though 30- cards already had it, and six months started boasting about how DLSS support was "coming this fall"?

I've been running PyTorch on my Radeon 7900 XT using ROCm. Is that not supposed to work?

No, it actually isn't supposed to work, it's not officially supported. https://sep5.readthedocs.io/en/latest/Installation_Guide/Ins...

Re: VUDA: A Vulkan Implementation of CUDA

#65

Post by einpoklum »

rootw0rm wrote: post #58
einpoklum wrote: post #36

1. This implements the clunky C-ish API; there's also the Modern-C++ API wrappers, with automatic error checking, RAII resource control etc.; see: https://github.com/eyalroz/cuda-api-wrappers (due disclosure: I'm the author) 2. Implementing the _runtime_ API is not the right choice; it's important to implement the _driver_ API, otherwise you can't isolate contexts, dynamically add newly-compiled JIT kernels via modul…

nice project. this is why HN kicks ass

Thanks for the compliment :-)

What I _really_ like to receive, though, is feedback from using the wrappers, ideas for changes/improvements, and of course messages volunteering to QA new versions before their release :-P

Re: VUDA: A Vulkan Implementation of CUDA

#66

Post by i80and »

graphe wrote: post #64
i80and wrote: post #62

Earlier quoted context omitted.

I've been running PyTorch on my Radeon 7900 XT using ROCm. Is that not supposed to work?

No, it actually isn't supposed to work, it's not officially supported. https://sep5.readthedocs.io/en/latest/Installation_Guide/Ins...

Fascinating. And yet.

Re: VUDA: A Vulkan Implementation of CUDA

#68

Post by slavik81 »

graphe wrote: post #64
i80and wrote: post #62

Earlier quoted context omitted.

I've been running PyTorch on my Radeon 7900 XT using ROCm. Is that not supposed to work?

No, it actually isn't supposed to work, it's not officially supported. https://sep5.readthedocs.io/en/latest/Installation_Guide/Ins...

The hardware that is officially supported is a subset of the hardware that works. You are correct that the RX 7900 XT is not officially supported, but I must point out that you are linking to a fork of the documentation from 2019. This is the official ROCm documentation: https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h...

Re: VUDA: A Vulkan Implementation of CUDA

#70

Post by JoshTriplett »

paulmd wrote: post #56
sanxiyn wrote: post #34

Earlier quoted context omitted.

Eh well, it is very close to just working. From "Training LLMs with AMD MI250 GPUs and MosaicML": > It all just works. No code changes were needed. https://www.mosaicml.com/blog/amd-mi250

“Just works” in this context means executing the compiled CUDA or the PTX bytecode without recompiling. Nobody is ever going to utilize ROCm if it requires distributing as source and recompiling. To make it even more insulting, even simply installing ROCm itself is a massive burden, even on an ostensibly-supported (as geohot discovered) and even just “it works out of the box if you distribute and compile it locally”…

> “Just works” in this context means executing the compiled CUDA or the PTX bytecode without recompiling. Nobody is ever going to utilize ROCm if it requires distributing as source and recompiling.

Even a source-compatible layer that let you just recompile CUDA code for an AMD GPU would be a huge improvement. That alone would eliminate the CUDA lock-in.

Post reply on HN