Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

VUDA: A Vulkan Implementation of CUDA

github.com

Post reply on HN
Print view View original

11–20 of 83 posts

Re: VUDA: A Vulkan Implementation of CUDA

#11

Post by jitl »

nightowl_games wrote: post #4

How does this relate to the goals outlined by George Hotz to bring ML to AMD chips and break the Nvidia dominance? I'm not an expert here but this approach seems powerful and important. But this system seems complex enough to doubt the ability of an individual to build. It seems like this would need a corporate sponsor to get off the ground. Perhaps AMD itself would be interested in paying engineers to iterate on thi…

Holtz is talking about drivers too, not just user space libraries.

> The software is terrible! There’s kernel panics in the driver. You have to run a newer kernel than the Ubuntu default to make it remotely stable. I’m still not sure if the driver supports putting two cards in one machine, or if there’s some poorly written global state. When I put the second card in and run an OpenCL program, half the time it kernel panics and you have to reboot.

He also talks about user space stuff but clearly he thinks the whole stack, above and below this kind of library also needs a lot of work.

Re: VUDA: A Vulkan Implementation of CUDA

#12

Post by roenxi »

Ballas wrote: post #8
AndrewKemendo wrote: post #7

This could be a big deal if we have an actual alternative to CUDA I can’t see NVIDIA letting this just exist

My opinion is that CUDA is not the mote keeping the others out - it's the CUDNN (and CUBLAS), more specifically the level to which they are optimized.

I'm not even certain optimisation matters. I can crash my machine (AMD graphics) with a stock Debian install by letting something attempt BLAS on the GPU.

The situation is starting to improve though. Installed a bunch of libraries from https://repo.radeon.com/rocm/apt/5.4 jammy main and the crashes got less frequent. I don't have a lot of faith in AMD to deliver reliable BLAS libraries at this point, but it could happen. The hardware is there, I just don't think they're prioritising supporting the right places in the distribution chain or supporting consumer-level graphics.

Re: VUDA: A Vulkan Implementation of CUDA

#13

Post by westurner »

josalhor wrote: post #3

As someone who has never programmed directly for a GPU, how does this compare to HIP? Can this be an efficient abstraction over Nvidia and AMD GPUs?

From https://news.ycombinator.com/item?id=34399633 :

>>> hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced. [...]

(Edit) CUDA APIs supported by hipify-clang: https://rocm.docs.amd.com/projects/HIPIFY/en/latest/supporte...

Re: VUDA: A Vulkan Implementation of CUDA

#14

Post by johndough »

DrNosferatu wrote: post #5

Sounds great! How far is it actually compatible right now? Are there any tests / benchmarks? Can this be used to run CUDA-accelerated LLMs?

The Vulkan spec does not enforce strict IEEE 754 floating point semantics, so perfect compatibility with CUDA is impossible.

https://registry.khronos.org/vulkan/specs/1.3-khr-extensions...

However, the deep learning field does currently not pay much attention to reproducibility, so this might not be a big issue.

Re: VUDA: A Vulkan Implementation of CUDA

#18

Post by my123 »

johndough wrote: post #14
DrNosferatu wrote: post #5

Sounds great! How far is it actually compatible right now? Are there any tests / benchmarks? Can this be used to run CUDA-accelerated LLMs?

The Vulkan spec does not enforce strict IEEE 754 floating point semantics, so perfect compatibility with CUDA is impossible. https://registry.khronos.org/vulkan/specs/1.3-khr-extensions... However, the deep learning field does currently not pay much attention to reproducibility, so this might not be a big issue.

Memory management in Vulkan is _very_ restricted. Nothing remotely like UVM. CUDA on Vulkan for these reasons will always stay a pet project at best, with no shot at usable quality whatsoever.

Re: VUDA: A Vulkan Implementation of CUDA

#19

Post by sheepscreek »

fancyfredbot wrote: post #17

It's not an implementation of CUDA, it's an implementation of the CUDA runtime API. The API is used to configure the card, allocate and copy memory, and run kernels. Importantly you cannot use this to write the actual kernels which run on the GPU!

I was half hoping this meant running CUDA code on AMD GPUs. Thanks for clarifying.

Re: VUDA: A Vulkan Implementation of CUDA

#20

Post by cwillu »

jitl wrote: post #11
nightowl_games wrote: post #4

How does this relate to the goals outlined by George Hotz to bring ML to AMD chips and break the Nvidia dominance? I'm not an expert here but this approach seems powerful and important. But this system seems complex enough to doubt the ability of an individual to build. It seems like this would need a corporate sponsor to get off the ground. Perhaps AMD itself would be interested in paying engineers to iterate on thi…

Holtz is talking about drivers too, not just user space libraries. > The software is terrible! There’s kernel panics in the driver. You have to run a newer kernel than the Ubuntu default to make it remotely stable. I’m still not sure if the driver supports putting two cards in one machine, or if there’s some poorly written global state. When I put the second card in and run an OpenCL program, half the time it kernel…

Par for the course with new kernel things: it's unusual for something new in the kernel to be stable in the distro kernels unless they've devoted a great deal of effort to backport things.
Post reply on HN