VUDA: A Vulkan Implementation of CUDA • Hacker News Board

Re: VUDA: A Vulkan Implementation of CUDA

#21

Post by syntaxers » Sat, Jul 01, 2023, 2:05 PM UTC

DrNosferatu wrote: post #5

Sounds great! How far is it actually compatible right now? Are there any tests / benchmarks? Can this be used to run CUDA-accelerated LLMs?

> > VUDA only takes kernels in SPIR-V format. VUDA does not provide any support for compiling CUDA C kernels directly to SPIR-V (yet). However, it does not know or care how the SPIR-V source was created - may it be GLSL, HLSL, OpenCL.

So the answer is no, it can't be used with kernels that use cublas or cudnn, which excludes almost all ML use-cases.

Re: VUDA: A Vulkan Implementation of CUDA

#22

Post by empyrrhicist » Sat, Jul 01, 2023, 2:21 PM UTC

sheepscreek wrote: post #19

fancyfredbot wrote: post #17

It's not an implementation of CUDA, it's an implementation of the CUDA runtime API. The API is used to configure the card, allocate and copy memory, and run kernels. Importantly you cannot use this to write the actual kernels which run on the GPU!

I was half hoping this meant running CUDA code on AMD GPUs. Thanks for clarifying.

I know AMD has a whole bunch of (related?) projects for GPU compute, but man - if they could just provide an interop layer that Just Works they'd get immediate access to so much more market share.

Re: VUDA: A Vulkan Implementation of CUDA

#23

Post by qwertox » Sat, Jul 01, 2023, 2:28 PM UTC

wtcactus wrote: post #10

Seems dead. Last commit was February 2022.

And that was only one line added. Most of the code is 3 and 5 years old.

Re: VUDA: A Vulkan Implementation of CUDA

#24

Post by roenxi » Sat, Jul 01, 2023, 2:33 PM UTC

nightowl_games wrote: post #4

How does this relate to the goals outlined by George Hotz to bring ML to AMD chips and break the Nvidia dominance? I'm not an expert here but this approach seems powerful and important. But this system seems complex enough to doubt the ability of an individual to build. It seems like this would need a corporate sponsor to get off the ground. Perhaps AMD itself would be interested in paying engineers to iterate on thi…

Hmm. I found

https://www.youtube.com/watch?v=Mr0rWJhv9jU

and

https://geohot.github.io/blog/jekyll/update/2023/06/07/a-div...

I feel a lot better about my journey with AMD now; there seemed to be some major issues with their GPU drivers. Now I know it wasn't just me.

Re: VUDA: A Vulkan Implementation of CUDA

#25

Post by themoonisachees » Sat, Jul 01, 2023, 2:33 PM UTC

flykespice wrote: post #16

AndrewKemendo wrote: post #7

This could be a big deal if we have an actual alternative to CUDA I can’t see NVIDIA letting this just exist

And on what legal ground would NVIDIA have to take this down?

When you have Nvidia money you don't need grounds to sue, the lawyers will think of something and drag any open-source devs through years-long suits.

The only saving grace would be Oracle v Google which established the de jure that an API isn't copyrightable.

Re: VUDA: A Vulkan Implementation of CUDA

#26

Post by iforgotpassword » Sat, Jul 01, 2023, 2:34 PM UTC

jitl wrote: post #11

nightowl_games wrote: post #4

How does this relate to the goals outlined by George Hotz to bring ML to AMD chips and break the Nvidia dominance? I'm not an expert here but this approach seems powerful and important. But this system seems complex enough to doubt the ability of an individual to build. It seems like this would need a corporate sponsor to get off the ground. Perhaps AMD itself would be interested in paying engineers to iterate on thi…

Holtz is talking about drivers too, not just user space libraries. > The software is terrible! There’s kernel panics in the driver. You have to run a newer kernel than the Ubuntu default to make it remotely stable. I’m still not sure if the driver supports putting two cards in one machine, or if there’s some poorly written global state. When I put the second card in and run an OpenCL program, half the time it kernel…

This is still so mind-boggling to me. AMD should be in a good financial position now that Zen was such a success, and that their GPUs are catching up too. Why are their drivers still a Clusterfuck across the board after all these years? Why not throw more manpower at the problem?

I'm sure even if their GPUs were twice as fast as Nvidia's, everybody would still buy team green because it's better to have a card that works than a broken piece of garbage. We tried to get an MI50 to work reliably at work with KVM, but that thing was a complete dumpster fire. A colleague just bought a 7900XTX for gaming and spent days getting it to work. This included three Windows reinstalls. And that use case is gaming on Windows, which supposedly is the best supported case. It only gets worse from there. Compute on Linux? Lol.

Now last time this topic came up, someone claimed that AMD is pretty much at their limits production wise, and there are a few unnamed large companies buying loads of their cards for compute and cloud gaming, and AMD basically has engineers dedicated to making sure things work exactly for their use case, so they don't have to really care about the rest. Sounds pretty wild, but not completely unrealistic...

Re: VUDA: A Vulkan Implementation of CUDA

#27

Post by hedora » Sat, Jul 01, 2023, 2:43 PM UTC

iforgotpassword wrote: post #26

jitl wrote: post #11

Earlier quoted context omitted.

Holtz is talking about drivers too, not just user space libraries. > The software is terrible! There’s kernel panics in the driver. You have to run a newer kernel than the Ubuntu default to make it remotely stable. I’m still not sure if the driver supports putting two cards in one machine, or if there’s some poorly written global state. When I put the second card in and run an OpenCL program, half the time it kernel…

This is still so mind-boggling to me. AMD should be in a good financial position now that Zen was such a success, and that their GPUs are catching up too. Why are their drivers still a Clusterfuck across the board after all these years? Why not throw more manpower at the problem? I'm sure even if their GPUs were twice as fast as Nvidia's, everybody would still buy team green because it's better to have a card that wo…

I've had the opposite experience under Linux.

My old nvidia 570's drivers went into severe bitrot. Basic stuff like screensavers and desktops broke badly, and games were flaky. The card is still more than powerful enough for what I used it for.

I switched to AMD, with open source drivers. I get windows-level performance on AAA (and indie) games in steam, and zero compatibility issues with the rest of the Linux ecosystem.

Re: VUDA: A Vulkan Implementation of CUDA

#28

Post by arthur2e5 » Sat, Jul 01, 2023, 2:43 PM UTC

cwillu wrote: post #20

jitl wrote: post #11

Earlier quoted context omitted.

Holtz is talking about drivers too, not just user space libraries. > The software is terrible! There’s kernel panics in the driver. You have to run a newer kernel than the Ubuntu default to make it remotely stable. I’m still not sure if the driver supports putting two cards in one machine, or if there’s some poorly written global state. When I put the second card in and run an OpenCL program, half the time it kernel…

Par for the course with new kernel things: it's unusual for something new in the kernel to be stable in the distro kernels unless they've devoted a great deal of effort to backport things.

The way conservative distros define "stable" is part of the problem. For things less than 3 years old, going for stale versions often runs counter to "stable".

Re: VUDA: A Vulkan Implementation of CUDA

#29

Post by heyoni » Sat, Jul 01, 2023, 2:44 PM UTC

empyrrhicist wrote: post #22

sheepscreek wrote: post #19

Earlier quoted context omitted.

I was half hoping this meant running CUDA code on AMD GPUs. Thanks for clarifying.

I know AMD has a whole bunch of (related?) projects for GPU compute, but man - if they could just provide an interop layer that Just Works they'd get immediate access to so much more market share.

Then they too could call themselves an AI company!

Re: VUDA: A Vulkan Implementation of CUDA

#30

Post by netheril96 » Sat, Jul 01, 2023, 2:46 PM UTC

Just in case other people who have AMD GPU and run Windows have the same needs as I have, that is, to train or run machine learning models, please checkout torch-directml and tensorflow-directml.