[dead]
Rust running on every GPU
151–160 of 211 posts
Re: Rust running on every GPU
#152This is a little crude still, but the fact that this is even possible is mind blowing. This has the potential, if progress continues, to break the vendor-locked nightmare that is GPU software and open up the space to real competition between hardware vendors. Imagine a world where machine learning models are written in Rust and can run on both Nvidia and AMD. To get max performance you likely have to break the abstra…
You might be interested in https://burn.dev, a Rust machine learning framework. It has CUDA and ROCm backends among others.
Re: Rust running on every GPU
#153Earlier quoted context omitted.
"Folks" as-in Rust stans, whom know very little about CUDA and what makes it nice in the first place, sure, but is there demand for Rust ports amongst actual CUDA programmers? I think not.
FYI, rust-cuda outputs nvvm so it can integrate with the existing cuda ecosystem. We aren't suggesting rewriting everything in Rust. Check the repo for crates that allow using existing stuff like cudnn and cuBLAS.
I take it you're the maintainer. Firstly, congrats on the work done, for the open source people are a small crowd, and determination of Rust teams here is commendable. On the other hand, I'm struggling to see the unique value proposition. What is your motivation with Rust-GPU? Graphics or general-purpose computing? If it's the latter, at least from my POV, I would struggle to justify going up against a daunting umbrella project like this; in view of it likely culminating in layers upon layers of abstraction. Is the long-term goal here to have fun writing a bit of Rust, or upsetting the entrenched status quo of highly-concurrent GPU programming? There's a saying that goes along like "pleasing all is a lot like pleasing none," and intuitively I would guess it should apply here.
Re: Rust running on every GPU
#154This is amazing. With time, we should be able to write GPU programs semantically identical to user land programs. The implications of this for inference is going to be huge.
CUDA programming consists of making a kernel parameterized by its thread id which is used to slightly alter its behavior while it executes on many hundreds of GPU cores; it's very different than general purpose programming. Memory and branching behave differently there. I'd say at best, it will be like traditional programs and libraries with multiple incompatible event loops.
Re: Rust running on every GPU
#155This is a little crude still, but the fact that this is even possible is mind blowing. This has the potential, if progress continues, to break the vendor-locked nightmare that is GPU software and open up the space to real competition between hardware vendors. Imagine a world where machine learning models are written in Rust and can run on both Nvidia and AMD. To get max performance you likely have to break the abstra…
You might be interested in https://burn.dev , a Rust machine learning framework. It has CUDA and ROCm backends among others.
I am interested, thanks for sharing!
Re: Rust running on every GPU
#156Earlier quoted context omitted.
First of all WebGPU has only been supported in Chrome for a few months and Firefox in the next release. And that's just Windows. We haven't had enough time to develop anything really. Secondly, the WebGPU standard is like Vulkan 1.0 and is cumbersome to work with. But that part is hearsay, I don't have much experience with it.
gpu is often cumbesome tho. i mean, openGL, Vulkan, they are not really trivial?
OpenGL is trivial compared to Vulkan. And apparently Vulkan has gotten much easier today compared to its initial release in 2015
Re: Rust running on every GPU
#157Earlier quoted context omitted.
FYI, rust-cuda outputs nvvm so it can integrate with the existing cuda ecosystem. We aren't suggesting rewriting everything in Rust. Check the repo for crates that allow using existing stuff like cudnn and cuBLAS.
I take it you're the maintainer. Firstly, congrats on the work done, for the open source people are a small crowd, and determination of Rust teams here is commendable. On the other hand, I'm struggling to see the unique value proposition. What is your motivation with Rust-GPU? Graphics or general-purpose computing? If it's the latter, at least from my POV, I would struggle to justify going up against a daunting umbre…
Thanks! I'm personally focused on compute, while other contributors are focused on graphics.
I believe GPUs are the future of computing. I think the tooling, languages, and ecosystems of GPUs are very bad compared to CPUs. Partially because they are newer, partially because they are different, and partially because for some reason the expectations are so low. So I intend to upset the status quo.
Re: Rust running on every GPU
#158If you're like me you want to write code that runs on the GPU but you don't, because everything about programming GPUs is pain. That's the use-case I see for rust-gpu. Make CPU devs into GPU devs with an acceptable performance penalty. If you're already a GPU programmer and know cuda/vulkan/metal/dx inside out then something like this will not be interesting to you.
Re: Rust running on every GPU
#159Earlier quoted context omitted.
The performance purist don't use Cuda either though (that's why Deepseek used PTX directly). Everything is an abstraction and choosing the right level of abstraction for your usecase is a tradeoff between your engineering capacities and your performance needs.
The issue in my mind is that this doesn’t seem to include any of the critical library functionality specific eg to NVIDIA cards, think reduction operations across threads in a warp and similar. Some of those don’t exist in all hardware architectures. We may get to a point where everything could be written in one language but actually leveraging the hardware correctly still requires a bunch of different implementation…
The features missing hardware support can fall back to software implementations.
In any case, ideally, the level of abstraction would be higher, with little application logic requiring GPU architecture awareness.
Re: Rust running on every GPU
#160This is amazing and there is already a pretty stacked list of Rust GPU projects. This seems to be at an even lower level of abstraction than burn[0] which is lower than candle[1]. I gueds whats left is to add backend(s) that leverage naga and others to the above projects? Feeks like everyone is building on different bases here, though I know the naga work is relatively new. [EDIT] Just to note, burn is the one that f…
You can check out https://rust-gpu.github.io/ecosystem/ as well, which mentions CubeCL.
Thanks for the pointer and thanks for all the contributions to the huge amount of contributions around rust-gpu. Outstanding work