CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU • Hacker News Board

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#41

Post by wingertge » Thu, Apr 24, 2025, 7:41 PM UTC

lostmsu wrote: post #40

wingertge wrote: post #33

Earlier quoted context omitted.

We don't yet support newer types like fp8 and fp4, that's actually my next project. I'm the only contributor with the hardware to actually use the new types, so it's a bit bottlenecked on a single person right now. But yes, the example is rather simplistic, should probably work on that some time once I'm done updating the feature set to Blackwell.

Isn't there a CPU-based "emulator" in Nvidia dev tools?

From what I can tell it's not accurate enough to catch a lot of errors in the real world. Maybe an illegal instruction, but not a race condition from a missing sync or a warp divergence on a uniform instruction or other potential issues like that.

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#42

Post by fc417fc802 » Thu, Apr 24, 2025, 7:57 PM UTC

wingertge wrote: post #35

fc417fc802 wrote: post #14

Earlier quoted context omitted.

This appears to be single source which would make it similar to SYCL. Given that it can target WGPU I'm really wondering why OpenCL isn't included as a backend. One of my biggest complaints about GPGPU stuff is that so many of the solutions are GPU only, and often only target the vendor compute APIs (CUDA, ROCm) which have much narrower ecosystem support (versus an older core vulkan profile for example). It's desirab…

There's infrastructure in the SPIR-V compiler to be able to target both OpenCL and Vulkan, but we don't currently use it because OpenCL would require a new runtime, while Vulkan can simply use the existing wgpu runtime and pass raw SPIR-V shaders. One thing I've never investigated is how performance OpenCL actually is for CPU. Do you happen to have any resources comparing it to a more native CPU implementation?

Sorry my interest there is debugging and I'm not immediately coming across good benchmarks. PoCL [0] seems to have added a TBB backend [1] so I'd expect it to be reasonable (otherwise why bother) but I haven't tested it.

It isn't really related to your question but I think the FluidX3D benchmarks [2] illustrate that OpenCL is at least viable across a wide variety of hardware.

As far as targeting CPUs in a release build it's not a particular backend that's important to me. The issue is at the source code level. Having single source is nice but you're still stuck with these two very different approaches. It means that the code is still clearly segmented and thus retargeting any given task (at least nontrivial ones) involves rewriting it to at least some extent.

Contrast that with a model like OpenMP where the difference between CPU and GPU is marking the relevant segment for offload. Granted that you'll often need to change algorithms when switching to achieve reasonable performance but it's still a really nice quality of life feature not to have to juggle more paradigms and libraries.

[0] https://github.com/pocl/pocl

[1] https://portablecl.org/docs/html/drivers.html

[2] https://github.com/ProjectPhysX/FluidX3D

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#43

Post by ashvardanian » Thu, Apr 24, 2025, 8:27 PM UTC

nathanielsimard wrote: post #31

ashvardanian wrote: post #22

Earlier quoted context omitted.

Agreed! I was looking through the summation example https://github.com/tracel-ai/cubecl/blob/main/examples/sum_t... > and it seems like the primary focus is on the more traditional pre-2018 GPU programming without explicit warp-level operations, asynchrony, atomics, barriers, or countless tensor-core operations. The project feels very nice and it would be great to have more notes in the README on the excluded functio…

We support warp operations, barriers for Cuda, atomics for most backends, tensor cores instructions as well. It's just not well documented on the readme!

Amazing! Would love to try them! If possible, would also ask for a table translating between CubeCL and CUDA terminology. It seems like CUDA Warps are called Planes in CubeCL, and it’s probably not the only difference.

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#44

Post by fc417fc802 » Thu, Apr 24, 2025, 8:35 PM UTC

nathanielsimard wrote: post #32

gitroom wrote: post #16

Gotta say, the constant dance between all these GPU frameworks kinda wears me out sometimes - always chasing that better build, you know?

The need to build CubeCL came from the Burn deep learning framework ( https://github.com/tracel-ai/burn ), where we want to easily build algorithms like in CUDA with a real programming language, while also being able to integrate those algorithms inside a compiler at runtime to fuse dynamic graphs. Since we don't want to rewrite everything multiple times, it also has to be multi-platform and optimal, so the feature s…

> I'm not aware of a tool that does that

Jax? But then you're stuck in python. SYCL?

But yeah not for Rust. This project is filling a prominent hole IMO.

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#45

Post by rowanG077 » Fri, Apr 25, 2025, 5:34 PM UTC

nathanielsimard wrote: post #32

gitroom wrote: post #16

Gotta say, the constant dance between all these GPU frameworks kinda wears me out sometimes - always chasing that better build, you know?

The need to build CubeCL came from the Burn deep learning framework ( https://github.com/tracel-ai/burn ), where we want to easily build algorithms like in CUDA with a real programming language, while also being able to integrate those algorithms inside a compiler at runtime to fuse dynamic graphs. Since we don't want to rewrite everything multiple times, it also has to be multi-platform and optimal, so the feature s…

Futhark immediately came to mind. It's designed to be able to be trivially integrated into a package.