Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

github.com

Post reply on HN
Print view View original

31–40 of 45 posts

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#31

Post by nathanielsimard »

ashvardanian wrote: post #22
rfoo wrote: post #20

I'd recommend having a "gemm with a twist" [0] example in the README.md instead of having an element-wise example. It's pretty hard to evaluate how helpful this is for AI otherwise. [0] For example, gemm but the lhs is in fp8 e4m3 and rhs is in bf16 and we want fp32 accumulation, output to bf16 after applying GELU.

Agreed! I was looking through the summation example https://github.com/tracel-ai/cubecl/blob/main/examples/sum_t... > and it seems like the primary focus is on the more traditional pre-2018 GPU programming without explicit warp-level operations, asynchrony, atomics, barriers, or countless tensor-core operations. The project feels very nice and it would be great to have more notes in the README on the excluded functio…

We support warp operations, barriers for Cuda, atomics for most backends, tensor cores instructions as well. It's just not well documented on the readme!

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#32

Post by nathanielsimard »

gitroom wrote: post #16

Gotta say, the constant dance between all these GPU frameworks kinda wears me out sometimes - always chasing that better build, you know?

The need to build CubeCL came from the Burn deep learning framework (https://github.com/tracel-ai/burn), where we want to easily build algorithms like in CUDA with a real programming language, while also being able to integrate those algorithms inside a compiler at runtime to fuse dynamic graphs.

Since we don't want to rewrite everything multiple times, it also has to be multi-platform and optimal, so the feature set must be per-device, not per-language. I'm not aware of a tool that does that, especially in Rust (which Burn is written in).

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#33

Post by wingertge »

rfoo wrote: post #20

I'd recommend having a "gemm with a twist" [0] example in the README.md instead of having an element-wise example. It's pretty hard to evaluate how helpful this is for AI otherwise. [0] For example, gemm but the lhs is in fp8 e4m3 and rhs is in bf16 and we want fp32 accumulation, output to bf16 after applying GELU.

We don't yet support newer types like fp8 and fp4, that's actually my next project. I'm the only contributor with the hardware to actually use the new types, so it's a bit bottlenecked on a single person right now. But yes, the example is rather simplistic, should probably work on that some time once I'm done updating the feature set to Blackwell.

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#34

Post by ttoinou »

fc417fc802 wrote: post #14
zekrioca wrote: post #2

Very interesting project! I am wondering how it compare against OpenCL, which I think adopts the same fundamental idea (write once, run everywhere)? Is it about CUbeCL's internal optimization for Rust that happens at compile time?

This appears to be single source which would make it similar to SYCL. Given that it can target WGPU I'm really wondering why OpenCL isn't included as a backend. One of my biggest complaints about GPGPU stuff is that so many of the solutions are GPU only, and often only target the vendor compute APIs (CUDA, ROCm) which have much narrower ecosystem support (versus an older core vulkan profile for example). It's desirab…

Who would use the OpenCL backend rather than the others targets provided ?

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#35

Post by wingertge »

fc417fc802 wrote: post #14
zekrioca wrote: post #2

Very interesting project! I am wondering how it compare against OpenCL, which I think adopts the same fundamental idea (write once, run everywhere)? Is it about CUbeCL's internal optimization for Rust that happens at compile time?

This appears to be single source which would make it similar to SYCL. Given that it can target WGPU I'm really wondering why OpenCL isn't included as a backend. One of my biggest complaints about GPGPU stuff is that so many of the solutions are GPU only, and often only target the vendor compute APIs (CUDA, ROCm) which have much narrower ecosystem support (versus an older core vulkan profile for example). It's desirab…

There's infrastructure in the SPIR-V compiler to be able to target both OpenCL and Vulkan, but we don't currently use it because OpenCL would require a new runtime, while Vulkan can simply use the existing wgpu runtime and pass raw SPIR-V shaders.

One thing I've never investigated is how performance OpenCL actually is for CPU. Do you happen to have any resources comparing it to a more native CPU implementation?

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#36

Post by adastra22 »

syl20bnr wrote: post #24
adastra22 wrote: post #12

Earlier quoted context omitted.

No it compiles indirectly through wgpu, which means it doesn’t have access to any Metal extensions not exposed by the wgpu interface.

I am the coder of the MSL dialect for the CubeCL CPP compiler. Since 0.5 release it directly compiles to MSL and support simdgroup matrix functions for instance. It does use wgpu for the runtime but without naga as we added msl pass through to wgpu just for this.

You should update the README.

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#38

Post by nathanielsimard »

DarkmSparks wrote: post #29

wow, what's the downsides to this? It feels like it could be one of the biggest leaps in programming in a long time, does it keep rusts safety aspects? How does it compare with say openCL?

We have safe and unsafe version for launching kernels where we can ensure that a kernel won't corrupt data elsewhere (and therefore won't create memory error or segfaults). But within a kernel ressources are mutable and shared between GPU cores, since that's how GPUs work.

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#39

Post by syl20bnr »

adastra22 wrote: post #36
syl20bnr wrote: post #24

Earlier quoted context omitted.

I am the coder of the MSL dialect for the CubeCL CPP compiler. Since 0.5 release it directly compiles to MSL and support simdgroup matrix functions for instance. It does use wgpu for the runtime but without naga as we added msl pass through to wgpu just for this.

You should update the README.

You are right we just released Burn and updated its readme, we were not thinking that CubeCL would be the one that could be featured. ^_^

Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU

#40

Post by lostmsu »

wingertge wrote: post #33
rfoo wrote: post #20

I'd recommend having a "gemm with a twist" [0] example in the README.md instead of having an element-wise example. It's pretty hard to evaluate how helpful this is for AI otherwise. [0] For example, gemm but the lhs is in fp8 e4m3 and rhs is in bf16 and we want fp32 accumulation, output to bf16 after applying GELU.

We don't yet support newer types like fp8 and fp4, that's actually my next project. I'm the only contributor with the hardware to actually use the new types, so it's a bit bottlenecked on a single person right now. But yes, the example is rather simplistic, should probably work on that some time once I'm done updating the feature set to Blackwell.

Isn't there a CPU-based "emulator" in Nvidia dev tools?
Post reply on HN