Where is the Metal love…
Why would anyone love something born out of pure spite for industry standards?
To be fair, the industry standards all suck except for CUDA.
CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
21–30 of 45 posts
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#22I'd recommend having a "gemm with a twist" [0] example in the README.md instead of having an element-wise example. It's pretty hard to evaluate how helpful this is for AI otherwise. [0] For example, gemm but the lhs is in fp8 e4m3 and rhs is in bf16 and we want fp32 accumulation, output to bf16 after applying GELU.
Agreed! I was looking through the summation example https://github.com/tracel-ai/cubecl/blob/main/examples/sum_t...> and it seems like the primary focus is on the more traditional pre-2018 GPU programming without explicit warp-level operations, asynchrony, atomics, barriers, or countless tensor-core operations.
The project feels very nice and it would be great to have more notes in the README on the excluded functionality to better scope its applicability in more advanced GPGPU scenarios.
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#23Earlier quoted context omitted.
For the same reason CUDA and ROCm are supported.
Apple is known to be not that great contributor to open source, unlike Nvidia, AMD, Intel.
You should check Linus opinion on those.
Also, to whom do you have to thank LLVM exists in first place, and has not fizzled out as yet another university compiler research project?
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#24Earlier quoted context omitted.
It also compiles directly to MSL, it is just missing from the post title.
No it compiles indirectly through wgpu, which means it doesn’t have access to any Metal extensions not exposed by the wgpu interface.
I am the coder of the MSL dialect for the CubeCL CPP compiler. Since 0.5 release it directly compiles to MSL and support simdgroup matrix functions for instance. It does use wgpu for the runtime but without naga as we added msl pass through to wgpu just for this.
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#25Praying to the kernel gods for some Rust FP8 training
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#26I'd recommend having a "gemm with a twist" [0] example in the README.md instead of having an element-wise example. It's pretty hard to evaluate how helpful this is for AI otherwise. [0] For example, gemm but the lhs is in fp8 e4m3 and rhs is in bf16 and we want fp32 accumulation, output to bf16 after applying GELU.
Agreed! I was looking through the summation example https://github.com/tracel-ai/cubecl/blob/main/examples/sum_t... > and it seems like the primary focus is on the more traditional pre-2018 GPU programming without explicit warp-level operations, asynchrony, atomics, barriers, or countless tensor-core operations. The project feels very nice and it would be great to have more notes in the README on the excluded functio…
CubeCL is the computation backend for Burn (https://burn.dev/) - ML framework done by the same team which does all the tensor magic like autodiff, op fusion and dynamic graphs.
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#27Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#28Earlier quoted context omitted.
It also compiles directly to MSL, it is just missing from the post title.
No it compiles indirectly through wgpu, which means it doesn’t have access to any Metal extensions not exposed by the wgpu interface.
wgpu has some options to access backend-specific types and shader passthrough (i.e., you provide your own shader for a backend directly).
Generally wgpu is open to supporting any Metal extensions you need. There's usually an analogous extension in one of the other backends (e.g., Vulkan, DX12) anyway.
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#29wow, what's the downsides to this? It feels like it could be one of the biggest leaps in programming in a long time, does it keep rusts safety aspects? How does it compare with say openCL?
Re: CubeCL: GPU Kernels in Rust for CUDA, ROCm, and WGPU
#30I'd recommend having a "gemm with a twist" [0] example in the README.md instead of having an element-wise example. It's pretty hard to evaluate how helpful this is for AI otherwise. [0] For example, gemm but the lhs is in fp8 e4m3 and rhs is in bf16 and we want fp32 accumulation, output to bf16 after applying GELU.
One of the main author here, the readme isn't really well up-to-date. We have our own gemm implementation based on CubeCL. It's still moving a lot, but we support tensor cores, use warp operations (Plane Operations in CubeCL), we even added TMA instructions for CUDA.