31–40 of 88 posts
Can you explain why you did the naive algorithm here and not any of the fast matrix multiplication ones that trade multiplications for more additions? Just for educational purposes or is there a performance benefit in the technique?
at least on my m2, the compiled kernel ends up using fast math anyways so using WGSL's fma didn't change anything about the actual kernel that gets run
Earlier quoted context omitted.
at least on my m2, the compiled kernel ends up using fast math anyways so using WGSL's fma didn't change anything about the actual kernel that gets run
inglor is probably referring to Strassen or Coppersmith–Winograd.
Great article! For context: this WebGPU version achieves ~17% of peak theoretical performance of M2. With CUDA (i.e. CuBLAS), you can reach ~75% of peak performance for same matrix config (without tensor core).
Not on the same computer, CUDA doesn’t run on the integrated GPU of the Apple M2 Pro.
Earlier quoted context omitted.
75% can't be the best we can do. What would reach 100% or nearly 100%? Handcoded assembly?
With GPUs it's not uncommon to run out of memory bandwidth before you max out the theoretical FLOPS. They may have a ton of bandwidth but it's never enough. That can lead you to some pretty counter-intuitive optimizations because it's often faster to do more compute work if it means you touch less memory in the process.
Earlier quoted context omitted.
at least on my m2, the compiled kernel ends up using fast math anyways so using WGSL's fma didn't change anything about the actual kernel that gets run
inglor is probably referring to Strassen or Coppersmith–Winograd.
How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance (https://siboehm.com/articles/22/CUDA-MMM)
(It's CUDA-specific, so there may be aspects that can't yet be ported to WGPU)
For a very deep dive into the subject, this is a great writeup: How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance ( https://siboehm.com/articles/22/CUDA-MMM ) (It's CUDA-specific, so there may be aspects that can't yet be ported to WGPU)
there are a few things that i wasn't able to figure out how to get access to/i wasn't sure if they were possible. for example, a lot of Simon's article takes advantage of the warp scheduler and warp tiling.
i had a hard time finding information on if that's even possible with my M2/metal and the general memory access patterns. it seems like CUDA does have better documentation in this regard
I wrote something similar a while back: https://github.com/FL33TW00D/wgpu-mm Also does quantized matmuls.
Great article! For context: this WebGPU version achieves ~17% of peak theoretical performance of M2. With CUDA (i.e. CuBLAS), you can reach ~75% of peak performance for same matrix config (without tensor core).