Can you explain why you did the naive algorithm here and not any of the fast matrix multiplication ones that trade multiplications for more additions? Just for educational purposes or is there a performance benefit in the technique?
Optimizing a WebGPU Matmul Kernel for 1 TFLOP
21–30 of 88 posts
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#22Can you explain why you did the naive algorithm here and not any of the fast matrix multiplication ones that trade multiplications for more additions? Just for educational purposes or is there a performance benefit in the technique?
[deleted]
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#23Earlier quoted context omitted.
> The smoothness of an iPhone map zoom, on any device. Any device except an iPhone, until Apple finally gets around to shipping WebGPU in Safari. Any year now...
Safari is officially enabling support for WebGPU in iOS 18.2, which is rolling out within the first weeks of December.
Where'd you hear that? It's not listed here:
https://developer.apple.com/documentation/safari-release-not...
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#24Great article! For context: this WebGPU version achieves ~17% of peak theoretical performance of M2. With CUDA (i.e. CuBLAS), you can reach ~75% of peak performance for same matrix config (without tensor core).
[deleted]
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#25Great article! For context: this WebGPU version achieves ~17% of peak theoretical performance of M2. With CUDA (i.e. CuBLAS), you can reach ~75% of peak performance for same matrix config (without tensor core).
75% can't be the best we can do. What would reach 100% or nearly 100%? Handcoded assembly?
The parameters of the matrix multiply, such as the size of the matrices, impose some limits to how close you can get to the peak theoretical performance in a particular GPU. Not all possible matrix multiplies are equally valuable to optimize a priori, so the hardware is designed to perform best on problems that are financially significant, such as modern LLMs.
As for handcoded assembly, do you believe that it would be financially sound to hand code and maintain thousands of kernels that way, even if you believed that they would be faster?
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#26Earlier quoted context omitted.
> The smoothness of an iPhone map zoom, on any device. Any device except an iPhone, until Apple finally gets around to shipping WebGPU in Safari. Any year now...
Safari is officially enabling support for WebGPU in iOS 18.2, which is rolling out within the first weeks of December.
Source?
Edit: I just pressed “Reset All to Defaults” under “WebKit Feature Flags” on my device running 18.2 beta, and the switch for WebGPU is on!! <3
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#27Can you explain why you did the naive algorithm here and not any of the fast matrix multiplication ones that trade multiplications for more additions? Just for educational purposes or is there a performance benefit in the technique?
at least on my m2, the compiled kernel ends up using fast math anyways so using WGSL's fma didn't change anything about the actual kernel that gets run
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#28Earlier quoted context omitted.
Safari is officially enabling support for WebGPU in iOS 18.2, which is rolling out within the first weeks of December.
Where'd you hear that? It's not listed here: https://developer.apple.com/documentation/safari-release-not...
Source is here, from a Unity WebGPU thread. Look at the comment from October 27 from Brendan Duncan, a Unity employee: https://discussions.unity.com/t/early-access-to-the-new-webg...
"I have found that WebGPU is enabled by default now with iOS 18.2. Apple has been working in the open on WebGPU. The WebKit source code has their latest WebGPU work in it. What hasn’t been known is their release schedule, but now with 18.2 it’s looking very promising that it will be on by default in that version."
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#29To clarify the title: TFLOP/s is the unit the author goes after, not TFLOP. People in the threads compare CUDA performance on GPUs to WebAssembly performance: please recall that H100 has a theoretical performance of about 1000 TFLOP/s for bfloat16, and even moderately complicated algorithms in typical modern transformer architectures can reach about half of that performance.
Re: Optimizing a WebGPU Matmul Kernel for 1 TFLOP
#30I wrote something similar a while back: https://github.com/FL33TW00D/wgpu-mm
Also does quantized matmuls.