Matmul on Blackwell: Part 2 – Using Hardware Features to Optimize Matmul • Hacker News Board

Re: Matmul on Blackwell: Part 2 – Using Hardware Features to Optimize Matmul

#11

Post by totalperspectiv » Sun, Sep 07, 2025, 2:58 PM UTC

subharmonicon wrote: post #9

totalperspectiv wrote: post #5

Earlier quoted context omitted.

I don’t follow your logic. Mojo can target multiple gpu vendors. What is the Modular specific lock in?

The blog post is about using an NVIDIA-specific tensor core API that they have built to get good performance. Modular has been pushing the notion that they are building technology that allows writing HW-vendor neutral solutions so that users can break free of NVIDIA's hold on high performance kernels. From their own writing: > We want a unified, programmable system (one small binary!) that can scale across architectu…

They allow you to write a kernel for Nvidia, or AMD, that can take full advantage of the Hardware of either one, then throw a compile time if-statement in there to switch which kernel to use based on the hardware available.

So, you can support either vendor with as-good-vendor-library performance. That’s not lock-in to me at least.

It’s not as good as the compiler being able to just magically produce optimized kernels for arbitrary hardware though, fully agree there. But it’s a big step forward from Cuda/HIP.

Re: Matmul on Blackwell: Part 2 – Using Hardware Features to Optimize Matmul

#12

Post by totalperspectiv » Sun, Sep 07, 2025, 2:59 PM UTC

whimsicalism wrote: post #8

smilekzs wrote: post #7

Earlier quoted context omitted.

Not OP but I think this could be an instance of leaky abstraction at work. Most of the time you hand-write an accelerator kernel hoping to optimize for runtime performance. If the abstraction/compiler does not fully insulate you from micro-architectural details affecting performance in non-trivial ways (e.g. memory bank conflict as mentioned in the article) then you end up still having per-vendor implementations, or…

Yes, it looks like they have some sort of metaprogramming setup (nicer than C++) for doing this: https://www.modular.com/mojo

I can confirm, it’s quite nice.

Re: Matmul on Blackwell: Part 2 – Using Hardware Features to Optimize Matmul

#13

Post by whimsicalism » Sun, Sep 07, 2025, 5:56 PM UTC

totalperspectiv wrote: post #12

whimsicalism wrote: post #8

Earlier quoted context omitted.

Yes, it looks like they have some sort of metaprogramming setup (nicer than C++) for doing this: https://www.modular.com/mojo

I can confirm, it’s quite nice.

jw: why do you use mojo here over triton or the new pythonic cute/cutlass?

Re: Matmul on Blackwell: Part 2 – Using Hardware Features to Optimize Matmul

#14

Post by totalperspectiv » Mon, Sep 08, 2025, 1:22 PM UTC

whimsicalism wrote: post #13

totalperspectiv wrote: post #12

Earlier quoted context omitted.

I can confirm, it’s quite nice.

jw: why do you use mojo here over triton or the new pythonic cute/cutlass?

Because I was originally writing some very CPU intensive SIMD stuff, which Mojo is also fantastic for. Once I got that working and running nicely I decided to try getting the same algo running on GPU since, at the time, they had just open sourced the GPU parts of the stdlib. It was really easy to get going with.

I have not used Triton/Cute/Cutlass though, so I can't compare against anything other than Cuda really.