Optimizing Matrix Multiplication on RDNA3

> Furthermore, performing custom ISA optimizations makes these changes RDNA3-specific

this is overblown at least wrt forward compatibility - all of the instructions used are in RDNA4 and most of them are even in CDNA3 (CDNA4 isn't public yet?) and the ones that aren't exactly there are only slightly renamed (ds_load -> ds_read). Sure it's annoying but it's not the end of the world to have some `#ifdef`s in your code (that's not very much different from what the compiler itself is going to do anyway).

Many of the optimizations here rely heavily on the size of matrix and it's relationship to hardware specific details, like LDS size, how they're banked and register count.

It's probably not surprising that you can grind a decent improvement over a general solution, and many of the improvements shown here will need to be re-balanced, or even simply not work, for kernels working on different matrix layouts. Similarly for trying to work on different hardware - even in the same architecture and generation these sort of details are often changing.

And all that required going down to the ISA level, which is a lot less easy (certainly less documented) for Nvidia - for example the "inspiration" post linked [0] on CUDA didn't beat cuBLAS also didn't try modifying the SASS directly, so there might be similar level gains unrealized there.

[0] https://siboehm.com/articles/22/CUDA-MMM

No, but it's not obviously clear that other sized kernels will hit the same bottlenecks seen in the post. It's not really shown one way or the other - is it that the rocm kernels are just inefficient, or just the author identified one that wasn't particularly well optimized? And do these opportunities for improvement really mean that the software is "Questionable", or just that you cannot really do an equivalent comparison at the level of ISA on other vendor's software stacks?

I'm not trying to minimize the work here, it's interesting and a good example of the sort of lengths you can go to in order to squeeze that last little bit of performance out (and again, showing the advantages of public ISA documentation and support for users working at that level), I just took issue to the parent comment seeming to use this work as evidence of a poor baseline.