Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizing Matrix Multiplication on RDNA3

seb-v.github.io

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Optimizing Matrix Multiplication on RDNA3

#11

Post by roenxi »

randomNumber7 wrote: post #5

Is the author a genius or has AMD questionable software?

ROCm multiplies in 4.5ms and the author multiplies in 2.8ms. The naive algorithm is 136ms. I don't think anyone at AMD is losing sleep over this; for a general purpose library this isn't horrible performance. It could be better, hand optimising to specific conditions often is. But as this blog post shows, optimising kernels is the sort of thing that people can do for fun and post blogs about if they care. They don't need AMD to be involved.

The problem with ROCm isn't that it only half-utilises the hardware, the problem was that someone trying to write this blog post in 2020 would have had (or at least the probability was rather high) a heading somewhere around implementing Kernel 0 talking about how the software crashed or the kernel panicked when they tried to run the benchmarks. That was what happened to me when I tried a conceptually similar exercise. I was wandering around HN posting comments about how there were no articles like this one to be found for AMD hardware and musing whether it was technically possible to do.

This makes me wish I'd bought an RDNA3 card instead of a Nvidia one for my last purchase. Not that I really regret the choice, AMD are going to have to show that they're interested in supporting consumer cards for a little longer to get me to trust them again although they're on the right path.

Re: Optimizing Matrix Multiplication on RDNA3

#13

Post by latchkey »

roenxi wrote: post #11
randomNumber7 wrote: post #5

Is the author a genius or has AMD questionable software?

ROCm multiplies in 4.5ms and the author multiplies in 2.8ms. The naive algorithm is 136ms. I don't think anyone at AMD is losing sleep over this; for a general purpose library this isn't horrible performance. It could be better, hand optimising to specific conditions often is. But as this blog post shows, optimising kernels is the sort of thing that people can do for fun and post blogs about if they care. They don't…

Follow Anush on Twitter and give him feedback. He's actively listening.

https://x.com/AnushElangovan

Re: Optimizing Matrix Multiplication on RDNA3

#14

Post by saagarjha »

roenxi wrote: post #11
randomNumber7 wrote: post #5

Is the author a genius or has AMD questionable software?

ROCm multiplies in 4.5ms and the author multiplies in 2.8ms. The naive algorithm is 136ms. I don't think anyone at AMD is losing sleep over this; for a general purpose library this isn't horrible performance. It could be better, hand optimising to specific conditions often is. But as this blog post shows, optimising kernels is the sort of thing that people can do for fun and post blogs about if they care. They don't…

AMD isn’t losing sleep over the fact that J. Random Blogger is beating their GEMM by 60% on 4096x4096? What universe are you living in? This company is fighting for their life against CUDA and you’re telling me their software stack being so bad it can’t use a third of the hardware on the the first and literally only thing people want it to do is somehow not a problem?

Re: Optimizing Matrix Multiplication on RDNA3

#16

Post by roenxi »

saagarjha wrote: post #14
roenxi wrote: post #11

Earlier quoted context omitted.

ROCm multiplies in 4.5ms and the author multiplies in 2.8ms. The naive algorithm is 136ms. I don't think anyone at AMD is losing sleep over this; for a general purpose library this isn't horrible performance. It could be better, hand optimising to specific conditions often is. But as this blog post shows, optimising kernels is the sort of thing that people can do for fun and post blogs about if they care. They don't…

AMD isn’t losing sleep over the fact that J. Random Blogger is beating their GEMM by 60% on 4096x4096? What universe are you living in? This company is fighting for their life against CUDA and you’re telling me their software stack being so bad it can’t use a third of the hardware on the the first and literally only thing people want it to do is somehow not a problem?

The point of a platform is for software engineers to provide key functionality independently. Your issue here is you don't understand why CUDA has been so dominant over the last decade - a ~50% software performance gap isn't that material when hardware capacity doubles every generation. If we've reached the point where J. Random Blogger can solve their own problems then the CUDA moat has quite possibly been broken.

If AMD was only 1 hardware generation behind Nvidia they'd be pretty competitive. People are happy using CPUs with a gap of several generations from the cutting edge. And it isn't even that bad because anyone who particularly cares can optimise their software and avoid using rocBLAS.

Re: Optimizing Matrix Multiplication on RDNA3

#18

Post by tgtweak »

Cuda has similar inefficiencies and many use cases can have equal uplifts by going lower level on the code.

I think this is what deepseek had done to get their speedups on older hardware.

Even way back in the days of GPU crypto mining - custom kernels hand built (mostly just unrolling loops) would yield 20% improvements over just running opencl and letting the drivers compile it down.

Re: Optimizing Matrix Multiplication on RDNA3

#20

Post by touisteur »

tgtweak wrote: post #18

Cuda has similar inefficiencies and many use cases can have equal uplifts by going lower level on the code. I think this is what deepseek had done to get their speedups on older hardware. Even way back in the days of GPU crypto mining - custom kernels hand built (mostly just unrolling loops) would yield 20% improvements over just running opencl and letting the drivers compile it down.

People have been trying to bypass CUDA and even PTX for a long time. One long rundown of optimizing gemm on NVIDIA hardware (https://salykova.github.io/sgemm-gpu) mentions 'maxas' (https://github.com/NervanaSystems/maxas/wiki/Introduction) - which was really a step forward in this space. I still blame Intel (buying NervanaSystems) for killing it...
Post reply on HN