I'm probably just ignorant but shouldn't the graphic of the tiled transpose have the green vector column-oriented in the final matrix?
Highly efficient matrix transpose in Mojo
41–50 of 70 posts
Re: Highly efficient matrix transpose in Mojo
#42Why do we ever need to transpose a matrix? Isn't it better to simply combine the transposition with whatever next operation one wishes to do with the matrix?
You're right that a good graph compiler will do this for you. There still may be times, like if you're interfacing with another library, where you'll need to switch a matrix between row major or column major layouts.
Serious linear algebra libraries expect a flag that tells if elements are column-major or row-major.
Re: Highly efficient matrix transpose in Mojo
#43Earlier quoted context omitted.
Matrix transpose is a canonical example of a memory bound operation and often used to showcase optimization in a particular programming language or library. See for example the cutlass matrix transpose tutorial from Jay Shah of flash attention 3 paper: https://research.colfax-intl.com/tutorial-matrix-transpose-i...
Unfortunately the issue (alluded to in the blog post you linked) is that transposes do absolutely no work but memory loads. Sure, they test that you can swizzle your accesses, but modern accelerators are all about pipelining and feeding matrix multiply units, which is considerably harder than loading from memory as fast as possible. Actually, even the Mojo post barely beats CUDA for most of its kernels, because you c…
I totally agree that the resulting kernel will be rarely useful. I just wanted to highlight that it is a commonly used educational exercise to showcase how to optimize for memory throughput. If the post showed how to fuse a transpose + rmsnorm epilogue to a gemm then the kernel would be more functional but the blog post would be much harder to follow for newcomers.
Jay Shah’s later articles contain examples that involve epilogue fusion. IMHO, understanding how to write an efficient transpose helps with following the more involved ones.
Re: Highly efficient matrix transpose in Mojo
#44Mojos compiler is closed source. Thats a big no-no
I work on Mojo. The whole compiler, runtime etc. will get open sourced, most likely within a year. It is just a matter of time and us getting all the required work done. https://docs.modular.com/mojo/faq/#open-source
,,will get open sourced'' means closed source, parent wrote the same
Re: Highly efficient matrix transpose in Mojo
#45> This kernel archives a bandwidth of 1056.08 GB/s which is faster than the 875.46 GB/s we archived using CUDA. I believe that to be the reason because we use the PTX api for TMA transfers in Mojo. I can't say for sure because I couldn't find the CUDA kernel but I kind of doubt this is true. You can hit memory bandwidth on Hopper without using TMA at all, which is mostly designed for accelerating asynchronous copies…
The kernels I mention in CUDA use all the equivalent logic like the Mojo kernels. You can find them on my GitHub: https://github.com/simveit/effective_transpose
You may want to provide a faster kernel on H100 via PR and I will merge after checking it’s faster.
Re: Highly efficient matrix transpose in Mojo
#46Earlier quoted context omitted.
Unfortunately the issue (alluded to in the blog post you linked) is that transposes do absolutely no work but memory loads. Sure, they test that you can swizzle your accesses, but modern accelerators are all about pipelining and feeding matrix multiply units, which is considerably harder than loading from memory as fast as possible. Actually, even the Mojo post barely beats CUDA for most of its kernels, because you c…
I totally agree that the resulting kernel will be rarely useful. I just wanted to highlight that it is a commonly used educational exercise to showcase how to optimize for memory throughput. If the post showed how to fuse a transpose + rmsnorm epilogue to a gemm then the kernel would be more functional but the blog post would be much harder to follow for newcomers. Jay Shah’s later articles contain examples that invo…
That was exactly my reason to write this blogpost and optimise transpose.
It is a simple educational yet not trivial example to learn the basics.
Re: Highly efficient matrix transpose in Mojo
#47I think Mojo’s lack of being a true open product and existing to drive profits at Modular has really held it back.
It’s just really impractical to use a licensed programming language in 2025.
Re: Highly efficient matrix transpose in Mojo
#48I'm probably just ignorant but shouldn't the graphic of the tiled transpose have the green vector column-oriented in the final matrix?
The colors are reading writing operations ;)
You have global memory and shared memory, the global is slower.
You read in rows in the global memory (faster than reading columns)
You write in columns in the shared memory (slower than in rows, but the shared memory is fast, this is the transpose operation)
You read in rows in the shared memory (very fast)
You write in rows in the global memory (faster than writing in columns)
The idea behind that tiling is to hide the slow part in a memory that is faster.
Re: Highly efficient matrix transpose in Mojo
#49Is the word archive used in place of achieve? I’m not sure if there is a terminology issue that I don’t understand in this post…
Re: Highly efficient matrix transpose in Mojo
#50I think Mojo’s lack of being a true open product and existing to drive profits at Modular has really held it back. It’s just really impractical to use a licensed programming language in 2025.
[deleted]