51–60 of 70 posts
As someone said below - you'd never write just a transpose kernel - it'll be fused into something else.
Look the frontier AI companies need something other than reversing binary trees to give interview candidates
I think Mojo’s lack of being a true open product and existing to drive profits at Modular has really held it back. It’s just really impractical to use a licensed programming language in 2025.
Possibly rose-tinted glasses on my part, but I’m optimistic for 2026. Chris Lattner has a pretty strong track record of getting these things right.
Great write up! I learned a lot!
I wonder if there is a reason for not using the high level abstractions provided by Modular
Earlier quoted context omitted.
Now I'm curious, how many times do you have to fully read the matrix in GPU for the total impact of reading columns to be higher than one-off actual transpose and then sequential row reads? I know it depends on lots of things, I'm after a rough estimate.
It's quite rare. Usually problems are tiled anyway and you can amortize the cost of having data in the "wrong" layout by loading coalesced in whatever is the best layout for your data and then transposing inside your tile, which gives you access to much faster memory.
I wonder if there is a reason for not using the high level abstractions provided by Modular
Most interesting algorithms (e.g. with dynamic shapes, mixed computation) are typically better scheduled by hand.
Earlier quoted context omitted.
I work on Mojo. The whole compiler, runtime etc. will get open sourced, most likely within a year. It is just a matter of time and us getting all the required work done. https://docs.modular.com/mojo/faq/#open-source
,,will get open sourced'' means closed source, parent wrote the same
He has a bit of a track record already.
I think Mojo’s lack of being a true open product and existing to drive profits at Modular has really held it back. It’s just really impractical to use a licensed programming language in 2025.
My impression is that this is on purpose on their part. They’ve repeatedly stated that by 2026 they will open source the compiler, and I think they’ve wanted a slow adoption ramp in order to spend some more time getting it right first. Possibly rose-tinted glasses on my part, but I’m optimistic for 2026. Chris Lattner has a pretty strong track record of getting these things right.
In the coarse graining code, you use an @parameter-for. Doesn’t that lead to some pretty large code size unrolling that? Or is that less of an issue on GPU? Great write up! I learned a lot!