Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels
nanduruganesh.github.io
Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels
1–6 of 6 posts
Re: Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels
#2World’s first?
Such lazy, much farming
https://github.com/fla-org/native-sparse-attention?utm_sourc...
Re: Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels
#3World’s first? Such lazy, much farming https://github.com/fla-org/native-sparse-attention?utm_sourc...
The Minimax paper was published in June 2026 coinciding with the Minimax M3 release - I’m not sure how the repo you posted here could have been an implementation of Minimax sparse attention when it was updated over a year ago?
Re: Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels
#4I’ve actually been really interested in Minimax M3 - seems like it flew under the radar but size wise might actually be runnable for local inference with a footprint somewhere between Deepseek V4 flash and pro.
Has anyone used the new Minimax M3 model? I’m curious how it compares with Deepseek V4 and GLM 5.2 and other larger open weights models.
Re: Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels
#5I’ve actually been really interested in Minimax M3 - seems like it flew under the radar but size wise might actually be runnable for local inference with a footprint somewhere between Deepseek V4 flash and pro. Has anyone used the new Minimax M3 model? I’m curious how it compares with Deepseek V4 and GLM 5.2 and other larger open weights models.
I have used it a little bit (0.5B tokens) for agentic tasks and coding. It is pretty nice and a serious step up from M2.7. I prefer it to GLM 5.2 for simpler tasks because of the cheaper token plan.
[Update: their cheapest token plan has been removed, I guess its back to GLM now]
Right now M3 is not far behind DS4, but I belive DS4 will improve much more with each round of training. It simply has a bigger brain, it just needs to fill it with more information.
Re: Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels
#6Speaking of minimax, has anyone used their video/music models? The demo page looks cool but I never heard of anyone actually using them in anything.