Viewing profile — korbip

This was done already here as well: https://arxiv.org/abs/2507.04239

I can share a similar PhD story (the result being visible here: https://github.com/NX-AI/flashrnn ). Back then I didn't find any tutorials that cover anything beyond the basics (wh…

Test it out here: https://github.com/NX-AI/mlstm_kernels https://huggingface.co/NX-AI/xLSTM-7b

There is a LOT of effort in the research community currently: 1. Improving the Self-Attention in the Transformer as is, keeping the quadratic complexity, which has some theoretical…

Thanks! I don't see any implementation there. In any case, we are planning a code release soon.

You mainly got it right. Usually one does have many scalar 'c' cells, that talk to each other via memory mixing. For the sLSTM, you group them into heads, talking only to cells wit…

This was formulated a bit unclear. It is not possible to parallelize in the sequence dimension for training as it is possible for Transformers. In the batch dimension you can alway…

For language in general it seems fine. But there might be specific tasks where it is necessary indeed.

Thank you! I can say that it is not really a diminishing factor at the scales reported in the paper. So, xLSTM[7:1] is pretty much on par with xLSTM[1:0] in speed. We show that it …

Disclaimer: I'm shared first author of this paper. As a clarification: The speed for training will be on par with FlashAttention-2, when fully optimized and only including the mLST…

Photovoltaic potential calculated live and privacy-friendly in your browser via WebGL. Using open LoD2 and laser point data from the Bavarian Public Agency for Digitization, High-S…

Feel free to add them. :)