Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A pure NumPy implementation of Mamba

github.com

Post reply on HN
Print view View original

31–39 of 39 posts

Re: A pure NumPy implementation of Mamba

#31

Post by exe34 »

1024core wrote: post #28
edflsafoiewq wrote: post #27

Earlier quoted context omitted.

The doc comment at the top of the .py file is sufficiently descriptive """Simple, minimal implementation of Mamba in one file of Numpy adapted from (1) and inspired from (2). Suggest reading the following before/while reading the code: [1] Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Albert Gu and Tri Dao) https://arxiv.org/abs/2312.00752 [2] The Annotated S4 (Sasha Rush and Sidd Karamcheti) http…

> The doc comment at the top of the .py file is sufficiently descriptive Which is the purpose of these doc comments. If you have the time to gripe on HN, you have the time to click on the link and do some reading. The "Usage" section in the link above is enough to help one disambiguate; if not, then there's always the doc comment.

i believe the gripe is a plea for others not to do the same thing, and instead to put some thought into presentation. the gripe is not about the specific case. if the poster is anything like me, if the first ten words of your post don't make sense to me, I'm just moving on to something else.

Re: A pure NumPy implementation of Mamba

#32

Post by volemo »

exe34 wrote: post #30
mint2 wrote: post #24

Okay so is Mamba also an llm? There’s too much name overloading! I’m familiar with mamba, the conda like thing in python, but a numpy implementation of that makes no sense.

maybe we should get rid of names entirely, and call it all software. "software: a super fast and fun retro-encabulator written in rust!"

Assign UUIDs to everything!

Re: A pure NumPy implementation of Mamba

#33

Post by TeMPOraL »

rsfern wrote: post #20
ptspts wrote: post #12

Contrary to what the title says, this is note a pure Pyhon + numpy implementation: in fact it also imports einops, transformers and torch. For me pure X means: to use this, all you have to install is X.

I don’t see a PyTorch import, and the transformers import is just for the tokenizer which I don’t really consider a nontrivial part of mamba So it’s just numpy and einops, which is pretty cool. I guess you could probably rewrite all the einops stuff in pure numpy if you want to trade readable code for eliminating the einops dependency Edit: found the torch import, but it’s just for a single torch.load to deserialize…

> Edit: found the torch import, but it’s just for a single torch.load to deserialize some data

Torch is quite heavy though, isn't it? All for that one deserialization call?

Re: A pure NumPy implementation of Mamba

#34

Post by ktm5j »

edflsafoiewq wrote: post #27
thoronton wrote: post #2

Why is it so difficult to write a short description what the project does? With too many open source projects people, who are not familiar with it, have to play detective to figure out what it actually is doing. "Wait a package manager based on numphy? That doesn't make any sense. Oh they mention LLM? So it must have something to do with AI"

The doc comment at the top of the .py file is sufficiently descriptive """Simple, minimal implementation of Mamba in one file of Numpy adapted from (1) and inspired from (2). Suggest reading the following before/while reading the code: [1] Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Albert Gu and Tri Dao) https://arxiv.org/abs/2312.00752 [2] The Annotated S4 (Sasha Rush and Sidd Karamcheti) http…

Okay, so why not just put that in the readme??

Re: A pure NumPy implementation of Mamba

#35

Post by [deleted] »

Tomte wrote: post #7
thoronton wrote: post #2

Why is it so difficult to write a short description what the project does? With too many open source projects people, who are not familiar with it, have to play detective to figure out what it actually is doing. "Wait a package manager based on numphy? That doesn't make any sense. Oh they mention LLM? So it must have something to do with AI"

The author did not post it to HN to confuse you. He did not post it here, at all. Why are you entitled to have every single GitHub repo explained, tailored to your individual knowledge? Many other people understood exactly what this is. Maybe the submitter could add a comment on HN with an explanation, but the author owes you nothing.

[deleted]

Re: A pure NumPy implementation of Mamba

#36

Post by 7thpower »

blagie wrote: post #15
Hugsun wrote: post #11

Is there a benefit to implementing it in numpy over pytorch or tf?

Yes. A numpy program will work tomorrow. ALL of the machine learning frameworks have incredible churn. I have code from two years ago which I can't make work reliably anymore -- not for lack of trying -- due to all the breaking changes and dependency issues. There are systems where each model runs in its own docker, with its own set of pinned library versions (many with security issues now). It's a complete and utter…

This was a very helpful explanation. I appreciate it.

Re: A pure NumPy implementation of Mamba

#37

Post by Hugsun »

blagie wrote: post #15
Hugsun wrote: post #11

Is there a benefit to implementing it in numpy over pytorch or tf?

Yes. A numpy program will work tomorrow. ALL of the machine learning frameworks have incredible churn. I have code from two years ago which I can't make work reliably anymore -- not for lack of trying -- due to all the breaking changes and dependency issues. There are systems where each model runs in its own docker, with its own set of pinned library versions (many with security issues now). It's a complete and utter…

That makes sense. I imagine that there are significant performance tradeoffs but those are probably worth it in many cases. I would be somewhat surprised if Mamba can be made usefully fast with NumPy, but it would be a pleasant surprise.

Re: A pure NumPy implementation of Mamba

#38

Post by blagie »

Hugsun wrote: post #37
blagie wrote: post #15

Earlier quoted context omitted.

Yes. A numpy program will work tomorrow. ALL of the machine learning frameworks have incredible churn. I have code from two years ago which I can't make work reliably anymore -- not for lack of trying -- due to all the breaking changes and dependency issues. There are systems where each model runs in its own docker, with its own set of pinned library versions (many with security issues now). It's a complete and utter…

That makes sense. I imagine that there are significant performance tradeoffs but those are probably worth it in many cases. I would be somewhat surprised if Mamba can be made usefully fast with NumPy, but it would be a pleasant surprise.

For me, there are plenty of old-school models which are still plenty useful and which run fine even on a modern, fast CPU and would do well with numpy, and even more so with cupy. Step through:

https://huggingface.co/learn/nlp-course/chapter1/1

And there's a pile of awesome. It feels pretty lame compared to GPT4o, ChatGPT, or even GPT3, but it's still super-useful a lot of the time, and not too resource-intensive.

(Disclaimer: That's the original Hugging Face course and is politely structured to work on reasonable machines too. They have other courses which require moderate GPU, and plenty of models which require crazy hardware.)

Re: A pure NumPy implementation of Mamba

#39

Post by exe34 »

volemo wrote: post #32
exe34 wrote: post #30

Earlier quoted context omitted.

maybe we should get rid of names entirely, and call it all software. "software: a super fast and fun retro-encabulator written in rust!"

Assign UUIDs to everything!

content-based addressing! Go big or go broke!
Post reply on HN