Trainable Llama-like transformer (with backpropagation) in numpy only (~600 lines)
Llama 3 implemented in pure NumPy
11–20 of 54 posts
Re: Llama 3 implemented in pure NumPy
#12The rotary embeddings bit is neat. I wonder if a complex representation would simplify vs complexify things (readability, performance, expressive power).
The tensor cores that do the bulk of the flops on the bulk of the gpus people use are just various sizes of floats, i think. We're in a funny position where progress in models and progress in hardware are kind of linked.
As far as expressive power goes, it shouldn't make a difference for the models in common use, but I could totally imagine models where it improves readability.
Re: Llama 3 implemented in pure NumPy
#13What is the difference to the llama.np repository credited in the README? https://github.com/hscspring/llama.np
Well, it supports Llama3. But the other question I have is about the license. The tokenizer.py file is identical, and the rest is very similar - just making minor adjustments here and there. Can they just take this Apache 2 licensed code, change it a bit and offer it as MIT? They are clearly not the original author.
Unfortunately, licenses are only worth as much as your lawyers.
Re: Llama 3 implemented in pure NumPy
#14`import jax.numpy as np`, then we also get a jax implemention after certain modifications: e.g. remove in-place index assignment, replace unsupported functions, etc
Re: Llama 3 implemented in pure NumPy
#15Earlier quoted context omitted.
Well, it supports Llama3. But the other question I have is about the license. The tokenizer.py file is identical, and the rest is very similar - just making minor adjustments here and there. Can they just take this Apache 2 licensed code, change it a bit and offer it as MIT? They are clearly not the original author.
Unfortunately, licenses are only worth as much as your lawyers.
DMCA takedowns are free.
Re: Llama 3 implemented in pure NumPy
#16I'll consider superintelligence achieved if AI can do such work faithfully.
[deleted]
Re: Llama 3 implemented in pure NumPy
#17Nice but the tricky part is the training data.
Re: Llama 3 implemented in pure NumPy
#18`import jax.numpy as np`, then we also get a jax implemention after certain modifications: e.g. remove in-place index assignment, replace unsupported functions, etc
...which should be much faster also on CPU, I assume.
Re: Llama 3 implemented in pure NumPy
#19Cool, instant cuda acceleration via cupy!
`import cupy as np`
Re: Llama 3 implemented in pure NumPy
#20How does the performance of llama3.np compare to other implementations, especially considering it's a pure NumPy implementation?