As someone who doesn’t work with languages like C, what’s the appeal of “in one file” or “header only”? Is it about dependency management?
Llama2.c: Inference llama 2 in one file of pure C
21–30 of 173 posts
Re: Llama2.c: Inference llama 2 in one file of pure C
#22As someone who doesn’t work with languages like C, what’s the appeal of “in one file” or “header only”? Is it about dependency management?
Yep! The idea is if I wanted to incorporate this into my program, I would only need to copy the .c/.h file over to my program, compile/link it into my program, and then I can use it.
Re: Llama2.c: Inference llama 2 in one file of pure C
#23As someone who doesn’t work with languages like C, what’s the appeal of “in one file” or “header only”? Is it about dependency management?
Not sure if there is a significant benefit, but I think its sort of Andrej's specialty as an educator to build things out from first principles. He has a habit of sharing his "from-scratch" version of important papers/methods. Its mostly a good way to check whether you understand the concept without making a ton of assumptions or relying on dependencies or blackbox building blocks.
Re: Llama2.c: Inference llama 2 in one file of pure C
#24FYI: this builds cleanly with WASI SDK and runs with no changes in a Wasm runtime if you're into that kind of thing
Re: Llama2.c: Inference llama 2 in one file of pure C
#25As someone who doesn’t work with languages like C, what’s the appeal of “in one file” or “header only”? Is it about dependency management?
It's helpful for dependency management, but I think in this case the goal is also having the user know that every aspect of the task is covered somewhere in this one file -- there is no "and then it goes into a library that I can't easily understand the workings of" limit to understanding how the tool works.
Re: Llama2.c: Inference llama 2 in one file of pure C
#26Is this for educational purposes only? Based on the success of llama.cpp and this one it appears that the industry is going in a direction of separate source code for every model that is released instead of general purpose frameworks like pytorch/tensorflow/onnxruntime?
Yes, this appears to be entirely educational.
No. Despite the name, llama.cpp supports more than just llama. It also isn’t an entirely bespoke thing as you indicate, since it is built on the more general purpose “ggml” tensor library/framework.
Re: Llama2.c: Inference llama 2 in one file of pure C
#27Is this for educational purposes only? Based on the success of llama.cpp and this one it appears that the industry is going in a direction of separate source code for every model that is released instead of general purpose frameworks like pytorch/tensorflow/onnxruntime?
Yes, since it's single-threaded.
Re: Llama2.c: Inference llama 2 in one file of pure C
#28Sounds like what Llama.cpp used to be.
I'm not sure what you mean by "used to be", the llama.cpp github repository was committed to just 4 hours ago.
This project cites llama.cpp as inspiration, but seems much-simplified. It only supports llama-2, only supports fp-32, and only runs on one CPU thread.
Re: Llama2.c: Inference llama 2 in one file of pure C
#29What are some uses for this?
Create a computer game about a small island with 100 people, with each person being politically aware, with llama2.c being their brain. Then you can simulate politics for a thousand years and see what happens. For instance.
Neat idea. Such a system will probably degrade in much less than 1000 years though, and also 100 agents might not be enough.
Re: Llama2.c: Inference llama 2 in one file of pure C
#30Is the trained model available on Hugging Face?