Llama2.c: Inference llama 2 in one file of pure C
1–10 of 173 posts
Re: Llama2.c: Inference llama 2 in one file of pure C
#2More details from Andrej here: https://twitter.com/karpathy/status/1683143097604243456?s=46...
Re: Llama2.c: Inference llama 2 in one file of pure C
#3"train a baby Llama 2 model in PyTorch, then inference it"
Re: Llama2.c: Inference llama 2 in one file of pure C
#4ohh thats some really nice readable c-code
Re: Llama2.c: Inference llama 2 in one file of pure C
#5Sounds like what Llama.cpp used to be.
Re: Llama2.c: Inference llama 2 in one file of pure C
#6What are some uses for this?
Re: Llama2.c: Inference llama 2 in one file of pure C
#7"make more better tests to decrease yolo" haha
Re: Llama2.c: Inference llama 2 in one file of pure C
#8neat!
note that gcc's default optimisation level is 0, which really isn't what people normally want.
adding -O2 to the gcc command line should improve performance quite a bit.
Re: Llama2.c: Inference llama 2 in one file of pure C
#9What are some uses for this?
Create a computer game about a small island with 100 people, with each person being politically aware, with llama2.c being their brain. Then you can simulate politics for a thousand years and see what happens. For instance.
Re: Llama2.c: Inference llama 2 in one file of pure C
#10To run a neural network, how much memory does one need?
Is it enought to load the first two layers from disk, calculate the activations for all nodes, discard the first layer, load the third layer from disk, calculate all the activations for all nodes, discard the second layer etc?
Then memory needs to be big enough to hold to 2 layers?