Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama2.c: Inference llama 2 in one file of pure C

github.com

Post reply on HN
Print view View original

171–173 of 173 posts

Re: Llama2.c: Inference llama 2 in one file of pure C

#171

Post by int_19h »

sanxiyn wrote: post #153
cultofmetatron wrote: post #79

Earlier quoted context omitted.

we need to kick the "ethical AI" people out. Its becoming increasingly clear they are damn annoying. I don't want safety scissors. restrict things running on your own servers, sure but don't give me a model I can't modify and use how i want on my machine.

If you want an unrestricted model, you should train one yourself. You don't want safety scissors, alas, we can't have all things we want, can we. Facebook is under no obligation to provide you one, after all it's Facebook's money, not yours.

Facebook does provide an unrestricted base model for Llama-2.

Re: Llama2.c: Inference llama 2 in one file of pure C

#172

Post by tomrod »

kgwgk wrote: post #165
tomrod wrote: post #161

Earlier quoted context omitted.

I can't speak for the p___g contest others want to engage in, but why not, lets make bare metal LLMs happen! https://github.com/rreilink/PiPyOS I appreciate the clarification earlier in the comment chain for what you meant by bare metal -- I had interpreted it as on-prem.

I didn’t know that there was a bare metal implementation of python, thanks for the link. I doubt it can run pytorch though.

As with all things, the key word is "yet" :)

Re: Llama2.c: Inference llama 2 in one file of pure C

#173

Post by nickpsecurity »

ljlolel wrote: post #145
GaggiX wrote: post #60

Earlier quoted context omitted.

>My next goal is to reduce 7B llama2 to 10-100M without making it much dumber. That is going to be hard as the 7B model was trained on 2T tokens. Maybe if you heavily restrict the range in which the model should operate.

1. It’s faster and cheaper to train a smaller model 2. Better than tokens is to train on probability distributions (distillation) and trees of probability distributions

I've never seen anything about training on probability distributions or trees of them. Do you have articles with examples you could share with us?

I did try a quick search for it. Found some interesting papers. The links to them are below in case anyone finds them interesting.

https://arxiv.org/abs/2212.11481

https://towardsdatascience.com/a-new-way-to-predict-probabil...

https://arxiv.org/pdf/1912.07913.pdf

https://dukespace.lib.duke.edu/dspace/bitstream/handle/10161...

Post reply on HN