Anyone know of a 65B instruct model I can run on llama.cpp? Or if it’s possible to train one on a 64gb Mac Studio? Haven’t come across either yet.
Llama has a 65B model and it runs (albeit slowly) on a 64gb Mac Studio with llama.cpp.
CodeAlpaca – Instruction following code generation model
41–50 of 50 posts
Re: CodeAlpaca – Instruction following code generation model
#42If someone were to add noise to the llama weights and then retrain a little, would anyone be able to tell? Could that org then pass it off as their own training, MIT licensed for the good of humanity?
Could be better to permute the neurons and weight matrices in such a way as to obtain the exact same network (an isomorphic graph).
Re: CodeAlpaca – Instruction following code generation model
#43"Model weights aren't part of the release for now, to respect OpenAI TOS and LLaMA license." I feel like the whole Open Source ML scene is slowed down by a strong chilling effect. Everyone seems to be afraid to release models. Meanwhile, other models are freely available up to alpaca 30b: https://github.com/underlines/awesome-marketing-datascience/...
Have you applied for the download? The first links were sent out pretty quickly.
Re: CodeAlpaca – Instruction following code generation model
#44>The model is not finetuned to be safe and harmless, so be cautious.
Ok I'll be careful, but if it says a bad word, I'm sending you my therapy bill.
Re: CodeAlpaca – Instruction following code generation model
#45Anyone know of a 65B instruct model I can run on llama.cpp? Or if it’s possible to train one on a 64gb Mac Studio? Haven’t come across either yet.
Llama has a 65B model and it runs (albeit slowly) on a 64gb Mac Studio with llama.cpp.
Yea I have that running. Looking specifically for a 65B instruction-tuned model. I just found a way to get alpaca 30B going though.
Re: CodeAlpaca – Instruction following code generation model
#46Earlier quoted context omitted.
Probably in the hundreds of dollar for 7B model, and may be a thousand or two for the 13B at worst
Far far less. Alpaca-7B's compute cost was around $60-$70 for Stanford and around $0.60 (yes 60 cents) for equivalent fine tunes using the Parameter Efficient Fine Tuning (PEFT) strategy of Low Rank Adapters (LoRA). The repo above can be replicated for similar costs. Easily less than $10 for up to 30B using LoRA (which requires only 24GB of VRAM for 30B/33B and smaller).
I am interested in this. What would be the cost for the best model possible by the public?
Re: CodeAlpaca – Instruction following code generation model
#47>The model is not finetuned to be safe and harmless, so be cautious. Ok I'll be careful, but if it says a bad word, I'm sending you my therapy bill.
Therapy bill?
Re: CodeAlpaca – Instruction following code generation model
#48Didn't explore much, but it seems alpaca-lora has better results for coding tasks. One example I've used was: "Implement quicksort in python.". This is the result with Code alpaca: def quicksort(arr): if len(arr) pivot: right_index -= 1 if left_index Alpaca lora result: def quicksort(arr): if len(arr) pivot] return quicksort(left) + [pivot] + quicksort(right) Shorter and much cleaner, not to mention it works (code al…
The Alpaca lora result is incorrect since, for example, if all elements in array are the same the result in just an array with one element.
You are correct also it's not quicksort as it's not in place array, but needs a lot of memory management
Re: CodeAlpaca – Instruction following code generation model
#49Earlier quoted context omitted.
Probably in the hundreds of dollar for 7B model, and may be a thousand or two for the 13B at worst
Far far less. Alpaca-7B's compute cost was around $60-$70 for Stanford and around $0.60 (yes 60 cents) for equivalent fine tunes using the Parameter Efficient Fine Tuning (PEFT) strategy of Low Rank Adapters (LoRA). The repo above can be replicated for similar costs. Easily less than $10 for up to 30B using LoRA (which requires only 24GB of VRAM for 30B/33B and smaller).
I thought so too, but for newcomers, they should expect to train model a dozen times or so :-)
Re: CodeAlpaca – Instruction following code generation model
#50"Model weights aren't part of the release for now, to respect OpenAI TOS and LLaMA license." I feel like the whole Open Source ML scene is slowed down by a strong chilling effect. Everyone seems to be afraid to release models. Meanwhile, other models are freely available up to alpaca 30b: https://github.com/underlines/awesome-marketing-datascience/...
> "Model weights aren't part of the release for now, to respect OpenAI TOS and LLaMA license."
Makes sense if they originally licensed the model weights from Meta. Fortunately you can get the weights via torrent without agreeing to the license by visiting facebook's repository and getting the magnet link yourself: https://github.com/facebookresearch/llama/pull/73/files