Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
1–10 of 303 posts
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#2Does it have any Apple M2 / ARM specific dependencies / optimizations, or will it generally run on any CPU (x86)?
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#3Alpaca model leaked again?
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#4Alpaca model leaked again?
No, this is an open reproduction of Alpaca on the leaked LLaMA 7B weight set.
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#5Does it have any Apple M2 / ARM specific dependencies / optimizations, or will it generally run on any CPU (x86)?
It inherits from ARM NEON optimizations in llama.cpp but I believe it should also work on x86.
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#6Looking forward to try it, but I don't have a macbook. I wonder if it runs on i7-11800h (8 core 16 thread CPU) with 64gb RAM
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#7@antimatter15 - how much ram does your M2 MacBook Air have? Performance looks very good. Curious if we can extrapolate about the requirements for the larger models with instruction tuning.
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#8Are there torrents for 13B and up as well?
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#9Alpaca model leaked again?
No, this is an open reproduction of Alpaca on the leaked LLaMA 7B weight set.
This isn't actually a direct reproduction. It's a bit missleading because it only says it at the bottom. This actually uses LORAs which train way faster and are much more flexible, but they perform a little worse.
There is someone that is doing a direct reproduction of Alpaca in the alpaca repo issues though.
> The weights are based on the published fine-tunes from alpaca-lora
Re: Show HN: Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook
#10Looking forward to try it, but I don't have a macbook. I wonder if it runs on i7-11800h (8 core 16 thread CPU) with 64gb RAM
Would that require porting llama to OpenVino?