Cool to see that the entire website, from the agreement to generating the predictions, is one Gradio [1] app written in Python
Alpaca: A strong open-source instruction-following model
11–20 of 313 posts
Re: Alpaca: A strong open-source instruction-following model
#12[deleted]
Re: Alpaca: A strong open-source instruction-following model
#13Cool to see that the entire website, from the agreement to generating the predictions, is one Gradio [1] app written in Python [1] https://gradio.dev/
would be better if it'd work on Firefox tho
Re: Alpaca: A strong open-source instruction-following model
#14>We are waiting for huggingface to officially support the llama models (i.e. this PR to be merged) before we release a stable version of the finetuning code.
Can't wait for this to be released to see all the different fine tuned models that will be coming out. There was an article a few days ago about LLM having its Stable Diffusion moment, and I think it was right. We went from LLama release > llama.cpp > Dalai and now an instruction tuned model in under 2 weeks.
Re: Alpaca: A strong open-source instruction-following model
#15interesting, looks like the web demo doesn't work at the moment though.
The prompt.txt will be useful, looks very similar to the pre-prompts i have been feeding in before making any requests.
Re: Alpaca: A strong open-source instruction-following model
#16>> Responses on par with txt-DaVinci-3
What is "on par" and how is it measured?
Re: Alpaca: A strong open-source instruction-following model
#17> Importantly, we have not yet fine-tuned the Alpaca model to be safe and harmless. We thus encourage users to be cautious when interacting with Alpaca, and to report any concerning behavior to help improve the safety and ethical considerations of the model.
Thanks but I would prefer you don't start injecting your "safeness" to gimp it. Or at least provide it as an alternative.
Re: Alpaca: A strong open-source instruction-following model
#18Cost to train this wouldn't be very high.
A regular CPU can generate or train about 2 words per second. 52,000 training examples of an average of 300 words would take 3 CPU months. And it is fully parallelizable.
And you can probably make it go far faster on a GPU.
Re: Alpaca: A strong open-source instruction-following model
#19https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci) I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.
They probably did but didn't release them for the public.
Why would they withold those ones but the the 65B ones? Seems like a weird choice IMO.
Re: Alpaca: A strong open-source instruction-following model
#20>> Responses on par with txt-DaVinci-3 What is "on par" and how is it measured?
>We performed a blind pairwise comparison between text-davinci-003 and Alpaca 7B, and we found that these two models have very similar performance: Alpaca wins 90 versus 89 comparisons against text-davinci-003.
Obviously not a completely foolproof comparison but it is at least clear it isn't much worse for the types of prompt they were testing.