Not bad.
State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
21–30 of 143 posts
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#22Earlier quoted context omitted.
That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?
You can use this command to apply the delta weights. ( https://github.com/lm-sys/FastChat#vicuna-13b ) The delta weights are hosted on huggingface and will be automatically downloaded.
Thanks! https://huggingface.co/lmsys/vicuna-13b-delta-v0
Edit, later: I found some instructive pages on how to use the vicuna weights with llama.cpp (https://lmsysvicuna.miraheze.org/wiki/How_to_use_Vicuna#Use_...) and pre-made ggml format compatible 4-bit quantized vicuna weights, https://huggingface.co/eachadea/ggml-vicuna-13b-4bit/tree/ma... (8GB ready to go, no 60+GB RAM steps needed)
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#23Earlier quoted context omitted.
Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.
https://github.com/facebookresearch/llama/pull/184
Nobody at Facebook approved it? Given the attention it has received, hard to imagine it has slipped through the cracks, but a deliberate decision to not address.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#24Earlier quoted context omitted.
A large array of uniquely-set floating point values. (AKA "parameters".) In a language model, a word is put in one end (as a numerical index to a wordlist), and then it and the weights multiplied together, and then a new word comes out (again as an index). Numbers in, numbers out, and a small bit of logic that maps words to numbers and back at either end. ("Encodings".) "Training" is the typically expensive process o…
> A large array of uniquely-set floating point values. How large? How many elements?
It's in the name of the model - "Vicuna-13B" implies there are 13 billion parameters.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#25It's actually very impressive. I gave it the task of converting a query and an OpenAPI spec into an API call, and it worked! I've not been succesful in getting GPT-3.5 to do this without rambling on about the reasoning for its decision.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#26what are model weights?
Essentially a computer neural network is just a lot of addition (and matrix multiplication) of floating point numbers. The parameters are the "strength" or "weights" of the connections between neurons on different layers and the "bias" of each neuron. If neuron Alice is connected to neuron Bob and Alice has a value of 0.7, and the weight of Alice's connection to bob is 0.5, then the value sent from Alice to Bob is 0.35. This value (and the values from all the other incoming connections) are summed at added to the neuron's negative bias.
I highly recommend checking out 3blue1brown series on how neural nets, gradient descent, and the dot product (implemented as a matrix multiplication) all tie together: https://www.youtube.com/watch?v=aircAruvnKk
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#27Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…
Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.
Very unlikely you'd face any legal action for usage of anything. If you share it, then it becomes less unlikely.
Edit: Also, judging by a comment from the team in the GitHub repository (https://github.com/lm-sys/FastChat/issues/86#issuecomment-14...), they seem to at least hint about been in contact with the llama team.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#28It's actually very impressive. I gave it the task of converting a query and an OpenAPI spec into an API call, and it worked! I've not been succesful in getting GPT-3.5 to do this without rambling on about the reasoning for its decision.
Usually if I want code from the GPT family I always add "Just show me the code, no extra words or explanation" in the end of the prompt, and it works 99% of the time.
Edit: just finished the conversion of Vicuna myself now and been doing some light testing, seems to work in ~80% of the cases for it, not as high success-rate as with GPT for sure. Probably there is a better way of structuring the prompt for Vicuna.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#29Earlier quoted context omitted.
That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?
If you follow this command in their instruction, the delta will be automatically downloaded and applied to the base model. https://github.com/lm-sys/FastChat#vicuna-13b : `python3 -m fastchat.model.apply_delta --base /path/to/llama-13b --target /output/path/to/vicuna-13b --delta lmsys/vicuna-13b-delta-v0`
I may have missed the detail, but it also expects the pytorch conversion rather than original LLaMa model.
Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
#30Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…
Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.
The llama-dl project actually helped you download the weights, whereas this just assumes you already have them. That feels like a pretty massive difference to me.