Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

twitter.com

Post reply on HN
Print view View original

11–20 of 143 posts

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#11

Post by swyx »

a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

so an extra licensing issue to get around the original non commercial license... this is just a research curiosity is it not?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#12

Post by 0cf8612b2e1e »

a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#13

Post by zhisbug »

0cf8612b2e1e wrote: post #12
a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.

https://github.com/facebookresearch/llama/pull/184

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#14

Post by detrites »

nerdchum wrote: post #5

what are model weights?

A large array of uniquely-set floating point values. (AKA "parameters".)

In a language model, a word is put in one end (as a numerical index to a wordlist), and then it and the weights multiplied together, and then a new word comes out (again as an index).

Numbers in, numbers out, and a small bit of logic that maps words to numbers and back at either end. ("Encodings".)

"Training" is the typically expensive process of feeding huge amounts of data into the model, to get it to choose the magic values for its weights that allow it to do useful stuff that looks and feels like that training data.

Something else that can be done with weights is they can be "fine-tuned", or "tweaked" slightly to give different overall results out of the model, therefore tailored to some new use-case. Often the model gets a new name after.

In this case, what's been released is not actually the weights. It's a set of these tweaks ("deltas"), which are intended to be added to Meta's LLaMA model weights to end up with the final intended LLaMA-based model, called "Vicuna".

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#15

Post by superkuh »

a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#16

Post by zhwu »

superkuh wrote: post #15
a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?

If you follow this command in their instruction, the delta will be automatically downloaded and applied to the base model. https://github.com/lm-sys/FastChat#vicuna-13b: `python3 -m fastchat.model.apply_delta --base /path/to/llama-13b --target /output/path/to/vicuna-13b --delta lmsys/vicuna-13b-delta-v0`

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#17

Post by MuffinFlavored »

detrites wrote: post #14
nerdchum wrote: post #5

what are model weights?

A large array of uniquely-set floating point values. (AKA "parameters".) In a language model, a word is put in one end (as a numerical index to a wordlist), and then it and the weights multiplied together, and then a new word comes out (again as an index). Numbers in, numbers out, and a small bit of logic that maps words to numbers and back at either end. ("Encodings".) "Training" is the typically expensive process o…

> A large array of uniquely-set floating point values.

How large? How many elements?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#18

Post by MMMercy2 »

superkuh wrote: post #15
a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?

You can use this command to apply the delta weights. (https://github.com/lm-sys/FastChat#vicuna-13b) The delta weights are hosted on huggingface and will be automatically downloaded.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#19

Post by [deleted] »

zhwu wrote: post #16
superkuh wrote: post #15

Earlier quoted context omitted.

That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?

If you follow this command in their instruction, the delta will be automatically downloaded and applied to the base model. https://github.com/lm-sys/FastChat#vicuna-13b : `python3 -m fastchat.model.apply_delta --base /path/to/llama-13b --target /output/path/to/vicuna-13b --delta lmsys/vicuna-13b-delta-v0`

[deleted]

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#20

Post by viraptor »

zhwu wrote: post #16
superkuh wrote: post #15

Earlier quoted context omitted.

That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?

If you follow this command in their instruction, the delta will be automatically downloaded and applied to the base model. https://github.com/lm-sys/FastChat#vicuna-13b : `python3 -m fastchat.model.apply_delta --base /path/to/llama-13b --target /output/path/to/vicuna-13b --delta lmsys/vicuna-13b-delta-v0`

This can be then quantized to the llama.cpp/gpt4all format, right? Specifically, this only tweaks the existing weights slightly, without changing the structure?
Post reply on HN