Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

twitter.com

Post reply on HN
Print view View original

41–50 of 143 posts

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#41

Post by eurekin »

MMMercy2 wrote: post #18
superkuh wrote: post #15

Earlier quoted context omitted.

That's what they say but I just spent 10 minutes searching the git repo, reading the relavent .py files and looking at their homepage and the vicuna-7b-delta and vicuna-13b-delta-v0 files are no where to be found. Am I blind or did they announce a release without actually releasing?

You can use this command to apply the delta weights. ( https://github.com/lm-sys/FastChat#vicuna-13b ) The delta weights are hosted on huggingface and will be automatically downloaded.

I did try, but got:

``` ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported. ```

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#43

Post by knodi123 »

meghan_rain wrote: post #39
sillysaurusx wrote: post #38

Earlier quoted context omitted.

(I work on llama-dl.) We’re fighting back against the DMCA requests on the basis that NN weights aren’t copyrightable. This thread has details: https://news.ycombinator.com/item?id=35393782 I don't think you have to worry about Facebook going after you. The worst that will happen is that they issue a DMCA, in which case your project gets knocked offline. I don’t think they’ll be going the RIAA route of suing individu…

keep up god's work!

> god's work

creating sentient life?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#44

Post by mdaniel »

superkuh wrote: post #26
nerdchum wrote: post #5

what are model weights?

Essentially a computer neural network is just a lot of addition (and matrix multiplication) of floating point numbers. The parameters are the "strength" or "weights" of the connections between neurons on different layers and the "bias" of each neuron. If neuron Alice is connected to neuron Bob and Alice has a value of 0.7, and the weight of Alice's connection to bob is 0.5, then the value sent from Alice to Bob is 0.…

To add to this excellent reply, I'll also point out that the reason folks want the weights is that they are the result of a massive search operation, akin to finding the right temperature to bake a cake from all possible floats. It takes a lot of wall clock time, and a lot of GPU energy, and a lot of input examples and counter-examples to find the "right" numbers. Thus, it really is better -- all things being equal -- to publish the results of that search to keep everyone else from having to repeat the search for themselves

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#45

Post by Jiocus »

phoenixreader wrote: post #36

Could someone explain how to test this? Applying the delta conversion requires 60GB of CPU RAM. Do you just have 60GB RAM on your machine?

Actually, yes..

2 x 32GiB (SDDR4 3200MHz) can be had for 170€ and probably less than that if doing the research. Took a bit of faith and a lot of impulse decision-making as this device was/is specified for up to 32GiB RAM only - but it went through.

This is precisely the use case I had in mind

*Lenovo 16ACH6H

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#46

Post by progman32 »

phoenixreader wrote: post #36

Could someone explain how to test this? Applying the delta conversion requires 60GB of CPU RAM. Do you just have 60GB RAM on your machine?

This struck me as well. Is the entire model being loaded before the deltas are applied? Would it be possible to apply the delta blockwise?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#47

Post by superkuh »

eurekin wrote: post #41
MMMercy2 wrote: post #18

Earlier quoted context omitted.

You can use this command to apply the delta weights. ( https://github.com/lm-sys/FastChat#vicuna-13b ) The delta weights are hosted on huggingface and will be automatically downloaded.

I did try, but got: ``` ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported. ```

> Unfortunately there's a mismatch between the model generated by the delta patcher and the tokenizer (32001 vs 32000 tokens). There's a tool to fix this at llama-tools (https://github.com/Ronsor/llama-tools). Add 1 token like (C controltoken), and then run the conversion script.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#48

Post by anonymousgpt1 »

sillysaurusx wrote: post #38
0cf8612b2e1e wrote: post #12

Earlier quoted context omitted.

Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.

(I work on llama-dl.) We’re fighting back against the DMCA requests on the basis that NN weights aren’t copyrightable. This thread has details: https://news.ycombinator.com/item?id=35393782 I don't think you have to worry about Facebook going after you. The worst that will happen is that they issue a DMCA, in which case your project gets knocked offline. I don’t think they’ll be going the RIAA route of suing individu…

Lemme save whoever is donating the legal here the time: model weights are definitely copyrightable.
Post reply on HN