Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

twitter.com

Post reply on HN
Print view View original

61–70 of 143 posts

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#61

Post by holoduke »

superkuh wrote: post #26
nerdchum wrote: post #5

what are model weights?

Essentially a computer neural network is just a lot of addition (and matrix multiplication) of floating point numbers. The parameters are the "strength" or "weights" of the connections between neurons on different layers and the "bias" of each neuron. If neuron Alice is connected to neuron Bob and Alice has a value of 0.7, and the weight of Alice's connection to bob is 0.5, then the value sent from Alice to Bob is 0.…

Why would a 4bit quantized model be less accurate than a 16?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#62

Post by m3kw9 »

taf2 wrote: post #40
phoenixreader wrote: post #36

Could someone explain how to test this? Applying the delta conversion requires 60GB of CPU RAM. Do you just have 60GB RAM on your machine?

I got the 64gb MacBook Pro but already realizing the 96gb laptop would have made sense now - I got it in Jan right before all the ai crazy really lite up - distinctly remember thinking who would ever need more then 64gb of ram…

Probably enough just wait till they find ways to optimize it. If 64 isn’t enough 96 is not gonna get you much more

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#63

Post by akiselev »

vlugorilla wrote: post #37

From the git repo: > This conversion command needs around 60 GB of CPU RAM. Ok. I don't have that. Has/will someone release the full weights with the deltas applied?

https://huggingface.co/jeffwan/vicuna-13b

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#64

Post by capableweb »

m3kw9 wrote: post #57
capableweb wrote: post #28

Earlier quoted context omitted.

Usually if I want code from the GPT family I always add "Just show me the code, no extra words or explanation" in the end of the prompt, and it works 99% of the time. Edit: just finished the conversion of Vicuna myself now and been doing some light testing, seems to work in ~80% of the cases for it, not as high success-rate as with GPT for sure. Probably there is a better way of structuring the prompt for Vicuna.

You can just say “no explain” and it won’t explain. Don’t waste tokens patronizing gpt lol

Doesn't always work. Sometimes you have to be more verbose or even repeat yourself in order to force the model to do something. This is one of those cases. Just adding "No explain" in the end of the prompt often doesn't stop it from adding an explanation anyways.

> write a function in JavaScript that turns a JavaScript array into JS DOM elements, like what Hiccup does in Clojure. No explain

Makes GPT-4 output text + code.

> write a function in JavaScript that turns a JavaScript array into JS DOM elements, like what Hiccup does in Clojure. Just show me the code, no extra words or explanation.

Makes GPT-4 output only code, nothing else.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#65

Post by oh_sigh »

sillysaurusx wrote: post #38
0cf8612b2e1e wrote: post #12

Earlier quoted context omitted.

Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.

(I work on llama-dl.) We’re fighting back against the DMCA requests on the basis that NN weights aren’t copyrightable. This thread has details: https://news.ycombinator.com/item?id=35393782 I don't think you have to worry about Facebook going after you. The worst that will happen is that they issue a DMCA, in which case your project gets knocked offline. I don’t think they’ll be going the RIAA route of suing individu…

If they aren't copyrightable, couldn't they still be classes as a trade secret and still fall under IP law? Though I'm not sure if distributing the weights to people who sign a simple agreement to not redistribute would count as taking reasonable precautions in maintaining secrecy.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#66

Post by gigel82 »

All of the llama derivatives are tainted by Meta's license, which makes commercial use illegal and even personal use dubious.

Is all the training material used for Llama available as open source? Maybe lots of folks can pool their resources and create fully open clean models / weights instead.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#67

Post by oh_sigh »

knodi123 wrote: post #43
meghan_rain wrote: post #39

Earlier quoted context omitted.

keep up god's work!

> god's work creating sentient life?

That can't be his work, since he only picked up that hobby about 0.000625% of the universe's timespan ago.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#68

Post by sbierwagen »

holoduke wrote: post #60
Animats wrote: post #49

Nice. You need a 28GB GPU, so it's not exactly something people can run on their laptop. Everybody's server costs are about to go the roof.

Wouldnt it be time for a somewhat older gpu with a lot of memory. Or is that hard to achieve?

NVidia segments product lines on VRAM size. Consumer cards top out at 24GB. If you want more than that you have to buy a datacenter card for 10x the price.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#69

Post by MuffinFlavored »

skrblr wrote: post #24
MuffinFlavored wrote: post #17

Earlier quoted context omitted.

> A large array of uniquely-set floating point values. How large? How many elements?

It's in the name of the model - "Vicuna-13B" implies there are 13 billion parameters.

the way these LLMs work, there is a weight for each parameter? 13 billion weights? what is an example of a parameter?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#70

Post by stcredzero »

a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

Vicuna at huggingface.com? This keeps making me think of "facehuggers" from Aliens and Vecna from Stranger Things.

(I know a vicuna is a llama like animal.)

Post reply on HN