Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

191–200 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#191

Post by hgsgm »

whimsicalism wrote: post #156
famouswaffles wrote: post #3

https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci) I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.

I am curious how you are expected to prove in a court of law that the weights you have are derived from some licensed weights if I take just a few gradient steps from the original weights.

Same way that court determines whether my book is a paraphrase of yours. Substantial similarity.

Re: Alpaca: A strong open-source instruction-following model

#192

Post by hgsgm »

redox99 wrote: post #29

They used OpenAI davinci to generate the dataset for instruction tuning. They mention one of the reasons this model is noncommercial is because OpenAI's terms of use prohibit developing models that compete with OpenAI. They also published the generated dataset. If one were to use this dataset to tune your own commercial model, would this be OK, or illegal? I would guess OpenAIs ToS no longer applies, as this is now A…

> OpenAI's terms of use prohibit developing models that compete with OpenAI.

People shouldn't collaborate with Ope(ratio)nAI's attempt to poison the English language with its name.

Government shouldn't allow lies to be trademarked brand names. It's fraud.

Re: Alpaca: A strong open-source instruction-following model

#193

Post by magicalhippo »

dongping wrote: post #188
magicalhippo wrote: post #175

Earlier quoted context omitted.

Some parameters would be more sensitive than others I suppose? So could you use 4 bits for most, and 8 bits, or even 16, for the remaining?

That's exactly why bitsandbytes has a threshold parameter to control the quantization.

Nice, good to know, thanks!

Re: Alpaca: A strong open-source instruction-following model

#194

Post by LeoPanthera »

pmoriarty wrote: post #172
LeoPanthera wrote: post #150

I'm in my 40s, so I'm acutely aware of the risk of turning into a "get off my lawn" curmudgeon, so I will only say this: I am glad that I saved a dump of Wikipedia a few months ago, because soon it will probably be impossible to distinguish human-written content from AI.

If it's indistinguishable, then how does it matter who/what it was written by?

Because I am far less likely to trust encyclopedic content written by an AI, which can lie so convincingly that it does not know that it is lying. I have even seen it generating fictional source references that are hard to verify.

I just don't trust it.

Re: Alpaca: A strong open-source instruction-following model

#195

Post by Vetch »

koheripbal wrote: post #151
dmw_ng wrote: post #72

Earlier quoted context omitted.

For 10 billion+ parameter models, the effects of quantization are relatively small, for smaller models like Llama 7B the effect becomes more dramatic, but there is ongoing research on new quantization methods (like GPTQ) that preserve significant performance even on the lower end. Quantization isn't the only technique available for downsizing a model, Llama itself is already the result of sizing the model and input d…

The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber. 8 bits, imo, is the minimum.

This is interesting. What sizes are you seeing this for?

Re: Alpaca: A strong open-source instruction-following model

#198

Post by DrSiemer »

koheripbal wrote: post #151
dmw_ng wrote: post #72

Earlier quoted context omitted.

For 10 billion+ parameter models, the effects of quantization are relatively small, for smaller models like Llama 7B the effect becomes more dramatic, but there is ongoing research on new quantization methods (like GPTQ) that preserve significant performance even on the lower end. Quantization isn't the only technique available for downsizing a model, Llama itself is already the result of sizing the model and input d…

The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber. 8 bits, imo, is the minimum.

So which is better, running 7B without quantization or running 13B with? They both require about the same amount of vRAM (10gb).

Re: Alpaca: A strong open-source instruction-following model

#199

Post by pmoriarty »

LeoPanthera wrote: post #194
pmoriarty wrote: post #172

Earlier quoted context omitted.

If it's indistinguishable, then how does it matter who/what it was written by?

Because I am far less likely to trust encyclopedic content written by an AI, which can lie so convincingly that it does not know that it is lying. I have even seen it generating fictional source references that are hard to verify. I just don't trust it.

It takes more than merely being human to earn my trust.

Re: Alpaca: A strong open-source instruction-following model

#200

Post by mellosouls »

LeoPanthera wrote: post #150

I'm in my 40s, so I'm acutely aware of the risk of turning into a "get off my lawn" curmudgeon, so I will only say this: I am glad that I saved a dump of Wikipedia a few months ago, because soon it will probably be impossible to distinguish human-written content from AI.

[flagged]
Post reply on HN