Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

41–50 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#41

Post by redox99 »

bogwog wrote: post #31
adam_arthur wrote: post #8

Pretty exciting to see the democratization of this tech. The next wave of innovation seems like it’ll be much less centralized into a handful of big companies. Even more powerful closed door LLMs will appear, but already so much utility out of these that can run on consumer hardware.

Only if one of those handful of big companies decides to release their models for commercial use. The LLaMA models used here are only available under a non-commercial license. Can’t build anything significant with leaked models. Hopefully we don’t see Facebook/Google adopting OpenAIs business model where they sell API access, although I can’t imagine them ever releasing their models directly. The last thing they woul…

I think a lot of people are going to just ignore the "license".

Re: Alpaca: A strong open-source instruction-following model

#42

Post by Traubenfuchs »

margorczynski wrote: post #17

> Importantly, we have not yet fine-tuned the Alpaca model to be safe and harmless. We thus encourage users to be cautious when interacting with Alpaca, and to report any concerning behavior to help improve the safety and ethical considerations of the model. Thanks but I would prefer you don't start injecting your "safeness" to gimp it. Or at least provide it as an alternative.

[flagged]

Re: Alpaca: A strong open-source instruction-following model

#43

Post by temp_account_32 »

Traubenfuchs wrote: post #42
margorczynski wrote: post #17

> Importantly, we have not yet fine-tuned the Alpaca model to be safe and harmless. We thus encourage users to be cautious when interacting with Alpaca, and to report any concerning behavior to help improve the safety and ethical considerations of the model. Thanks but I would prefer you don't start injecting your "safeness" to gimp it. Or at least provide it as an alternative.

[flagged]

What if you accidentally generate a racial slur offline on your PC? Who will protect you then?

Re: Alpaca: A strong open-source instruction-following model

#44

Post by chaxor »

nwoli wrote: post #36
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

A lot of them aren’t very good though at the same stable diffusion vram level unfortunately (and we’ve had large non consumer level gpu llms open sourced for a while eg gpt-j)

That is likely because "good" is a higher bar in language than images, because people don't mind or notice the longer range artifacts in image models as much.

Re: Alpaca: A strong open-source instruction-following model

#45

Post by Animats »

bryananderson wrote: post #4
famouswaffles wrote: post #3

https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci) I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.

I am not a lawyer, but the consensus (around here anyway) seems to be that weights cannot be copyrighted since they are the output of an automated process, so perhaps the license cannot limit how the weights are used. The code can be copyrighted, but the code can also be reimplemented from scratch and used to load the weights.

The US doesn't have database copyright, but some countries, including the entire European Union, do.[1]

[1] https://en.wikipedia.org/wiki/Database_right

Re: Alpaca: A strong open-source instruction-following model

#46

Post by 2bitencryption »

simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

Here is one question I have not seen answered yet:

All the magic of "7B LLaMA running on a potato" seems to involve lowering precision down to f16 and then further quantizing to int4.

Clearly this quantized model still outputs something resembling human language, at the very least.

But I haven't seen anyone show what effect this quantizing has on the quality of the output. If the quality of the output is bad, it's unclear if it's because the model needs to be finetuned (as Stanford did here) or if it's because the quanitizing reduced the quality, or both.

If this fine-tuned Stanford model still has excellent output after quantizing it to run on a Raspberry Pi 4GB, that would be awesome!

Re: Alpaca: A strong open-source instruction-following model

#47

Post by bitL »

adam_arthur wrote: post #37
bogwog wrote: post #31

Earlier quoted context omitted.

Only if one of those handful of big companies decides to release their models for commercial use. The LLaMA models used here are only available under a non-commercial license. Can’t build anything significant with leaked models. Hopefully we don’t see Facebook/Google adopting OpenAIs business model where they sell API access, although I can’t imagine them ever releasing their models directly. The last thing they woul…

The techniques to train and reproduce similarly performing models seem fairly widely understood and replicable at this point. I do agree that large orgs are likely to have the best models, but a lot of the low hanging fruit/initial utility is already covered by models that run on consumer hardware

Can you post a link on how to train a gigantic LLM on consumer hardware with all detailed steps please?

EDIT: You seem to have deleted your response but I am posting my reply anyway:

Microsoft has mentioned today that it cost them many hundreds of millions of USD to accommodate OpenAI models. I studied NLP at Stanford including LLMs. Only Google had a smaller transformer that could somewhat compete (Reformer) but the rest went with the brute force GPT-like models and 1000+ GPUs, $10M+/training run etc. I haven't seen a single post anywhere explaining how to train such large models (a smaller model on 4-8 GPUs doesn't count). "Facts are clear"

Re: Alpaca: A strong open-source instruction-following model

#49

Post by mlyle »

Traubenfuchs wrote: post #42
margorczynski wrote: post #17

> Importantly, we have not yet fine-tuned the Alpaca model to be safe and harmless. We thus encourage users to be cautious when interacting with Alpaca, and to report any concerning behavior to help improve the safety and ethical considerations of the model. Thanks but I would prefer you don't start injecting your "safeness" to gimp it. Or at least provide it as an alternative.

[flagged]

On the other hand, LLM's without some degree of safety work will frequently advocate for genocide/eugenics; perpetuate terrible stereotypes; mock ethnic dialects of language; veer randomly into erotica; etc. Not things in "trigger warning" territory but stuff that is broadly considered unacceptable.

Re: Alpaca: A strong open-source instruction-following model

#50

Post by mellosouls »

Traubenfuchs wrote: post #42
margorczynski wrote: post #17

> Importantly, we have not yet fine-tuned the Alpaca model to be safe and harmless. We thus encourage users to be cautious when interacting with Alpaca, and to report any concerning behavior to help improve the safety and ethical considerations of the model. Thanks but I would prefer you don't start injecting your "safeness" to gimp it. Or at least provide it as an alternative.

[flagged]

Tay
Post reply on HN