Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

61–70 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#61

Post by mx20 »

nickthegreek wrote: post #34

Quick compare with this vs 7B LLama. Input: Tell me something about owls. Output 7B LLama: Asked by kimberly at 12:54 PM on May. 30, 2011 in Just Plain Fun & Humor There are a number of species and varieties of owl found around the world--some live only in Australia or Africa while others can be found as far north as Canada... The one thing that all have in common is their ability to adapt. Owls will find homes in pl…

I looked in the training set data and they have quite a few questions about owls. Also it got " downward curved beak" from davinci and got it still wrong.

Like:

"instruction": "Describe the sound an owl makes.",

"instruction": "Summarize the differences between an owl and a hawk.",

"instruction": "Find a fact about the bird of the following species", "input": "Species: Great Horned Owl",

"instruction": "What is the binomial nomenclature of the barn owl?",

"instruction": "Generate a riddle about an owl.",

Re: Alpaca: A strong open-source instruction-following model

#62

Post by gamegoblin »

2bitencryption wrote: post #46
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

Here is one question I have not seen answered yet: All the magic of "7B LLaMA running on a potato" seems to involve lowering precision down to f16 and then further quantizing to int4. Clearly this quantized model still outputs something resembling human language, at the very least. But I haven't seen anyone show what effect this quantizing has on the quality of the output. If the quality of the output is bad, it's un…

Decrease in accuracy is negligible and decreases as model size increases. That is, larger models quantize even better than smaller models.

https://arxiv.org/abs/2210.17323

Re: Alpaca: A strong open-source instruction-following model

#63

Post by margorczynski »

numeri wrote: post #54
Traubenfuchs wrote: post #42

Earlier quoted context omitted.

[flagged]

I personally think this is important completely aside from any cultural propriety/censoring, as it's the one of the only parts of AI ethics that we can currently directly work on and test. If we can't get models not to say racist or otherwise terrible things, we can't make any guarantees about our ability to control or guide some future AGI. A very much secondary reason I appreciate these (admittedly annoying) attemp…

> If we can't get models not to say racist or otherwise terrible things, we can't make any guarantees about our ability to control or guide some future AGI.

This is a very bold assumption that the current LLMs function and "think" in the same way some future AGI would. They do not even reason, just make up words that fit some context - thus they "hallucinate".

There is no reason the approach taken here by injecting some bias or word filtering would apply to the real thing. And AI safety and aligment is not (at least it was not until getting hijacked) and was not about some model saying mean words but something really threatening like the paperclip maker problem - an agent choosing a path to a goal which is not aligned with what humans find acceptable (e.g. solving world hunger by killing everyone)

Re: Alpaca: A strong open-source instruction-following model

#64

Post by freedmand »

redox99 wrote: post #29

They used OpenAI davinci to generate the dataset for instruction tuning. They mention one of the reasons this model is noncommercial is because OpenAI's terms of use prohibit developing models that compete with OpenAI. They also published the generated dataset. If one were to use this dataset to tune your own commercial model, would this be OK, or illegal? I would guess OpenAIs ToS no longer applies, as this is now A…

The instruction tuning dataset is only 52,000 rows. It shouldn't be too hard to crowdsource high-quality human answers to this many questions and retrain the model, nixing the dependency on OpenAI.

Re: Alpaca: A strong open-source instruction-following model

#65

Post by beepbooptheory »

Traubenfuchs wrote: post #51
temp_account_32 wrote: post #43

Earlier quoted context omitted.

What if you accidentally generate a racial slur offline on your PC? Who will protect you then?

[flagged]

Its sarcastic but how are you literally not that person right now? Being angry at an invisible evil, not once opening your eyes to any other thought, totally incapable of making any real point except to whine... It's quite a good descriptor actually!

I hope you can find a model for yourself that will assure you there are only two genders. Just to save the rest us from this childishness.

Re: Alpaca: A strong open-source instruction-following model

#66

Post by bitL »

oceanplexian wrote: post #58
bogwog wrote: post #31

Earlier quoted context omitted.

Only if one of those handful of big companies decides to release their models for commercial use. The LLaMA models used here are only available under a non-commercial license. Can’t build anything significant with leaked models. Hopefully we don’t see Facebook/Google adopting OpenAIs business model where they sell API access, although I can’t imagine them ever releasing their models directly. The last thing they woul…

I think anyone can train one of these models. “anyone” being someone who can throw $1-5M in cloud resources at compute. Or a bored sysadmin with access to a few thousand GPUs and a cool boss. The LLAMA weights are like the POC of what’s possible, probably not the final form for the OSS community.

There seems to be a big difference between efficiently training a "large-ish" model on 4-8 GPUs and a gigantic model on 1000+ GPUs. The same techniques might not work due to different warm up steps, gradient overlaps etc.

All you can see running in the wild are quantized LLaMA variants (4 or 8-bit) whereas the original model is 32-bit.

Re: Alpaca: A strong open-source instruction-following model

#67

Post by oezi »

oceanplexian wrote: post #58
bogwog wrote: post #31

Earlier quoted context omitted.

Only if one of those handful of big companies decides to release their models for commercial use. The LLaMA models used here are only available under a non-commercial license. Can’t build anything significant with leaked models. Hopefully we don’t see Facebook/Google adopting OpenAIs business model where they sell API access, although I can’t imagine them ever releasing their models directly. The last thing they woul…

I think anyone can train one of these models. “anyone” being someone who can throw $1-5M in cloud resources at compute. Or a bored sysadmin with access to a few thousand GPUs and a cool boss. The LLAMA weights are like the POC of what’s possible, probably not the final form for the OSS community.

Whats possible now (March 2023).

The field is moving so quick.

Re: Alpaca: A strong open-source instruction-following model

#68

Post by rjb7731 »

nickthegreek wrote: post #34

Quick compare with this vs 7B LLama. Input: Tell me something about owls. Output 7B LLama: Asked by kimberly at 12:54 PM on May. 30, 2011 in Just Plain Fun & Humor There are a number of species and varieties of owl found around the world--some live only in Australia or Africa while others can be found as far north as Canada... The one thing that all have in common is their ability to adapt. Owls will find homes in pl…

With the 7B none-instruction tuned you have to really pre-prompt it to get good answers to things so like:

SYSTEM: You are an AI research assistant. You use a tone that is technical and scientific.

USER: Hello, who are you?

ASSISTANT: Hi! I am an AI research assistant. How can I help you today?

USER: Can you tell me something about owls.

Re: Alpaca: A strong open-source instruction-following model

#70

Post by soVeryTired »

margorczynski wrote: post #63
numeri wrote: post #54

Earlier quoted context omitted.

I personally think this is important completely aside from any cultural propriety/censoring, as it's the one of the only parts of AI ethics that we can currently directly work on and test. If we can't get models not to say racist or otherwise terrible things, we can't make any guarantees about our ability to control or guide some future AGI. A very much secondary reason I appreciate these (admittedly annoying) attemp…

> If we can't get models not to say racist or otherwise terrible things, we can't make any guarantees about our ability to control or guide some future AGI. This is a very bold assumption that the current LLMs function and "think" in the same way some future AGI would. They do not even reason, just make up words that fit some context - thus they "hallucinate". There is no reason the approach taken here by injecting s…

> They do not even reason, just make up words that fit some context - thus they "hallucinate".

But they can explain their 'reasoning' in a way that makes sense to humans a lot of the time. Serious question: how do you know if something does or doesn't reason?

Post reply on HN