Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

231–240 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#231

Post by int_19h »

mlyle wrote: post #49
Traubenfuchs wrote: post #42

Earlier quoted context omitted.

[flagged]

On the other hand, LLM's without some degree of safety work will frequently advocate for genocide/eugenics; perpetuate terrible stereotypes; mock ethnic dialects of language; veer randomly into erotica; etc. Not things in "trigger warning" territory but stuff that is broadly considered unacceptable.

On the gripping hand, LLMs that were forced into being "safe" just end up regurgitating biases to that effect. One particular example that I ran into: https://news.ycombinator.com/item?id=34987074

Re: Alpaca: A strong open-source instruction-following model

#232

Post by notpushkin »

zitterbewegung wrote: post #212
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

LLAMA.cpp with 65B parameters runs on a MacBook M1 Max with 64GB of RAM. See https://gist.github.com/zitterbewegung/4787e42617aa0be6019c3...

What's the correlation between parameter count and RAM usage? Will LLaMA-13B fit on my MacBook Air with 8 GB of RAM or am I stuck with 7B?

Re: Alpaca: A strong open-source instruction-following model

#233

Post by barnabee »

karmasimida wrote: post #153
bryananderson wrote: post #4

Earlier quoted context omitted.

I am not a lawyer, but the consensus (around here anyway) seems to be that weights cannot be copyrighted since they are the output of an automated process, so perhaps the license cannot limit how the weights are used. The code can be copyrighted, but the code can also be reimplemented from scratch and used to load the weights.

Not copy right, but doesn't mean this isn't someone's property.

Information can’t be property.

IP “rights” are a perverse legal fiction that have inflicted untold damage.

Re: Alpaca: A strong open-source instruction-following model

#235

Post by notpushkin »

leobg wrote: post #75
londons_explore wrote: post #23

The training data doesn't seem to be great quality... "instruction": "Calculate the sum of each column in the following table.", "input": "2 | 3 | 4\n 5 | 6 | 7", "output": "2+3+4 = 9\n5+6+7 = 18" I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of huma…

Don’t even need human time. Just run it through davinci. Is this correct, yes or no? Or MTurk. Can cost less than $500.

Well, MTurk is selling human time, so...

Re: Alpaca: A strong open-source instruction-following model

#237

Post by mirekrusin »

kir-gadjello wrote: post #98
azeirah wrote: post #92

Earlier quoted context omitted.

Is there a reason why we aren't using BLOOM's model? It's huge, 160B parameters. It's open, it's free, it's been out there for a while. Is something wrong with it? Using outdated training data? Using outdated training techniques? Or is it just not well known enough -- ie, not in the news at the right time.

BLOOM, while being a useful model for underrepresented languages, underperforms openai's GPT-3 models of similar parameter count on english benchmarks. Which is a pity. And OpenAI's known models (at least the ones for which the architecture is publicly known) https://platform.openai.com/docs/model-index-for-researchers aren't pinnacles of parameter efficiency either: Deepmind's Chinchilla and LLaMA beat them on bench…

Incentives already exist for being efficient - but of course it doesn't mean less carbon footprint, it means more powerful models.

In other words every carbon footprint reduction optimisation is used to yield more efficient output with the same (all available) carbon footprint.

Re: Alpaca: A strong open-source instruction-following model

#238

Post by sanxiyn »

est wrote: post #230
sanxiyn wrote: post #206

Earlier quoted context omitted.

Note that Alpaca is NOT using RLHF. It explicitly states it used supervised finetuning.

It says > We train the Alpaca model on 52K instruction-following demonstrations generated in the style of self-instruct using text-davinci-003 Which leads to self-instruct https://github.com/yizhongw/self-instruct From a glimpse they used a LM to classify instructions & train the model which IMHO is very similar to RLHF

No, it is not RLHF because there is no reward model involved. See also OpenAI's explanation here: https://platform.openai.com/docs/model-index-for-researchers

Re: Alpaca: A strong open-source instruction-following model

#239

Post by meghan_rain »

nl wrote: post #196

Note that the OpenAssistant project has a 100k+ multilingual instruction tuning dataset they are in the process of releasing. Stats from March 9 are 50K English prompts and 40K Spanish.

Wow that was quick, they only started in what, December?

Re: Alpaca: A strong open-source instruction-following model

#240

Post by MacsHeadroom »

tysam_and wrote: post #222
MacsHeadroom wrote: post #215

Earlier quoted context omitted.

WRONG. Research shows effectively imperceptible performance difference at 4-bit and even 3-bit with GPTQ quantization. You cannot tell the difference and if you think you do you're wrong, because it barely even registers on any benchmark. (Note: llama.cpp's 4bit is naive, not GPTQ, and sucks but they are refactoring it to use GPTQ quantization) References: https://arxiv.org/abs/2210.17323 - GPTQ: Accurate Post-Traini…

Good points, though I would gently encourage not starting a post with "WRONG." in the middle of a nuanced discussion. I remember 'way back when' when there was a .5-2% flat performance drop for UINT8 on some models when it was first introduced (depends upon the modality). Like, 4 bit quantization really is probably enough for a number of usecases and likely beats smaller models with precision enough to make it the eq…

Good points, I didn't mean to come off abrasive but I can see why I would. My attention was to get attention on a thread where my new comment would be buried under the 8 other replies, so I put a big attention grabber at the start.

But again good points about the nuances of lower precision. For LLMs at least 'The Case for 4-bit Precision' and 'GPTQ' seem fairly conclusive that over ~10B parameters even 3-bit precision has virtually undetectable loss with the right trircks. Levels which, if they even mattered, can easily be overcome with a little additional training.

Newer ongoing research on LLaMA specifically[0] shows we can reduce the model's size around 84% without any meaningful performance loss through a combination of GPTQ, binning, and 3-bit.

[0] https://nolanoorg.substack.com/p/int-4-llama-is-not-enough-i...

Post reply on HN