Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

291–300 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#291

Post by leodriesch »

frozenwind wrote: post #277

From what I understood they used 175 seed tasks as an example for da-vinci which then generated the 52K instructions used to fine-tune LLama. What I don't understand is how did they achieve giving this context to GPT, since afaik the context window is just 2048 tokens? It seems that it's not fine-tuning, but few-shot.

They show an example of this under "Training Recipe". They prompt Davinci to give them instruction following examples, 20 at a time. And that a lot of times. The context window is also 4000 tokens, at least that is the maximum amount you can pick in the OpenAI playground.

Re: Alpaca: A strong open-source instruction-following model

#292

Post by est »

sebzim4500 wrote: post #287
est wrote: post #229

Earlier quoted context omitted.

what does "1.3B parameters" mean in this context? Does it mean we load 175B gpt-3 model first, then overwrite 1.3B parameters with InstructGPT? I find this sentence difficult to understand > Our labelers prefer outputs from our 1.3B InstructGPT model over outputs from a 175B GPT-3 model https://openai.com/research/instruction-following I am a newbie, plz correct me if I am wrong.

They mean that they took a 1.3B parameter model, applied the InstructGPT finetuning model and found that it worked better for their usecase than a 175B parameter model which had not gone through that process.

Ah I got it now. Thanks.

From the gpt-3 paper it looks like they have many variants like

- GPT-3-350M

- GPT-3-1.3B

- GPT-3-2.7B

- GPT-3-6.7B

- GPT-3-13B

- GPT-3-175B

Ada, Babbage, Curie and Davinci line up closely with 350M, 1.3B, 6.7B, and 175B respectively. The names are pretty suggestive.

Re: Alpaca: A strong open-source instruction-following model

#294

Post by pradn »

nodemaker wrote: post #283
counttheforks wrote: post #281

Earlier quoted context omitted.

Same, until I realized that about 60% of the information it gives me is either subtly wrong or 100% factually incorrect. Yet it's so, so confident.

And in that way its actually more overall correct than the most knowledgeable person on earth. With google you also get fed some very dangerously wrong info (recent example masks) but you think its correct. With ChatGPT you have to actually use your critical thinking skills and get to the truth which in my opinion a huge advancement over google.

No, the crucial thing is that a good human will tell you if they don't know something, or if they are simply unsure.

Re: Alpaca: A strong open-source instruction-following model

#295

Post by dmw_ng »

pradn wrote: post #294
nodemaker wrote: post #283

Earlier quoted context omitted.

And in that way its actually more overall correct than the most knowledgeable person on earth. With google you also get fed some very dangerously wrong info (recent example masks) but you think its correct. With ChatGPT you have to actually use your critical thinking skills and get to the truth which in my opinion a huge advancement over google.

No, the crucial thing is that a good human will tell you if they don't know something, or if they are simply unsure.

It's adorable seeing this kind of critique in the context of HN, I wonder how many folk knew my heavily upvoted comment above ("For 10 billion+ ...") was from someone who has only been looking at this stuff for a few weeks. ChatGPT is no better or worse than any consultant I've ever met (including myself), or most of the commenters you find here every single day.

Re: Alpaca: A strong open-source instruction-following model

#296

Post by kkielhofner »

zitterbewegung wrote: post #290
aortega wrote: post #270

Earlier quoted context omitted.

That is still a 4000 usd computer. You can get 2 RTX3900 used for ~1000 usd and run 65B much faster. I have a discord server up serving almost 500 users with 65B. https://twitter.com/ortegaalfredo/status/1635402627327590400 For some things is better than GPT3, for other even Alpaca is better.

How do you make it load on two GPUs or does llama.cpp does it automatically? I have a setup with a threadrippper and a RTX3090 and a Titan RTX. I haven't had the time to set it up so that's why I have been using my Mac.

llama.cpp doesn't use the GPU at all. The genius *.cpp (whisper.cpp, llama.cpp) projects are specifically intended to optimize/democratize otherwise GPU only models to run on CPU/non-GPU (CUDA, ROCm). Technically speaking the released models are capable of running on GPU via standard framework (PyTorch, TensorFlow) support for CPU but in practice without a lot of optimization they are incredibly slow to the point of useless, hence *.cpp.

You want something along these lines (warning: unnecessarily potentially offensive):

https://rentry.org/llama-tard-v2

Re: Alpaca: A strong open-source instruction-following model

#297

Post by akrymski »

LeoPanthera wrote: post #150

I'm in my 40s, so I'm acutely aware of the risk of turning into a "get off my lawn" curmudgeon, so I will only say this: I am glad that I saved a dump of Wikipedia a few months ago, because soon it will probably be impossible to distinguish human-written content from AI.

Have you every tried to edit the Wikipedia? Spamming it is incredible hard for humans (whether AI is involved or not) because it has a pretty good system of moderators based on the social trust model. The only way to bypass it is to effectively bribe a mod, similarly to how spammers pay for backlinks to increase their PageRank. This is what prevents it from turning into a spam farm, not the lack of text content.

Re: Alpaca: A strong open-source instruction-following model

#299

Post by jhbadger »

atleastoptimal wrote: post #133
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

Question: what percentage of the hype and momentum for this is so people can run sex chatbots on their local machine?

Or really just any text generation that chatGPT dislikes. It's nice not to be judged by a program (and perhaps logged somewhere that you asked for something "inappropriate").

Re: Alpaca: A strong open-source instruction-following model

#300

Post by sdenton4 »

amoss wrote: post #256
Animats wrote: post #45

Earlier quoted context omitted.

The US doesn't have database copyright, but some countries, including the entire European Union, do.[1] [1] https://en.wikipedia.org/wiki/Database_right

Those protections rely on "systematic arrangement", i.e. that some effort has been put into finding, verifying and collection a set of facts / individual pieces of data. It is not obvious that would apply to the set of weights that come out of a training algorithm. (standard I'm not a lawyer, I don't know but I suspect it is unclear enough that it would need to be argued in court).

I assure you that if you scramble the weights of a neutral networks, the model will perform poorly... Training something new cost millions of dollars in accelerator and research salary, and evaluation ('verifying') is a big part of the work.
Post reply on HN