Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

281–290 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#281

Post by counttheforks »

dmw_ng wrote: post #224
Havoc wrote: post #216

Earlier quoted context omitted.

Curious about the google traffic comment. Are you saying people are visiting sites less because they can stay on Bing/OpenAI?

There is some very natural split regarding what I'll send to ChatGPT vs. what goes to Google. For example "six nations fixtures" obviously Google, but anything of depth or where recency is irrelevant goes the other direction. Asked it a few Linux questions today, how to interpret the title of a particular FRED chart, and a ton more sessions that Firefox history somehow didn't manage to correctly track the title for.…

Same, until I realized that about 60% of the information it gives me is either subtly wrong or 100% factually incorrect. Yet it's so, so confident.

Re: Alpaca: A strong open-source instruction-following model

#282

Post by chesneyc »

aortega wrote: post #270
zitterbewegung wrote: post #212

Earlier quoted context omitted.

LLAMA.cpp with 65B parameters runs on a MacBook M1 Max with 64GB of RAM. See https://gist.github.com/zitterbewegung/4787e42617aa0be6019c3...

That is still a 4000 usd computer. You can get 2 RTX3900 used for ~1000 usd and run 65B much faster. I have a discord server up serving almost 500 users with 65B. https://twitter.com/ortegaalfredo/status/1635402627327590400 For some things is better than GPT3, for other even Alpaca is better.

Thanks for taking the time to set this up. I will definitely give it a go later today. I don't have access to hardware that I can run LLaMA on and I'm really curious to see what the 65B model has to offer.

Re: Alpaca: A strong open-source instruction-following model

#283

Post by nodemaker »

counttheforks wrote: post #281
dmw_ng wrote: post #224

Earlier quoted context omitted.

There is some very natural split regarding what I'll send to ChatGPT vs. what goes to Google. For example "six nations fixtures" obviously Google, but anything of depth or where recency is irrelevant goes the other direction. Asked it a few Linux questions today, how to interpret the title of a particular FRED chart, and a ton more sessions that Firefox history somehow didn't manage to correctly track the title for.…

Same, until I realized that about 60% of the information it gives me is either subtly wrong or 100% factually incorrect. Yet it's so, so confident.

And in that way its actually more overall correct than the most knowledgeable person on earth. With google you also get fed some very dangerously wrong info (recent example masks) but you think its correct. With ChatGPT you have to actually use your critical thinking skills and get to the truth which in my opinion a huge advancement over google.

Re: Alpaca: A strong open-source instruction-following model

#284

Post by counttheforks »

nodemaker wrote: post #283
counttheforks wrote: post #281

Earlier quoted context omitted.

Same, until I realized that about 60% of the information it gives me is either subtly wrong or 100% factually incorrect. Yet it's so, so confident.

And in that way its actually more overall correct than the most knowledgeable person on earth. With google you also get fed some very dangerously wrong info (recent example masks) but you think its correct. With ChatGPT you have to actually use your critical thinking skills and get to the truth which in my opinion a huge advancement over google.

Not really, with google you get multiple sources at a glance. Sure, they can still be wrong, but some critical thinking + multiple sources = more likely to be correct than relying on a single unreliable source.

Re: Alpaca: A strong open-source instruction-following model

#285

Post by zh217 »

simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

Also today: ChatGLM released by Tsinghua University. I've made a separate submission for it: https://news.ycombinator.com/item?id=35150190

The GitHub page is https://github.com/THUDM/ChatGLM-6B. The GitHub description is all in Chinese, but the model itself can handle English queries on a single consumer GPU well. Considering its size, I'd say the quality of its responses are outstanding.

Re: Alpaca: A strong open-source instruction-following model

#286

Post by danielbln »

IanCal wrote: post #166
simonw wrote: post #25

Earlier quoted context omitted.

They say they trained it for $600.

Under $100. > Fine-tuning a 7B LLaMA model took 3 hours on 8 80GB A100s, which costs less than $100 on most cloud compute providers. They also spent $500 creating the dataset using openais APIs.

Those $500 would be $50 if they use gpt3.5-turbo, which is a magnitude cheaper than davinci-3.

Re: Alpaca: A strong open-source instruction-following model

#287

Post by sebzim4500 »

est wrote: post #229
sebzim4500 wrote: post #203

Earlier quoted context omitted.

InstructGPT isn't a single model, it's a set of techniques for finetuning a foundation model

what does "1.3B parameters" mean in this context? Does it mean we load 175B gpt-3 model first, then overwrite 1.3B parameters with InstructGPT? I find this sentence difficult to understand > Our labelers prefer outputs from our 1.3B InstructGPT model over outputs from a 175B GPT-3 model https://openai.com/research/instruction-following I am a newbie, plz correct me if I am wrong.

They mean that they took a 1.3B parameter model, applied the InstructGPT finetuning model and found that it worked better for their usecase than a 175B parameter model which had not gone through that process.

Re: Alpaca: A strong open-source instruction-following model

#288

Post by seydor »

kossTKR wrote: post #149

A Tangent, but how long before we will see half the population having relationships with their AI assistants like in the sci-fi movie "Her". Add cutting edge voice synthesis and maybe a realistic video stream in a few years and we're there. Japan already has problems with the concept of "2d Waifus" and lack of intimacy between the younger generation. Maybe the downfall will not just be climate catastrophe but hyper i…

I don't know about relationships,but as far as online discussions, AIs are better because they don't argue in bad faith like all of the internet. Unfortunately they are too censored currently but locally running bots will arrive

Re: Alpaca: A strong open-source instruction-following model

#289

Post by Jack5500 »

losteric wrote: post #137
Tiberium wrote: post #113

Earlier quoted context omitted.

Such a thing already exists and there were some results - https://open-assistant.io I'm not sure why the authors of Alpaca didn't try to train it on this dataset.

There's the OIG dataset ( https://laion.ai/blog/oig-dataset/ ) which was used to train a NeoX 20B ChatBot ( https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-... ). The dataset is larger and publicly available. I want to try finetuning LLaMa on this tonight.

how did it go?

Re: Alpaca: A strong open-source instruction-following model

#290

Post by zitterbewegung »

aortega wrote: post #270
zitterbewegung wrote: post #212

Earlier quoted context omitted.

LLAMA.cpp with 65B parameters runs on a MacBook M1 Max with 64GB of RAM. See https://gist.github.com/zitterbewegung/4787e42617aa0be6019c3...

That is still a 4000 usd computer. You can get 2 RTX3900 used for ~1000 usd and run 65B much faster. I have a discord server up serving almost 500 users with 65B. https://twitter.com/ortegaalfredo/status/1635402627327590400 For some things is better than GPT3, for other even Alpaca is better.

How do you make it load on two GPUs or does llama.cpp does it automatically? I have a setup with a threadrippper and a RTX3090 and a Titan RTX. I haven't had the time to set it up so that's why I have been using my Mac.
Post reply on HN