Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

121–130 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#121

Post by valine »

superkuh wrote: post #96

I've played a lot with davinci 3 ($25 of credits worth) and it can do some impressive rhyming and interpretation of concepts as emoji sequences. From the 3 times I've interacted with this fine tuned llama 7B it is clear it cannot do that. I've also run the "vanilla" 7B, 13B, and 30B on my home computer with llama.cpp modified for interactive "chat" mode with various pre-prompt and these can't do it either. I have no…

7B parameters is next to nothing when compared to gpt3. If 7B works as well as it does here, A fine tuned 65B model could very easily achieve chatGPT level performance.

Re: Alpaca: A strong open-source instruction-following model

#123

Post by Jackson__ »

simonw wrote: post #9

Here's a link that opens their training data (52,000 rows) in Datasette Lite: https://lite.datasette.io/?json=https://github.com/tatsu-lab... Means you can run SQL LIKE queries against it to try and get a feel for what's in there. This link includes a query for running searches for terms: https://lite.datasette.io/?json=https://github.com/tatsu-lab...

Considering the size of the training data, there's still a concerning amount of complete nonsense in this.

Example 1:

>"instruction": "Using the PDF below, edit the text to make it more effective.",

>"input": "https://www.example.com/example.pdf",

>"output": "We recently launched a new product and wanted to(...)

Example 2:

>"instruction": "Explain why you think this painting is meaningful.",

>"input": "https://i.pinimg.com/originals/67/2a/d1/672ad1ed7e8d71e3e7fd...",

>"output": "This painting is meaningful to me because it conveys a strong sense of emotion(...)

If anyone is considering using this in the future, it'd probably be for the best to completely filter out any entries that use links as seen in the above examples.

Re: Alpaca: A strong open-source instruction-following model

#124

Post by bitL »

kir-gadjello wrote: post #108
bitL wrote: post #47

Earlier quoted context omitted.

Can you post a link on how to train a gigantic LLM on consumer hardware with all detailed steps please? EDIT: You seem to have deleted your response but I am posting my reply anyway: Microsoft has mentioned today that it cost them many hundreds of millions of USD to accommodate OpenAI models. I studied NLP at Stanford including LLMs. Only Google had a smaller transformer that could somewhat compete (Reformer) but the…

"Accomodate" is the word to scrutinize here. Yes, it will cost a lot to outright buy physical HPC infrastructure to train and infer a series of large models deployed for customers all over the globe. No, it won't cost nearly as much to rent cloud infra to train a similarly-sized model. No, you won't be able to train a large model on a single multi-GPU node, you will need a cluster containing a respectable power of tw…

You are missing the point. Extremely large LLMs don't train the same way as your BERT_Large x8 variety of LLMs. Your whole training procedure is different. Also Microsoft spent so much initially because their Azure Cloud was unable to cope with it electrically and they had to rewire a datacenter for it. So it's not even a question of just renting 1000 GPUs. Do you have actual experience training GPT-3+ sized models?

Re: Alpaca: A strong open-source instruction-following model

#125

Post by layer8 »

wkat4242 wrote: post #82
nickthegreek wrote: post #14

>We are waiting for huggingface to officially support the llama models (i.e. this PR to be merged) before we release a stable version of the finetuning code. Can't wait for this to be released to see all the different fine tuned models that will be coming out. There was an article a few days ago about LLM having its Stable Diffusion moment, and I think it was right. We went from LLama release > llama.cpp > Dalai and…

Yeah this whole industry seems to suddenly have moved from the pace of a pebble creek to a waterfall. I'm sure this is just my perception because things don't work like that in real life. Is it really just a ton of slowly developed things happening to mature close together?

Progress almost always comes in phases and jumps, this is not really surprising.

Re: Alpaca: A strong open-source instruction-following model

#126

Post by isoprophlex »

cerol wrote: post #122

Can someone provide a guide on how run LLaMA on a fairly average CPU/Nvidia GPU?

Check the git repo for llama.cpp.

Model weights in torrent magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352

Runs slowly but without trouble on an intel i7 with 32 gb ram.

Re: Alpaca: A strong open-source instruction-following model

#127

Post by IanCal »

Tiberium wrote: post #113
freedmand wrote: post #64

Earlier quoted context omitted.

The instruction tuning dataset is only 52,000 rows. It shouldn't be too hard to crowdsource high-quality human answers to this many questions and retrain the model, nixing the dependency on OpenAI.

Such a thing already exists and there were some results - https://open-assistant.io I'm not sure why the authors of Alpaca didn't try to train it on this dataset.

That dataset isn't released yet.

> Can I download the data?

> You will be able to, under CC BY 4.0, but it's not released yet. We want to remove spam and PII before releasing it.

Re: Alpaca: A strong open-source instruction-following model

#128

Post by mattkrause »

IanCal wrote: post #106
rjb7731 wrote: post #68

Earlier quoted context omitted.

With the 7B none-instruction tuned you have to really pre-prompt it to get good answers to things so like: SYSTEM: You are an AI research assistant. You use a tone that is technical and scientific. USER: Hello, who are you? ASSISTANT: Hi! I am an AI research assistant. How can I help you today? USER: Can you tell me something about owls.

I thought I'd put some in as an example. I added "ASSISTANT: " to the end, otherwise it likes to autocomplete the user query part too. This is all with 4 bit quantisation from lambda.cpp, using `--repeat_last_n 256 --repeat_penalty 1.178 --temp 0.7 --top_k 40 --top_p 0`. I've not experimented too much. 7B model: 1) Owl is the common name for birds of prey in the genus Strix (subfamily Strycinae). They have large forw…

It would be even more impressive if it were correct!

- Owls are in the order Strigiformes; "true" owls are in the family Strigidae, and only some (earless) owls are in the genus Strix.

- As for size, https://largest.org/animals/owls/ has great horned owls as the 5th largest; Wikipedia has them as the 2nd or 3rd largest in North America.

- Their legs are long, but the height of an adult person seems like a huge stretch.

Elf owls are adorable though!

Re: Alpaca: A strong open-source instruction-following model

#129

Post by all2 »

wkat4242 wrote: post #85
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

I know, this is crazy!! I can't fathom how development has suddenly seemed to accelerate.

The timing of the Facebook leak seems suspect.

Re: Alpaca: A strong open-source instruction-following model

#130

Post by barking_biscuit »

z3t4 wrote: post #91
londons_explore wrote: post #23

The training data doesn't seem to be great quality... "instruction": "Calculate the sum of each column in the following table.", "input": "2 | 3 | 4\n 5 | 6 | 7", "output": "2+3+4 = 9\n5+6+7 = 18" I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of huma…

Training a model to do math is impossible. If you tell a child that 1+1=2 10+10=20 100+100=200 an "AI" can not figure out that 1000+1000=2000 like a child can.

>Training a model to do math is impossible.

https://www.deepmind.com/blog/discovering-novel-algorithms-w...

Post reply on HN