Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Orca 2: Teaching Small Language Models How to Reason

arxiv.org

Post reply on HN
Print view View original

61–70 of 82 posts

Re: Orca 2: Teaching Small Language Models How to Reason

#61

Post by svaha1728 »

kgeist wrote: post #46
FFP999 wrote: post #21

Earlier quoted context omitted.

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

Just yesterday I saw an example of a person asking GPT what "fluftable" means. The word was invented by their little daughter and they didn't know what it meant. GPT reasoned it was a portmaneau of"fluffy" and "comfortable", and it made sense because it was used in reference to a pillow. If it's just regurgitation, I'd like to know how it's able to understand novel words not found in the training data...

I would read Francois Chollet's explanation of this. It's very good: https://fchollet.substack.com/p/how-i-think-about-llm-prompt...

For words that are not in the model's vocabulary, like 'fluftable', the model uses a subword tokenization strategy. It breaks down the word into smaller known subunits (subwords or characters) and represents each subunit with its own vector. By understanding the context in which 'fluftable' appears and comparing it to known words with similar subunits, the model can infer a plausible meaning for the word. This is done by analyzing the vector space in which these representations exist, observing how the vectors align or differ from those of known words.

'As always, the most important principle for understanding LLMs is that you should resist the temptation of anthropomorphizing them.'

Re: Orca 2: Teaching Small Language Models How to Reason

#62

Post by imhoguy »

btbuildem wrote: post #10

Are we beginning to see "specialized SLMs"? We've already seen some pretend-agent based solutions (where the same model is given several different roles and made to act as eg. ceo / architect / dev / sales in a startup). I wonder if the way forward is to train smaller models with different sets of "skills" or "neural affinities". One for reasoning, one for summarization, one for math, one for code, etc - then combini…

Specialized LLMs, and likely SLMs too, are really the future. I use them mostly to aid programming and really just stopped paying for GPT-4. Phind and others are really on par now in my coding needs.

Re: Orca 2: Teaching Small Language Models How to Reason

#63

Post by sharemywin »

svaha1728 wrote: post #61
kgeist wrote: post #46

Earlier quoted context omitted.

Just yesterday I saw an example of a person asking GPT what "fluftable" means. The word was invented by their little daughter and they didn't know what it meant. GPT reasoned it was a portmaneau of"fluffy" and "comfortable", and it made sense because it was used in reference to a pillow. If it's just regurgitation, I'd like to know how it's able to understand novel words not found in the training data...

I would read Francois Chollet's explanation of this. It's very good: https://fchollet.substack.com/p/how-i-think-about-llm-prompt... For words that are not in the model's vocabulary, like 'fluftable', the model uses a subword tokenization strategy. It breaks down the word into smaller known subunits (subwords or characters) and represents each subunit with its own vector. By understanding the context in which 'flufta…

isn't 'infer' another word for reason?

Re: Orca 2: Teaching Small Language Models How to Reason

#64

Post by kristianp »

sa-code wrote: post #58
hmottestad wrote: post #24

Earlier quoted context omitted.

All the llama models, including the 70B one can run on consumer hardware. You might be able to fit GPT-3 (175B) at Q4 or Q3 on a Mac Studio, but that's probably the limit for consumer hardware. At 4-bit a 7B model requires some 4GB of ram, so that should probably be possible to run on a phone, just not very fast.

Gpt 3.5 turbo is 20B

I doubt that. What's your source?

Re: Orca 2: Teaching Small Language Models How to Reason

#65

Post by sharemywin »

pelorat wrote: post #37
FFP999 wrote: post #21

Earlier quoted context omitted.

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

Reasoning is a learnt concept that involves retrieving memories and running them though an algorithm, also retrieved from memory, and then you loop the process until a classifier deems the result to be adequate to the given goal.

I asked GPT 4 and it had some counter points:

Reasoning blends learned skills and natural cognition. It integrates new information, not just past memories. Reasoning is adaptable, not rigidly algorithmic. Emotions and context also shape reasoning.

which seemed to make sense.

Re: Orca 2: Teaching Small Language Models How to Reason

#66

Post by svaha1728 »

sharemywin wrote: post #63
svaha1728 wrote: post #61

Earlier quoted context omitted.

I would read Francois Chollet's explanation of this. It's very good: https://fchollet.substack.com/p/how-i-think-about-llm-prompt... For words that are not in the model's vocabulary, like 'fluftable', the model uses a subword tokenization strategy. It breaks down the word into smaller known subunits (subwords or characters) and represents each subunit with its own vector. By understanding the context in which 'flufta…

isn't 'infer' another word for reason?

vector math in a 1536-dimensional space?

Re: Orca 2: Teaching Small Language Models How to Reason

#67

Post by ganeshkrishnan »

T-A wrote: post #35
alecco wrote: post #16

> Progressive Learning: We start with LLaMA-2-7B or LLaMA-2-13B checkpoint and finetune it on the train split of FLAN-v2 dataset for one epoch. Note that FLAN-v2 dataset contains both zero-shot and few-shot problems. We then train on 5 million ChatGPT data from Orca 1 for 3 epochs. Then we train on the combination of 1 million GPT-4 data from Orca 1 and Orca 2’s 817K data for 4 epochs. I think people are missing why…

...and quantized ones from the usual suspect: https://huggingface.co/TheBloke/Orca-2-7B-GGUF https://huggingface.co/TheBloke/Orca-2-13B-GGUF The 7B Q5_K_M one is small enough to run on an 8GB consumer GPU.

All the 13B files seems to be quantized.

Re: Orca 2: Teaching Small Language Models How to Reason

#68

Post by jug »

This sounds quite exciting! Like Mistral all over again, only more transparent, open, and major backing probably as Microsoft are looking to significantly reduce costs now that they're expanding AI wide across their platforms? The approach truly feels like a next step in LLM design.

Re: Orca 2: Teaching Small Language Models How to Reason

#69

Post by avion23 »

sharemywin wrote: post #65
pelorat wrote: post #37

Earlier quoted context omitted.

Reasoning is a learnt concept that involves retrieving memories and running them though an algorithm, also retrieved from memory, and then you loop the process until a classifier deems the result to be adequate to the given goal.

I asked GPT 4 and it had some counter points: Reasoning blends learned skills and natural cognition. It integrates new information, not just past memories. Reasoning is adaptable, not rigidly algorithmic. Emotions and context also shape reasoning. which seemed to make sense.

I hope this will be found in history books and some students will point the irony that people are relying on gpt4's arguments about reasoning in a thread where it's proclaimed that said model can't reason

Re: Orca 2: Teaching Small Language Models How to Reason

#70

Post by lucubratory »

svaha1728 wrote: post #61
kgeist wrote: post #46

Earlier quoted context omitted.

Just yesterday I saw an example of a person asking GPT what "fluftable" means. The word was invented by their little daughter and they didn't know what it meant. GPT reasoned it was a portmaneau of"fluffy" and "comfortable", and it made sense because it was used in reference to a pillow. If it's just regurgitation, I'd like to know how it's able to understand novel words not found in the training data...

I would read Francois Chollet's explanation of this. It's very good: https://fchollet.substack.com/p/how-i-think-about-llm-prompt... For words that are not in the model's vocabulary, like 'fluftable', the model uses a subword tokenization strategy. It breaks down the word into smaller known subunits (subwords or characters) and represents each subunit with its own vector. By understanding the context in which 'flufta…

I'm sorry, but that's absurd. Being able to explain the precise mechanism behind reasoning would make anything sound like it's not reasoning, because of our prior experiences. If we understood human reasoning well enough to explain exactly what happens in our brain, you would conclude that we're not really reasoning because you can provide an explanation of how we're reasoning about novel, out of distribution data. This is "God of the gaps" for thought.
Post reply on HN