Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Orca 2: Teaching Small Language Models How to Reason

arxiv.org

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Orca 2: Teaching Small Language Models How to Reason

#41

Post by RationalDino »

coderaptor wrote: post #25
FFP999 wrote: post #21

Earlier quoted context omitted.

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

I haven’t heard a definition of “reasoning” or “thinking” that proves humans aren’t doing exactly that same probabilistic regurgitation. I don’t think it’s possible to prove; feels like a philosophical question.

I won't define reasoning, just call out one aspect.

We have the ability to follow a chain of reasoning, say "that didn't work out", backtrack, and consider another. ChatGPT seems to get tangled up when its first (very good) attempt goes south.

This is definitely a barrier that can be crossed by computers. AlphaZero is better than we are at it. But it is a thing we do which we clearly don't simply do with the probabilistic regurgitation method that ChatGPT uses.

That said, the human brain combines a bunch of different areas that seem to work in different ways. Our ability to engage in this kind of reason, for example, is known to mostly happen in the left frontal cortex. So it seems likely that AGI will also need to combine different modules that work in different ways.

On that note, when you add tools to ChatGPT, it suddenly can do a lot more than it did before. If those tools include the right feedback loops, the ability to store/restore context, and so on, what could it then do? This isn't just a question of putting the right capabilities in a box. They have to work together for a goal. But I'm sure that we haven't achieved the limit of what can be achieved.

Re: Orca 2: Teaching Small Language Models How to Reason

#42

Post by xanderlewis »

coderaptor wrote: post #25
FFP999 wrote: post #21

Earlier quoted context omitted.

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

I haven’t heard a definition of “reasoning” or “thinking” that proves humans aren’t doing exactly that same probabilistic regurgitation. I don’t think it’s possible to prove; feels like a philosophical question.

It feels like humans do do a similar regurgitation as part of a reasoning process, but if you play around with LLMs and ask them mathematical questions beyond the absolute basics it doesn’t take long before they trip up and reveal a total lack of ‘understanding’ as we would usually understand it. I think we’re easily fooled by the fact that these models have mastered the art of talking like an expert. Within any domain you choose, they’ve mastered the form. But it only takes a small amount of real expertise (or even basic knowledge) to immediately spot that it’s all gobbledygook and I strongly suspect that when it isn’t it’s just down to luck (and the fact that almost any question you can ask has been asked before and is in the training data). Given the amount of data being swallowed, it’s hard to believe that the probabilistic regurgitation you describe is ever going to lead to anything like ‘reasoning’ purely through scaling. You’re right that asking what reasoning is may be a philosophical question, but you don’t need to go very far to empirically verify that these models absolutely do not have it.

Re: Orca 2: Teaching Small Language Models How to Reason

#43

Post by borg16 »

intended wrote: post #8

I really really want this to work. However at this point - benchmark success is about as effective as results from someone who has been “taught the test” If say… Merck wanted to use this same model to reason out a logistics issue, or apply it to some business problem at scale - you’d have to deal with hallucinations all over the place. The best analogy I have right now is that improved results on benchmarks are like…

> Merck wanted to use this same model to reason out a logistics issue, or apply it to some business problem at scale - you’d have to deal with hallucinations all over the place.

I wouldn't think Merck would leave it all to the model? There will be humans still in the loop ensuring that the output is valid for their use case? I don't think we are still there yet where we can completely productionalize these models without any human involvement later on whatsoever.

Re: Orca 2: Teaching Small Language Models How to Reason

#45

Post by intended »

FFP999 wrote: post #31
intended wrote: post #29

Earlier quoted context omitted.

It’s possible to prove. Use an LLM to do a real world task that you should be able to achieve by reasoning.

> Use an LLM to do a real world task that you should be able to achieve by reasoning. Such as explaining the logical fallacies in this argument and the one above?

Take anything, see how far you get before you have to really grapple with hallucination.

Once that happens, your mitigation strategy will end up being the proof.

Re: Orca 2: Teaching Small Language Models How to Reason

#46

Post by kgeist »

FFP999 wrote: post #21
intended wrote: post #8

I really really want this to work. However at this point - benchmark success is about as effective as results from someone who has been “taught the test” If say… Merck wanted to use this same model to reason out a logistics issue, or apply it to some business problem at scale - you’d have to deal with hallucinations all over the place. The best analogy I have right now is that improved results on benchmarks are like…

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

Just yesterday I saw an example of a person asking GPT what "fluftable" means. The word was invented by their little daughter and they didn't know what it meant. GPT reasoned it was a portmaneau of"fluffy" and "comfortable", and it made sense because it was used in reference to a pillow. If it's just regurgitation, I'd like to know how it's able to understand novel words not found in the training data...

Re: Orca 2: Teaching Small Language Models How to Reason

#47

Post by titaniumtown »

davidkunz wrote: post #4

For smaller models, I'm impressed by Mistral-7b or fine-tuned variants like Zephyr. I use it regularly in Neovim[1] for mundane tasks (grammar correction, summaries, ...). I'm curious how Orca 2 performs, downloading it right now. [1]: with https://github.com/David-Kunz/gen.nvim

Haven't seen this neovim plugin before! I'm setting this up right now.

Re: Orca 2: Teaching Small Language Models How to Reason

#48

Post by nodja »

GaggiX wrote: post #7
davidkunz wrote: post #4

For smaller models, I'm impressed by Mistral-7b or fine-tuned variants like Zephyr. I use it regularly in Neovim[1] for mundane tasks (grammar correction, summaries, ...). I'm curious how Orca 2 performs, downloading it right now. [1]: with https://github.com/David-Kunz/gen.nvim

Also OpenChat-3.5v model (It has 7B parameters, I think it is also a Mistral finetuning), demo: https://openchat.team/

oh wow this model is kinda amazing, it passes my "creative" tests that only chatgpt 3.5 did decently well with, I've recently been disillusioned that open source has been moving the wrong way due to the focus on benchmarks, but this model seems to hit the spot in usefulness in more whacky prompts ("write X in the style of Y" kinda prompts)

Re: Orca 2: Teaching Small Language Models How to Reason

#49

Post by jjtheblunt »

amelius wrote: post #44

This is why imho Microsoft is way cooler than Apple. They have tons of published research. In Apple, even speaking about your research with a friend may result in severe punishment.

Apple publishes too, search for it for example, but much less.

Re: Orca 2: Teaching Small Language Models How to Reason

#50

Post by gnaritas99 »

FFP999 wrote: post #21
intended wrote: post #8

I really really want this to work. However at this point - benchmark success is about as effective as results from someone who has been “taught the test” If say… Merck wanted to use this same model to reason out a logistics issue, or apply it to some business problem at scale - you’d have to deal with hallucinations all over the place. The best analogy I have right now is that improved results on benchmarks are like…

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

You are simply incorrect. They can reason.
Post reply on HN