Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Orca 2: Teaching Small Language Models How to Reason

arxiv.org

Post reply on HN
Print view View original

51–60 of 82 posts

Re: Orca 2: Teaching Small Language Models How to Reason

#51

Post by cloverich »

coderaptor wrote: post #25
FFP999 wrote: post #21

Earlier quoted context omitted.

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

I haven’t heard a definition of “reasoning” or “thinking” that proves humans aren’t doing exactly that same probabilistic regurgitation. I don’t think it’s possible to prove; feels like a philosophical question.

On the other hand, it seems rather intuitive we have a logic based component? Its the underpinning of science. We have to be taught when we've stumbled upon something that needs tested. But we can be taught that. And then once we learn to recognize it, we intuitively do so in action. ChatGPT can do this in a rudimentary way as well. It says a program should work a certain way. Then it writes it. Then it runs it. Then when the answer doesn't come out as expected (at this point, probably just error cases), it goes back and changes it.

It seems similar to what we do, if on a more basic level. At any rate, it seems like a fairly straight forward 1-2 punch that, even if not truly intelligent, would let it break through its current barriers.

Re: Orca 2: Teaching Small Language Models How to Reason

#52

Post by azmodeus »

RationalDino wrote: post #41
coderaptor wrote: post #25

Earlier quoted context omitted.

I haven’t heard a definition of “reasoning” or “thinking” that proves humans aren’t doing exactly that same probabilistic regurgitation. I don’t think it’s possible to prove; feels like a philosophical question.

I won't define reasoning, just call out one aspect. We have the ability to follow a chain of reasoning, say "that didn't work out", backtrack, and consider another. ChatGPT seems to get tangled up when its first (very good) attempt goes south. This is definitely a barrier that can be crossed by computers. AlphaZero is better than we are at it. But it is a thing we do which we clearly don't simply do with the probabil…

Instead of going bank you can construct a tree of different reasonings with an LLM then take a vote or synthesise see Tee of thought prompting

Re: Orca 2: Teaching Small Language Models How to Reason

#53

Post by amelius »

jjtheblunt wrote: post #49
amelius wrote: post #44

This is why imho Microsoft is way cooler than Apple. They have tons of published research. In Apple, even speaking about your research with a friend may result in severe punishment.

Apple publishes too, search for it for example, but much less.

Much, much, less. They are definitely not in the same league.

Re: Orca 2: Teaching Small Language Models How to Reason

#54

Post by jpdus »

alecco wrote: post #16

> Progressive Learning: We start with LLaMA-2-7B or LLaMA-2-13B checkpoint and finetune it on the train split of FLAN-v2 dataset for one epoch. Note that FLAN-v2 dataset contains both zero-shot and few-shot problems. We then train on 5 million ChatGPT data from Orca 1 for 3 epochs. Then we train on the combination of 1 million GPT-4 data from Orca 1 and Orca 2’s 817K data for 4 epochs. I think people are missing why…

It isn't.

Compared to the original Orca model and method which spawned many of the current SotA OSS models, Orca 2 models seem to perform underwhelming, below outdated 13b models and below Mistral 7b base models (e.g. [1]; didn't test myself yet, ymmv).

[1] https://twitter.com/abacaj/status/1727004543668625618?t=R_vV...

Re: Orca 2: Teaching Small Language Models How to Reason

#55

Post by sorokod »

nodja wrote: post #48
GaggiX wrote: post #7

Earlier quoted context omitted.

Also OpenChat-3.5v model (It has 7B parameters, I think it is also a Mistral finetuning), demo: https://openchat.team/

oh wow this model is kinda amazing, it passes my "creative" tests that only chatgpt 3.5 did decently well with, I've recently been disillusioned that open source has been moving the wrong way due to the focus on benchmarks, but this model seems to hit the spot in usefulness in more whacky prompts ("write X in the style of Y" kinda prompts)

Always surprised how poorly these models do on the benchmarks they claim to do well. OpenChat has a benchmark radar diagram[1] but but often fails on actual samples.

[1] https://github.com/imoneoi/openchat

Re: Orca 2: Teaching Small Language Models How to Reason

#56

Post by trash_cat »

btbuildem wrote: post #10

Are we beginning to see "specialized SLMs"? We've already seen some pretend-agent based solutions (where the same model is given several different roles and made to act as eg. ceo / architect / dev / sales in a startup). I wonder if the way forward is to train smaller models with different sets of "skills" or "neural affinities". One for reasoning, one for summarization, one for math, one for code, etc - then combini…

Yes, this is the trend. OAIs marketplace of GPTs is a confirmation of this. BabyAGI, AutoGen, AutoGPT are all multiple LLM/SLM architectures under the hood. While we don´t have access to proprietary data or the ability to run bigger models, the natural direction is to combine them with specialized tasks like you just described. The issue is then the interface - making it good and communicate seamlessly between models and what roles they + the architecture the models are operating in. The last point is up to your imagination.

Re: Orca 2: Teaching Small Language Models How to Reason

#57

Post by esafak »

worldsayshi wrote: post #12
btbuildem wrote: post #10

Are we beginning to see "specialized SLMs"? We've already seen some pretend-agent based solutions (where the same model is given several different roles and made to act as eg. ceo / architect / dev / sales in a startup). I wonder if the way forward is to train smaller models with different sets of "skills" or "neural affinities". One for reasoning, one for summarization, one for math, one for code, etc - then combini…

Isn't this the whole idea with Mixture Of Experts approach that is GPT-4 is using?

Is GPT-4's MOE based on combining specialized models?

Re: Orca 2: Teaching Small Language Models How to Reason

#58

Post by sa-code »

hmottestad wrote: post #24
yujian wrote: post #20

I'm not sure if I'm missing something from the paper, but are multi-billion parameter models getting called "small" language models now? And when did this paradigm shift happen?

All the llama models, including the 70B one can run on consumer hardware. You might be able to fit GPT-3 (175B) at Q4 or Q3 on a Mac Studio, but that's probably the limit for consumer hardware. At 4-bit a 7B model requires some 4GB of ram, so that should probably be possible to run on a phone, just not very fast.

Gpt 3.5 turbo is 20B

Re: Orca 2: Teaching Small Language Models How to Reason

#59

Post by Davidzheng »

RationalDino wrote: post #41
coderaptor wrote: post #25

Earlier quoted context omitted.

I haven’t heard a definition of “reasoning” or “thinking” that proves humans aren’t doing exactly that same probabilistic regurgitation. I don’t think it’s possible to prove; feels like a philosophical question.

I won't define reasoning, just call out one aspect. We have the ability to follow a chain of reasoning, say "that didn't work out", backtrack, and consider another. ChatGPT seems to get tangled up when its first (very good) attempt goes south. This is definitely a barrier that can be crossed by computers. AlphaZero is better than we are at it. But it is a thing we do which we clearly don't simply do with the probabil…

these are things we can teach children to do when they don't do it at first. I don't see why we can't teach this behavior to AI. Maybe we should teach LLM's to play games or something. or do those proof thingys that they teach in US high school geometry or something like that. To learn some formal structure within which they can think about the world

Re: Orca 2: Teaching Small Language Models How to Reason

#60

Post by GenericPoster »

FFP999 wrote: post #21
intended wrote: post #8

I really really want this to work. However at this point - benchmark success is about as effective as results from someone who has been “taught the test” If say… Merck wanted to use this same model to reason out a logistics issue, or apply it to some business problem at scale - you’d have to deal with hallucinations all over the place. The best analogy I have right now is that improved results on benchmarks are like…

At the moment I read "how to reason" in the headline my bullshit detector started to go off. LLMs do not reason, they do not think, they are not AGI. They generate by regurgitating.

Did you only read the title? Because the abstract gives you a pretty good idea of what they mean when they say reason. It's pretty easy to understand. No need to immediately call bullshit just because of a minor semantic disagreement.

>ThEY DON'T tHiNk. They'rE JuSt STochAStiC pARrotS. It'S not ReAL AGi.

It doesn't even matter if these claims are true or not. They're missing the point of the conversation and the paper. Reason is a perfectly valid word to use. So is think. If you ask it a question and then follow up with 'think carefully' or 'explain carefully'. You'll get the same response.

inb4 AcTUALLy LlMS Can'T do aNYtHIng CaRefUlly BECaUse pRogRAms ARen'T caRefUl

Post reply on HN