Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

81–90 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#81

Post by photon_lines »

CooCooCaCha wrote: post #61
photon_lines wrote: post #44

Earlier quoted context omitted.

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

I’m curious how this would work considering knowledge can be fuzzy. Like if I’m out camping and I sit on a log or a rock those things are not what people usually think of as chairs but they can serve as chairs in that situation.

You can get models to actually show that 'logs' could function as 'chairs.' You're forgetting that we humans also learn this as well, but we learn this in a much simpler manner than LLMs though so someone has to explicitly let models know what assumptions they can make. You get the LLM to write Prolog programs and learn associations in this manner. As the model gets better at logically modelling the problems - the solutions to prompted problems like this should get better.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#82

Post by michaelfeathers »

This is a good talk about the problem: https://youtu.be/hGXhFa3gzBs?si=15IJsTQLsyDvBFnr

Key takeaway, LLMs are abysmal at planning and reasoning. You can give them the rules of planning task and ask them for a result but, in large part, the correctness of their logic (when it occurs) depends upon additional semantic information rather then just the abstract rules. They showed this by mapping nouns to a completely different domain in rule and input description for a task. After those simple substitutions, performance fell apart. Current LLMs are mostly pattern matchers with bounded generalization ability.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#83

Post by vaylian »

sys32768 wrote: post #57

As a non-coder I can get away with asking this: Why is it so hard to simulate reason? Logic and reason are based on rules. Then you add values to steer the conclusions based on the available data. Why not have separate systems for values and logic and memory working together as an AI brain to generate truly reasoned responses? You could even have adversarial parts that duke it out (left-wing vs right-wing, Jefferson…

LLMs operate on word co-occurances. Given an initial prompt (consisting of words), it tries to predict the most likely next word. And then it goes on to predict the next most likely word based on the previous output. There is also a bit of randomization involved, so that it doesn't always produce the same results. Some people call these LLMs "statistical parrots", because the LLMs simply repeat the word combinations that they were trained with. But despite this very simple approach, we get very convincing outputs for things that do not require reasoning. LLMs might have encountered examples of logical reasoning in their training data, but they can't generalize it to arbitrary logical reasoning, because that's not something they are designed to do.

Disclaimer: I'm not an expert in AI and I do not follow the developments on a deep technical level.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#84

Post by elpocko »

gwbas1c wrote: post #52

I just played the game and sent ChatGPT (free, I think 3.5) "Alice has 5 sisters and 3 bothers. How many sister's does Alice's bother have?" The whole thing felt like interacting with your typical support rep who's friendly but otherwise has no common sense and intuition about the thing they're supporting. In other words, it felt like I was interacting with a typical "not so smart but friendly and overconfident" huma…

>"Alice has 5 sisters and 3 bothers. How many sister's does Alice's bother have?"

To be fair -- you wrote "bother" twice, and "sister's" with an apostrophe. Technically the question makes no sense like that.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#85

Post by sosuke »

llm_trw wrote: post #23
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

>Worth noting that the prompts from the experiment include "To answer the question, DO NOT OUTPUT ANY TEXT EXCEPT following format that contains final answer: ### Answer:" so it appears that they are stopping the models from 'thinking out loud'. If I add that to the prompt, GPT4o gets it consistently wrong... Yes this is a common thing I see people who think LLMs are idiots do. The more an LLM talks the smarter it ge…

New option needed besides concise. Think quietly on your own, then answer concisely.

Sometimes I think I'd prefer it to "think" before answering anyhow. The immediate thinking out loud text can be irritating for some irrational reason.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#86

Post by michael-ax »

toxik wrote: post #7
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

Seems like an odd thing not to state prominently. ChatGPT 4o solved it every time for me. Seems to work OK even with the "hands tied behind back" prompt. Wonder if this paper is just inaccurate or if OpenAI adjusted the model -- seems unlikely. Actually, 4o is reported to solve this riddle 60% of the time. I guess I have to test more, but with my about a dozen tests so far, all were correct.

Alice has N Brothers, and she has M sisters. How many sisters do Alice’s brothers have?

I have not gotten the correct answer to the question as phrased above in one go from Gpt4o yet! (and today was not the first day i tried.)

Phrase it as shown above and you'll likely need 5 or more interactions to get it to generate the correct output. With Gemini i could not get it below 8 without feeling like i was cheating. fwiw.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#87

Post by wtbdqrs »

nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

I appear to be reasoning at times but I have mostly no idea what I am talking about. I hit a bunch of words and concepts in the given context and thus kind of hallucinate sense.

Given a few months of peace of mind and enough money for good enough food, I could actually learn to reason without sounding like a confused babelarian.

Reasoning is mostly a human convention supported by human context that would have been a different one if the Fascists had won the war or the Soviet Union wouldn't have gotten corrupted.

But none of that has anything to do with pulling up a whiteboard to draw some flowcharts and run some numbers, all of which is why I am certain there is nothing the devs have "to fix". It took most reasonable humans many generations to learn stuff. Very few of us did the actual work.

It's all just a matter of time.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#88

Post by monsieurbanana »

sosuke wrote: post #85
llm_trw wrote: post #23

Earlier quoted context omitted.

>Worth noting that the prompts from the experiment include "To answer the question, DO NOT OUTPUT ANY TEXT EXCEPT following format that contains final answer: ### Answer:" so it appears that they are stopping the models from 'thinking out loud'. If I add that to the prompt, GPT4o gets it consistently wrong... Yes this is a common thing I see people who think LLMs are idiots do. The more an LLM talks the smarter it ge…

New option needed besides concise. Think quietly on your own, then answer concisely. Sometimes I think I'd prefer it to "think" before answering anyhow. The immediate thinking out loud text can be irritating for some irrational reason.

Just telling the llm to preface it's conclusion with "Answer:" in a parseable way would be better.

You're still paying for those thinking tokens, or at the very least have to wait for them to be generated.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#89

Post by Closi »

ModernMech wrote: post #76
Closi wrote: post #71

Earlier quoted context omitted.

ChatGPT 4o seems to occasionally get this correct, but still thinks it's a puzzle and will often get this wrong... https://chatgpt.com/share/b8901f4c-7313-4461-acad-9e561c1e6e... Still seems like it fares slightly better than some of the other models.

Yeah, the overfitting is apparent with these kinds of edge cases, which makes you wonder where else it's over fitting. I can actually tell this when I'm working in a programming domain where I'm an expert and where a compiler holds the source of truth, but not in others, so I'm wary to use it there.

Interestingly you get a great answer if you change the prompt as follows which implies that it is not a puzzle:

> I am on the shore of a river with a goat and a boat. How do I get to the other side? I work in commercial agriculture logistics.

This gets a great answer even with GPT3.5

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#90

Post by tsunamifury »

pawelmurias wrote: post #59
llm_trw wrote: post #23

Earlier quoted context omitted.

>Worth noting that the prompts from the experiment include "To answer the question, DO NOT OUTPUT ANY TEXT EXCEPT following format that contains final answer: ### Answer:" so it appears that they are stopping the models from 'thinking out loud'. If I add that to the prompt, GPT4o gets it consistently wrong... Yes this is a common thing I see people who think LLMs are idiots do. The more an LLM talks the smarter it ge…

LLMs are idiots. They can't reason properly and only parrot stuff https://chatgpt.com/share/dcb4ff4e-e8a2-463b-86ec-9caf10b6e6... Sometimes they get the answer right to something really complex because it fits a pattern, but sometimes they answer with something really really stupid.

Why are so many people so insistent on saying this?

I’m guessing you are in denial that we can make a simulated reasoning machine?

Post reply on HN