Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

121–130 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#121

Post by dragonwriter »

ip26 wrote: post #113
nerdjon wrote: post #54

Earlier quoted context omitted.

I honestly thought about this recently when I was trying to see the limits of Claude Opus. Some of the problems I gave it, what if instead of telling it to solve the problem I asked it to write the script and then give me the command and inputs needed to properly run it to get the answer I needed. That way instead of relying on the LLM to do properly analysis of the numbers it just needs to understand enough to write…

I don’t understand why LLM’s aren’t already set up to do what you describe automatically behind the curtain. Extract a math equation from text (LLMs are good at translating between languages right?) and immediately evaluate it on the host CPU. LLM is the equivalent of recalling your times tables. Computer arithmetic is the equivalent of re-computing your times tables.

> I don’t understand why LLM’s aren’t already set up to do what you describe automatically behind the curtain.

LLM-based systems with tool use (which this is an application of) often are, to an extent, the issue is tuning the (behind the scenes, system) prompting so that they use appropriate tools in every case where they should, and do so correctly. (There's also a cost factor involved since behind-the-scenes tool use means multiple LLM round trips to answer the question, so tuning the system to use tools more aggressively makes the system more expensive.)

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#122

Post by gs17 »

visarga wrote: post #45

This prompt was often discussed on forums in the last year, we knew models fail it

No reason not to evaluate it more in depth, though. Understanding not just that they fail but if some models fail more/less and under what conditions is still useful.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#123

Post by kveykva »

sollewitt wrote: post #105
layer8 wrote: post #94

Earlier quoted context omitted.

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

Have there ever been studies that demonstrate that those individuals don't simulate possible state transitions they'll go through in a different modality? I'd be curious if they visualize actions they'll take still, just not verbally.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#124

Post by gs17 »

mvkel wrote: post #26

Isn't this just saying what we already knew, that tokenization has these fundamental flaws? Did we need a paper for it?

It is, I'd even say it's just "LLMs aren't very good at math" rather than a "Complete Reasoning Breakdown". Still, evaluating when/how they fail could be useful.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#125

Post by more_corn »

ericmcer wrote: post #118
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

If you really think about what an LLM is you would think there is no way that leads to general purpose AI. At the same time though they are already doing way more than we thought they could. Maybe people were surprised by what OpenAI achieved so now they are all just praying that with enough compute and the right model AGI will emerge.

LLMs are the language center of an AI. They in no way purport to be: long term memory, critical thinking engines, calculators.

If we want those things we can build them. Building them into the language center would be absurd and weird.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#126

Post by Karellen »

sdiepend wrote: post #20
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

The right answer depends on how Alice identifies I guess? :)

> > Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?"

> The right answer depends on how Alice identifies I guess? :)

Given that the wording of the question specifically identifies Alice as "she", rather than using a gender-neutral pronoun or no pronoun at all, I think inferring that she identifies as female is reasonable.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#127

Post by [deleted] »

Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

[deleted]

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#128

Post by solidasparagus »

ericmcer wrote: post #118
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

If you really think about what an LLM is you would think there is no way that leads to general purpose AI. At the same time though they are already doing way more than we thought they could. Maybe people were surprised by what OpenAI achieved so now they are all just praying that with enough compute and the right model AGI will emerge.

> If you really think about what an LLM is you would think there is no way that leads to general purpose AI

It is an autoregressive sequence predictor/generator. Explain to me how humans are fundamentally different

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#129

Post by ElevenLathe »

sollewitt wrote: post #105
layer8 wrote: post #94

Earlier quoted context omitted.

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

The existence of an "inner monologue" isn't really a falsifiable claim. Some people claim to have one while other people claim not to, but we can't test the truth of these claims.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#130

Post by voxic11 »

wtbdqrs wrote: post #87
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

I appear to be reasoning at times but I have mostly no idea what I am talking about. I hit a bunch of words and concepts in the given context and thus kind of hallucinate sense. Given a few months of peace of mind and enough money for good enough food, I could actually learn to reason without sounding like a confused babelarian. Reasoning is mostly a human convention supported by human context that would have been a…

Yeah, I think these chatbots are just too sure of themselves. They only really do "system 1 thinking" and only do "system 2 thinking" if you prompt them to. If I ask gpt-4o the riddle in this paper and tell it to assume its reasoning contains possible logical inconsistencies and to come up with reasons why that might be then it does correctly identify the problems with its initial answer and arrives at the correct one.

Here is my prompt:

I have a riddle for you. Please reason about possible assumptions you can make, and paths to find the answer to the question first. Remember this is a riddle so explore lateral thinking possibilities. Then run through some examples using concrete values. And only after doing that attempt to answer the question by reasoning step by step.

The riddle is "Alice has N brothers and she also has M sisters. How many sisters does Alice’s brother have?"

After you answer the riddle please review your answer assuming that you have made a logical inconsistency in each step and explain what that inconsistency is. Even if you think there is none do your best to confabulate a reason why it could be logically inconsistent.

Finally after you have done this re-examine your answer in light of these possible inconsistencies and give what you could consider a second best answer.

Post reply on HN